"Choppy" message timing behavior seen with fast but not cyclone

[dcconner]

Bug report

Required Info:

• Operating System:

  • Ubuntu 22.04

• Installation type:

  • Rolling binaries (+ custom rclpy fix)

• Version or commit hash:

  • binary - ros-rolling

ii ros-rolling-fastrtps 2.11.0-1jammy.20230711.200844 amd64 eprosima Fast DDS (formerly Fast RTPS) is a C++ implementation of the DDS (Data Distribution Service) standard of the OMG (Object Management Group).
ii ros-rolling-fastrtps-cmake-module 3.2.0-1jammy.20230711.195833 amd64 Provide CMake module to find eProsima FastRTPS.
ii ros-rolling-rmw-fastrtps-cpp 7.3.0-1jammy.20230711.212546 amd64 Implement the ROS middleware interface using eProsima FastRTPS static code generation in C++.
ii ros-rolling-rmw-fastrtps-shared-cpp 7.3.0-1jammy.20230711.211804 amd64 Code shared on static and dynamic type support of rmw_fastrtps_cpp.
ii ros-rolling-rosidl-dynamic-typesupport-fastrtps 0.1.0-1jammy.20230711.203915 amd64 FastDDS serialization support implementation for use with C/C++.
ii ros-rolling-rosidl-typesupport-fastrtps-c 3.2.0-1jammy.20230711.204421 amd64 Generate the C interfaces for eProsima FastRTPS.
ii ros-rolling-rosidl-typesupport-fastrtps-cpp 3.2.0-1jammy.20230711.204023 amd64 Generate the C++ interfaces for eProsima FastRTPS.

• DDS implementation:
  • Fast-RTPS
• Client library (if applicable):
  • rclpy (patched Fix an inherent race in execution vs. destruction. rclpy#1150)

Steps to reproduce issue

Using a custom version of the py_pubsub tutorials
https://github.com/dcconner/py_pubsub

Initially created to debug ros2/rclpy#1147

Also use the rclpy patch ros2/rclpy#1150

Build and run the two commands:

ros2 run py_pubsub talker
ros2 run py_pubsub listener

Without the rclpy patch ros2/rclpy#1150 eventually the subscriptions will stop responding as reported in ros2/rclpy#1147, but we don't think that is related to this issue.

If I run with Cyclone DDS the performance is consistent with expectations.
Messages received, paused for a bit, then messages restart. Timing is consistent

If I run with default RMW then the output is "choppy" and timing is not as predictable.

See ros2/rclpy#1147 (comment) for a sample output.
Notice loss missing data around time = 1691767104.650 in above block, and then again later at time = 1691767116.840

It seemed to recover, but this behavior seems problematic. I did NOT see this with Cyclone

I never saw any indication of issues with the publisher as it seemed to consistently publish and echo to screen at a consistent rate.

Expected behavior

Timing and number of received messages is consistent

Actual behavior

Timing and number of received messages varies significantly.