Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ROS2] Pausing Cameras #150

Closed
InvincibleRMC opened this issue Jan 30, 2024 · 24 comments
Closed

[ROS2] Pausing Cameras #150

InvincibleRMC opened this issue Jan 30, 2024 · 24 comments

Comments

@InvincibleRMC
Copy link

Hello is it possible to pause a flir camera using the ros-driver? Currently cannot run three flir cameras on my network. I can run two just fine but, am being limited by a 1 gig network. I only need 2 feeds at the same time so I am curios about somehow pausing a camera to free up network bandwidth and cycling through which cameras I display. I messed around with trigger_mode = "Software" but, did not get anywhere with it.

@berndpfrommer
Copy link
Collaborator

Are you using the ROS1 or the ROS2 driver?

@InvincibleRMC
Copy link
Author

Using the ROS2 driver with Iron.

@berndpfrommer
Copy link
Collaborator

So at the moment this is not possible, but if you commit to testing, I will implement it. We have 3 options:

  1. Service call to start/stop the camera. Not my favorite because pretty inconvenient.
  2. Stop the camera the when the last subscriber is gone. The cleanest but with some potential to cause crashes. Could be difficult to implement but from a technical point my favorite
  3. Have a boolean rosparam "run" that toggles start/stop.
    What are your thoughts?

@InvincibleRMC
Copy link
Author

InvincibleRMC commented Jan 31, 2024

To me I believe a service or rosparam makes more sense. Option number 2 requires that a subscriber be destroyed then recreated to pause and unpause wouldn't that cause lots of unnecessary overhead/latency?

@berndpfrommer
Copy link
Collaborator

True, latency is there, but I'm more worried about the stability of the system. A Spinnaker start/stop is a pretty heavy weight operation and I'm starting to lose track of all the threads that are running. I did a draft implementation of 2) and in the process hosed my laptop such that none of the spinnaker cameras would start again, not even with SpinView, after replugging the cameras. I fear implementing a service call or a toggle parameter won't be much safer. The only upside there is that the code path is less frequently exercised than subscribe/unsubscribe. But that also means it's a feature that is likely to be not well tested and causes driver crashes every now and then.

@InvincibleRMC
Copy link
Author

That makes sense. My use case doesn't require lots of toggles so if there is worse latency I could live with that. Is there anything else you want from me like the testing you had mentioned?

@berndpfrommer
Copy link
Collaborator

I need a few days to finish up another project I'm on. Hope to get to this one on the weekend.

@berndpfrommer
Copy link
Collaborator

Looking busy so must push this by another week, but not forgotten!

@berndpfrommer
Copy link
Collaborator

Please test the driver in this branch and let me know how it goes. It has a feature connect_while_subscribed that you must set to True in the launch file (default is False).

@berndpfrommer berndpfrommer changed the title Pausing Cameras [ROS2] Pausing Cameras Feb 6, 2024
@InvincibleRMC
Copy link
Author

So having tested the branch it definitely destroys and creates the node/publisher. However, my network usage remains the same. So this sadly does not seem to be helping with my network bandwidth limitations.

@berndpfrommer
Copy link
Collaborator

What do you mean with "destroys and creates the node/publisher"? The ROS node should be left intact, it is just that when you unsubscribe it should disconnect the ROS node from the camera, then reconnect it when you subscribe to the image again. Can you positively assure with "ros2 node info" and "ros2 topic info" that there are no more subscribers to the image?
When testing this with my USB3 cameras the CPU usage of the ROS node dropped to completely zero upon unsubscribing, indicating that there were no more images coming from the SDK to the ROS node.

@InvincibleRMC
Copy link
Author

InvincibleRMC commented Feb 6, 2024

Sorry I was mistaken the ROS node was not destroyed. However it is still not closing the publisher.

ros2 topic info /surface/bottom_cam/image_raw
Type: sensor_msgs/msg/Image
Publisher count: 1
Subscription count: 0

My launch files looks like this. Do I have to do something in the .yaml file I am using as well?

import os

from ament_index_python.packages import get_package_share_directory
from launch.launch_description import LaunchDescription
from launch_ros.actions import Node
from launch_ros.parameter_descriptions import Parameter


def generate_launch_description() -> LaunchDescription:

    parameter_file = os.path.join(get_package_share_directory('rov_flir'), 'config',
                                  'blackfly_s.yaml')

    parameters = {
        'debug': False,
        'compute_brightness': False,
        'adjust_timestamp': True,
        'dump_node_map': False,
        # set parameters defined in blackfly_s.yaml
        'gain_auto': 'Continuous',
        'pixel_format': 'BayerRG8',
        'exposure_auto': 'Continuous',
        # These are useful for GigE cameras
        'device_link_throughput_limit': 125000000,
        'gev_scps_packet_size': 9000,
        # ---- to reduce the sensor width and shift the crop
        #   'image_width': 1280,
        #   'image_height': 720,
        # 'offset_x': 16,
        # 'offset_y': 0,
        'binning_x': 3,
        'binning_y': 3,
        'connect_while_subscribed': True,
        'frame_rate_auto': 'Off',
        'frame_rate': 30.0,
        'frame_rate_enable': True,
        'buffer_queue_size': 1,
        'trigger_mode': 'Off',
        'chunk_mode_active': True,
        'chunk_selector_frame_id': 'FrameID',
        'chunk_enable_frame_id': True,
        'chunk_selector_exposure_time': 'ExposureTime',
        'chunk_enable_exposure_time': True,
        'chunk_selector_gain': 'Gain',
        'chunk_enable_gain': True,
        'chunk_selector_timestamp': 'Timestamp',
        'chunk_enable_timestamp': True
    }

    # launches node to run front flir camera
    front_cam = Node(
        package='spinnaker_camera_driver',
        executable='camera_driver_node',
        name='front_cam',
        emulate_tty=True,
        output='screen',
        parameters=[Parameter('serial_number', '23473577'),
                    Parameter('parameter_file', parameter_file),
                    parameters]
    )

    # launches node to run bottom flir camera
    bottom_cam = Node(
        package='spinnaker_camera_driver',
        executable='camera_driver_node',
        name='bottom_cam',
        emulate_tty=True,
        output='screen',
        parameters=[Parameter('serial_number', '23473566'),
                    Parameter('parameter_file', parameter_file),
                    parameters]
    )

    return LaunchDescription([
        front_cam,
        bottom_cam
    ])

Another thing to note is I am using the POE mode for the cameras not usb.

@berndpfrommer
Copy link
Collaborator

Your launch file has the connect_while_subscribed flag set correctly, so that should work. The yaml file needs no adjustment. The publisher should also be there, and nobody is subscribing to it, so it should definitely disconnect.

For good measure I suggest you put a LOG_INFO("stopping camera") into stopCamera() in camera.cpp, just to be really sure the camera is stopped.

Also you could move the call to deinitCamera() from stop() to stopCamera() and see if that stops the stream of data. It will break the reconnect then but at least we'd learn if that shuts down the network stream.

Apparently GigE vision is based on UDP. You should be able to see a UDP listener appear (netstat -ap) as the camera connects, then see a stream of udp packets coming (tcpdump is helpful here). The UDP listener should go away when the driver exits (or successfully disconnects from the SDK).

@InvincibleRMC
Copy link
Author

Ok it seems to be working great now. Apparently when I cloned with branch command I goofed and cloned the wrong branch.

@berndpfrommer
Copy link
Collaborator

Great! Please use it for maybe a week or two, and if you don't see crashes/hangs I'll merge it into the main development branch.

@InvincibleRMC
Copy link
Author

An unrelated question but is there a way to turn off Camera::PrintStatus()? I see that the Camera constructor takes a useStatus parameter but, I can't see if there is a way to set it from the launch file.

@berndpfrommer
Copy link
Collaborator

The answer is: No. Please open a new issue if you want this implemented. I don't want to conflate the necessary changes with the connect_while_subscribed pull request.

@InvincibleRMC
Copy link
Author

image

Bad news sometimes while still subscribed the image stream stops and doesn't resume without re-listening on the topic. This new issue was quite intermittent issue and took between 5-10 minutes to show up.

@berndpfrommer
Copy link
Collaborator

The only thing I can suggest is to put debug printout statements into checkSubscriptions() to print out the subscription count. Also put statements into startCamera() and stopCamera() so you know if the camera is actually running (meaning the SDK is sending images).

I've had my share of negative experiences with ROS2 transports aka rmw. Weird stuff like this happens to me sometimes and I blame it on myself because hey, <sarcasm>such basic stuff would be noticed by everyone else and it would be fixed, right? </sarcasm>. Bottom line: if this is not a problem with the driver, this may well be a rmw feature. The log you are showing seems to indicate that packets are coming in from the SDK. Why would the driver not publish? Quite possibly your listener somehow can't connect to the topic.

@berndpfrommer
Copy link
Collaborator

I don't really want to understand the details of this, but from what I understand others have experienced similar problems, too. How this can ever be acceptable, I don't know. I guess it's an engineering tradeoff.

ros2/rmw_fastrtps#677

@InvincibleRMC
Copy link
Author

InvincibleRMC commented Feb 16, 2024

Ok as a solution I just added a watch dog that just checks every second to see if the frame updated on my subscriber so if it gets stuck it resets the subscriber. So this branch has solved my use case.

@InvincibleRMC
Copy link
Author

@berndpfrommer Would it be possible to get this merged into iron-devel aswell? That is the ros distro I was testing on.

@berndpfrommer
Copy link
Collaborator

I'll merge it into iron but it requires an update to rosdistro (just submitted the PR for it) because I also added the new synchronized camera driver package. It could take weeks/months until this feature finds its way into the apt repository.

@InvincibleRMC
Copy link
Author

Thanks so much. I don't mind building from source in the meantime.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants