vhost_user: Add support for SHMEM_MAP/UNMAP backend requests #251

mtjhrc · 2024-08-27T15:35:25Z

Summary of the PR

This PR adds support for backend requests SHMEM_MAP and SHMEM_UNMAP.
This allows the device to memory map a file descriptor into a Shared Memory Region.

The SHMEM_MAP and SHMEM_UNMAP are part of this QEMU RFC:

[RFC PATCH v2 0/5] vhost-user: Add SHMEM_MAP/UNMAP requests
https://mail.gnu.org/archive/html/qemu-devel/2024-06/msg05736.html

I don't anticipate these specific requests to change much, the discussion for this RFC is mainly about a situation where one device is accessing shared memory mapped by another device.

These requests are required for implementing blob resource support in vhost-user-gpu.
The current gpu PR, doesn't implement blob resources and as such doesn't require this, but I am working on creating another PR with blob resource support.

This is based on a commit by Albert Esteve [email protected] in his branch/fork.

Requirements

Before submitting your PR, please make sure you addressed the following
requirements:

All commits in this PR have Signed-Off-By trailers (with
git commit -s), and the commit message has max 60 characters for the
summary and max 75 characters for each description line.
All added/changed functionality has a corresponding unit/integration
test.
All added/changed public-facing functionality has entries in the "Upcoming
Release" section of CHANGELOG.md (if no such section exists, please create one).
Any newly added unsafe code is properly documented.

stefano-garzarella

Author and SoB are different, can you fix it, or add @aesteve-rh SoB if he is a co-author.

Please check also the coverage and add a line in the changelog.

vhost/src/vhost_user/backend_req.rs

vhost/src/vhost_user/frontend_req_handler.rs

vhost/src/vhost_user/message.rs

Extract creating a backend and a frontend from a socketpair to a utility function. Signed-off-by: Matej Hrica <[email protected]>

mtjhrc · 2024-08-28T15:27:32Z

I have addressed the feedback, you can take another look.

Also, should I add some comment that this feature is not really "stable"? Or will this PR have to wait until it gets merged in QEMU?

mtjhrc · 2024-08-28T15:31:45Z

The coverage was too low, so I wanted to add a test for the default impl of the frontend request handlers, but I also added coverage for all the methods, since that seemed quite trivial. This has increased the coverage too much, so I had to increase the coverage config value.

stefano-garzarella · 2024-08-28T15:35:01Z

The code LGTM, but before merging this PR, I'd like to wait at least the spec merged in QEMU.
We recently removed some code that was never merged in the spec, so I'd like to avoid making the same mistake.
If the QEMU RFC still requires discussion about the code, maybe we can try to merge at least the changes to the specification so we can move forward here as well.

WDYT?

mtjhrc · 2024-08-29T09:14:11Z

I understand the concern. I'm talking to @aesteve-rh to see if we can figure something out

vhost/src/vhost_user/message.rs

Add request defintions and methods for using the new SHMEM_MAP/UNMAP backend requests. Note that at the time of writing this, these requests are part of this RFC in QEMU: https://mail.gnu.org/archive/html/qemu-devel/2024-06/msg05736.html Co-authored-by: Albert Esteve <[email protected]> Co-authored-by: Matej Hrica <[email protected]> Signed-off-by: Albert Esteve <[email protected]> Signed-off-by: Matej Hrica <[email protected]>

Add tests to assert return values (and existence) of default implemenation of VhostUserFrontendReqHandler and VhostUserFrontendReqHandlerMut trait methods. Signed-off-by: Matej Hrica <[email protected]>

Signed-off-by: Matej Hrica <[email protected]>

dorindabassey · 2024-09-03T11:10:36Z

vhost/src/vhost_user/message.rs

+        (self.flags & !VhostUserMMapFlags::all().bits()) == 0
+            && self.fd_offset.checked_add(self.len).is_some()
+            && self.shm_offset.checked_add(self.len).is_some()
+    }


do you want to add more checks relevant to the struct? e.g a check based on valid values for shmid?

From virtio spec:

A device may have multiple shared memory regions associated with it. Each region has a shmid to identify it, the meaning of which is device-specific.

According to the virtio (not vhost) spec the valid values of shmid are dependent on the device type (And probably the advertised device features...), so I don't think you could check it here generally.

ok, fair enough for shmid, you can leave it as it is.

dorindabassey · 2024-09-03T11:15:15Z

vhost/src/vhost_user/frontend_req_handler.rs

+
+        fn shmem_map(&mut self, req: &VhostUserMMap, _fd: &dyn AsRawFd) -> HandlerResult<u64> {
+            assert_eq!({ req.shmid }, 0);
+            Ok(!self.shmem_mappings.insert((req.shm_offset, req.len)) as u64)


i think it's better to remove the negation (!) and use a direct check to return meaningful values. e.g if self.shmem_mappings.insert(req.shm_offset, req.len) { Ok(1) } else { Ok(0) } and why the double braces? missing &?

By braces do you mean ( ) or { }? The (req.shm_offset, req.len) is a tuple.

In regards to the negation with cast instead of explicit if, I considered to write it like you did, I mainly wrote it like this, because it is symmetric with the existing shared_object_add, shared_object_remove above:

fn shared_object_add(&mut self, uuid: &VhostUserSharedMsg) -> HandlerResult<u64> { Ok(!self.shared_objects.insert(uuid.uuid) as u64) } fn shared_object_remove(&mut self, uuid: &VhostUserSharedMsg) -> HandlerResult<u64> { Ok(!self.shared_objects.remove(&uuid.uuid) as u64) }

I am not sure if I should change it.

The (req.shm_offset, req.len) is a tuple.

ah right!

because it is symmetric with the existing shared_object_add, shared_object_remove above:

doesn't necessarily mean it's the correct approach.

dorindabassey · 2024-09-03T11:15:33Z

vhost/src/vhost_user/frontend_req_handler.rs

+
+        fn shmem_unmap(&mut self, req: &VhostUserMMap) -> HandlerResult<u64> {
+            assert_eq!({ req.shmid }, 0);
+            Ok(!self.shmem_mappings.remove(&(req.shm_offset, req.len)) as u64)


vhost/src/vhost_user/backend_req.rs

dorindabassey · 2024-09-03T11:33:51Z

The code LGTM, but before merging this PR, I'd like to wait at least the spec merged in QEMU. We recently removed some code that was never merged in the spec, so I'd like to avoid making the same mistake. If the QEMU RFC still requires discussion about the code, maybe we can try to merge at least the changes to the specification so we can move forward here as well.

WDYT?

are you talking about the vhost-user specification https://qemu-project.gitlab.io/qemu/interop/vhost-user.html#message-specification?
If so, I think it make sense to atleast have just the spec merged, since the patches are still RFC it might take a while before it's merged.

stefano-garzarella · 2024-09-03T12:13:16Z

The code LGTM, but before merging this PR, I'd like to wait at least the spec merged in QEMU. We recently removed some code that was never merged in the spec, so I'd like to avoid making the same mistake. If the QEMU RFC still requires discussion about the code, maybe we can try to merge at least the changes to the specification so we can move forward here as well.
WDYT?

are you talking about the vhost-user specification https://qemu-project.gitlab.io/qemu/interop/vhost-user.html#message-specification?

@dorindabassey yep

If so, I think it make sense to atleast have just the spec merged, since the patches are still RFC it might take a while before it's merged.

yeah, if it's possible, I'd go in that direction.
So, I'm marking this PR as draft to be sure we don't merge this till we have the spec changes merged in QEMU.

stefano-garzarella reviewed Aug 28, 2024

View reviewed changes

vhost/src/vhost_user/backend_req.rs Outdated Show resolved Hide resolved

vhost/src/vhost_user/frontend_req_handler.rs Show resolved Hide resolved

vhost/src/vhost_user/message.rs Outdated Show resolved Hide resolved

vhost/src/vhost_user/message.rs Outdated Show resolved Hide resolved

backend_req::tests: Extract test setup to a function

fe8b6f5

Extract creating a backend and a frontend from a socketpair to a utility function. Signed-off-by: Matej Hrica <[email protected]>

mtjhrc force-pushed the shmem branch 13 times, most recently from 80c4bf1 to 630cd66 Compare August 28, 2024 15:06

mtjhrc marked this pull request as ready for review August 28, 2024 15:16

mtjhrc requested review from Ablu, eryugey, germag, jiangliu, sboeuf and slp as code owners August 28, 2024 15:16

alyssais reviewed Aug 29, 2024

View reviewed changes

vhost/src/vhost_user/message.rs Outdated Show resolved Hide resolved

mtjhrc force-pushed the shmem branch 2 times, most recently from cd80826 to 5f05fec Compare August 29, 2024 13:53

mtjhrc added 2 commits August 29, 2024 17:24

Add tests for default impls of Frontend req handlers

d0220a8

Add tests to assert return values (and existence) of default implemenation of VhostUserFrontendReqHandler and VhostUserFrontendReqHandlerMut trait methods. Signed-off-by: Matej Hrica <[email protected]>

Increase test coverage config value

bf692de

Signed-off-by: Matej Hrica <[email protected]>

mtjhrc force-pushed the shmem branch from 5f05fec to bf692de Compare August 29, 2024 15:24

dorindabassey reviewed Sep 3, 2024

View reviewed changes

vhost/src/vhost_user/backend_req.rs Show resolved Hide resolved

stefano-garzarella marked this pull request as draft September 3, 2024 12:13

alyssais mentioned this pull request Sep 25, 2024

virtiofs Dax cloud-hypervisor/cloud-hypervisor#5591

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

vhost_user: Add support for SHMEM_MAP/UNMAP backend requests #251

vhost_user: Add support for SHMEM_MAP/UNMAP backend requests #251

mtjhrc commented Aug 27, 2024 •

edited

Loading

stefano-garzarella left a comment

mtjhrc commented Aug 28, 2024 •

edited

Loading

mtjhrc commented Aug 28, 2024

stefano-garzarella commented Aug 28, 2024

mtjhrc commented Aug 29, 2024

dorindabassey Sep 3, 2024

mtjhrc Sep 3, 2024 •

edited

Loading

dorindabassey Sep 3, 2024

dorindabassey Sep 3, 2024

mtjhrc Sep 3, 2024 •

edited

Loading

dorindabassey Sep 3, 2024

dorindabassey Sep 3, 2024

dorindabassey commented Sep 3, 2024

stefano-garzarella commented Sep 3, 2024

vhost_user: Add support for SHMEM_MAP/UNMAP backend requests #251

Are you sure you want to change the base?

vhost_user: Add support for SHMEM_MAP/UNMAP backend requests #251

Conversation

mtjhrc commented Aug 27, 2024 • edited Loading

Summary of the PR

Requirements

stefano-garzarella left a comment

Choose a reason for hiding this comment

mtjhrc commented Aug 28, 2024 • edited Loading

mtjhrc commented Aug 28, 2024

stefano-garzarella commented Aug 28, 2024

mtjhrc commented Aug 29, 2024

dorindabassey Sep 3, 2024

Choose a reason for hiding this comment

mtjhrc Sep 3, 2024 • edited Loading

Choose a reason for hiding this comment

dorindabassey Sep 3, 2024

Choose a reason for hiding this comment

dorindabassey Sep 3, 2024

Choose a reason for hiding this comment

mtjhrc Sep 3, 2024 • edited Loading

Choose a reason for hiding this comment

dorindabassey Sep 3, 2024

Choose a reason for hiding this comment

dorindabassey Sep 3, 2024

Choose a reason for hiding this comment

dorindabassey commented Sep 3, 2024

stefano-garzarella commented Sep 3, 2024

mtjhrc commented Aug 27, 2024 •

edited

Loading

mtjhrc commented Aug 28, 2024 •

edited

Loading

mtjhrc Sep 3, 2024 •

edited

Loading

mtjhrc Sep 3, 2024 •

edited

Loading