Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

User dialog appears even if no device matches the filter #105

Open
tbessie opened this issue Jan 25, 2023 · 15 comments
Open

User dialog appears even if no device matches the filter #105

tbessie opened this issue Jan 25, 2023 · 15 comments

Comments

@tbessie
Copy link

tbessie commented Jan 25, 2023

I don't know if this is the proper place to mention this, but it seems nonsensical for requestDevice() to bring up the dialog even if the filter isn't matched. As a developer wanting to use this feature, we only care to ask the user if a given device type is attached. If not, we don't want to bother them (in our case, this is a foot pedal used by transcriptionists - if the user doesn't have one attached, then asking them is unnecessary and annoying).

As a side note, why does this API not allow a querying of what HID devices are attached? it would be useful to know this in advance before asking users to approve access.

@chengweih001
Copy link
Contributor

but it seems nonsensical for requestDevice() to bring up the dialog even if the filter isn't matched

Even at the moment requestDevice() brings up the dialog doesn't have any matched device, later on when a matched device gets plugged in, it will show up in the dialog.

As a side note, why does this API not allow a querying of what HID devices are attached?

There is getDevices() for this purpose. However, in consideration of security & privacy, it is designed not to expose what HID devices available on the system until the user grant the permission through requestDevice().

@softworkz
Copy link

I came here to ask exact the same question. I implemented support for an IR remote control receiver and it worked out pretty well.
Though, I've been under the impression that no dialog would be shown when the filter doesn't match (in the initial phase I had many such cases, but these must have been due to something else than not matching. 

Essentially, this makes the whole API somewhat useless. I do not quite get the point about devices appearing in the dialog "later" when they get plugged in. 

What "later" would that be? How long is the user supposed to sit and wait? And how should a user know that they are supposed to plug-in a device? 
I mean - the application cannot tell them before something like "you need to plugin device XY" beause the application doesn't even know whether it's plugged in or not. But when the dialog wouldn't be shown, the app could show a prompt to plug in the device and then there can be a re-try button or something. This allows to properly interact and instruct users.

But the way it's currently done means that countless users across the planer will keep seeing those empty dialogs - which just doesn't make any sense - what does that tell?  "Choose something of nothing!" or "We know you can't do anything in that dialog, but we still want to let you actively do nothing"?

I understand that the information about whether a device is available or not should not be exposed that way - but there really needs to be a better way than showing empty dialogs for the purpose of privacy protection.

@nondebug
Copy link
Collaborator

I implemented support for an IR remote control receiver and it worked out pretty well.

Can you provide more information about this project? I'd like to take a look so I can better understand the UI constraints.

The API behavior is intended to mitigate browser fingerprinting.

WebHID can be used for active fingerprinting. It's common for a desktop or laptop computer to have at least one connected HID device, and many computers have built-in devices for keyboards, trackpads, touchscreens, and other sensors. We should expect that malicious actors will use device characteristics as signals to identify the user. To mitigate against this risk, we designed the API to access only the entropy necessary. In this case it means we only expose information about a device after the user has granted permission to access that device.

I do not quite get the point about devices appearing in the dialog "later" when they get plugged in. What "later" would that be?

This means the chooser dialog supports hotplugging. If the chooser dialog is empty and the user connects a device that matches the filter then the dialog will update to add the device. The application doesn't need to call requestDevice again.

But when the dialog wouldn't be shown, the app could show a prompt to plug in the device and then there can be a re-try button or something. This allows to properly interact and instruct users.

I think it's good practice for applications to explain to the user why a permission is needed before requesting that permission. The requestDevice method requires a user activation, so your application must already require user interaction of some form before showing the chooser the first time. Is the user clicking a "Connect IR receiver" button or something similar to open the chooser? Instead of showing the chooser directly, could the app instead show an interstitial dialog that asks them to plug-in the receiver before clicking the "Connect IR receiver" button?

I suggest looking at how Google Meet integrated support for call controls using WebHID. In Meet's settings dialog, Audio tab, there's a "Call control" feature with a "+ Connect device" button that opens the chooser. The settings dialog has some brief help text to suggest which types of devices might work and which features are supported.

@softworkz
Copy link

I implemented support for an IR remote control receiver and it worked out pretty well.

Can you provide more information about this project? I'd like to take a look so I can better understand the UI constraints.

Yes of course. It's about Windows Media Center remote controls: https://www.google.com/search?q=wmc+remote+control&source=lnms&tbm=isch

And the applications are the client applications for Emby Media Server (https://emby.media/). Many of the client applications are sharing the same html/js core code (and you can run it in the browser of course), that's why WebHID seems to be ideal for supporting those remote controls. I haven't tested non-Windows platforms yet and I'm not sure whether it will work there because the remote is using HID pages in the vendor-specific range, but it's working perfectly on Windows/Chrome already.

What makes this case a bit special is that the remote control's HID support is hybrid (or more precisely: the receiver is modeled as a composite device), which means that half of the buttons on the remote are already working without needing the WebHID API, like up/down/left/right/ok/back, numeric keys 0-9, play/pause/skip, volume etc.
But other buttons (channel+/-, Live TV, Music, Videos, Subtitles, Home etc.) can only be supported through the WbHID API, which is the purpose of this (sub-)project.

Now, the extended button support is not an elementary precondition to use the apps, which means that we cannot put the setup procedure into some settings menu as most users wouldn't ever notice that it exists.  Ideally, it should just work when a user has the receiver installed ("eHome Infrared Transceiver"). But as that's not possible and use of HID devices via WebHID API requires user consent (surely for good reasons), the next best option which would still be acceptable would be to show the requestDevice() dialog, in case that device would be available, even though it's quite inconvenient when a user needs to do that each time when using the application.

As far as I have understood, it's not possible to persist that consent for subsequent uses, right?

But what we can't do is to show an empty dialog for each and every user of the application and each time when a user starts the application, when a user doesn't have such receiver connected,

Maybe there should be a separate kind of consent that allows the applications to query for the existence of (only) a specific device, which would be a kind of consent that can be persisted?

@nondebug
Copy link
Collaborator

As far as I have understood, it's not possible to persist that consent for subsequent uses, right?

This behavior depends on the device. The problem is we don't want to persist permissions for devices that can't be reliably re-identified after restarting the browser or reconnecting the device. For WebHID, permissions are persisted if device is USB and has a non-empty serial number string descriptor and non-empty product name. If the device isn't eligible for persistent permissions then we store an ephemeral permission that's revoked when the device is disconnected or the browser is closed. Chrome logs the serial number string in chrome://device-log, there should be a "HID device added" log message with the device information.

It sounds like eHome Infrared Transceiver isn't eligible for persistent permissions which means you'll have to grant the permission each time the browser is restarted. If you're the machine owner you can get around this by setting enterprise policy to automatically grant permissions. See WebHidAllowDevicesForUrls and WebHidAllowDevicesWithHidUsagesForUrls.

Maybe there should be a separate kind of consent that allows the applications to query for the existence of (only) a specific device, which would be a kind of consent that can be persisted?

I think adding a new type of device permission would complicate the permission model too much.

Perhaps we can extend the API to enable applications to register interest in a device without calling requestDevice directly. Suppose we had a method that works like requestDevice but doesn't return a promise. It would delay showing the chooser until there's at least one connected device that matches the filters. When the user grants the permission a deviceconnected event is dispatched. If the user decides not to grant the permission then no event is dispatched.

@softworkz
Copy link

softworkz commented Feb 14, 2023

This behavior depends on the device. The problem is we don't want to persist permissions for devices that can't be reliably re-identified after restarting the browser or reconnecting the device. For WebHID, permissions are persisted if device is USB and has a non-empty serial number string descriptor and non-empty product name. I

It does have a serial number (other HID tools show it), but it doesn't show a product name (I don't know why it doesn't):

In Device Manager:

Why doesn't the serial number alone suffice to be eligible for persistence?

Perhaps we can extend the API to enable applications to register interest in a device without calling requestDevice directly. Suppose we had a method that works like requestDevice but doesn't return a promise. It would delay showing the chooser until there's at least one connected device that matches the filters. When the user grants the permission a deviceconnected event is dispatched. If the user decides not to grant the permission then no event is dispatched.

I'd say that makes perfect sense.

Essentially, we don't need to know whether the device is there or not. We just need to make sure that users get the opportunity to activate and use it when it's there but won't get bothered otherwise.

@softworkz
Copy link

Essentially, we don't need to know whether the device is there or not. We just need to make sure that users get the opportunity to activate and use it when it's there but won't get bothered otherwise.

And for those (few) who might have it but never want to use it, we can provide an option somewhere to disable our issuing of the request. So that would be fine for all cases that need to be covered.

@nondebug
Copy link
Collaborator

Why doesn't the serial number alone suffice to be eligible for persistence?

Even though we're confident the tuple (vendor ID, product ID, serial number) is sufficient to uniquely identify the device, none of those are user-facing identifiers. We need a user-facing identifier so we can give the user an opportunity to review and revoke granted permissions (chrome://settings/content/hidDevices). If the user can't figure out which permission corresponds to the device then they can't easily revoke it.

It's interesting that Device Manager has the name "Microsoft eHome Infrared Transceiver" but we don't get that name when we call HidD_GetProductString. Can you share more information about the device? USB Device Tree Viewer does a pretty good job of grabbing all the relevant details, maybe it will show where that string comes from.

@softworkz
Copy link

This is the output I get for the device from the tool (already had it installed 😄 ):

    =========================== USB Port3 ===========================

Connection Status        : 0x01 (Device is connected)
Port Chain               : 1-11-3

      ========================== Summary =========================
Vendor ID                : 0x0471 (Philips Consumer Lifestyle BV)
Product ID               : 0x0815
USB Version              : 1.1
Port maximum Speed       : High-Speed
Device maximum Speed     : Full-Speed
Device Connection Speed  : Full-Speed
Self powered             : no
Demanded Current         : 100 mA
Used Endpoints           : 3

      ======================== USB Device ========================

        +++++++++++++++++ Device Information ++++++++++++++++++
Device Description       : eHome Infrared Receiver (USBCIR)
Device Path 1            : \\?\USB#VID_0471&PID_0815#PH00HNKU#{a5dcbf10-6530-11d2-901f-00c04fb951ed} (GUID_DEVINTERFACE_USB_DEVICE)
Device Path 2            : \\?\USB#VID_0471&PID_0815#PH00HNKU#{064f8c82-77b2-445e-b85d-c4e20f942fe1}
Kernel Name              : \Device\USBPDO-12
Device ID                : USB\VID_0471&PID_0815\PH00HNKU
Hardware IDs             : USB\VID_0471&PID_0815&REV_0000 USB\VID_0471&PID_0815
Driver KeyName           : {36fc9e60-c465-11cf-8056-444553540000}\0015 (GUID_DEVCLASS_USB)
Driver                   : \SystemRoot\System32\drivers\usbcir.sys (Version: 10.0.22621.1  Date: 2022-05-07)
Driver Inf               : C:\WINDOWS\inf\usbcir.inf
Legacy BusType           : PNPBus
Class                    : USB
Class GUID               : {36fc9e60-c465-11cf-8056-444553540000} (GUID_DEVCLASS_USB)
Service                  : usbcir
Enumerator               : USB
Location Info            : Port_#0003.Hub_#0006
Location IDs             : PCIROOT(0)#PCI(1400)#USBROOT(0)#USB(11)#USB(3), ACPI(_SB_)#ACPI(PC00)#ACPI(XHCI)#ACPI(RHUB)#ACPI(HS11)#ACPI(HSS3)
Container ID             : {51a4d0cf-9193-5604-ab45-7954df75c436}
Manufacturer Info        : Microsoft
Capabilities             : 0x94 (Removable, UniqueID, SurpriseRemovalOK)
Status                   : 0x0180600A (DN_DRIVER_LOADED, DN_STARTED, DN_DISABLEABLE, DN_REMOVABLE, DN_NT_ENUMERATOR, DN_NT_DRIVER)
Problem Code             : 0
Address                  : 3
HcDisableSelectiveSuspend: 0
EnableSelectiveSuspend   : 0
SelectiveSuspendEnabled  : 0
EnhancedPowerMgmtEnabled : 0
IdleInWorkingState       : 0
WakeFromSleepState       : 1
Power State              : D0 (supported: D0, D1, D2, D3, wake from D0, wake from D1, wake from D2)

        ---------------- Connection Information ---------------
Connection Index         : 0x03 (Port 3)
Connection Status        : 0x01 (DeviceConnected)
Current Config Value     : 0x01 (Configuration 1)
Device Address           : 0x0E (14)
Is Hub                   : 0x00 (no)
Device Bus Speed         : 0x01 (Full-Speed)
Number Of Open Pipes     : 0x02 (2 pipes to data endpoints)
Pipe[0]                  : EndpointID=1  Direction=OUT  ScheduleOffset=0  Type=Interrupt  wMaxPacketSize=0x10    bInterval=1   -> 270 Bits/ms = 33750 Bytes/s
Pipe[1]                  : EndpointID=1  Direction=IN   ScheduleOffset=0  Type=Interrupt  wMaxPacketSize=0x10    bInterval=1   -> 270 Bits/ms = 33750 Bytes/s
Data (HexDump)           : 03 00 00 00 12 01 10 01 00 00 00 10 71 04 15 08   ............q...
                           00 00 01 02 03 01 01 01 00 0E 00 02 00 00 00 01   ................
                           00 00 00 07 05 01 03 10 00 01 00 00 00 00 07 05   ................
                           81 03 10 00 01 00 00 00 00                        .........

        --------------- Connection Information V2 -------------
Connection Index         : 0x03 (3)
Length                   : 0x10 (16 bytes)
SupportedUsbProtocols    : 0x03
 Usb110                  : 1 (yes, port supports USB 1.1)
 Usb200                  : 1 (yes, port supports USB 2.0)
 Usb300                  : 0 (no, port not supports USB 3.0)
 ReservedMBZ             : 0x00
Flags                    : 0x00
 DevIsOpAtSsOrHigher     : 0 (Device is not operating at SuperSpeed or higher)
 DevIsSsCapOrHigher      : 0 (Device is not SuperSpeed capable or higher)
 DevIsOpAtSsPlusOrHigher : 0 (Device is not operating at SuperSpeedPlus or higher)
 DevIsSsPlusCapOrHigher  : 0 (Device is not SuperSpeedPlus capable or higher)
 ReservedMBZ             : 0x00
Data (HexDump)           : 03 00 00 00 10 00 00 00 03 00 00 00 00 00 00 00   ................

    ---------------------- Device Descriptor ----------------------
bLength                  : 0x12 (18 bytes)
bDescriptorType          : 0x01 (Device Descriptor)
bcdUSB                   : 0x110 (USB Version 1.1)
bDeviceClass             : 0x00 (defined by the interface descriptors)
bDeviceSubClass          : 0x00
bDeviceProtocol          : 0x00
bMaxPacketSize0          : 0x10 (16 bytes)
idVendor                 : 0x0471 (Philips Consumer Lifestyle BV)
idProduct                : 0x0815
bcdDevice                : 0x0000
iManufacturer            : 0x01 (String Descriptor 1)
 Language 0x0409         : "Philips"
iProduct                 : 0x02 (String Descriptor 2)
 Language 0x0409         : "eHome Infrared Transceiver"
iSerialNumber            : 0x03 (String Descriptor 3)
 Language 0x0409         : "PH00HNKU"
bNumConfigurations       : 0x01 (1 Configuration)
Data (HexDump)           : 12 01 10 01 00 00 00 10 71 04 15 08 00 00 01 02   ........q.......
                           03 01                                             ..

    ------------------ Configuration Descriptor -------------------
bLength                  : 0x09 (9 bytes)
bDescriptorType          : 0x02 (Configuration Descriptor)
wTotalLength             : 0x0020 (32 bytes)
bNumInterfaces           : 0x01 (1 Interface)
bConfigurationValue      : 0x01 (Configuration 1)
iConfiguration           : 0x00 (No String Descriptor)
bmAttributes             : 0xA0
 D7: Reserved, set 1     : 0x01
 D6: Self Powered        : 0x00 (no)
 D5: Remote Wakeup       : 0x01 (yes)
 D4..0: Reserved, set 0  : 0x00
MaxPower                 : 0x32 (100 mA)
Data (HexDump)           : 09 02 20 00 01 01 00 A0 32 09 04 00 00 02 FF FF   .. .....2.......
                           FF 00 07 05 01 02 10 00 00 07 05 81 02 10 00 00   ................

        ---------------- Interface Descriptor -----------------
bLength                  : 0x09 (9 bytes)
bDescriptorType          : 0x04 (Interface Descriptor)
bInterfaceNumber         : 0x00 (Interface 0)
bAlternateSetting        : 0x00
bNumEndpoints            : 0x02 (2 Endpoints)
bInterfaceClass          : 0xFF (Vendor Specific)
bInterfaceSubClass       : 0xFF
bInterfaceProtocol       : 0xFF
iInterface               : 0x00 (No String Descriptor)
Data (HexDump)           : 09 04 00 00 02 FF FF FF 00                        .........

        ----------------- Endpoint Descriptor -----------------
bLength                  : 0x07 (7 bytes)
bDescriptorType          : 0x05 (Endpoint Descriptor)
bEndpointAddress         : 0x01 (Direction=OUT EndpointID=1)
bmAttributes             : 0x02 (TransferType=Bulk)
wMaxPacketSize           : 0x0010 (16 bytes)
bInterval                : 0x00 (ignored)
Data (HexDump)           : 07 05 01 02 10 00 00                              .......

        ----------------- Endpoint Descriptor -----------------
bLength                  : 0x07 (7 bytes)
bDescriptorType          : 0x05 (Endpoint Descriptor)
bEndpointAddress         : 0x81 (Direction=IN EndpointID=1)
bmAttributes             : 0x02 (TransferType=Bulk)
wMaxPacketSize           : 0x0010 (16 bytes)
bInterval                : 0x00 (ignored)
Data (HexDump)           : 07 05 81 02 10 00 00                              .......

      -------------------- String Descriptors -------------------
             ------ String Descriptor 0 ------
bLength                  : 0x04 (4 bytes)
bDescriptorType          : 0x03 (String Descriptor)
Language ID[0]           : 0x0409 (English - United States)
Data (HexDump)           : 04 03 09 04                                       ....
             ------ String Descriptor 1 ------
bLength                  : 0x10 (16 bytes)
bDescriptorType          : 0x03 (String Descriptor)
Language 0x0409          : "Philips"
Data (HexDump)           : 10 03 50 00 68 00 69 00 6C 00 69 00 70 00 73 00   ..P.h.i.l.i.p.s.
             ------ String Descriptor 2 ------
bLength                  : 0x36 (54 bytes)
bDescriptorType          : 0x03 (String Descriptor)
Language 0x0409          : "eHome Infrared Transceiver"
Data (HexDump)           : 36 03 65 00 48 00 6F 00 6D 00 65 00 20 00 49 00   6.e.H.o.m.e. .I.
                           6E 00 66 00 72 00 61 00 72 00 65 00 64 00 20 00   n.f.r.a.r.e.d. .
                           54 00 72 00 61 00 6E 00 73 00 63 00 65 00 69 00   T.r.a.n.s.c.e.i.
                           76 00 65 00 72 00                                 v.e.r.
             ------ String Descriptor 3 ------
bLength                  : 0x12 (18 bytes)
bDescriptorType          : 0x03 (String Descriptor)
Language 0x0409          : "PH00HNKU"
Data (HexDump)           : 12 03 50 00 48 00 30 00 30 00 48 00 4E 00 4B 00   ..P.H.0.0.H.N.K.
                           55 00                                             U.

@nondebug
Copy link
Collaborator

It looks like the device does have a product name string:

iProduct                 : 0x02 (String Descriptor 2)
 Language 0x0409         : "eHome Infrared Transceiver"

Probably we can work around this by walking up the device tree to find the USB device and fetching the product name from there.

@softworkz
Copy link

Yea, it seems that just the subdevices are unnamed.

@nondebug
Copy link
Collaborator

I created https://crbug.com/1416405 for the product name bug

@softworkz
Copy link

Thanks so much for your support!

@softworkz
Copy link

If anybody would need such device for dev or test, I can organize that.

@softworkz
Copy link

@nondebug - Hi, I've seen there hasn't been much progress on the product name issue.

We have completed a Windows app with a WebView2 and would rather use the WebHID implementation I already did instead of writing native code for it, but the problem still stands with the dialog popping up each time, which is not acceptable for a local application.
I wonder whether any ways exist in this context to disable the dialog and auto-allow device usage?

I've seen the policies, but I suppose these won't work since the web-side code is running from file:// URLs, which most likely won't be seen as "valid URLs" (haven't tried though).
We're running the engine with --disable-web-security and we can set any other flag. Is there some way to get this working without prompt dialogs?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants