Turning on CMCD Query Args corrupts existing query args containing certain URLEncoded values.

[wilaw]

Environment

• The MPD passes the DASH-IF Conformance Tool on https://conformance.dashif.org/
• The stream has correct Access-Control-Allow-Origin headers (CORS)
• There are no network errors such as 404s in the browser console when trying to play the stream
• The issue observed is not mentioned on https://github.com/Dash-Industry-Forum/dash.js/wiki/FAQ
• The issue occurs in the latest reference client on http://reference.dashif.org/dash.js/ and not just on my page

• Link to playable MPD file:
• Dash.js version: latest
• Browser name/version:
• OS name/version:

Steps to reproduce

I can see a bug where, when CMCD is being used with an existing URL-encoded query arg that includes %20 (spaces), these are changed to '+'.  This can be illustrated with the following manifest playback

https://reference.dashif.org/dash.js/latest/samples/dash-if-reference-player/index.html?mpd=https%3A%2F%2Fdash.akamaized.net%2Fakamai%2Fbbb_30fps%2Fbbb_30fps.mpd%3Fa%3Dsomehting%20with%20spaces+&debug.logLevel=4&streaming.delay.liveDelayFragmentCount=NaN&streaming.delay.liveDelay=NaN&streaming.buffer.initialBufferLevel=NaN&streaming.liveCatchup.maxDrift=NaN&streaming.liveCatchup.playbackRate.min=NaN&streaming.liveCatchup.playbackRate.max=NaN

which shows in the javascript panel the correct request to the server of

https://dash.akamaized.net/akamai/bbb_30fps/bbb_30fps.mpd?a=somehting%20with%20spaces

However, if we turn on CMCD with query-arg mode and try the same manifest

https://reference.dashif.org/dash.js/latest/samples/dash-if-reference-player/index.html?mpd=https%3A%2F%2Fdash.akamaized.net%2Fakamai%2Fbbb_30fps%2Fbbb_30fps.mpd%3Fa%3Dsomehting%20with%20spaces+&debug.logLevel=4&streaming.delay.liveDelayFragmentCount=NaN&streaming.delay.liveDelay=NaN&streaming.buffer.initialBufferLevel=NaN&streaming.liveCatchup.maxDrift=NaN&streaming.liveCatchup.playbackRate.min=NaN&streaming.liveCatchup.playbackRate.max=NaN&streaming.cmcd.enabled=true

then we see that the request to the server looks like this:

https://dash.akamaized.net/akamai/bbb_30fps/bbb_30fps.mpd?a=somehting+with+spaces&CMCD=ot%3Dm%2Csid%3D%22d336fd7a-178d-4dfe-b180-809fa58af3c6%22

where the original query arg has been modified.

We can also see with a change in capitalization. If we try to play the URL

https://dash.akamaized.net/akamai/bbb_30fps/bbb_30fps.mpd?a=%3d

without CMCD-query args active, then the player makes the correct request to the server. If however we turn on CMCD query args, then it makes this request to the server

https://dash.akamaized.net/akamai/bbb_30fps/bbb_30fps.mpd?a=%3D&CMCD=ot%3Dm%2Csid%3D%22b7b542be-8f66-4834-aedf-4fcd4883a449%22

in which the %3d had been converted to %3D. This breaks any server which is hashing the URL in order to decide if an access token is valid.

[dsilhavy]

Just had a quick look, and it is related to the function addAditionalQueryParameterToUrl in Utils.js:

    static addAditionalQueryParameterToUrl(url, params) {
        try {
            if (!params || params.length === 0) {
                return url;
            }

            let modifiedUrl = new URL(url);

            params.forEach((param) => {
                if (param.key && param.value) {
                    modifiedUrl.searchParams.set(param.key, param.value);
                }
            });

            return modifiedUrl.href;


        } catch (e) {
            return url;
        }
    }

The native URLSearchParams implementation automatically replaces whitespaces with + signs as soon as we add a new query parameters using modifiedUrl.searchParams.set(param.key, param.value).

In addition, I found that (https://http.dev/percent-encoding) :

Percent-encoding uses three characters to represent a reserved character. The first is the percent sign %, and this is followed by the two-character hexadecimal value of the reserved ASCII character. The two-character hexadecimal value is not case sensitive.

The conversion from %3d to %3D also happens automatically once we call modifiedUrl.searchParams.set(param.key, param.value).

To support the behavior described in this issue we probably need to replace the logic that sets the query parameters. It is to be highlighted though that URLSearchParams (https://developer.mozilla.org/en-US/docs/Web/API/URLSearchParams) and URL (https://developer.mozilla.org/en-US/docs/Web/API/URL/URL) are native browser implementation and other clients might use it as well.

[dsilhavy]

I also checked in Shaka player:

• The first issue with the + signs instead of %20 does not appear: https://dash.akamaized.net/akamai/bbb_30fps/bbb_30fps.mpd?a=somehting%20with%20spaces&CMCD=mtp%3D76700%2Cot%3Dm%2Csf%3Dd%2Csid%3D%22afe52941-94bd-4f08-b3d5-5f1c1f6add54%22%2Csu
• The second behavior is similar to the dash.js behavior. The resulting request URL is https://dash.akamaized.net/akamai/bbb_30fps/bbb_30fps.mpd?a=%3D&CMCD=ot%3Dm%2Csf%3Dd%2Csid%3D%223a321f71-22bb-4481-972b-e3d65d6f1510%22%2Csu if CMCD is enabled