Compatibility with Hardened JavaScript - start of built-ins

[phoddie]

This PR is a continuation of the effort to improve test262 compatibility with Hardened JavaScript after lockdown. The initial PR focused on language tests. This one starts the work on built-in tests.

The approach is to modify the tests as little as possible to allow them to pass before and after lockdown. None of the changes so far require the tests themselves to have knowledge of Hardened JavaScript.

The Writable and Configurable attributes of built-ins are a significant source of failures under lockdown. Lockdown increases the integrity level of built-ins to FROZEN, causing over 1500 tests that verify the attributes of built-ins to fail.

There are three ways that the tests verify the attributes of built-ins:

• Using the verifyProperty helper function in propertyHelper.js. This is the preferred method, according to the test262 sources. Some test that do not use verifyProperty are marked for update.
• Using the verifyConfigurable, verifyWritable, verifyEnumerable, verifyNotConfigurable, verifyNotWritable, and verifyMotEnumerable helper functions. These are marked as deprecated in the test262 sources. They perform more-or-less the same checks as verifyProperty as they use the same helper functions, isWritable and isConfigurable.
• Ad-hoc tests in-line in the test code. This is done primarily by older tests.

The approach taken is to add the new verifyBuiltinProperty function to propertyHelper.js. This function calls through to verifyProperty after setting the writable and configurable options, when present, to false when running under lockdown. This accounts for the expected raised integrity level of built-ins. A new function is necessary because verifyProperty is used with objects created at runtime, not only built-ins.

With the new verifyBuiltinProperty function in place, the tests are modified as follows:

• Calls to verifyProperty that operate on a built-in are replaced with calls to verifyBuiltinProperty.
• Calls to the deprecated helper functions (verifyConfigurable, etc.) are replaced with calls to verifyProperty or verifyBuiltinProperty as appropriate.
• Ad-hoc checks for configurable and writable properties of built-ins are replaced with calls to verifyBuiltinProperty.

The benefits of these changes include:

• Many of the built-in test failures under lockdown are fixed
• The use of the deprecated property helper functions (verifyConfigurable, etc.) is significantly reduced (perhaps eliminated...) from test262
• The intention of the tests is clearer by expressly calling verifyBuiltinProperty, a function that expects a built-in object.

Having an explicit check for built-in properties may open the door to more extensive and consistent tests of built-in properties. For example, test262 is inconsistent in checking:

• Built-in functions are native functions
• The order of enumeration of length and name properties of built-in functions matches the order required by CreateBuiltinFunction abstract operation
• Other requirements for built-in functions

The $262.isLockedDown() function is introduced to determine if lockdown has completed. It is only invoked by verifyBuiltinProperty at this time and only if isLockedDown is present on $262.

There was a small bug in verifyProperty. When checking that the configurable attribute is false, verifyProperty is destructive - it deletes the property. The implementation of verifyProperty uses Array.prototype.join. Once the test for Array.prototype.join was updated to use verifyProperty the test harness failed. The solution is to cache Array.prototype.join in a local variable and invoke it using .call. This approach mirrors how isConfigurable() in propertyHelper.js caches Object.prototype.hasOwnProperty.

[ljharb]

To be clear, some of this won't be fixable - a fully locked-down JS realm is not simply 262-compliant.

[Ms2ger]

You should probably add some documentation in tree that explains when to use verifyBuiltinProperty vs verifyProperty

[gibson042]

With the same caveat here as in #4088 (review) about not formally approving work that I already consulted on, I love this! And I agree with @Ms2ger about wanting to document use of the new helper, which I guess would be a new RFC.

[gibson042]

This is also going to fail when the test for Function.prototype.call is updated to use verifyProperty, and likewise for some other functions and methods used in these helpers. I'd leave verifyProperty alone in this PR (since it isn't updating the test for Array.prototype.join), and a followup can add starting code like

// Capture primordial functions and receiver-uncurried primordial methods that
// are used in verification but might be destroyed *by* that process itself.
var __isArray = Array.isArray;
var __defineProperty = Object.defineProperty;
var __join = Function.prototype.call.bind(Array.prototype.join);
var __push = Function.prototype.call.bind(Array.prototype.push);
var __hasOwnProperty = Function.prototype.call.bind(Object.prototype.hasOwnProperty);
var __propertyIsEnumerable = Function.prototype.call.bind(Object.prototype.propertyIsEnumerable);

[gibson042]

Such changes turned out to be independent, so we don't even need to wait for followup: #4235

[phoddie]

Understood. I considered caching join at the top-level like propertyHelper.js/isWritable does for Array.isArray but followed the model of propertyHelper.js/isConfigurable instead. Agreed that this is better addressed separately from this PR.

[gibson042]

This name is potentially misleading, e.g. the IteratorResult object returned from (function*(){})().next() and the { proxy, revoke } object returned from Proxy.revocable({}, {}) (and both of its values, for that matter) are also built-in objects but their properties are not expected to be non-writable or non-configurable even in locked-down environments.

Suggested change

	function verifyBuiltinProperty(obj, name, desc, options) {
	function verifyPrimordialProperty(obj, name, desc, options) {

(cf. TC39 - How We Work: Glossary)

[phoddie]

That makes sense.

[phoddie]

You should probably add some documentation in tree that explains when to use verifyBuiltinProperty vs verifyProperty

Thanks for the suggestion, @Ms2ger. I've added documentation to the source since I didn't find a better place.

FWIW – the suggestion by @gibson042 to change the name to verifyPrimordialProperty makes the purpose more precise. I also linked to the definition of primordial.

[gibson042]

Suggested change

	* For non-priordial objects, use verifyProperty.
	* For non-primordial objects, use verifyProperty.

[phoddie]

Well... your recommended name change will give me more practice typing "primordial."

[lahma]

Sorry for the unrelated comment, but I have to say as a keen observer of development in this repository... As a non-native speaker I have to google (merriam-websterize) these things 😄 This is very interesting direction especially for runtimes allowing interop access (Jint is a .NET interpreter that allows injecting .NET services and objects into runtime - there are sandboxing needs for big players wanting to support user-supplied scripting).

[ptomato]

I think making the distinction between verifyProperty and verifyPrimordialProperty everywhere is good, and I'm especially happy that you are motivated to go through and update all those old tests that didn't use verifyProperty yet.

I'd prefer one thing to be different in implementation: I'd rather not add $262.isLockedDown and other custom code specifically for Hardened JS. Primoridals being locked-down is not compliant with ECMA-262, and in the past our position has been to not add customizations specifically for intentional non-compliance (a non-Hardened JS example is implementations that prefer not to ship a with statement.)

INTERPRETING.md says this:

In addition, consumers may choose to override any of the functions defined by test harness files as they see fit.

This seems like a case where Hardened JS should do exactly that, with verifyPrimordialProperty. test262's upstream implementation of it can simply be var verifyPrimordialProperty = verifyProperty; or similar, and the special case can go into Hardened JS's test262 harness. That wouldn't be possible if we didn't make the distinction between verifyProperty and verifyPrimordialProperty upstream, so I'm fine with doing that, I'd just rather not have lockdown-specific code in the upstream harness.

[phoddie]

@ptomato – thanks for the feedback.

I think making the distinction between verifyProperty and verifyPrimordialProperty everywhere is good, and I'm especially happy that you are motivated to go through and update all those old tests that didn't use verifyProperty yet.

Great. That's the main point of this PR. It will make test262 more consistent and, potentially, even more thorough.

Primordials being locked-down is not compliant with ECMA-262...

Yes and no. Locking-down primordials is a proposal for ECMA-262. The proposal deliberately creates incompatibilities with strict mode, just like strict mode deliberately created incompatibilities with sloppy mode. A benefit of this work on test262 is that it allows us to characterize those incompatibilities much more precisely because test262 is so thorough.

I'd prefer one thing to be different in implementation: I'd rather not add $262.isLockedDown and other custom code specifically for Hardened JS

Sure, until the relevant Hardened JavaScript proposals make their way to Stage 2.7, let's keep the Hardened JavaScript specific harness changes outside the test262 main branch.

[ljharb]

@phoddie right, but test262 doesn't include proposals until they're at stage 2.7 - so we need to make sure that the tests aren't testing for something that isn't currently allowed.

[phoddie]

@ljharb – Based on the discussion to this point, in particular my response above to @ptomato , what is being tested that is not at Stage 2.7?

[ljharb]

for example, there are no compliant differences allowed in these attributes. writability and configurability is strictly defined by the spec.

[phoddie]

@ljharb – Please read the full thread, specifically:

Sure, until the relevant Hardened JavaScript proposals make their way to Stage 2.7, let's keep the Hardened JavaScript specific harness changes outside the test262 main branch.

[ljharb]

ah ok, well then that'll be great once implemented :-)

[phoddie]

I believe all of the changes discussed have been applied. In addition, there is an update to one of the linter checks.

One question: the repository contains two copies of propertyHelper.js -- one for the harness and one for the linter. I have applied my changes to both. The recent update to propertyHelper.js in the harness (__join and friends) is not reflected in the linter copy. Maybe that's OK, but what is the norm for updates to these two?

[ptomato]

One question: the repository contains two copies of propertyHelper.js -- one for the harness and one for the linter. I have applied my changes to both. The recent update to propertyHelper.js in the harness (__join and friends) is not reflected in the linter copy. Maybe that's OK, but what is the norm for updates to these two?

I am mystified by the presence of this copy. I wonder if it's because the location of the harness folder is hardcoded relative to the location of the tests?