-
-
Notifications
You must be signed in to change notification settings - Fork 191
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
React-Native support #313
Comments
I managed to make it (almost) work with this: import SQLite from 'react-native-sqlite-2';
import setGlobalVars from 'indexeddbshim/src/setGlobalVars';
import Dexie from 'dexie';
const win = {};
setGlobalVars(
win,
{
checkOrigin: false,
win: SQLite,
deleteDatabaseFiles: false,
useSQLiteIndexes: true,
}
);
class MyDexie extends Dexie {
constructor(name) {
super(name, {
indexedDB: win.indexedDB,
IDBKeyRange: win.IDBKeyRange,
});
}
} I say almost because Babel seems to be ignoring an arrow function in the output bundle: Original: Object.defineProperty(IDBVersionChangeEvent, Symbol.hasInstance, {
value: obj => util.isObj(obj) && 'oldVersion' in obj && typeof obj.defaultPrevented === 'boolean'
}); Output: Object.defineProperty(IDBVersionChangeEvent, typeof Symbol === 'function' ? Symbol.hasInstance : '@@hasInstance', {
value: obj => util.isObj(obj) && 'oldVersion' in obj && typeof obj.defaultPrevented === 'boolean'
}); So there is a syntax error in Android, but at least it works in remote debugger for me. |
@nizkp Unfortunately testing with the remote debugger is not an option in this case because that makes all JS execute remotely in Chrome instead of through React-Native's own runtime. What that means is that normal Indexeddb should even work there without the shim. Getting this Babel problem fixed could solve everything though. |
True, but it's actually working (data is stored in mobile via react-native-sqlite-2). But of course it's not useful if it can't run in the old Android JSC Engine. |
Though the ES2015 preset should normally convert, FWIW, there is a Babel plugin for arrow functions: https://babeljs.io/docs/plugins/transform-es2015-arrow-functions/ |
I'm using // file: configureDB.js
export default function configureDB() {
// Make indexeddbshim happy with React Native's environment
if (global.window.navigator.userAgent === undefined) {
global.window.navigator = { ...global.window.navigator, userAgent: '' };
}
// Import after RN initilization otherwise we get an exception about process not being defined
const SQLite = require('react-native-sqlite-2').default;
global.window.openDatabase = SQLite.openDatabase;
// babel-polyfill is Needed on Android otherwise indexeddbshim complains
// about Object.setPrototypeOf missing
require('babel-polyfill');
require('indexeddbshim');
global.shimIndexedDB.__setConfig({ checkOrigin: false });
const Dexie = require('dexie');
const db = new Dexie('my_db');
// schema
db.version(1).stores({
friends: 'name,shoeSize'
});
return db;
}
// file: index.js
import React from 'react';
import {
AppRegistry,
View,
} from 'react-native';
import configureDB from './configureDB';
function setup() {
db = configureDB();
// [...]
class App extends React.Component {
render() {
return (
<View style={styles.container}>
{...}
</View>
);
}
}
return App;
}
AppRegistry.registerComponent('MyApp', setup); This is a quite hackish and I wish we would have proper support for React Native directly in Also this setup is working (mostly) for my use case up until I say mostly as I have had incorrect results when querying multi-entry indexes with I thought those issues might be fixed by upgrading So I had to keep using Example: // When saving the following object with IndexedDBShim:
{ foo: "my value", bar: undefined }
// It is being stored within the sqlite DB as
{"foo":"my value","bar":null,"$types":{"bar":"userObject"}}
// And then when read back from the DB, I get this:
{ foo: "my value", bar: {} } I tracked it down to var Typeson = require('typeson');
var reg = require('typeson-registry');
// Simulates what happens in https://github.com/axemclion/IndexedDBShim/blob/6c4ed0694edefc37ce814140240d6f58f1e0f316/src/Sca.js
var structuredCloning = reg.presets.structuredCloningThrowing;
function traverseMapToRevertToLegacyTypeNames (obj) {
if (Array.isArray(obj)) {
return obj.forEach(traverseMapToRevertToLegacyTypeNames);
}
if (obj && typeof obj === 'object') { // Should be all
Object.entries(obj).forEach(([prop, val]) => {
if (prop in newTypeNamesToLegacy) {
const legacyProp = newTypeNamesToLegacy[prop];
delete obj[prop];
obj[legacyProp] = val;
}
});
}
}
var newTypeNamesToLegacy = {
IntlCollator: 'Intl.Collator',
IntlDateTimeFormat: 'Intl.DateTimeFormat',
IntlNumberFormat: 'Intl.NumberFormat',
userObject: 'userObjects',
undef: 'undefined',
negativeInfinity: 'NegativeInfinity',
nonbuiltinIgnore: 'nonBuiltInIgnore',
arraybuffer: 'ArrayBuffer',
blob: 'Blob',
dataview: 'DataView',
date: 'Date',
error: 'Error',
file: 'File',
filelist: 'FileList',
imagebitmap: 'ImageBitmap',
imagedata: 'ImageData',
infinity: 'Infinity',
map: 'Map',
nan: 'NaN',
regexp: 'RegExp',
set: 'Set',
int8array: 'Int8Array',
uint8array: 'Uint8Array',
uint8clampedarray: 'Uint8ClampedArray',
int16array: 'Int16Array',
uint16array: 'Uint16Array',
int32array: 'Int32Array',
uint32array: 'Uint32Array',
float32array: 'Float32Array',
float64array: 'Float64Array'
};
// console.log('StructuredCloning1', JSON.stringify(structuredCloning));
traverseMapToRevertToLegacyTypeNames(structuredCloning);
// console.log('StructuredCloning2', JSON.stringify(structuredCloning));
var TSON = new Typeson().register(structuredCloning);
var r = TSON.stringifySync({foo: "my value", bar: undefined });
console.log('==result', r);
var r2 = TSON.parse(r);
console.log('==result2', r2);
So this effectively suggests the issue is within the minified Hope it will help someone else. |
Great to have this info, thanks! I don't know when I may be able to get to looking at this to see what we can do to fix. Please keep us updated if you would if things change on the React Native side... |
@jeanregisser , can I ask if you also tested the non-minified bundle and whether that had problems for you too? |
I'd really like to better understand the issues in the latest IndexedDBShim regarding serialization/deserialization of objects with I suppose there is a chance though that the changes to |
This also works: // IndexedDB hackery begins
Object.setPrototypeOf = Object.setPrototypeOf || function(obj, proto) {
obj.__proto__ = proto;
return obj;
}
// Make indexeddbshim happy with React Native's environment
if (global.window.navigator.userAgent === undefined) {
global.window.navigator = { ...global.window.navigator, userAgent: '' };
}
const SQLite = require('react-native-sqlite-2').default;
global.window.openDatabase = SQLite.openDatabase;
require('indexeddbshim');
global.shimIndexedDB.__setConfig({ checkOrigin: false }); if you don't wanna do shrugs |
…porarily circumvent current limitations in Babel until <babel/babel#5978> addressed; (see also <indexeddbshim#311 (comment)>); fixes indexeddbshim#311 - Linting: Expand ESLint file coverage and apply minor linting fix to test file - npm: Update `eventtargeter` to avoid automatic `Object.setPrototypeOf` calls (make conditional on `CFG.fullIDLSupport`); fixes indexeddbshim#313 - npm: Bump to 3.6.0
…porarily circumvent current limitations in Babel until <babel/babel#5978> addressed; (see also <indexeddbshim#311 (comment)>); fixes indexeddbshim#311 - Linting: Expand ESLint file coverage and apply minor linting fix to test file - Testing: Update web-platform-tests and our tests accordingly - npm: Update `eventtargeter` to avoid automatic `Object.setPrototypeOf` calls (make conditional on `CFG.fullIDLSupport`); fixes indexeddbshim#313 - npm: Bump to 3.6.0
In 3.6.0, the issue with Note that as far as the previous comment, you don't need to polyfill But the latter example was a helpful reminder to check Anyways, feel free to report back on whether the fix works for everybody... |
@brettz9 doing this:
Am I missing something? |
Doing a log like this:
produces:
so the |
@vladikoff : What do you get if you try this instead: import setGlobalVars from 'indexeddbshim/src/setGlobalVars';
console.log(setGlobalVars); |
@brettz9 the loading via With The way
above won't help to save it. |
How about const setGlobalVars = require('indexeddbshim/dist/indexeddbshim-noninvasive.js');
console.log(setGlobalVars); Btw, I can easily push a fix to check the existence of |
(If you wanted to go the |
So what's the full currently established way of doing this? |
@zaptrem : See (or answer) my comments above to @vladikoff |
I’m not sure which of those is experimentation/broken(due to some
displaying crash screens)/intended for something else. My goal is to
enable(?) the shim so that any other code that assumes IndexedDB exists can
run with no modifications.
…On Fri, Feb 7, 2020 at 9:44 PM Brett Zamir ***@***.***> wrote:
@zaptrem <https://github.com/zaptrem> : See (or answer) my comments above
to @vladikoff <https://github.com/vladikoff>
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#313>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAMJTRVE4TN7VUXXZ5AKJADRBYMBRANCNFSM4ELGCFLA>
.
|
Also interested in using an IndexedDB shim on React Native/Expo. (Just realized that Cloud Firestore wasn't doing Offline Persistence due to the lack of it) I've tried doing what was mentioned in this thread firebase/firebase-js-sdk#436 by @zwily (Gist here : https://gist.github.com/zwily/e9e97e0f9f523a72c24c7df01d889482), and sure enough, like others, it seems to work on iOS, and crashes on Android. The crash shows a message I've identified to come from the function IDBObjectStore.prototype.__validateKeyAndValueAndCloneValue (inside '/indexeddbshim/dist/indexeddbshim-noninvasive.js') and the error is thrown after the following checks var _clonedValue = Sca.clone(value);
key = Key.extractKeyValueDecodedFromValueUsingKeyPath(_clonedValue, me.keyPath); // May throw so "rethrow"
...
if (key.failure) {
if (!cursorUpdate) {
if (!me.autoIncrement) {
throw (0, _DOMException.createDOMException)('DataError', 'Could not evaluate a key from keyPath and there is no key generator');
//--------> This here is the error that's happening on android
} I'm wondering if anyone could help shed some light on what is happening in this part of the code. The source is kind of full of semi-minified code and it's kind of hard to trace and understand them all. |
I think Android uses some special Facebook implementation of the JS Engine while iOS is required to use JavaScriptCore/WebKit. |
Thank you for reporting on what you have tried and where things are going wrong (the example code you have at https://gist.github.com/zwily/e9e97e0f9f523a72c24c7df01d889482 looks pretty sound, and may be helpful to others. (FWIW, I have added in a branch some code to avoid the need for the As far as the minified code, probably what you are seeing is from The code you have excerpted (and much of the codebase) tries to follow the spec pretty closely, even in terminology as possible. The code beginning The subsequent checks are a part of our internal A failure can occur when the method If the key path is an array, the array items will be individually checked and if any fail, that call will fail. The other cases that can fail can be seen at https://github.com/axemclion/IndexedDBShim/blob/master/src/Key.js#L609-L613 (the lines preceding also set These lines basically tell us that if the keypath sequence can't find a component in the sequence, then there will be a failure. The issue could occur with You can get usage help on Stackoverflow, but if there is a problem where cloning has not occurred properly, you can report that here (though it may need to go to typeson/typeson-registry, though I should also be able to help over there if that is the case). |
Thanks for the helpful info :) Scratch that : _clonedValue and value are the same. What I've found is that when state updates are performed (I guess the add, or put function is called), the value parameter is simply an empty object! Update : Empty object only occurs when 'add' is called; 'put' calls have objects with valid, expected keyPath property inside them. |
@swittk : I am guessing it is a bug in typeson or typeson-registry with the type of data you are trying to clone. What version of indexeddbshim are you using? I would update to 6.0.1 (or current However, it is puzzling to me that a simple object should have this issue. Is the Since those |
I'm so sorry, in my prior comment I wasn't logging the object correctly (logged clonedValue instead of _clonedValue). I've since edited the post. I've found that value and _clonedValue are identical in all cases (So Sca.clone works, no problem). However I've found that the crash occurs when the value provided is an empty object ( {} ), while keyPath is some value (keypath="batchId" to be exact; I guess it's something from Firestore) At first start of the application; one 'add' operation is successful
after that there are multiple calls to 'put', which seem to behave normally as expected Then just before the crash the condition referenced before happens
I'm not sure if I understand this correctly, but it seems either batchId came from somewhere unexpected within firebase, or the object itself isn't really a typical object. However, I've compared the logs to the react-native iOS version (which do work) and found that the same log occurs in IDBObjectStore.add, however it doesn't crash...?
In the iOS version, after the curious 'add' with the empty object call, normal 'put' calls are executed after that with no issue, and no crash occurs. update: This behavior probably coincides with creating a new document with no data in firestore then assigning data to it (to be specific in my case; creating a new DocumentReference, getting the id of it, then calling .set(value) to the DocumentReference). However, I still do not fully understand how batchId came to exist while the value object itself is completely empty.. |
I've since come to discover that the iOS and Android versions return identical results, and fail the same Key.extractKeyValueDecodedFromValueUsingKeyPath test (both return {"failure":true}). The only difference is in the autoIncrement property; iOS has it being true, while Android has it being false, and thus it chugs along on one and fails on the other.
Update: Seems autoIncrement is part of the firestore's invocation of createObjectStore, and looking in the firestore/firebase source's minified code; all instances of autoIncrement are set to !0 (true). So I'm not sure why the autoIncrement check is showing false here (on the me.autoIncrement check). |
Just found another interesting thing in case it would help; I logged storeProperties.autoInc inside IDBObjectStore.__createInstance and the results were surprising; on iOS almost all calls are autoInc:false, with very, very few calls having autoInc:true |
The logs show that the autoInc value provided to IDBObjectStore.__createInstance are the exact opposite in these two platforms .__. I don't know where to look for next; I'm guessing that the true/false values after the numerical values are due to the values being converted to Boolean somewhere, then copied around. But the initial numerical values being 0 instead of 1 and vice versa is confusing me. |
Looking at https://github.com/firebase/firebase-js-sdk/blob/542240d093ad54f3c8e5739dcb8449e8b9cab131/packages/firestore/src/local/indexeddb_schema.ts#L468-L470 , it appears that |
Please note that as of version 6.1.0 just released, the code at https://gist.github.com/zwily/e9e97e0f9f523a72c24c7df01d889482#file-expo-firestore-persistence-hack-js-L31-L34 should no longer be needed. I guess I'll keep this issue open in case someone can write up a section for a README explaining usage which is not tied to a particular library but may work for React Native (if such a thing is possible). But the README already details usage, so tbh, I'm not sure even this is necessary. (The Firebase issue appears to be a bug in Firebase, so you might try submitting a PR which tries adding |
I'm afraid I don't have much time to look into this now. IIUC, the default for OTOH, you may be right that the mirroring is due to something internal to IndexedDBShim and the environment. I think you need to find out where the calls to I did at least investigate the possibilities, and I suspect it is no. 2 below. What I might therefore try if I were you is changing the But in case that doesn't solve it, here is the full list of possibilities for the sources of the calls to
|
Just a little update; I have found the solution :) Then when I saw your latest comment, I figured that it might be worth trying to mess with that particular field ("autoInc BOOLEAN" in sys), that maybe Android SQLite really doesn't work well with Boolean Long story short; I changed two things to make it work (all within the insertStoreInfo inside IDBObjectStore.__createObjectStore)
Changed BOOLEAN to NUMERIC I did this first, but the resulting calls to IDBDatabase.__createInstance still showed storeProperties being the inverse of what was called to insertStoreInfo on first creation (Example)
Second run log : Shows IDBDatabase.__createInstance being called (I logged the storeProperties value)
So obviously, autoInc is still wrong. So I did the next logical thing in case Booleans are really not playing nice with Android
Seems dumb, right? I really didn't expect it to work, but hey.. the results speak for itself First app launch :
Second app launch:
AutoIncrement is now correct! And firestore offline persistence is now working BEAUTIFULLY :) Oh, one more thing, I really, really love the changes you made in the current release with the cleanup of the minified code base. However, the require('fs') line makes React Native crash due to it not supporting dynamic requires (Probably due to sad, probably security-related reasons) Right now I've changed it to this Sorry for the long post! I'm really excited to have solved this problem, and I'm very, very thankful to you for maintaining this library, and answering to issues with edge cases like this :) |
Great work finding the problem! I plan to look into a fix for that, and for the Rollup change (how Thank you for your persistence in helping us coming to a resolution. I guess it was not an underspecification within SQLite but that Android just has a bug in following this from the SQlite docs: "Boolean values are stored as integers 0 (false) and 1 (true)." Btw, would you mind testing to see whether we even need to change the BOOLEAN type in the CREATE TABLE syntax? It sounds like SQLite is documented as converting this automatically to NUMERIC. While I wouldn't expect making the type explicitly NUMERIC to be a breaking change for other environments if Android requires it, I'd rather not take the risk if it is not necessary. Btw, it looks like from the issue you describe in requiring "fs" that you mean the problem with React Native is in handling static (fs) requires rather than dynamic ones. If I'm understanding it correctly, this comment speaks to your workaround as well. |
Hi. I've just verified that changing from BOOLEAN to NUMERIC is not necessary; just the Number() cast is enough to retain the correct data. |
Also, I tried the latest committed code. Using constant strings causes react native to complain too, so the lines Currently I've converted it to this to work. |
Seems the version on npm isn't updating. I noticed something with the TravisCI build failing? |
Apologies, need a little time to do a final review. |
See #352 (comment) . In short, if folks could test |
Hey, I am using the import * as SQLite from 'expo-sqlite';
import setGlobalVars from 'indexeddbshim/dist/indexeddbshim-noninvasive';
setGlobalVars(window, { checkOrigin: false, win: SQLite }); |
Is there any way to use Indexed DB Polyfill in React Native Expo? WARN [2022-11-14T20:48:04.371Z] @firebase/app: Firebase: Error thrown when reading from IndexedDB. Original error: undefined is not a function (near '...(0, _idb.openDB)...'). (app/idb-get).
WARN [2022-11-14T20:48:04.402Z] @firebase/app: Firebase: Error thrown when reading from IndexedDB. Original error: undefined is not a function (near '...(0, _idb.openDB)...'). (app/idb-get).
WARN [2022-11-14T20:48:04.415Z] @firebase/app: Firebase: Error thrown when writing to IndexedDB. Original error: undefined is not a function (near '...(0, _idb.openDB)...'). (app/idb-set). |
I would like to use this library together with something like https://github.com/craftzdog/react-native-sqlite-2 (which provides a WebSQL compatible interface through SQLite) to get Indexeddb support on React-Native.
I was hopeing this would be easy and that something like the following should work:
But this fails when trying to import indexeddbshim with the following error:
I think that maybe it is trying to do some fancy stuff because it might think it is running on Node. I passed the SQLite object as win in the config because it would not find openDatabase any other way.
What can I try to make this work?
The text was updated successfully, but these errors were encountered: