-
Notifications
You must be signed in to change notification settings - Fork 85
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Rkyv 0.8 migration #575
Rkyv 0.8 migration #575
Commits on Sep 18, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 56e5d53 - Browse repository at this point
Copy the full SHA 56e5d53View commit details -
Configuration menu - View commit details
-
Copy full SHA for 0be556a - Browse repository at this point
Copy the full SHA 0be556aView commit details -
Configuration menu - View commit details
-
Copy full SHA for 11e112d - Browse repository at this point
Copy the full SHA 11e112dView commit details -
Configuration menu - View commit details
-
Copy full SHA for e004ce2 - Browse repository at this point
Copy the full SHA e004ce2View commit details -
Turns out, this is the wrong approach. It's still useful code because it's a direct method for passing strings that doesn't rely on rkyv (so it could potentially be more efficient for very efficiency-oriented applications). But I think in the end, after writing this and seeing the errors that come from it, I think we should just abandon String<N> altogether because it doesn't get us anything.
Configuration menu - View commit details
-
Copy full SHA for 3dd4e2a - Browse repository at this point
Copy the full SHA 3dd4e2aView commit details
Commits on Sep 22, 2024
-
Configuration menu - View commit details
-
Copy full SHA for a506e7c - Browse repository at this point
Copy the full SHA a506e7cView commit details -
Configuration menu - View commit details
-
Copy full SHA for 6d954d6 - Browse repository at this point
Copy the full SHA 6d954d6View commit details -
clean up a warning in loader crate
minor cleanup because the warning is distracting from chasing down all the other warnings
Configuration menu - View commit details
-
Copy full SHA for 423bfe9 - Browse repository at this point
Copy the full SHA 423bfe9View commit details -
Configuration menu - View commit details
-
Copy full SHA for 6470e1c - Browse repository at this point
Copy the full SHA 6470e1cView commit details -
regex search/replace instances of xous_ipc::String to String
split into a couple commits for ease of committing
Configuration menu - View commit details
-
Copy full SHA for a1bf70b - Browse repository at this point
Copy the full SHA a1bf70bView commit details -
regex search/replace instances of xous_ipc::String to String
split into multiple commits for easier committing
Configuration menu - View commit details
-
Copy full SHA for 6d7a8fd - Browse repository at this point
Copy the full SHA 6d7a8fdView commit details -
regex search/replace instances of xous_ipc::String to String
split into multiple commits for easier committing
Configuration menu - View commit details
-
Copy full SHA for 52ffd88 - Browse repository at this point
Copy the full SHA 52ffd88View commit details -
regex search/replace instances of xous_ipc::String to String
split into multiple commits for easier committing
Configuration menu - View commit details
-
Copy full SHA for 916c98f - Browse repository at this point
Copy the full SHA 916c98fView commit details -
regex search/replace instances of xous_ipc::String to String
core xous services cluster
Configuration menu - View commit details
-
Copy full SHA for 1d9316f - Browse repository at this point
Copy the full SHA 1d9316fView commit details -
regex search/replace instances of xous_ipc::String to String
more regex results
Configuration menu - View commit details
-
Copy full SHA for f2fbc7f - Browse repository at this point
Copy the full SHA f2fbc7fView commit details -
regex search/replace instances of xous_ipc::String to String
codegenerator fixes
Configuration menu - View commit details
-
Copy full SHA for 849ae5f - Browse repository at this point
Copy the full SHA 849ae5fView commit details -
regex search/replace instances of xous_ipc::String to String
includes some light touch-up to the API to clean up a consequence of String not being copy-able anymore
Configuration menu - View commit details
-
Copy full SHA for 3eb61d3 - Browse repository at this point
Copy the full SHA 3eb61d3View commit details -
Configuration menu - View commit details
-
Copy full SHA for c4f67cc - Browse repository at this point
Copy the full SHA c4f67ccView commit details -
Configuration menu - View commit details
-
Copy full SHA for 0ef93e3 - Browse repository at this point
Copy the full SHA 0ef93e3View commit details -
Configuration menu - View commit details
-
Copy full SHA for 15722a3 - Browse repository at this point
Copy the full SHA 15722a3View commit details -
regex xous_ipc::String to String + cleanup idioms
some idioms are improved with String, this cleans up a bit of code in addition to the search/replace
Configuration menu - View commit details
-
Copy full SHA for d2f13d9 - Browse repository at this point
Copy the full SHA d2f13d9View commit details -
Configuration menu - View commit details
-
Copy full SHA for 1b0fa6e - Browse repository at this point
Copy the full SHA 1b0fa6eView commit details -
cleanup a typecast ambiguity due to new API
Rust can't tell how to cast some u32_le into u32/usize automatically due to confusion with the num_derive trait, so we have to explicitly specify a chain of typecasts to clean up the conflict with the new rkyv
Configuration menu - View commit details
-
Copy full SHA for 0fbca1c - Browse repository at this point
Copy the full SHA 0fbca1cView commit details -
Configuration menu - View commit details
-
Copy full SHA for b514d31 - Browse repository at this point
Copy the full SHA b514d31View commit details -
Configuration menu - View commit details
-
Copy full SHA for 1fd59f0 - Browse repository at this point
Copy the full SHA 1fd59f0View commit details -
security-relevant: clear_volatile() API missing on String
xous_ipc::<String> had a clear-volatile() API for zeroizing sensitive strings. This is now missing in String, so it has been manually implemented here by casting the String to a slice-u8 and then iterating through it using raw pointers and volatile writes with a compiler barrier to hopefully avoid all of the landmines I know of for zeroizing a string... commit is called out separately because this one is one of the more "dangerous" security-related changes in the API migration.
Configuration menu - View commit details
-
Copy full SHA for f1d2633 - Browse repository at this point
Copy the full SHA f1d2633View commit details -
Configuration menu - View commit details
-
Copy full SHA for f8719d7 - Browse repository at this point
Copy the full SHA f8719d7View commit details -
manual cleanup Copy derive: gam crate
There isn't a simple regex to deal with the loss of Copy when deprecating xous_ipc, so this has to be a manual pass. Losing the copy trait means (maybe) some loss of efficiency in the API because we have to clone some structures instead of copying them, although, I think actually what might have been happening under the hood anyways is the copy was implemented as a clone-under-the-hood for the xous-ipc. Either way, this happens in the high level management structures for modals, so it's not the end of the world to incur a clone on the description of the structures -- we'd have to pay more attention if this happened e.g. at the pixel shuffling level, but a few UI strings at human-interaction rates isn't going to kill us.
Configuration menu - View commit details
-
Copy full SHA for 58527ea - Browse repository at this point
Copy the full SHA 58527eaView commit details -
review-needed: modify the alert modal prompt loop
The original loop takes advantage of the copy-trait that is no longer available. I think the intent of the loop was to mutate a field within the copy through each iteration so you can update the prompt to handle field validations that failed without re-allocating the ManagedPromtWithTextResponse structure. This has been rewritten to alloc that each iteration to avoid the borrow error, and instead the `prompt` itself is a string that's initialized with the initial prompt message and is updated on each iteration if the form validation fails. @gsora git seems to say you wrote this code, maybe I could use your help to review this patch.
Configuration menu - View commit details
-
Copy full SHA for dfd7272 - Browse repository at this point
Copy the full SHA dfd7272View commit details -
Configuration menu - View commit details
-
Copy full SHA for af35f6e - Browse repository at this point
Copy the full SHA af35f6eView commit details -
bunnie-review: fix borrows in modals main loop
note for more review by bunnie down the road to make sure that I have not changed any behaviors accidentally in fixing the loss of Copy on a core structure
Configuration menu - View commit details
-
Copy full SHA for f6f7cbd - Browse repository at this point
Copy the full SHA f6f7cbdView commit details -
a bunch of code has to be re-formatted because spacing changed and we have a strict formatting rule
Configuration menu - View commit details
-
Copy full SHA for 0cdbfdf - Browse repository at this point
Copy the full SHA 0cdbfdfView commit details -
formatting changes that resulted from the regex search
a bunch of lines are shorter or different length now which triggers a mandatory reformat. this patch cleans that up, so that going forward the manual rework patches are clearer and not tied up with formatting patches.
Configuration menu - View commit details
-
Copy full SHA for 34e6f46 - Browse repository at this point
Copy the full SHA 34e6f46View commit details -
removed sized strings from pddb interchange structures
I think this is safe to do because no strings can be directly serialized into the PDDB's structures. The place to check for this is inside the backend/ directory. For example, `DictName` is where the name field is serialized into a structure that goes onto the disk. It's explicitly length-checked. What this means is that we could have calls into the pddb crate with invalid length strings, however, if you check the lib implementation, all of the APIs also pre-flight the length of arguments to ensure they fit into the destination string. So mostly what we're losing is the constraints on the intermediate IPC representation, but the length is still checked on either side.
Configuration menu - View commit details
-
Copy full SHA for be227ad - Browse repository at this point
Copy the full SHA be227adView commit details -
Configuration menu - View commit details
-
Copy full SHA for 1b5eacd - Browse repository at this point
Copy the full SHA 1b5eacdView commit details -
Configuration menu - View commit details
-
Copy full SHA for 6e4eec9 - Browse repository at this point
Copy the full SHA 6e4eec9View commit details -
ah yes. This was the kind of xous-ipc abuse I was worried I would hav…
…e done. And I did it! and it's now fixed. I was using xous-ipc String to make a fixed-length string in a structure that's not strictly an IPC structure. The solution to this is to just implement a fixed length string directly.
Configuration menu - View commit details
-
Copy full SHA for 7141a02 - Browse repository at this point
Copy the full SHA 7141a02View commit details -
Configuration menu - View commit details
-
Copy full SHA for 8399239 - Browse repository at this point
Copy the full SHA 8399239View commit details -
Configuration menu - View commit details
-
Copy full SHA for 4eb504b - Browse repository at this point
Copy the full SHA 4eb504bView commit details -
Configuration menu - View commit details
-
Copy full SHA for 15aaf55 - Browse repository at this point
Copy the full SHA 15aaf55View commit details -
Configuration menu - View commit details
-
Copy full SHA for e423bfd - Browse repository at this point
Copy the full SHA e423bfdView commit details -
Configuration menu - View commit details
-
Copy full SHA for 95a31a3 - Browse repository at this point
Copy the full SHA 95a31a3View commit details -
cleanup
replace
, and make into_inner/from_inner methodsthe into/from inner methods are useful for using the Buffer abstraction as a way to invoke rkyv on a structure to serialize it and then manipulate the serialized data directly
Configuration menu - View commit details
-
Copy full SHA for 1bd8be2 - Browse repository at this point
Copy the full SHA 1bd8be2View commit details -
Configuration menu - View commit details
-
Copy full SHA for ef71885 - Browse repository at this point
Copy the full SHA ef71885View commit details -
fixup API to not rely on copy, instead use borrow
also some more regex fixup
Configuration menu - View commit details
-
Copy full SHA for 8424d19 - Browse repository at this point
Copy the full SHA 8424d19View commit details -
final cleanup on shellchat xous-ipc abstractions
the implementation relied on a quirk of xous-ipc where the message ID is 0 for string send/receives. This cleans it up so that the API is transparent and consistent with other API calls in the system.
Configuration menu - View commit details
-
Copy full SHA for 3c0140b - Browse repository at this point
Copy the full SHA 3c0140bView commit details -
Configuration menu - View commit details
-
Copy full SHA for 5cad1a3 - Browse repository at this point
Copy the full SHA 5cad1a3View commit details -
Configuration menu - View commit details
-
Copy full SHA for 4c7c931 - Browse repository at this point
Copy the full SHA 4c7c931View commit details -
Configuration menu - View commit details
-
Copy full SHA for ada196d - Browse repository at this point
Copy the full SHA ada196dView commit details -
Configuration menu - View commit details
-
Copy full SHA for 840964d - Browse repository at this point
Copy the full SHA 840964dView commit details -
Configuration menu - View commit details
-
Copy full SHA for d78cbff - Browse repository at this point
Copy the full SHA d78cbffView commit details -
Configuration menu - View commit details
-
Copy full SHA for 65e3a03 - Browse repository at this point
Copy the full SHA 65e3a03View commit details -
Configuration menu - View commit details
-
Copy full SHA for 89bc7c1 - Browse repository at this point
Copy the full SHA 89bc7c1View commit details -
attempt to update rkyv use in TLS library
@nworbnhoj would love it if you could have a look at the implementation and let me know if I've ported this correctly. I think what was going on is you were using rkyv to serialize a structure to the PDDB. I've just updated the ser/de to use the latest `alloc` API (which is much simpler than the API used in xous-ipc because it is no-alloc). I'll try and run some tests that involve the TLS crate too down the road under the theory that it would suss out any porting errors.
1Configuration menu - View commit details
-
Copy full SHA for 44b9041 - Browse repository at this point
Copy the full SHA 44b9041View commit details -
Configuration menu - View commit details
-
Copy full SHA for 5973e48 - Browse repository at this point
Copy the full SHA 5973e48View commit details -
Configuration menu - View commit details
-
Copy full SHA for 2f86042 - Browse repository at this point
Copy the full SHA 2f86042View commit details -
Configuration menu - View commit details
-
Copy full SHA for a3933a7 - Browse repository at this point
Copy the full SHA a3933a7View commit details -
Configuration menu - View commit details
-
Copy full SHA for 888cd22 - Browse repository at this point
Copy the full SHA 888cd22View commit details -
Configuration menu - View commit details
-
Copy full SHA for 7d69e16 - Browse repository at this point
Copy the full SHA 7d69e16View commit details -
update formatting on the published crates
not doing a reformat of the entire project to match nightly formatting changes, but capturing the published crates so we can maybe save a version bump later on to get the project clean.
Configuration menu - View commit details
-
Copy full SHA for bb72095 - Browse repository at this point
Copy the full SHA bb72095View commit details -
Configuration menu - View commit details
-
Copy full SHA for 75954de - Browse repository at this point
Copy the full SHA 75954deView commit details -
Configuration menu - View commit details
-
Copy full SHA for e01326c - Browse repository at this point
Copy the full SHA e01326cView commit details -
Configuration menu - View commit details
-
Copy full SHA for d39ca54 - Browse repository at this point
Copy the full SHA d39ca54View commit details -
Configuration menu - View commit details
-
Copy full SHA for 838bd58 - Browse repository at this point
Copy the full SHA 838bd58View commit details
Commits on Sep 23, 2024
-
Configuration menu - View commit details
-
Copy full SHA for cbe9aa0 - Browse repository at this point
Copy the full SHA cbe9aa0View commit details -
Configuration menu - View commit details
-
Copy full SHA for 489c614 - Browse repository at this point
Copy the full SHA 489c614View commit details -
Configuration menu - View commit details
-
Copy full SHA for e9ab865 - Browse repository at this point
Copy the full SHA e9ab865View commit details -
Configuration menu - View commit details
-
Copy full SHA for 9ebb61c - Browse repository at this point
Copy the full SHA 9ebb61cView commit details -
Configuration menu - View commit details
-
Copy full SHA for ec43c46 - Browse repository at this point
Copy the full SHA ec43c46View commit details -
Configuration menu - View commit details
-
Copy full SHA for cdc8411 - Browse repository at this point
Copy the full SHA cdc8411View commit details -
Configuration menu - View commit details
-
Copy full SHA for e5bd1cf - Browse repository at this point
Copy the full SHA e5bd1cfView commit details -
Configuration menu - View commit details
-
Copy full SHA for f4a4dfb - Browse repository at this point
Copy the full SHA f4a4dfbView commit details -
Configuration menu - View commit details
-
Copy full SHA for d581bd4 - Browse repository at this point
Copy the full SHA d581bd4View commit details -
Configuration menu - View commit details
-
Copy full SHA for 4239176 - Browse repository at this point
Copy the full SHA 4239176View commit details -
Configuration menu - View commit details
-
Copy full SHA for 553f98f - Browse repository at this point
Copy the full SHA 553f98fView commit details -
Configuration menu - View commit details
-
Copy full SHA for 7ea1400 - Browse repository at this point
Copy the full SHA 7ea1400View commit details -
Configuration menu - View commit details
-
Copy full SHA for 1ec6613 - Browse repository at this point
Copy the full SHA 1ec6613View commit details -
Configuration menu - View commit details
-
Copy full SHA for 3ce6baa - Browse repository at this point
Copy the full SHA 3ce6baaView commit details -
Configuration menu - View commit details
-
Copy full SHA for c699e06 - Browse repository at this point
Copy the full SHA c699e06View commit details -
Configuration menu - View commit details
-
Copy full SHA for 048e8d8 - Browse repository at this point
Copy the full SHA 048e8d8View commit details -
length of a record stored is the
pos
field for rkyv-to-pddbprior use of `rkyv` reserves two bytes to record `pos` pre-pended to the rkyv record the new rkyv works without having to do that because it indexes the rkyv data off the end of the stored records. Thus, all you need to know is the total length of the byte slice that stores the serialized data. This is available as the return argument when you read a PDDB key, so we don't have to explicitly store it anymore.
Configuration menu - View commit details
-
Copy full SHA for 3a9a7eb - Browse repository at this point
Copy the full SHA 3a9a7ebView commit details -
Configuration menu - View commit details
-
Copy full SHA for e61a2e3 - Browse repository at this point
Copy the full SHA e61a2e3View commit details -
add bounds checking to a TextView string length
Before xous-ipc's fixed-length string guaranteed this thing fit into a single page. Now, we can send potentially unbounded amounts of text between servers. I don't think it makes sense to send more than a few k of text to render on the screen at once, given our screen sizes, so I've added a patch that truncates the text to the previous length that things would truncate to. The behavior is the same as before, except that you will over-allocate memory on the heap in the sending process as you shove more string data into a TextView than can be rendered. Previously, the TextView would just truncate the data and discard the data. The truncation happens at the point where you try to hand the record off to another process, so, one could in theory try to make this recoverable by adding a layer that splits the textview into more manageable chunks or pages the data, but that is an exercise for another day.
Configuration menu - View commit details
-
Copy full SHA for 921be1c - Browse repository at this point
Copy the full SHA 921be1cView commit details -
we don't have to pass a "size" field anymore; since we're end-offset from pos, just pass pos and use that to chain records.
Configuration menu - View commit details
-
Copy full SHA for 8cd1cef - Browse repository at this point
Copy the full SHA 8cd1cefView commit details -
fail, don't panic on serialization failure
this allows us to handle the error. probably should propagate on to xous-ipc
Configuration menu - View commit details
-
Copy full SHA for 2e9b19d - Browse repository at this point
Copy the full SHA 2e9b19dView commit details -
Configuration menu - View commit details
-
Copy full SHA for 047ffe3 - Browse repository at this point
Copy the full SHA 047ffe3View commit details -
change rkyv behavior in xous-ipc from panic to failure
this allows for more graceful error handling in case of serialization failures
Configuration menu - View commit details
-
Copy full SHA for 0ecfa80 - Browse repository at this point
Copy the full SHA 0ecfa80View commit details -
Configuration menu - View commit details
-
Copy full SHA for f494da2 - Browse repository at this point
Copy the full SHA f494da2View commit details -
Configuration menu - View commit details
-
Copy full SHA for 9c2ee16 - Browse repository at this point
Copy the full SHA 9c2ee16View commit details -
Configuration menu - View commit details
-
Copy full SHA for 67e0004 - Browse repository at this point
Copy the full SHA 67e0004View commit details -
Configuration menu - View commit details
-
Copy full SHA for e68fde8 - Browse repository at this point
Copy the full SHA e68fde8View commit details