Reducing size of contract spec/interface in contract wasm files #1551
Replies: 1 comment 4 replies
-
|
💡 A low complex approach to reducing the size would be to GZ compress the specs. Here's an assessment of what the saving would be to GZ compress the contracts installed on mainnet. Tools that need to read the specs would need to decompress them. Most languages and environments support GZ in their stdlib, including browsers.
Ref: spec-size-gz.csv |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for the numbers, that's very useful! However, the table is a bit hard to digest, so here are some stats to help putting things into perspective:
We also should take into account the cumulative effect of every contract being smaller - if every contract had ~10% reduced size on average, every transaction would cost ~10% less in terms of read (unless it also reads very large non-contract code entries) and the ledger read capacity will be freed up to the same degree. While I think the impact doesn't look that large in absolute terms, it's still a meaningful cost saving in a lot of cases without any additional effort beyond the initial implementation. So I think the main question is whether there is easy access to the safe gz libraries that do sufficient sandboxing and can unzip data from untrusted sources. If safety is a hard issue to solve, then these savings are probably not worth it and we should think of alternative ways of reducing the spec size. |
Beta Was this translation helpful? Give feedback.
-
That's a very very very wrong way to interpret the numbers from the table. Focusing on averages and medians without analyzing reasons behind the size overhead makes no sense. If you check several relatively large contracts with larger spec/contract size ratio from the table head (e.g. this or this one) you can see that their meta contains detailed Rust Doc comments, while similar in size contracts from the bottom of the table have no docs included into the meta (for example, this contract). When we deployed Reflector contracts on Pubnet, we had to replace all doc comments with regular comments ( I'd say that the documentation comments make the biggest impact on the contract meta size variations in this table. Of course, if we ignore micro-contracts with simple internal logic, where the size of the interface meta might be simply dictated by the number of exported functions/types. The spec format is more-or-less efficient, so I think it makes sense to focus on reducing the doc comments size. |
Beta Was this translation helpful? Give feedback.
-
|
Extracting autogenerated docs for the contract interface and storing them elsewhere may solve the problem of contract meta size growth. Docs are not essential for the contract execution, discovery, and interoperability. Unlike functions/types meta. A file with docs may be stored in GitHub repo or on IPFS. We can agree on the discoverability of the docs meta file and standardize it in a SEP -- there are several option here. The file itself might look something like this: {
"functions": {
"blip": {
"doc": "It blips like no other smart contract"
"inputs": [
{
"name": "message",
"doc": "Message to send while blipping"
},
{
"name": "iterations",
"doc": "How many times is should blip"
}
],
"outputs": {
"doc": "Returns the number of blips executed"
}
}
},
"structs": [
"BlipInfo": {
"fields": {
"message": {
"doc": "Message sent"
},
"iteration": {
"doc": "Current iteration"
}
}
}
]
}JSON is the simplest serialization form, directly accessible by all web apps. Since this meta will be loaded in async manner, its size and compression won't matter for tools that will be displaying the docs, so JSON will work just fine. The format was inspired by the contract-wasm-interface-parser lib, but any other format will do just fine as well. |
Beta Was this translation helpful? Give feedback.
-
|
Good point about docs, we should look at them specifically. I just did an analysis of all contracts on mainnet and this is what I found:
Data: specgznodoc.csv (data is from leighmcculloch/stellar-contract-wasms@a320867) With the above findings, I'm leaning towards:
Realistically what I see is folks making great websites to contain docs about their contracts, so in reality docs are maybe not the win they were imagined to be, but given GZ makes them significantly more affordable, it might be worth giving them a little more life before doing (3). |
Beta Was this translation helpful? Give feedback.
-
From time time I hear folks talk about the size that contract specs take up in deployed contracts, and it seems like an issue that is a concern to some.
I'm starting this conversation to discuss different ideas on how the contract specs might be compacted, compressed, or changed to reduce their impact to .wasm sizes. It is also an opportunity to voice an opinion on if this is a material issue, or not an issue.
To keep the conversation organised, a few asks for participating in this thread:
To comment on existing ideas that have already been posted in a comment, please post a sub-comment on the comment.
For new ideas that don't already have a thread, please post a top-level comment proposing the idea and showing example numbers for what size the contract spec would become. There are a few hundred contracts on pubnet that can be used as examples. See the notes below for how to get and analyse them.
To indicate that contract specs size is a problem for you, press the emoji button at the top of the discussion and select 👍🏻 .
To indicate that contract specs size is not a problem for you, press the emoji button at the top of the discussion and select 👎🏻 .
Take less structured discussion to Discord here.
Note
There is no existing SEP that captures how specs are defined and work, there probably should be, but in its absence here's a quick summary:
A contract spec details the exported interface of the contract, including the:
A contract's contract spec is stored in the custom section of the Wasm file, in the section with name
contractspecv0.The format of contract specs are a stream of XDR
SCSpecEntryentities appended. The XDR is defined in Stellar-contract-spec.x.Developer tooling like the soroban-sdk, stellar-cli, the stellar sdks (e.g. js-stellar-sdk, etc), and other tools (e.g. stellar.expert), to just name a few, extract contract specs out of .wasm files to understand what functions exist, what parameters they require, how to construct the types, and what values they return. Contract specs are used for bindings generation to make it possible to call a contract with types native in the language in use that get converted into their Stellar representations.
Tip
Looking for contracts to analyze? I've uploaded a few hundred contracts that are deployed on mainnet here:
https://github.com/leighmcculloch/stellar-contract-wasms
Tip
If you'd like to explore the spec data is stored in a contract wasm file, or any of the files above, the wasm-cs and stellar-cli tools can inspect them, see the examples below:
Examples
Inspect what custom sections exist in your contract wasm, and how much space they are consuming:
Inspect the contents of the contractspecv0 custom section:
Decode the contract specs:
As a starting point, here's a csv of contracts on mainnet at ledger
53458367, their size, and the size of their spec. Note this list includes all contracts installed. There's no assessment of whether they are actively used, or valuable to the community, and are just one way to consider the problem.Ref: spec-size.csv
Discord discussion thread: https://discord.com/channels/897514728459468821/1283687871491997716
Beta Was this translation helpful? Give feedback.
All reactions