-
Notifications
You must be signed in to change notification settings - Fork 52
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Rubygems and bundler #14
Comments
I've been working on a couple of different approaches to this, if you'd like to talk about it more, feel free to dm me (on twitter, in bio). Basically, I've made it (kind of) work by building from source after adding gems to the default gems in 'ruby-wasm32-wasi/usr' as mounted to '/usr' which is kind of clunky 🙈 , and that's why I am also trying to find a more elegant solution. I would guess the maintainers here have some better thoughts on how to approach this 😄 . |
It still needs some hacks, but here is how to use bundler and rubygems on Wasm. The main important points are:
|
@kateinoigakukun - can you also lend some direccions on running C-compiled libraries? |
And looks like your branch #15 merged; |
I'm not well familiar with nokogiri, but it seems using mkmf. So what needs to be done to support wasm target are:
|
I updated the instruction, but not much changed 👍 |
I began running your recipe using a couple changes: I cloned today's release of curl -LO https://github.com/ruby/ruby.wasm/releases/download/2022-07-05-a/ruby-head-wasm32-unknown-wasi-full.tar.gz
tar xfz ruby-head-wasm32-unknown-wasi-full.tar.gz
rbenv local 3.2.0-dev I prepared an example running a simpler rubygem, bundle init
bundle add ox
bundle config set --local path 'vendor/bundle'
bundle install I added a simple program, require 'bundler/setup'
require 'net/http'
require 'ox'
Ox.default_options = { mode: :generic, effort: tolerant, smart: true, }
page = Ox.parse Net::HTTP.get URI.parse 'http://example.com'
puts page.locate('*/a')[0].attributes['href'] And some helper code in #!/bin/bash
run() {
wasmtime run \
--mapdir /usr::./head-wasm32-unknown-wasi-full/usr \
--mapdir /root::./ \
--mapdir /dev::/dev \
--env BUNDLE_GEMFILE=/root/Gemfile \
head-wasm32-unknown-wasi-full/usr/local/bin/ruby -- \
"$@"
}
run --version
run '-e' 'puts Dir.glob("/root/vendor/bundle/ruby/3.2.0+1/gems/*")'
run /root/ox.rb I seemed prepared enough, only by some quirk my rubygems had gone missing. [grace@chesapeake ruby]$ ./run.sh
ruby 3.2.0dev (2022-07-06) [wasm32-wasi]
/root/vendor/bundle/ruby/3.2.0+1/gems/ox-2.14.11
Could not find ox-2.14.11 in any of the sources
Run `bundle install` to install missing gems. Perhaps in a side discussion, can you explain the [grace@chesapeake ruby]$ run -e "require 'net/http'"
Ignoring debug-1.5.0 because its extensions are not built. Try: gem pristine debug --version 1.5.0
Ignoring rbs-2.6.0 because its extensions are not built. Try: gem pristine rbs --version 2.6.0
<internal:/usr/local/lib/ruby/3.2.0+1/rubygems/core_ext/kernel_require.rb>:85:in `require': cannot load such file -- socket (LoadError)
from <internal:/usr/local/lib/ruby/3.2.0+1/rubygems/core_ext/kernel_require.rb>:85:in `require'
from /usr/local/lib/ruby/3.2.0+1/net/protocol.rb:22:in `<top (required)>'
from <internal:/usr/local/lib/ruby/3.2.0+1/rubygems/core_ext/kernel_require.rb>:85:in `require'
from <internal:/usr/local/lib/ruby/3.2.0+1/rubygems/core_ext/kernel_require.rb>:85:in `require'
from /usr/local/lib/ruby/3.2.0+1/net/http.rb:23:in `<top (required)>'
from <internal:/usr/local/lib/ruby/3.2.0+1/rubygems/core_ext/kernel_require.rb>:85:in `require'
from <internal:/usr/local/lib/ruby/3.2.0+1/rubygems/core_ext/kernel_require.rb>:85:in `require'
from -e:1:in `<main>' |
@c4lliope It looks like |
I had been guessing my problem is misaligned ruby version numbers; |
@c4lliope You can see which paths are in the search path by:
And you will see Also you can see more verbose log by replacing
Please note that Again, ox has C extension library and it's not compiled in your build script. That's why it fails. |
Ah, incredible description; these commands are useful for digging deeper in my build. |
Hello! I'm seeing some progress, and some remaining errors. I had an example going with the simple and dependency-less I began seeing an odd error with the gem he chose as an example, though - $ wasmtime run \
--mapdir /usr::3_2-wasm32-unknown-wasi-full/usr \
--mapdir /src::./src \
--env BUNDLE_GEMFILE=/src/Gemfile \
ruby.wasm -- /src/tree.rb
Could not find fiddle-1.1.1 in locally installed gems
Run `bundle install` to install missing gems. Looking closer, the $ bundle install
Using bundler 2.4.1
Using colorize 0.8.1
Using fiddle 1.1.1
Using prettier_print 1.2.0
Using syntax_tree 5.3.0
Bundle complete! 3 Gemfile dependencies, 5 gems now installed.
Bundled gems are installed into `./bundle`
$ bundle show syntax_tree
/home/grace/src/ruby-wasm/src/bundle/ruby/3.2.0/gems/syntax_tree-5.3.0
$ bundle show fiddle
/home/grace/.rbenv/versions/3.2.0/lib/ruby/gems/3.2.0/gems/fiddle-1.1.1 Since my Ruby versions are the same, I'd imagine $ ruby -v
ruby 3.2.0 (2022-12-25 revision a528908271) [x86_64-linux]
$ wasmtime ruby.wasm -- -v
ruby 3.2.0 (2022-12-25 revision a528908271) [wasm32-wasi] Sadly, no $ ls 3_2-wasm32-unknown-wasi-full/usr/local/lib/ruby/gems/3.2.0/gems/
abbrev-0.1.1 etc-1.4.2 net-pop-0.1.2 psych-5.0.1 singleton-0.1.1
base64-0.1.1 fcntl-1.0.2 net-protocol-0.2.1 racc-1.6.2 stringio-3.0.4
benchmark-0.2.1 fileutils-1.7.0 net-smtp-0.3.3 rake-13.0.6 strscan-3.0.5
bigdecimal-3.1.3 find-0.1.1 nkf-0.1.2 rbs-2.8.2 syntax_suggest-1.0.2
bundler-2.4.1 forwardable-1.3.3 observer-0.1.1 rdoc-6.5.0 tempfile-0.1.3
cgi-0.3.6 getoptlong-0.2.0 open3-0.1.2 readline-0.0.3 test-unit-3.5.7
csv-3.2.6 ipaddr-1.2.5 openssl-3.1.0 reline-0.3.2 time-0.2.1
date-3.3.3 irb-1.6.2 open-uri-0.3.0 resolv-0.2.2 timeout-0.3.1
debug-1.7.1 json-2.6.3 optparse-0.3.1 resolv-replace-0.1.1 tmpdir-0.1.3
delegate-0.3.0 logger-1.5.3 ostruct-0.5.5 rexml-3.2.5 tsort-0.1.1
did_you_mean-1.6.3 matrix-0.4.2 pathname-0.2.1 rinda-0.1.1 typeprof-0.21.3
digest-3.1.1 minitest-5.16.3 power_assert-2.0.3 rss-0.2.9 un-0.2.1
drb-2.1.1 mutex_m-0.1.2 pp-0.4.0 ruby2_keywords-0.0.5 uri-0.12.0
english-0.7.2 net-ftp-0.2.0 prettyprint-0.1.1 securerandom-0.2.2 weakref-0.1.2
erb-4.0.2 net-http-0.3.2 prime-0.1.2 set-1.0.3 yaml-0.2.1
error_highlight-0.5.1 net-imap-0.3.4 pstore-0.1.2 shellwords-0.1.0 zlib-3.0.0 Here are the gems in my normal Ruby 3.2.0: $ ls ~/.rbenv/versions/3.2.0/lib/ruby/gems/3.2.0/gems/
abbrev-0.1.1 getoptlong-0.2.0 ostruct-0.5.5 ruby2_keywords-0.0.5
addressable-2.8.1 io-console-0.6.0 pathname-0.2.1 rubygems-update-3.4.5
base64-0.1.1 io-nonblock-0.2.0 power_assert-2.0.3 securerandom-0.2.2
benchmark-0.2.1 io-wait-0.3.0 pp-0.4.0 set-1.0.3
bigdecimal-3.1.3 ipaddr-1.2.5 prettyprint-0.1.1 shellwords-0.1.0
bundler-2.4.1 irb-1.6.2 prime-0.1.2 singleton-0.1.1
bundler-2.4.5 json-2.6.3 pry-0.14.2 stringio-3.0.4
capybara-3.38.0 logger-1.5.3 pstore-0.1.2 strscan-3.0.5
cgi-0.3.6 matrix-0.4.2 psych-5.0.1 syntax_suggest-1.0.2
coderay-1.1.3 method_source-1.0.0 public_suffix-5.0.1 syslog-0.1.1
csv-3.2.6 mini_mime-1.1.2 racc-1.6.2 tempfile-0.1.3
date-3.3.3 minitest-5.16.3 rack-3.0.4.1 test-unit-3.5.7
debug-1.7.1 mutex_m-0.1.2 rack-test-2.0.2 time-0.2.1
delegate-0.3.0 net-ftp-0.2.0 rake-13.0.6 timeout-0.3.1
did_you_mean-1.6.3 net-http-0.3.2 rbs-2.8.2 tmpdir-0.1.3
digest-3.1.1 net-imap-0.3.4 rdoc-6.5.0 torch-rb-0.12.0
drb-2.1.1 net-pop-0.1.2 readline-0.0.3 torch-rb-0.12.1
english-0.7.2 net-protocol-0.2.1 readline-ext-0.1.5 tsort-0.1.1
erb-4.0.2 net-smtp-0.3.3 regexp_parser-2.6.2 typeprof-0.21.3
error_highlight-0.5.1 nkf-0.1.2 reline-0.3.2 un-0.2.1
etc-1.4.2 nokogiri-1.14.0-x86_64-linux resolv-0.2.2 uri-0.12.0
fcntl-1.0.2 observer-0.1.1 resolv-replace-0.1.1 weakref-0.1.2
fiddle-1.1.1 open3-0.1.2 rexml-3.2.5 xpath-3.2.0
fileutils-1.7.0 openssl-3.1.0 rice-4.0.4 yaml-0.2.1
find-0.1.1 open-uri-0.3.0 rinda-0.1.1 zlib-3.0.0
forwardable-1.3.3 optparse-0.3.1 rss-0.2.9 There's much |
In short, |
I tried the example in your gist without success, in fact I'm not sure I even understand it (shouldn't it need to build with the vfs first?). Anyway, I attached my experience to that. Has anyone actually managed to do this at all, or should I just inline code (pure ruby only, obviously) to add additional libraries? FYI the lib I was trying with was a pure-ruby msgpack gem. |
I added an experimental CLI tool to package gems into .wasm #358. It supports pure Ruby gems and very simple extension libraries at this moment. We can support more complex extension libraries like nokogiri after dynamic linking support. |
I just wrote a FAQ entry query, for gem support. I did not know this thread exists before.
Hey there kateinoigakukun - could you add a FAQ entry perhaps, and simple usage example, at the least for a simple gem? Perhaps one day more complex gems work but for me I'd be happy to just have small gems that do things be supported. I want to find out how many of my gems I can make work via wasm.ruby, and that includes super-simple gems like hello world or perhaps a bit more complicated ones. Also, if anyone has time, it would be nice to show limitations; I assume we can not create directories or files via a .html file that has ruby.wasm? If so it would be GREAT, but if not then I understand it, the browser is a limited environment. The FAQ could mention this though - I desperately need documentation. I am really bad at figuring out things on my own (except for a bit of trial and error, and reading other people's examples; that would also be nice to have for ruby.wasm, examples by other people so we can learn from that). |
Hello again - I had some success running Ruby code in an online page,
as you can see on ://ruby.assembled.app.
This is good progress, and I'm now curious if there is an approach
for requiring ruby gem libraries inside the
.wasm
package,so my code can use helpers such as
1.second
and"hello".titlecase
,among others.
I recognize some libraries including C extensions, such as Nokogiri,
are unlikely to be runnable inside
.wasm
unless specially compiled.For non-C gems, I'm curious if there is a non-compilation approach,
though if compiling a new
.wasm
binary is necessary, I'd be glad for directions.The text was updated successfully, but these errors were encountered: