Ruby C Extensions, Explained
- How To Use This Repository
- Strategy Combinations
Hello there! Welcome to Ruby C Extensions, Explained.
This repository, and the example ruby gems in it, were originally written as companion materials for some conference talks being given in 2021 by Mike Dalessio (@flavorjones).
RubyKaigi 2021 talk 2021-09-11
This talk focuses mostly on the mechanics of the gems in this repository.
RubyConf 2021 talk 2021-11-08
This talk may cover slightly different material, including an approach to rigorous testing, how to think about Trust in the context of precompilation, and what industrializing precompilation might look like.
How To Use This Repository
Each of the gems in this repository demonstrate a different strategy for integrating with (or packaging) third-party libraries. A brief explanation of the strategies is below, and the code for each gem is in a distinct subdirectory, along with a README that contains a more complete explanation.
Each gem builds on the previous one if you tackle them in this order:
These are actual, working gems! And they've all been pushed to rubygems.org:
gem install rcee_isolated
gem install rcee_system
gem install rcee_packaged_source
gem install rcee_packaged_tarball
gem install rcee_precompiled
Or they can all be installed as dependencies of the meta-gem
gem install rcee
Also note that these gems are tested. In particular, the precompiled native gems have a rather complex setup to build, install, and test across platforms using
rake-compiler-dock and Github Actions. See the Actions running at https://github.com/flavorjones/ruby-c-extensions-explained/actions
A few commonly-followed approaches have emerged in the Ruby ecosystem. Broadly categorized, the strategies are:
Strategy 0, "isolated"
A self-contained C extension (no third-party libraries).
See subdirectory /isolated.
Strategy 1, "system"
Find and use an third-party library already installed on the target system.
See subdirectory /system.
Strategy 2a, "packaged_source"
Package the source code for the third-party library, and compile it into the C extension during installation.
See subdirectory /packaged_source.
Strategy 2b, "packaged_tarball"
Package the tarball distribution for the third-party library, use its configuration toolchain and build a library that the C extension can link against.
See subdirectory /packaged_tarball.
Strategy 3, "precompiled"
Precompile the library and C extension ahead-of-time for a subset of platforms; fall back to
packaged_tarball strategy for other platforms.
See subdirectory /precompiled
These strategies can also be combined!
"system", fallback to "packaged"
"precompiled", fall back to "packaged", leave option for "system"
I'll try to answer questions asked during conference Q&A here.
How do you test cross-compiling gems on Github Actions?
Nokogiri uses the
rake-compiler-dock OCI images directly to build the gem; and then uses the upload-artifact/download-artifact actions to install that gem on the target system.
See https://github.com/sparklemotion/nokogiri/blob/main/.github/workflows/gem-install.yml for the concrete implementation!
What's the significance of the flowers you're using as a background image for your RubyKaigi slides?
I tweeted a brief teaser video in which I explained the joke!
When I edited my RubyKaigi talk about C extensions, some bits had to be cut for time. Here's a fun teaser that I put together from some of those bits. [video link] pic.twitter.com/wZjSepHNpq— mike dalessio (@flavorjones) September 9, 2021
In short, my friend Mark once sent me flowers for my birthday along with a card that read:
Fetching nokogiri-1.8.0.gem (100%) Building native extensions. This could take a while...
Mark is funny. This was the inspiration for the talk's title.