A few months ago I’ve read a book by Pat Shaugnessy called Ruby Under Microscope. It taught me a lot about Ruby’s internals and inspired to dive a bit deeper than normally and try building an extension to the language.
As most of Rubyists know Ruby is a language originally written in C language by Yukihiro Matsumoto. The C implementation is called MRI (Matz’s Ruby Interpreter) and it supports writing plugins that can “talk” to any C program of choice. I’d like to show you how I built my first extension, that it really isn’t that difficult and potentially can bring you huge benefits in terms of performance.
Before we start, keep in mind that apart from MRI there’s a bunch of other Ruby language implementations, e.g. jRuby written in Java, Rubinius written in… Ruby and many others, so the C extension that I’m presenting in the article will most probably not work with other platforms than MRI.
Ladies and gentlemen, I present you the MatrixBoost library 👏
I didn’t want to create YAHW (Yet Another Hello World) extension, so I spent quite some time thinking what exactly should I implement, but finally after some brainstorming I came up with an idea. I’ve noticed that the Matrix gem which is a part of Ruby’s standard library is completely written in Ruby, so I decided to write a gem that’d reimplement some of the common matrix operations in pure C. Such library should come in handy for people whose programs heavily rely on Ruby’s Matrix library and perform lots of matrix operations.
The main idea is that:
- The C extension will convert a Ruby matrix into yasML C matrix
- yasML will perform the desired matrix operation
- C extension will convert the result C matrix back to the format understandable by Ruby
In the next paragraphs you’ll see what were the necessary steps to achieve it.
Step 1: Set up the gem
gemspec file make sure to add
ext folder to the list of required paths, like this:
spec.require_paths = ["lib", "ext"]. It'll be necessary for the gem to load the extension code we're going to add.
Step 2: Prepare
ext directory create a file called
extconf.rb with the following content:
When you execute the
extconf.rb file then the
mkmf library (part of the Ruby extension build system) will prepare a Makefile required to compile the C part of the gem.
extconf.rb in my snippet is the simplest possible config, but of course if needed it can be extended to do things like generating header files, checking other necessary C dependencies or configure C compiler options.
You can check out what
mkmf is capable of here.
Step 3: Write Ruby <-> C glue code
Here’s the most interesting and at the same time the most difficult part. We’ll need to write a C file which’ll serve as an entry point from the Ruby code. It’ll add a Ruby module with functions that’ll convert a Ruby arrays into C arrays, perform desired matrix operations and convert the results back into a Ruby array.
Entry point — Init_extension
In order to initialize the extension Ruby will search for a method called
Init_extension(void) in a file named
ext/matrix_boost/extension.c. This function will define a Ruby module with functions performing matrix operations, like
mul_matrix. These functions will be executed in our C extension.
Here’s how a basic
Init_extension will look like:
This content of
Init_extension is rather straightforward:
- First line defines a
- Second line defines a Ruby class called
MatrixBoost::NativeHelpers. Last argument defines a base class for our class.
rb_cObjectcorresponds to the basic ruby
- Third line defines a Ruby method called
MatrixBoost::NativeHelpersmodule. This function will execute C function called
mul_matrix. The last argument indicates the number of arguments (
- Fourth line defines a Ruby method called
MatrixBoost::NativeHelpersmodule. This function will execute C function called
inv_matrixand has one argument.
Ruby <-> C communication
Let’s now take a look at
inv_matrix function implementation to see how to Ruby objects can "talk" to the C matrix library:
First, it calls
Check_Type function from Ruby C API that'll verify that the
m argument is a Ruby array. If the check fails, a following error will be raised:
Next, it executes a function
rb_array_to_matrix in order to convert the input ruby array to a
Matrix type from
matrices.c library and assign it to
mc variable. I'll show you the definition of
rb_array_to_matrix in a sec.
Matrix struct we can use it to feed the
matrix_invert function from the
matrices.c library. The function will return a new
Matrix struct which we'll assign to
Next thing we need to do is to call
matrix_destroy(mc) in order to free the memory after
mc variable to prevent memory leaks. Remember, it's C, there's no Garbage Collection here ;)
If the inversion operation failed (returned a null result), we’ll return
Qnil which corresponds to Ruby's
If it succeeded, we’ll copy the inverted matrix to a new Ruby array and free the memory after the matrix struct.
Notice that the return type of our function is
VALUE. VALUE type is defined by MRI and is basically a pointer to a Ruby object.
Translating Ruby arrays to C matrices (and opposite)
In previous paragraph we skipped the part about converting the arrays from Ruby to C and opposite. Let’s now take a look at these functions and explain some of more interesting parts.
First, there’s the
rb_array_to_matrix, defined as follows:
In order to initialize the C matrix we need the number of rows and columns from the Ruby array. To get the length of a Ruby array from C level we’ll use a method called
RARRAY_LEN from Ruby's C API.
Next, we’ll iterate over each cell of the array, getting the element values by calling
rb_ary_entry and converting the Ruby
Numbers to C
doubles by calling the
NUM2DBL function. The resulting number will be assign to a proper cell in the C matrix.
matrix_to_rb_array we're doing pretty much the opposite:
First, we initialize a new Ruby array by calling
rb_ary_new(). We'll then fill it by copying the values from the C matrix.
for loop we're getting the value of each matrix's cell, converting it from
double to Ruby
Number by calling
DBL2NUM (opposite to what we called in
rb_array_to_matrix and appending the value to the current row by calling
Step 4: Compile the extension
The extension’s code is ready, but before we can use it in a Ruby program we’ll have to compile the C code.
In order to do it we’ll add a following rake task to our gem’s
And run it by calling
Step 5: Test it 🧪
Now, if we call
require "matrix_boost/extension" in our gem we'll be able to use the C extension from the Ruby code.
We’ll define our
lib/matrix_boost.rb as follows:
Now in terminal run the console by calling
bin/console and compare the results of original Ruby method with
Even though the original method returned the values as
Rationals and our gem returned
Floats the results are equal 🎉
Step 6: Benchmark it 🤓
Great, our extension is working as expected! Now let’s benchmark it to see how much we really gained by performing the matrix operations in C.
Let’s add a benchmark rake task that’ll compute 1000000 inversions of a random 4-dimension matrix (most common size in 3d graphics).
It turns out that C implementation is ~5.2x faster! It should make a difference for programs heavily relying on matrix operations, like ray tracers (who writes ray tracers in Ruby is another question 🙈).
Even though C language might be scary for some people compared to Ruby and though Ruby’s C API is a bit messy and not very well documented I think it’s worth giving it a shot and trying to write a simple C. Especially when a part of your program needs a performance Boost or if you want to integrate a C library into your codebase.
Full code for
matrix_boost library can be found here.
Knowledge about C extensions in Ruby is a bit scattered around the Internet, so I collected some links worth checking if you’re struggling with writing your own extension: