Use batch loading & lazy relationships to eliminate N+1 queries in Ruby on Rails

TLDR: Use ams_lazy_relationships gem ( https://github.com/Bajena/ams_lazy_relationships) to eliminate N+1 queries in ActiveModel::Serializers 🎉

Quick intro to batch loading

A few months ago I came across a post by exAspArk explaining how they built and used the BatchLoader gem to optimize the number of database queries in their apps.

The topic of batch loading was new to me back then, but it sounded like a perfect fit for our Rails app in Leadfeeder, because the recommended Rails way of using wasn’t flexible enough for many of our complex endpoints.

Quick explanation for people who just asked “what the hell is batch loading”:
Batch loading is a method for preventing unnecessary DB calls.
If we need to load a collection of records (e.g. for each blog post load its author), then instead of calling the DB to get every author we can collect s, run one DB query and then assign the author to every blog post.
This example should give you a better idea of how it works.

Limitations of BatchLoader gem

After getting more familiar with the topic it turned out that there’s already a lot of work done in the GraphQL world, but unfortunately if you’re still an old-fashioned REST API guy like me you’ll quickly notice that there’s no Ruby gem that’d nicely wrap the batch loading functionality, so I started experimenting with it myself.

First I followed Usama Ashraf’s approach from this post and it worked quite well, however I stumbled upon two problems with this code:

  • It generates a lot of repeatable “boilerplate” code that pollutes the serializers a lot.
  • When requesting deeply nested relationships (e.g. blog_post.comments.author) ActiveModel::Serializers gem will still perform N+1 queries for the author relationship due to the way it serializes the records.

I decided to write my own gem based on BatchLoader to tackle these issues.

Lazy Relationships to the rescue!

The gem is called ams_lazy_relationships. It’s an extension for ActiveModel::Serializers gem based on the above-mentioned BatchLoader gem.

It introduces the concept of lazy relationships. Lazy relationships are methods () that wrap relationship methods provided by ActiveModel::Serializers.

Lazy relationships are cool because:

  • They prevent N+1 queries when serializing complex object tress by using batch loading
  • They do not load excessive data (like Rails when improperly used).
  • They let you remove N+1s even when not all relationships are ActiveRecord models (e.g. some records are stored in a MySQL DB and other models are stored in Cassandra)

Probably this thing isn’t still 100% clear to you, so in the next paragraphs I’ll show you how to install the gem and explain the concept on a few examples.

Installation

The installation process is rather simple:

  1. Include module in your base serializer:

2. As this gem uses heavily I highly recommend clearing the batch loader's cache between HTTP requests. To do so add a following middleware: to your app’s .

For more info about the middleware check out BatchLoader gem docs: exAspArk/batch-loader#caching

Example 1: Basic ActiveRecord relationships

If the relationships in your serializers are plain old ActiveRecord relationships you’re lucky, because ams_lazy_relationships by default assumes that the relationship is an ActiveRecord relationship, so you can use the simplest syntax.

Imagine you have an endpoint that renders a list of blog posts and includes their comments.

The N+1 prone way of defining the serializer would be:

To prevent loading comments using a separate DB query for each post just change it to:

Example 2: Modifying the relationship before rendering

Sometimes it may happen that you need to process the relationship before rendering, e.g. decorate the records. In this case the gem provides a special method (in our case ) for each defined relationship. Check out the example — we’ll decorate every comment before serializing:

Example 3: Introducing loader classes

Under the hood ams_lazy_relationships uses special loader classes to batch load the relationships. By default the gem uses serializer class names and relationship names to instantiate correct loaders, but it may happen that e.g. your serializer’s class name doesn’t match the model name (e.g. your model’s name is but the serializer’s name is ).

In this case you can define the lazy relationship by passing a correct param:

You can find all the available loader classes here.

Example 4: Non ActiveRecord -> ActiveRecord relationships

This one is interesting. It may happen that the root record is not an ActiveRecord model (e.g. a Cequel model), however its relationship is an AR model.

Imagine that is not an AR model and is a standard AR model. The lazy relationship would look like this:

Example 5: Use lazy relationship without rendering it

Sometimes you may just want to make use of lazy relationship without rendering the whole nested record.
For example imagine that your serializer is supposed to render attribute. You can define the lazy relationship and just use it in other attribute evaluator:

Summary

I hope my idea of lazy relationships appealed to you. If you managed to make use of the gem in your project or have any improvement ideas let me know in the comments or create an issue/PR here: https://github.com/Bajena/ams_lazy_relationships

Full stack developer @Leadfeeder. Working on random stuff in my free time.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store