Building a custom Google Data Studio connector from A-Z. Part 4 — Continuous Integration

Welcome to the last and final part of my tutorial! In the previous one I showed you how to unit test your connector using Jest framework, generate code coverage and check the code quality using eslint. Now you’ll learn how to automate all this stuff using a CI platform. In this article I’ll be using CircleCI, however these steps should be pretty similar for other tools like Semaphore or Travis.

What we’ll do in this article is we’ll:

  • Make CircleCI run the unit test suite on every new commit
  • Run eslint checks automatically
  • Generate the code coverage and send it to codecov.io
  • After successful build push the new code to Google Scripts

The build part

The build part is pretty much straightforward. The only tricky part for me was sending the code coverage to CodeCov. Still I’ll describe all the steps, so here we go…

First let’s register in CircleCI and add the repository as a “project” in the CI tool. Now we’ll need to add .circleci/config.yml file to our repository. We’ll include all the required build/deploy steps there. Our initial file will be responsible only for running the tests. Here’s how it looks like:

Now if I commit the file and push the master branch to GitHub the build will start automatically:

First successful build!

Now let’s make use of the fact that we’ve setup eslint and that Jest is capable of generating code coverage. We’ll modify the steps to check the code quality and run test suite with the code coverage report. I’ve replaced yarn test command with the following:

- run:
name: "Lint the code"
command: yarn lint
- run:
name: "Run tests (with coverage)"
command: yarn coverage

Now whenever any of eslint’s rules are violated the build will fail with appropriate output:

Oops, I’ve violated the rules :O

This one is super optional. I decided that I want to display a nicely looking badge with the code coverage percentage in my repository’s readme. I opted for CodeCov tool that’s free for open source projects. Their badge looks like this:

CodeCov badge

First I, of course, registered and setup my repository in CodeCov’s interface. Then, basing on this document, hoped that CircleCI will somehow automagically send the coverage to CodeCov. This didn’t really happen :D

The solution wasn’t really that hard — it turned out that there’s an npm package that does the thing I need, so I only had to:

  1. Add CODECOV_TOKEN to my CircleCI project

2. Add the following step to the .circleci/config.yml file

- run:
name: "Send coverage to CodeCov"
command: yarn add --dev codecov && ./node_modules/.bin/codecov

The deployment part

The deployment part is much more interesting. I couldn’t find any tutorial for this around the Internet, so it looks like I’m a pioneer in this field. If you know a better way to setup automatic GDS connector deployments give me some feedback :)

My solution utilizes CLASP package for script version management. I already described the idea behind script versions and deployments and how to setup CLASP in part 1 of the series, so if you’re new to these concepts take a break here, read the first part and come back in a few minutes :)

As you may remember I’ve already setup the “Production” deployment back then. We’ll now write a script that’s gonna do the following things:

  1. Push the code to Google Scripts
  2. Create a new script version
  3. Modify the “Production” deployment to point at the newly created version

Here’s how the bash script I invented looks like:

The deployment bash script

The worst part is now done. The only thing that’s left is to make CircleCI run the deployment script on every push to master.

First let’s add a new script to our package.json file: "deploy-production": "./deploy.sh 'Production'".

Then we’ll need to allow CircleCI job modify our Google Script. If you haven’t done it before run clasp login in src folder and login using your Google credentials. Clasp will create a .clasprc.json file containing access tokens to Google services in your home directory.

I haven’t found any better way of doing Google authentication on the CI side than just copying the contents of your local file to the CI application. In case of CircleCI we’ll need to add a new environment variable in our CircleCI project and call it CLASPRC. The value of the variable should simply be the content of your local clasprc.json file.

Last thing is adding the deployment step to our .circleci/config.yml file so that:

  • It writes the content of CLASPRC variable to ~/.clasprc.json file
  • It runs the deploy.sh script we wrote before
  • It runs the deploy step only after a successful master branch build

Our final config file looks like this:

Ok, so this is the moment we’ve been waiting for — let’s commit something to master, push it to GitHub and let CircleCI do its magic!

Deploy in progress

The job completed successfully. Now if we verify the result in Google Scripts interface we’ll see that indeed — a new version was created:

New version is here

And the “Production” deployment points at the newly created version:

Updated “Production” deployment

Summary

That’s it! Our connector got even more mature in this part. Now there will be no place for random bugs — every new line of code will be thoroughly examined by the CI tool and will instantly appear for the users thanks to the deployment script we wrote.

You may say that doing all these things I did in parts 3 and 4 is an overkill for such a small application and… you may probably be right. However, when your connector gets more advanced you’ll quickly notice that manual debugging and deployments take a looot of time. I’m pretty sure that after a week of such “manual” development you’ll waste as much time on debugging as I did on writing unit tests and setting up the CI :)

Hope you learned a lot during this series of posts and that it’ll help you develop great products.

As always you can check the full code of this part in my GitHub repository or test the connector using this link.

If you liked the series do not hesitate to 👏!

Full stack developer @Leadfeeder. Working on random stuff in my free time.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store