Building a custom Google Data Studio connector from A-Z. Part 4 — Continuous Integration
Welcome to the last and final part of my tutorial! In the previous one I showed you how to unit test your connector using Jest framework, generate code coverage and check the code quality using eslint. Now you’ll learn how to automate all this stuff using a CI platform. In this article I’ll be using CircleCI, however these steps should be pretty similar for other tools like Semaphore or Travis.
What we’ll do in this article is we’ll:
- Make CircleCI run the unit test suite on every new commit
- Run eslint checks automatically
- Generate the code coverage and send it to codecov.io
- After successful build push the new code to Google Scripts
The build part
The build part is pretty much straightforward. The only tricky part for me was sending the code coverage to CodeCov. Still I’ll describe all the steps, so here we go…
Simple build setup
First let’s register in CircleCI and add the repository as a “project” in the CI tool. Now we’ll need to add .circleci/config.yml file to our repository. We’ll include all the required build/deploy steps there. Our initial file will be responsible only for running the tests. Here’s how it looks like:
Now if I commit the file and push the master branch to GitHub the build will start automatically:
Eslint & code coverage
Now let’s make use of the fact that we’ve setup eslint and that Jest is capable of generating code coverage. We’ll modify the steps to check the code quality and run test suite with the code coverage report. I’ve replaced yarn test command with the following:
- run:
name: "Lint the code"
command: yarn lint- run:
name: "Run tests (with coverage)"
command: yarn coverage
Now whenever any of eslint’s rules are violated the build will fail with appropriate output:
Send the coverage to CodeCov
This one is super optional. I decided that I want to display a nicely looking badge with the code coverage percentage in my repository’s readme. I opted for CodeCov tool that’s free for open source projects. Their badge looks like this:
First I, of course, registered and setup my repository in CodeCov’s interface. Then, basing on this document, hoped that CircleCI will somehow automagically send the coverage to CodeCov. This didn’t really happen :D
The solution wasn’t really that hard — it turned out that there’s an npm package that does the thing I need, so I only had to:
- Add
CODECOV_TOKENto my CircleCI project
2. Add the following step to the .circleci/config.yml file
- run:
name: "Send coverage to CodeCov"
command: yarn add --dev codecov && ./node_modules/.bin/codecovThe deployment part
The deployment part is much more interesting. I couldn’t find any tutorial for this around the Internet, so it looks like I’m a pioneer in this field. If you know a better way to setup automatic GDS connector deployments give me some feedback :)
Prepare the deployment script
My solution utilizes CLASP package for script version management. I already described the idea behind script versions and deployments and how to setup CLASP in part 1 of the series, so if you’re new to these concepts take a break here, read the first part and come back in a few minutes :)
As you may remember I’ve already setup the “Production” deployment back then. We’ll now write a script that’s gonna do the following things:
- Push the code to Google Scripts
- Create a new script version
- Modify the “Production” deployment to point at the newly created version
Here’s how the bash script I invented looks like:
Run the script on CircleCI
The worst part is now done. The only thing that’s left is to make CircleCI run the deployment script on every push to master.
First let’s add a new script to our package.json file: "deploy-production": "./deploy.sh 'Production'".
Then we’ll need to allow CircleCI job modify our Google Script. If you haven’t done it before run clasp login in src folder and login using your Google credentials. Clasp will create a .clasprc.json file containing access tokens to Google services in your home directory.
I haven’t found any better way of doing Google authentication on the CI side than just copying the contents of your local file to the CI application. In case of CircleCI we’ll need to add a new environment variable in our CircleCI project and call it CLASPRC. The value of the variable should simply be the content of your local clasprc.json file.
Last thing is adding the deployment step to our .circleci/config.yml file so that:
- It writes the content of
CLASPRCvariable to~/.clasprc.jsonfile - It runs the
deploy.shscript we wrote before - It runs the
deploystep only after a successfulmasterbranch build
Our final config file looks like this:
Check the results!
Ok, so this is the moment we’ve been waiting for — let’s commit something to master, push it to GitHub and let CircleCI do its magic!
The job completed successfully. Now if we verify the result in Google Scripts interface we’ll see that indeed — a new version was created:
And the “Production” deployment points at the newly created version:
Summary
That’s it! Our connector got even more mature in this part. Now there will be no place for random bugs — every new line of code will be thoroughly examined by the CI tool and will instantly appear for the users thanks to the deployment script we wrote.
You may say that doing all these things I did in parts 3 and 4 is an overkill for such a small application and… you may probably be right. However, when your connector gets more advanced you’ll quickly notice that manual debugging and deployments take a looot of time. I’m pretty sure that after a week of such “manual” development you’ll waste as much time on debugging as I did on writing unit tests and setting up the CI :)
Hope you learned a lot during this series of posts and that it’ll help you develop great products.
As always you can check the full code of this part in my GitHub repository or test the connector using this link.
If you liked the series do not hesitate to 👏!
