Sha (Joe) Zhu | Some of my favourite CI tools and why

Do you use GitHub collaboratively with others?
Are you afraid of break others code or you are more afraid of others to break your code.
Do you develop software for others to use?
does your code have a lot of dependencies? And you afraid of some development will break your code?

If any of the above is true, I hope this blog will be useful for you.

To over-simplify things, a continuous integration workflow aims to

test your code in an independent/freshly built virtual environment;
check if your code runs, so it will likely to work on others new installs.
seperate the testing process from DevOps. Continuously check your code by machines, without too much disruption for your current work momentum.

A CI tool is simply connected with your GitHub repo with a webhook. Whenever it detects a new commit, the CI tool will take the configuration (yaml) file, and follow your instruction to build a virtual machine, install, execute and test your code and making sure everything works as you expected at least.

If you are still not convinced, check out this blog that Gitlab tells you why you should use continuous integration.

The following is a list of CI tools that I use on daily basis, hope you find them useful.

Travis CI

TravisCI is one of the oldest and probably the most widely used CI. Let me take the example at DEploid to explain what the configuration file does.

The software package DEploid is written in c++. The software is designed for deconvoluting mixed genomes with unknown proportions. You can find more information here. For the TravisCI configuration, the following defines the language, operating system, and dependencies that you will need to install on the OS: homebrew for MacOS, and apt for Linux.

language: cpp
os:
  - osx
  - linux

addons:
  homebrew:
    update: true
    brewfile: true
  apt:
    packages:
      - libcppunit-dev
      - valgrind
      - r-base-core
      - lcov
      - python-pip
      - doxygen
      - graphviz

The following defines that we test to use both gcc and clang compiler to compile DEploid. You don’t have to do that. For most programs, people just use one compiler. The before install section defines a few extra things that I check with the Linux system, such as coding style and test coverage.

compiler:
  - gcc
  - clang
before_install:
  - echo $LANG
  - echo $LC_ALL
  - if [ $TRAVIS_OS_NAME == linux ]; then pip install --user cpp-coveralls cpplint; fi
  - if [ $TRAVIS_OS_NAME == linux ]; then apt-cache policy zlib*; fi
  - if [ $TRAVIS_OS_NAME == linux ]; then .ci/style.sh; fi

The following part defines how to install DEploid, and many checks including: checking input/output, results backwards compatible, direct download from GitHub and install. After successful compile and install, it sends the coverage report to coverall.

before_script:
  - ./bootstrap

script:
  - make
  - make check
  - if [ $TRAVIS_OS_NAME == linux ]; then ./tests/test_binary.sh; fi
  - ./tests/testPOS.sh
  - ./tests/test_binaryVcfVsTxt.sh
  - ./tests/test-against-previous-version.sh
  - ./tests/test_binaryReproducible.sh
  - if [ $TRAVIS_OS_NAME == linux ]; then cd docs/doxygen; doxygen Doxyfile; cd ../..; fi
  - rm -r *
  - wget --no-check-certificate https://github.com/DEploid-dev/DEploid/archive/master.tar.gz -o /dev/null
  - tar -xf master.tar.gz
  - cd DEploid-master
  - ./bootstrap
  - make; make check
  - cd ..; rm -r *
  - wget --no-check-certificate https://github.com/DEploid-dev/DEploid/archive/master.tar.gz -o /dev/null
  - tar -xf master.tar.gz
  - mv DEploid-master/* .
  - ./bootstrap
  - make; make check
after_success:
  - coveralls --exclude lib --exclude tests --exclude src/random --exclude src/codeCogs/ --exclude src/export/ --exclude src/gzstream/ --gcov-options '\-lp'

If you build R packages, TravisCI is a very good choice. It allows to use three different testing R versions. From basic to development version, which is pretty handy. For more information, you can find at its documentation

Circleci

The reason I started using of circleci was because I was looking for things can support for private repo builds for free (this is way before Github CI become a thing). Back in the days, circleci was very similar to TravisCI, the configuration file does not differ much. A few years ago, the launch of Circleci-II was a game changer.

Circleci-II has many nice feature, it supports more complex compiling matrix, i.e. different versions of gcc compiler, different versions of Ubuntu. The nice thing of Circleci-II is that it integrates with Docker very well. So the process can be easily cashed or stacked up, which makes things much much faster.

AppVeyor

Support windows is good enough reason to be listed here. Probably the first CI tool to support Windows OS. It takes me the longest to figure out how to fix things. Not an AppVeyor fault, probablyjust Windows :p

Github CI

Despite the facts that GitHub is the most favoured platform for coders and everything else is hooked with Github, Github only announced its CICD pipeline (beta) until 2019. However, a late entrance does not make the Github CICD framework any less likeable. My guess Github has taken its time for everyone to settle. It has really learned what developers would want and how to best achieve the goal. Things are just much easier to use now.

Fist of all, it supports public and private, you won’t have to worry about the cost of hosting many repos. For someone is cheap and have dozens of repos like me, this can’t be any better.

Second of all, it’s really fast! TBH, I have not benchmarked this with a stopwatch. But from experiences, actions are almost triggered the second that commits are made. Unlike other CI platforms, depends on the number of online users, there will be a delay, unless you pay for the service.

It is really easy to use the CICD framework with secrets, repos can interact with each other. It makes automated coding much easier.

Summary

For the future, I think I will continue to use all the listed CI tools, but leaning towards github CI more and more. Just because it’s

Fast and easy to use
Support private repo
Easily support complex workflow (CICD)
Apparently it supports any OS, I mean *nix is probably easy. But hey, if I can use the same script for Windows (now MS owns github, this is by default?). Why not! Don’t know if it’s possible yet, but it should.

For more information, you can find more stuff from this blog written by google, where they have compared more tools.