Background

If you buy into the concept that Unit Tests and Continuous Integration is a good thing then you will eventually bump into Travis-CI (https://travis-ci.org/).  The concept is that Travis-CI will take your github project (in my case https://github.com/Zomojo/compiletools) and run the unit tests.  All you need to do is add a small .travis.yml file that contains details like the language and commands to run your tests then the rest is supposed to be automagic.

Reality vs Marketing

With hindsight, I bet that for many projects Travis-CI works like a charm.  However I immediately failed. Now that I’ve finally cracked it, I can say that one of the reason I initially failed was due to the python requirements of my project and the other was a difference in hyphen handling by argparse.

My project was developed on Fedora 24 and Arch Linux, however the container that Travis-CI provides is Ubuntu based. As a specific example of dependency issues, I depend on configargparse. On F24 (Oct 2016), installing python-configargparse gives you version 0.10.  Ubuntu doesn’t provide configargparse in a package so you get  configargparse from PyPi (via pip install configargparse) which is now at version 0.11.  It turns out that there is a change in API between those two versions that broke my project.  That wasn’t the only difference between the two environments but it is a good example.

Experienced pythonistas will simply point out that I could have pinned the version of configargparse in my requirements.txt. And while that is true, it’s 1) not obvious to somebody with only hundreds of hours of python experience, and 2) wasn’t my only problem.

At this point a good bit of advice to my future self is “Use Travis-CI from the beginning”. If I had have done that my project would not have evolved into a place where it took almost two days to get running on Travis-CI.

Travis-CI tips

My (finally working) .travis.yml file is

language: python
python:
    - "2.7" 
    - "3.4"
    - "3.5"
addons:
    apt:
        sources:
            - ubuntu-toolchain-r-test
        packages:
            - g++-5
            - gcc-5

# command to install dependencies 
install:
    - pip install -r requirements.txt 
    - export CXX="g++-5" CC="gcc-5" 

# command to run tests
script:
    - nosetests

The language and versions of python are fairly obvious.  The addons section is only necessary because my project (python-compiletools) is about compiling C++ code so I need a C++ compiler available to run the tests.

The install section took a lot to learn.  There seems to be many ways of doing this section and one of my own hangups was that I didn’t want to write a requirements.txt that got out of sync with my setup.py.  I eventually learnt that if you have a requirements.txt that contains simply

-e .

then the requirements are extracted from setup.py.

The export line in the install section is a project specific requirement. That is, I need a compiler that can compile C++11 in order to complete my specific unittests.

Initial Giveup

It’s worth pointing out that at the time I gave up on using Travis-CI my project was (locally) passing unit tests and in active use on Fedora 22, 24, Arch Linux and Centos 7. So in one sense the code was “good”. Looking over my git logs I can see that I had 9 commits trying to make it work on Travis-CI before I threw in the towel. The process of reading Travis errors, making a commit that I think will work, waiting for Travis, reading more errors, was too slow and the big reason that I gave up.

Ubuntu

For unrelated reasons I set up an Ubuntu virtual machine on F24 using virt-manager. Having my hands on this environment then allowed me to fix a lot of the errors that Travis-CI was reporting. One of the stranger errors that occurred seems to be a difference in the algorithm that argparse uses to tell if an option starting with a hyphen is actually an option in its own right or if it was meant to be data for a prior option.  For example,

myexe --foo -bar

Is that meant to be a variable called foo is set to “-bar”  or is -bar another option?   A realworld example that I was facing is passing around options for g++

ct-cake --CXXFLAGS=-std=c++11

so the -std is not an option in its own right but a setting for my CXXFLAGS variable. That worked fine on F24 but failed on Ubuntu.

I eventually got around this particular problem by implementing a quote stripping phase in my program.  I now use a command line like

ct-cake --CXXFLAGS="-std=c++11"

Then my python code looks like

cap = configargparse.getArgumentParser()
args = cap.parse_args()
strip_quotes(args)

And the strip_quotes function removes the extraneous double quotes on any of the arguments. See around line 332 in https://github.com/Zomojo/compiletools/blob/master/ct/apptools.py if you are interested in how I implement the quote stripping.

As mentioned earlier there was also the difference in versions of python modules which I took care of by a try statement around the import.

At this point all my unit tests worked on Ubuntu so I thought I’d give Travis-CI another crack.  Unexpectedly, I still failed!

Docker

A bit more learning led me to the fact that the Travis-CI team make available a docker image that you can use to test on.  This answers my previous objection that I was taking too long in the guess solution, commit, push, wait, blowup, cycle.

I installed docker using

sudo dnf install docker-client docker --allowerasing 

and started docker with

sudo systemctl start docker

and proved that I’d got this far correctly via

sudo docker run hello-world

The docker image from Travis-CI came by running

docker run -it -u travis quay.io/travisci/travis-python /bin/bash

switched to the travis user (tip: user name is also the password)

su - travis

checked out my project

mkdir Zomojo
cd Zomojo/
git clone https://github.com/Zomojo/compiletools.git
cd compiletools/

I can’t remember why or from where I got this line of magic

travis compile

Travis-CI does its unittesting in a python virtualenv so we need to activate that

source ~/virtualenv/python2.7/bin/activate

Remember from my earlier .travis.yml I needed some specific environment variables so time to set those

export CXX="g++-5" CC="gcc-5"

And finally we can test and fix quickly

nosetests

Conclusion

Despite the rather large learning hurdles that needed to be vaulted, overall I’d recommend using Travis-CI.  In particular I recommend setting Travis-CI up very early on in your project so that you never reach the point of a wide divergence between tests that pass on your local environment and tests that pass on Travis-CI. Your code will be more robust by having the different environment running it and you are more likely to encounter and fix problems before your users run into them.