Continuous Integration

We use travis-ci and codecov for continuous integration and tox to automate the process at the repository level.

When a commit is pushed in a pull request, a call to $ tox is made by TravisCI which triggers the following chain of events:

  1. A test environment is spun up for each version of python tested (defined in tox.ini).
  2. Code styles verifications, and quality checks are run (flake8, pylint, doc8). The documentation is also built at this time (docs).
  3. The test suite is run completely with coverage, including the dedicated backward compatibility tests.
  4. The structure of the repository is validated by check-manifest and readme_renderer.
  5. The results of the coverage check are reported directly in the pull request.

The coverage results show the difference of coverage introduced by the changes. We always aim to have changes that improve coverage.

If a step fails at any point in any environment, the build will be immediately stopped, marked as failed and reported to the pull request and repository. In such case, the maintainers and relevant contributors will be alerted.