Continuous Integration (CircleCI)
What is continuous integration?
Continuous integration (CI) is a software development strategy that increases the speed of development while ensuring the quality of the code that teams deploy. Developers continually commit code in small increments (at least daily, or even several times a day), which is then automatically built and tested before it is merged with the shared repository.
Source: CircleCI documentation
Configuration
Our CircleCI configuration resides in https://github.com/digitalfabrik/lunes-cms/blob/develop/.circleci/config.yml. It contains all the jobs listed below. See Configuring CircleCI for a full reference.
Workflow develop
This workflow gets triggered everytime a commit is pushed to the develop
branch or a PR is opened.
install
This job executes pip install -e .[dev,doc,test]
and makes use of the CircleCI Dependency Cache.
It passes the virtual environment .venv
to the subsequent jobs.
setup-test-reporter
This job sets up the test coverage reporter for CodeClimate.
test
This job runs the tests. It sets up a temporary postgres database and runs the migrations
before testing. It runs pytest and passes the coverage in the test-results
directory to the build artifacts.
upload-test-coverage
This job joins all separate coverage data files generated from the previous steps and uploads it to CodeClimate.
compile-translations
This job compiles the translation file and passes the resulting django.mo
to the testing and packaging job.
check-translations
This job uses the dev-tool https://github.com/digitalfabrik/lunes-cms/blob/develop/tools/check_translations.sh to check whether the translation file is up to date and does not contain any empty or fuzzy entries.
build-dev-package
This job creates a python package and passes the resulting files in dist
to the publish-dev-package job.
See Packaging for more information.
publish-dev-package
This job publishes the built package to TestPyPI via Twine.
This is only executed on develop (usually after PRs have been merged) or on branches that end with -publish-dev-package
.
build-documentation
This job checks whether the documentation can be generated without any errors by running https://github.com/digitalfabrik/lunes-cms/blob/develop/tools/build_documentation.sh.
black
This job executes black --check .
, which checks whether the code matches the Black code style.
pylint
This job executes pylint_runner
, which checks whether the Linting throws any errors or warnings.
shellcheck/check
This job makes use of the ShellCheck CircleCI Orb and
executes the pre-defined job shellcheck/check
. It is configured to check the directory https://github.com/digitalfabrik/lunes-cms/blob/develop/tools
and to allow external sources because all dev tools source one common function script.
Workflow main
This workflow gets executed when a commit is pushed to the main
branch. Typically, this is a release PR from develop
.
install-main
See install.
bump-version
This job authenticates as the deliverino app and runs bumpver update
to bump the version and commit the
changes to the main branch. Additionally, it merges the version bump commit into the develop
branch.
Workflow deploy
This workflow gets executed when a commit is tagged.
install-deploy
See install.
compile-translations-deploy
See compile-translations.
publish-package
See publish-dev-package. The only difference is that PyPI is used as repository instead of TestPyPI.
create-release
This job authenticates as Deliverino app and creates a GitHub release with https://github.com/digitalfabrik/lunes-cms/blob/develop/.circleci/scripts/create_release.py.
notify-mattermost
This job sends a release notification to Mattermost into the releases
channel. It needs the Mattermost
webhook which is injected via the mattermost
context.
Debugging with SSH
If you encounter any build failures which you cannot reproduce on your local machine, you can SSH into the build server and examine the problem. See Debugging with SSH for more information.