Apache Cassandra | Apache Cassandra Documentation

Continuous Integration

The project has three separate CI surfaces. Knowing which one applies saves a lot of time.

System What it runs When it runs

System	What it runs	When it runs
GitHub Actions	`ant check`: licence (rat) and checkstyle checks. Changes under `.build/run-ci*` or `.jenkins/` have their own checks.	Automatically, on every push to any branch of any fork
Pre-commit CI: pre-ci.cassandra.apache.org, or any clone of it	The test pipeline against your branch, at the profile you choose	On demand, via UI, or with `.build/run-ci`
Post-commit CI: ci-cassandra.apache.org	The full `post-commit` pipeline against the official branches	Automatically, after every push on an `apache/cassandra` branch

GitHub Actions

ant check: licence (rat) and checkstyle checks.
Changes under .build/run-ci* or .jenkins/ have their own checks.

Automatically, on every push to any branch of any fork

Pre-commit CI:
pre-ci.cassandra.apache.org,
or any clone of it

The test pipeline against your branch, at the profile you choose

On demand, via UI, or with .build/run-ci

Post-commit CI:
ci-cassandra.apache.org

The full post-commit pipeline against the official branches

Automatically, after every push on an apache/cassandra branch

Testing a patch before it is committed is the author’s and the reviewer’s job, and it is done with pre-commit CI.

Pre-commit CI

Pre-commit CI is a clone of ci-cassandra.apache.org that runs in any Kubernetes cluster. The Jenkinsfile, the docker images, and the test scripts are the same ones used post-commit and locally, so a failure seen in one place is reproducible in the others.

You have three ways to get at one:

pre-ci.cassandra.apache.org, hosted by the ASF and paid for by a donation of AWS credits from Amazon. It is intended for committers and trusted contributors who do not have access to a company-private CI clone. Ask on the dev mailing list, or contact a PMC member, for an account.
Your own cluster, provisioned with .build/run-ci --only-setup. See Provisioning your own instance.
A clone operated by your employer. Several companies run one; the command line and the results are identical.

Contributors without an account should say so on the ticket. The reviewer or committer then runs CI on the patch’s behalf.

Using `run/ci` – One-time setup

The .build/run-ci script needs its Python dependencies:

pip install -r .build/run-ci.d/requirements.txt

helm and kubectl are needed only when provisioning a cluster, not when submitting builds to an existing Jenkins.

The Jenkins URL and user can be kept in .build/.run-ci.env (or in the JENKINS_URL and JENKINS_USER environment variables) instead of being passed on every invocation. The script prompts for the password.

Using `run/ci` – Running a build

The script takes the branch of a pushed fork, not your working tree; commit and push first. It defaults to the current branch and its tracking remote.

# run the current branch, on the cluster in your current kubeconfig context
.build/run-ci

# run the current branch on a Jenkins you have no kubectl access to
.build/run-ci --url pre-ci.cassandra.apache.org --user myuser

# a full pre-commit run of someone else's branch
.build/run-ci -r https://github.com/jrwest/cassandra.git -b jwest/15452-5.0 -p pre-commit

# only the jvm-dtest and stress stages, on JDK 11
.build/run-ci -p custom -e 'stress.*|jvm-dtest.*' -j 11

.build/run-ci --help, and .build/run-ci.d/README.md, list every option.

Profiles

A profile is a named set of pipeline stages. The definitive list is pipelineProfiles() in .jenkins/Jenkinsfile.

Profile Purpose

Profile	Purpose
`skinny`	The default. Lint, unit tests, cqlsh tests, jvm-dtests and dtests: enough to catch the obvious, and quick.
`packaging`	Artifacts, deb and rpm packages, lint. Used when only the build or packaging changed.
`pre-commit`	What a patch is expected to pass before it is committed.
`pre-commit w/ upgrades`	As above, plus the upgrade tests. Required whenever the patch could affect upgrades, serialization, or on-disk formats.
`post-commit`	Everything: 140–200k tests, including the large and novnode dtests. This is what runs on ci-cassandra.apache.org.
`custom`	The stages matching the regexp given with `-e`. Useful for iterating on one failure.

skinny

The default. Lint, unit tests, cqlsh tests, jvm-dtests and dtests: enough to catch the obvious, and quick.

packaging

Artifacts, deb and rpm packages, lint. Used when only the build or packaging changed.

pre-commit

What a patch is expected to pass before it is committed.

pre-commit w/ upgrades

As above, plus the upgrade tests. Required whenever the patch could affect upgrades, serialization, or on-disk formats.

post-commit

Everything: 140–200k tests, including the large and novnode dtests. This is what runs on ci-cassandra.apache.org.

custom

The stages matching the regexp given with -e. Useful for iterating on one failure.

Wall-clock time is a function of how far the cluster can scale out, not of the profile alone; a post-commit run finishes within a few hours on a cluster that can reach a few hundred nodes.

Results

On completion the script writes two files under build/ci/:

ci_summary_<fork>_<branch>_<build>.html, an aggregated summary of the run
results_details_<fork>_<branch>_<build>.tar.xz, the individual test results

Attach both to the Jira ticket. They are the record that the patch was tested, they outlive any CI instance, and reviewers can read them instead of having to browse your Jenkins instance (which might even be private). Use .build/run-ci -o <build number> to fetch the artefacts of an earlier build.

Provisioning your own instance

.build/run-ci --only-setup installs the Jenkins helm chart into the current Kubernetes context. With this approach you never have to open the Jenkins UI, and the cluster scales back to a single controller node when idle, so the cost tracks what you use.

The one prerequisite is a cluster whose node groups match the agent sizes the pipeline asks for; the commands for GKE and EKS are in .jenkins/k8s/README.md.

Tear it down with .build/run-ci --only-tear-down.

Reproducing a failure locally

Because CI runs the in-tree scripts and images, any stage can be run on a laptop:

.build/docker/run-tests.sh -a test -c 1/64 -j 17
.build/docker/run-tests.sh -a jvm-dtest -t BooleanTest

See .build/README.md and testing.

Post-commit CI

ci-cassandra.apache.org runs the post-commit profile against trunk and each supported release branch, on hardware donated by NetApp, DataStax, Huawei and iland. Its results are archived at nightlies.apache.org/cassandra, and Butler shows them over time, which is how you tell a new regression from a long-standing flaky test.

When a post-commit failure is not obviously your patch, check Butler and the issue tracker before assuming it is; when it is, a Jira ticket for the failure is expected.