Checking your work: validating the kernel by building and testing in CI

The Linux kernel is one of the most complex pieces of software ever written. Being in ring 0, bugs in the kernel are a big problem, so having confidence in the correctness and robustness of the kernel is incredibly important. This is difficult enough for a single version and configuration of the kernel, but becomes exponentially more difficult as you consider the need to validate the kernel across various configurations, architectures, trees, and branches.

While there are a number of great tools available to write tests for the kernel, and a number of great resources to run those tests and publish the results, our general testing and CI story is inconsistent. Some subsystems run a 100% stable kselftest suite on patchwork, others run regression tests in datacenters maintained by private companies, and yet others run tests on machines that reside in maintainers’ private datacenters (basements?).

The aim of this talk is to paint a relatively complete picture of the kernel testing story as it exits today, to highlight some of the pain-points felt by maintainers and contributors, and to suggest a way forward towards a more comprehensive and consistent testing story for the upstream community.

David Vernet