Automation on Forges and CI

GitHub Actions and GitLab CI/CD can automatically perform arbitrary operations, typically when someone pushes to a repository.

These are used for:

• gatekeeping/enforcing coding standard on contributions from others

• parts of the development cycle that do not fit on the machine you use to develop code.

Which typically means:

• Build and publish your work on the web (including documentation)

• Run all unit and integration tests, and static analysis tools

• Measure test coverage

• Check code style and practices (Linting, auto-formatting)

Feedback loops in Software Development

Consider these activities we can perform on a software project:

• type checks

• incremental compilation

• full compilation

• test suite (unit)

• test suite (end-to-end)

• linting

• code format check

• building documentation

• spell checking

For each of these activities:

• How much information can it give us while developing software?

• How “expensive” is it?

• How important is it?

• Should it be done on a software forge, or on the machine where you develop?

• How often should it be performed?

What about test coverage?

Writer’s recommendation (but let us discuss first)

Urgency and Importance of Practices in Software Development

Practice	Information Gain	Time Cost	Importance	Where and When?
type checks	medium	low	low	local, remote (commit hooks?)
incremental compilation	medium	low	low	local
full compilation	medium	high	high	remote (local)
test suite (unit)	medium	high	very high	local,remote
test suite (end-to-end)	high	high	very high	remote (merge?) local (sparsely)
linting	medium	low	medium	local, remote (commit hooks)
building documentation	low	medium	high	remote
spell check	low	low	medium	remote (local)

Typically test coverage measurement and display can be performed remotely. Coverage data is gathered when running a test suite.

Git Hooks

Git has a mechanism to trigger actions when some event happen, called hooks.

Most notable hooks:

• pre-commit: launches a process when you launch the command git commit. If the process terminates with a non-success exit code, it will prevent you from committing your changes. Typical use cases:

  • auto formatting and formatting checks

  • linting

  Note

  There is a Python framework called pre-commit created to manage (e.g., install and run) useful pre-commit hooks.

• post-receive: this hook can be used to launch a process on the machine where the remote repository is hosted, as a fundamental form of CI.

Basic CX with hooks

We can reproduce the fundamental functionality of automation platforms by setting up a bare repository on a remote machine and the relevant hooks.

For this exercise:

1. clone this repository:

git clone https://gitlab.com/michele.mesiti/cx-course.git

2. switch to the hooks branch

3. follow the instructions in the README.

Start with the “local” version of the exercise, then - if you have a remote machine you can connect to via SSH - try the “remote” version.

Note that this simple mechanism is very limited, which is why in most cases automation platforms like GitHub actions or Gitlab CI/CD are used.

“CI” and HPC

Some issues in the development of HPC code are addressed by regularly running jobs that:

• Test compilation with different compilers and libraries (e.g., MPIs, CUDA/ROCm, …)

• Test and benchmark the performance on specific hardware and infrastructure

• build/package software for various compilers, MPI versions, architectures

• … all using standardized environments in conjunction with containers

This might require “special” set up, especially performance benchmarking.