CI/CD Pipelines

Hearthforge runs pipelines in Docker or Podman containers. A pipeline is described by a .hearthforge-ci.toml file at the root of the repository. The Pipelines tab of a repository shows the run history, a short reference, and a link to the full template.

How a run works

A push, a tag, or the Run pipeline button creates a run. The config is read from the pushed commit, so every branch can carry its own pipeline.
Hearthforge pulls image, creates the cache volumes, and starts one container named hearthforge-ci-<run id> with work_dir as its working directory. Files listed under [[copy]] are taken from their source images first.
The commit is exported with git archive on the host and uploaded into clone_project_to. The image needs no git. No .git directory reaches the container.
Steps run in file order, in that one container, so files and installed packages persist from step to step. Each step is one docker exec of shell with shell_setup prepended.
Published files are copied out of the container into $DATA_DIR/ci/artifacts/<run id>/. The container is removed. Cache volumes stay.

Everything up to the first step is shown as the step pipeline setup.

Setup

Hearthforge talks to the container engine over its Unix socket. It checks, in order:

CI_DOCKER_SOCKET, when set
/var/run/docker.sock
/run/podman/podman.sock
/run/user/<uid>/podman/podman.sock

Without a socket the Pipelines tab still works, and every run ends in one skipped pipeline setup step that names the problem.

Host install

Docker needs no extra setup. Rootless Podman needs the API socket enabled:

systemctl --user enable --now podman.socket

Hearthforge in a container

The socket must be mounted into the Hearthforge container. compose.yml carries commented-out bindings for both engines. Mount the socket at /var/run/docker.sock and the auto-detection finds it, or mount it elsewhere and set CI_DOCKER_SOCKET.

Mounting the socket gives Hearthforge full control over the container engine. That is equivalent to root on the host for Docker, and to the socket owner's account for rootless Podman.

Cache volumes, image pulls, and the CI containers live on the engine side. Nothing about them refers to paths inside the Hearthforge container, so a containerised Hearthforge works the same as a host install.

Server settings

Variable	Default	Description
`CI_DOCKER_SOCKET`	(auto-detected)	Path to the Docker/Podman socket
`CI_MAX_CONCURRENT`	`2`	Runs executed at once. Further runs wait in a queue.
`CI_MAX_HISTORY`	`50`	Runs kept per repository, artifacts included
`CI_DEFAULT_TIMEOUT`	`3600`	Step timeout in seconds when the config sets none

Example

image = "docker.io/debian:stable"
work_dir = "/ci/build"
clone_project_to = "/ci/build/project"   # must be absolute
shell_setup = "set -euo pipefail"
timeout = 3600                           # per-step default, in seconds
memory_limit = "2g"

cache = [
  { path = "/root/.cache/pip", max_size = "1g" },
]

[on]
push = ["main", "release/*"]   # branch globs, or ["*"]
tag = true

[variables]
  [variables.PYTHON]
  default = "3.12"
  description = "Interpreter version, overridable from the Run pipeline form"

[[steps]]
name = "setup"
run_sh = "apt-get update -qq && apt-get install -y --no-install-recommends python3 python3-pip"
timeout = 300

[[steps]]
name = "test"
run_sh = "cd project && python3 -m pytest"

[[steps]]
name = "package"
run_if = 'test -n "${CI_COMMIT_TAG}"'
run_sh = "cd project && python3 -m build"
publish_gzip = ["/ci/build/project/dist/"]

[[steps]]
name = "report"
always = true
run_sh = "cat /ci/build/project/.pytest_cache/v/cache/lastfailed || true"

The template in the Pipelines tab lists every key with a comment.

Step options

Key	Effect
`run_sh`	The command. Runs through `shell`, after `shell_setup`.
`timeout`	Seconds. Falls back to the top-level `timeout`, then `CI_DEFAULT_TIMEOUT`.
`run_if`	Shell expression. A non-zero exit skips the step.
`always`	Run even after an earlier step failed.
`warn_on_fail`	A failure marks the step `warning` and the run continues.
`clear`	Delete `clone_project_to` and extract a fresh checkout before the step.
`publish_*`	`publish_file`, `publish_tar`, `publish_gzip`, `publish_zstd`, `publish_zip`

Environment

Every step sees CI=true plus:

Variable	Value
`CI_PIPELINE_ID`	numeric run ID
`CI_REPO_NAME`	repository name
`CI_SERVER_URL`	Hearthforge `BASE_URL`
`CI_TRIGGER_SOURCE`	`push`, `tag`, or `manual`
`CI_COMMIT_SHA`	full commit hash
`CI_COMMIT_SHORT_SHA`	first 8 characters
`CI_COMMIT_BRANCH`	branch name, empty for tags
`CI_COMMIT_TAG`	tag name, empty for branches
`CI_COMMIT_REF_NAME`	branch or tag name

[variables] defaults come next, then overrides from the Run pipeline form, then secrets. A later source wins on a name clash.

Secrets

Admins add secrets under Repository settings → CI Secrets. They are injected as environment variables. Their values are replaced with [MASKED] in step logs. The masking is a plain text match: a secret printed in encoded or split form is not caught.

Status badge

![CI](https://your-forge.example.com/REPO_NAME/ci/badge.svg)

Caches

Each cache path becomes a named volume, hearthforge-ci-cache-<hash>, labelled com.hearthforge.repo=<repository>. The volume is mounted at the same path in every run of that repository.

A cache without max_size grows without bound.
A cache with max_size is deleted after the run that takes it over the limit. The run detail page shows a cache step naming the dropped volume. The next run starts cold. Size the cap at about twice the working set.
A volume the current config no longer lists is deleted after a run on the default branch. Other branches keep it.
Purge caches on the Pipelines tab deletes all volumes of the repository. Deleting or renaming a repository does the same.
Everything else needs the engine's own tools:

docker volume ls --filter label=com.hearthforge.repo=REPO_NAME
docker volume rm $(docker volume ls -q --filter label=com.hearthforge.repo)   # all Hearthforge caches

The size check reads the daemon's reported usage. A daemon that does not report volume sizes never triggers the cap.

Gotchas

clone_project_to must be absolute, and no cache path may lie inside it or above it. The checkout is extracted over that directory and a clear step deletes it. The config is rejected otherwise.
The image needs no git, and the checkout has no .git. git describe or a version from the log needs a step that installs git and fetches the history itself, for example over the forge's HTTP clone URL. Pushing back from a pipeline needs the admin's credentials, kept as a secret.
The image is pulled on every run. Even with a local copy, the daemon contacts the registry. There is no registry authentication, so private images must already be present and pullable without login. Pin a tag; a moving tag changes the toolchain under the pipeline.
Match the libc for [[copy]]. A binary from an -alpine image is musl and fails on a glibc image with an error that looks like a missing file.
publish_* uses tools inside the image. publish_gzip, publish_tar, and publish_zstd need tar (plus zstd). publish_zip needs zip. publish_file needs nothing.
Steps share one container. A file left behind by one step is visible to the next. Use clear on a step that must start from a clean tree. Do not rm -rf work_dir itself: the container's working directory is gone and every later step fails to start.
Logs are cut at 2 MB per step. Redirect verbose output to a file and publish the file instead.
A Hearthforge restart cancels running runs. Their containers are force-removed on startup. The runs show as cancelled and can be retried.
The container is on the engine's default network. It reaches the internet and any other container on that network. Set cpu_limit and memory_limit; there are no defaults.
Old runs are deleted. Only the newest CI_MAX_HISTORY runs per repository are kept, and their artifacts go with them. Attach anything permanent to a release.