# CI/CD Pipelines

Hearthforge runs pipelines in Docker or Podman containers. A pipeline is
described by a `.hearthforge-ci.toml` file at the root of the repository.
The **Pipelines** tab of a repository shows the run history, a short
reference, and a link to the full [template](public/assets/hearthforge-ci-template.toml).

## How a run works

1. A push, a tag, or the **Run pipeline** button creates a run. The config is
   read from the pushed commit, so every branch can carry its own pipeline.
2. Hearthforge pulls `image`, creates the cache volumes, and starts one
   container named `hearthforge-ci-<run id>` with `work_dir` as its working
   directory. Files listed under `[[copy]]` are taken from their source images
   first.
3. The commit is exported with `git archive` on the host and uploaded into
   `clone_project_to`. The image needs no git. No `.git` directory reaches the
   container.
4. Steps run in file order, in that one container, so files and installed
   packages persist from step to step. Each step is one `docker exec` of
   `shell` with `shell_setup` prepended.
5. Published files are copied out of the container into
   `$DATA_DIR/ci/artifacts/<run id>/`. The container is removed. Cache volumes
   stay.

Everything up to the first step is shown as the step `pipeline setup`.

## Setup

Hearthforge talks to the container engine over its Unix socket. It checks,
in order:

1. `CI_DOCKER_SOCKET`, when set
2. `/var/run/docker.sock`
3. `/run/podman/podman.sock`
4. `/run/user/<uid>/podman/podman.sock`

Without a socket the Pipelines tab still works, and every run ends in one
skipped `pipeline setup` step that names the problem.

### Host install

Docker needs no extra setup. Rootless Podman needs the API socket enabled:

```bash
systemctl --user enable --now podman.socket
```

### Hearthforge in a container

The socket must be mounted into the Hearthforge container. `compose.yml`
carries commented-out bindings for both engines. Mount the socket at
`/var/run/docker.sock` and the auto-detection finds it, or mount it elsewhere
and set `CI_DOCKER_SOCKET`.

Mounting the socket gives Hearthforge full control over the container engine.
That is equivalent to root on the host for Docker, and to the socket owner's
account for rootless Podman.

Cache volumes, image pulls, and the CI containers live on the engine side.
Nothing about them refers to paths inside the Hearthforge container, so a
containerised Hearthforge works the same as a host install.

### Server settings

| Variable             | Default           | Description                                          |
|----------------------|-------------------|------------------------------------------------------|
| `CI_DOCKER_SOCKET`   | _(auto-detected)_ | Path to the Docker/Podman socket                     |
| `CI_MAX_CONCURRENT`  | `2`               | Runs executed at once. Further runs wait in a queue. |
| `CI_MAX_HISTORY`     | `50`              | Runs kept per repository, artifacts included         |
| `CI_DEFAULT_TIMEOUT` | `3600`            | Step timeout in seconds when the config sets none    |

## Example

```toml
image = "docker.io/debian:stable"
work_dir = "/ci/build"
clone_project_to = "/ci/build/project"   # must be absolute
shell_setup = "set -euo pipefail"
timeout = 3600                           # per-step default, in seconds
memory_limit = "2g"

cache = [
  { path = "/root/.cache/pip", max_size = "1g" },
]

[on]
push = ["main", "release/*"]   # branch globs, or ["*"]. `*` does not cross a `/`
tag = true

[variables]
  [variables.PYTHON]
  default = "3.12"
  description = "Interpreter version, overridable from the Run pipeline form"

[[steps]]
name = "setup"
run_sh = "apt-get update -qq && apt-get install -y --no-install-recommends python3 python3-pip"
timeout = 300

[[steps]]
name = "test"
run_sh = "cd project && python3 -m pytest"

[[steps]]
name = "package"
run_if = 'test -n "${CI_COMMIT_TAG}"'
run_sh = "cd project && python3 -m build"
publish_gzip = ["/ci/build/project/dist/"]

[[steps]]
name = "report"
always = true
run_sh = "cat /ci/build/project/.pytest_cache/v/cache/lastfailed || true"
```

The template in the Pipelines tab lists every key with a comment.

### Step options

| Key            | Effect                                                                       |
|----------------|------------------------------------------------------------------------------|
| `run_sh`       | The command. Runs through `shell`, after `shell_setup`.                      |
| `timeout`      | Seconds. Falls back to the top-level `timeout`, then `CI_DEFAULT_TIMEOUT`.   |
| `run_if`       | Shell expression. A non-zero exit skips the step.                            |
| `always`       | Run even after an earlier step failed.                                       |
| `warn_on_fail` | A failure marks the step `warning` and the run continues.                    |
| `clear`        | Delete `clone_project_to` and extract a fresh checkout before the step.      |
| `publish_*`    | `publish_file`, `publish_tar`, `publish_gzip`, `publish_zstd`, `publish_zip` |

### Environment

Every step sees `CI=true` plus:

| Variable              | Value                          |
|-----------------------|--------------------------------|
| `CI_PIPELINE_ID`      | numeric run ID                 |
| `CI_REPO_NAME`        | repository name                |
| `CI_SERVER_URL`       | Hearthforge `BASE_URL`         |
| `CI_TRIGGER_SOURCE`   | `push`, `tag`, or `manual`     |
| `CI_COMMIT_SHA`       | full commit hash               |
| `CI_COMMIT_SHORT_SHA` | first 8 characters             |
| `CI_COMMIT_BRANCH`    | branch name, empty for tags    |
| `CI_COMMIT_TAG`       | tag name, empty for branches   |
| `CI_COMMIT_REF_NAME`  | branch or tag name             |

`[variables]` defaults come next, then overrides from the **Run pipeline**
form, then secrets. A later source wins on a name clash.

### Secrets

Admins add secrets under **Repository settings → CI Secrets**. They are
injected as environment variables. Their values are replaced with `[MASKED]`
in step logs. The masking is a plain text match: a secret printed in encoded
or split form is not caught.

### Status badge

```
![CI](https://your-forge.example.com/REPO_NAME/ci/badge.svg)
```

## Caches

Each `cache` path becomes a named volume, `hearthforge-ci-cache-<hash>`,
labelled `com.hearthforge.repo=<repository>`. The volume is mounted at the
same path in every run of that repository.

- A cache without `max_size` grows without bound.
- A cache with `max_size` is deleted after the run that takes it over the
  limit. The run detail page shows a `cache` step naming the dropped volume.
  The next run starts cold. Size the cap at about twice the working set.
- A volume the current config no longer lists is deleted after a run on the
  default branch. Other branches keep it.
- **Purge caches** on the Pipelines tab deletes all volumes of the
  repository. Deleting or renaming a repository does the same.
- Everything else needs the engine's own tools:

```bash
docker volume ls --filter label=com.hearthforge.repo=REPO_NAME
docker volume rm $(docker volume ls -q --filter label=com.hearthforge.repo)   # all Hearthforge caches
```

The size check reads the daemon's reported usage. A daemon that does not
report volume sizes never triggers the cap.

## Gotchas

- **`clone_project_to` must be absolute**, and no `cache` path may lie inside
  it or above it. The checkout is extracted over that directory and a `clear`
  step deletes it. The config is rejected otherwise.
- **The image needs no git, and the checkout has no `.git`.** `git describe`
  or a version from the log needs a step that installs git and fetches the
  history itself, for example over the forge's HTTP clone URL. Pushing back
  from a pipeline needs the admin's credentials, kept as a secret.
- **The image is pulled on every run.** Even with a local copy, the daemon
  contacts the registry. There is no registry authentication, so private
  images must already be present and pullable without login. Pin a tag; a
  moving tag changes the toolchain under the pipeline.
- **Match the libc for `[[copy]]`.** A binary from an `-alpine` image is
  musl and fails on a glibc image with an error that looks like a missing
  file.
- **`publish_*` uses tools inside the image.** `publish_gzip`, `publish_tar`,
  and `publish_zstd` need `tar` (plus `zstd`). `publish_zip` needs `zip`.
  `publish_file` needs nothing.
- **Steps share one container.** A file left behind by one step is visible to
  the next. Use `clear` on a step that must start from a clean tree. Do not
  `rm -rf` `work_dir` itself: the container's working directory is gone and
  every later step fails to start.
- **Logs are cut at 2 MB per step.** Redirect verbose output to a file and
  publish the file instead.
- **A Hearthforge restart cancels running runs.** Their containers are
  force-removed on startup. The runs show as `cancelled` and can be retried.
- **The container is on the engine's default network.** It reaches the
  internet and any other container on that network. Set `cpu_limit` and
  `memory_limit`; there are no defaults.
- **Old runs are deleted.** Only the newest `CI_MAX_HISTORY` runs per
  repository are kept, and their artifacts go with them. Attach anything
  permanent to a release.