Fix findings from a full codebase review
Security and resource limits: - Registry Basic auth counts every argon2 check before it runs and caches verified credentials, so parallel guesses no longer bypass the limiter. - A global semaphore caps parallel argon2 runs (64 MiB each). - Rate-limit keys are never empty and IPv6 is keyed by /64. - Password and passkey changes sign out other sessions. Adding or revoking a passkey and removing the password need a recent login. - The current-password check is rate-limited per user. - init generates a random admin password when ADMIN_PASSWORD is unset. - The SSH server has an idle timeout. - BASE_URL is normalized, so CSRF and WebAuthn origins match browsers. - The reserved admin name is matched case-insensitively. - Passkeys require user verification on registration and login. - Markdown allows only the classes the renderer emits. Git and file views: - Commit diffs are read up to MAX_RENDER_BYTES and binary sizes come from one batched call. Big READMEs show a download notice. - Tree listings use ls-tree -z and diffs decode quoted paths, so non-ASCII file names work. - Signature badges gain an "unverified" state and revoked keys show bad. - UI edits and deletes carry the base commit. A change on top of a push that touched the same file is refused and the editor keeps the text. - Writes use a private index and full branch refs. Create and rename refuse existing paths and deleting a missing file makes no commit. - Branches can start from annotated tags and suffix byte ranges work. - Bare conversion skips a symlinked .git and recovers a leftover tmp dir. CI: - Cancel marks the active step cancelled and never ends as success. - Engine calls have timeouts, containers are labelled and swept at startup, and stale names are removed before create. - Deleting a repo stops its runs and removes artifacts and caches. - Secrets are masked longest first, also at log cut points. - CI_REGISTRY is lowercase and pruning keeps recently finished runs. Issues and patches: - Authors cannot delete or edit closed or merged items. - Diff, file and markdown caches have byte limits. - Multi-label filters survive pagination. - Reactions check that the comment belongs to the target. - Patch-check errors are cached briefly and survive client disconnects. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
MCI.md
@@ -25,7 +25,8 @@ reference, and a link to the full [template](public/assets/hearthforge-ci-templa
image needs no tar, gzip, zstd or zip. The container is removed. Cache
volumes stay.
Everything up to the first step is shown as the step `pipeline setup`.
Everything up to the first step is shown as the step `pipeline setup`. It
fails after 30 minutes.
The image must provide a POSIX shell at `/bin/sh` (or whatever `shell`
names), `sleep` for the container's main process, and `rm` plus `mkdir` if a
@@ -130,8 +131,8 @@ The template in the Pipelines tab lists every key with a comment.
| Key | Effect |
|----------------|------------------------------------------------------------------------------|
| `run_sh` | The command. Runs through `shell`, after `shell_setup`. |
| `timeout` | Seconds. Falls back to the top-level `timeout`, then `CI_DEFAULT_TIMEOUT`. |
| `run_if` | Shell expression. A non-zero exit skips the step. |
| `timeout` | Seconds. Falls back to the top-level `timeout`, then `CI_DEFAULT_TIMEOUT`. Also bounds `run_if` and `clear`. |
| `run_if` | Shell expression. A non-zero exit skips the step. A timeout fails it. |
| `always` | Run even after an earlier step failed. |
| `warn_on_fail` | A failure marks the step `warning` and the run continues. |
| `clear` | Delete `clone_project_to` and extract a fresh checkout before the step. |
@@ -153,7 +154,7 @@ Every step sees `CI=true` plus:
| `CI_COMMIT_BRANCH` | branch name, empty for tags |
| `CI_COMMIT_TAG` | tag name, empty for branches |
| `CI_COMMIT_REF_NAME` | branch or tag name |
| `CI_REGISTRY` | `<BASE_URL host>/<repo>`, the image name prefix for the built-in registry |
| `CI_REGISTRY` | `<BASE_URL host>/<repo>` with the repo name lowercased, the image name prefix for the built-in registry |
`[variables]` defaults come next, then overrides from the **Run pipeline**
form, then secrets. A later source wins on a name clash.
@@ -268,11 +269,14 @@ report volume sizes never triggers the cap.
every later step fails to start.
- **Logs are cut at 2 MB per step.** Redirect verbose output to a file and
publish the file instead.
- **A Hearthforge restart cancels running runs.** Their containers are
force-removed on startup. The runs show as `cancelled` and can be retried.
- **A Hearthforge restart cancels running runs.** Every CI container carries
the label `com.hearthforge.ci`, and all of them are force-removed on
startup. The runs show as `cancelled` and can be retried.
- **Deleting a repository cancels its runs** and deletes their artifacts and
cache volumes.
- **The container is on the engine's default network.** It reaches the
internet and any other container on that network. Set `cpu_limit` and
`memory_limit`; there are no defaults.
- **Old runs are deleted.** Only the newest `CI_MAX_HISTORY` runs per
repository are kept, and their artifacts go with them. Attach anything
- **Old runs are deleted.** Only the `CI_MAX_HISTORY` most recently finished
runs per repository are kept, and their artifacts go with them. Attach anything
permanent to a release.
MREADME.md
@@ -36,8 +36,8 @@ go build -o hearthforge ./cmd/hearthforge
./hearthforge # http://localhost:3000, SSH on port 2222
```
Default admin credentials: `admin` / `changeme`
Set the `ADMIN_PASSWORD` environment variable **before** running `db:init` to choose your own.
`init` creates the user `admin`. Set the `ADMIN_PASSWORD` environment variable **before** running `init` to choose its password.
Without it, `init` generates a random password and prints it once. In the container it appears in the log of the first start.
### Docker / Podman
@@ -52,7 +52,7 @@ The container stores all persistent data (repos, database, avatars, releases, SS
### Manual repository import
Existing repositories can be copied into the `data/repos` directory.
Non-bare repos are automatically converted to bare repos on startup (uncommitted changes and worktrees are discarded).
Non-bare repos are automatically converted to bare repos on startup (uncommitted changes and worktrees are discarded). A repo whose `.git` is a symlink is skipped.
Pushing directly to the on-disk repositories (bypassing the bundled HTTP/SSH endpoints) generally works as well.
### SSH access
@@ -74,7 +74,7 @@ All settings are environment variables:
| `PORT` | `3000` | HTTP port |
| `SSH_PORT` | `2222` | SSH port |
| `DATA_DIR` | `./data` | Repos, database, uploads |
| `ADMIN_PASSWORD` | `changeme` | Initial admin password (only used by `init`) |
| `ADMIN_PASSWORD` | _(random, printed once)_ | Initial admin password (only used by `init`) |
| `OWNER_DISPLAY_NAME` | `Admin` | Display name for the owner |
| `BASE_URL` | `http://localhost:$PORT` | Used in clone URLs and links |
| `REGISTRATION_TYPE` | `enabled` | `enabled`, `disabled`, or `queue` (admin approval) |
@@ -106,7 +106,7 @@ All settings are environment variables:
| `CI_NETWORK` | _(engine default)_ | Engine network for CI containers, e.g. one created with IPv6 |
| `REGISTRY_PULL` | `admin` | Who may pull container images: `admin`, `users`, or `public` |
\* Set `TRUSTED_PROXY=1` only when Hearthforge is behind a reverse proxy that strips any incoming `X-Forwarded-For` from clients. Caddy and Traefik do this by default; nginx requires `proxy_set_header X-Forwarded-For $remote_addr;` (rather than the common `$proxy_add_x_forwarded_for`, which appends to a client-supplied value). Setting `TRUSTED_PROXY=1` in front of a proxy that does not strip means rate limits and any audit logging are spoofable per request.
\* Set `TRUSTED_PROXY=1` only when Hearthforge is behind a reverse proxy that overwrites any incoming `X-Forwarded-For` from clients with the real client address. Without the header, the socket address is used. Caddy and Traefik do this by default; nginx requires `proxy_set_header X-Forwarded-For $remote_addr;` (rather than the common `$proxy_add_x_forwarded_for`, which appends to a client-supplied value). Setting `TRUSTED_PROXY=1` in front of a proxy that does not strip means rate limits and any audit logging are spoofable per request. Rate limits key IPv6 clients by their /64 prefix.
### Reverse proxy deployment
Mcmd/hearthforge/main.go
@@ -6,6 +6,7 @@ package main
import (
"context"
"crypto/rand"
"errors"
"flag"
"fmt"
@@ -32,7 +33,7 @@ const usage = `Usage: hearthforge [command]
Commands:
(none) start the HTTP and SSH servers
init create the admin account (password from ADMIN_PASSWORD, default "changeme")
init create the admin account (password from ADMIN_PASSWORD, else random and printed once)
Configuration comes from environment variables. See README.md.
`
@@ -71,17 +72,20 @@ func main() {
if command == "init" {
pw := os.Getenv("ADMIN_PASSWORD")
if pw == "" {
pw = "changeme"
log.Println("ADMIN_PASSWORD not set, using default 'changeme'")
generated := pw == ""
if generated {
pw = rand.Text()
}
created, err := database.InitAdmin(ctx, pw)
if err != nil {
log.Fatal(err)
}
if created {
switch {
case created && generated:
fmt.Printf("admin account created, password: %s\n", pw)
case created:
fmt.Println("admin account created")
} else {
default:
fmt.Println("admin account already exists")
}
return
Mcompose.yml
@@ -17,7 +17,7 @@ services:
SSH_PORT: 2222
OWNER_DISPLAY_NAME: Admin
BASE_URL: "http://localhost:3000"
# ADMIN_PASSWORD: change-me
# ADMIN_PASSWORD: "" # unset: the first start prints a random admin password to the log
# CI_DOCKER_SOCKET: /var/run/docker.sock # only when mounted at another path
# SSH_DISABLED: 0
# SCANNED_REPO_PRIVATE: 1
Minternal/ci/docker.go
@@ -348,14 +348,18 @@ func (r *Runner) createContainer(ctx context.Context, runID int64, repoName stri
if workDir == "" {
workDir = "/"
}
// A retry reuses the name. An earlier attempt cancelled mid-create can
// leave a container the runner never learned the id of.
r.removeContainer(ctx, containerName(runID))
resp, body, err := r.doJSON(ctx, http.MethodPost,
fmt.Sprintf("/containers/create?name=hearthforge-ci-%d", runID),
"/containers/create?name="+containerName(runID),
map[string]any{
"Image": cfg.Image,
"Cmd": []string{"sleep", "infinity"},
"Env": envVars,
"WorkingDir": workDir,
"HostConfig": hostConfig,
"Labels": ciContainerLabels,
})
if err != nil {
return "", err
@@ -381,13 +385,50 @@ func (r *Runner) startContainer(ctx context.Context, containerID string) error {
return nil
}
// removeContainer force-removes a container. Cleanup is best effort.
// ciContainerLabels marks every container the runner creates, so the startup
// sweep finds the ones a crash left behind.
var ciContainerLabels = map[string]string{"com.hearthforge.ci": "1"}
// cleanupTimeout bounds each engine call made after a run ends or is cancelled.
const cleanupTimeout = 2 * time.Minute
// removeContainer force-removes a container by id or name. Cleanup is best
// effort. It still runs when ctx is cancelled.
func (r *Runner) removeContainer(ctx context.Context, containerID string) {
if _, _, err := r.doJSON(ctx, http.MethodDelete, "/containers/"+containerID+"?force=true", nil); err != nil {
log.Printf("remove container %s: %v", containerID, err)
ctx, cancel := context.WithTimeout(context.WithoutCancel(ctx), cleanupTimeout)
defer cancel()
resp, body, err := r.doJSON(ctx, http.MethodDelete, "/containers/"+containerID+"?force=true", nil)
switch {
case errors.Is(err, errNoSocket):
case err != nil:
log.Printf("[ci] remove container %s: %v", containerID, err)
case resp.StatusCode >= 300 && resp.StatusCode != http.StatusNotFound:
log.Printf("[ci] remove container %s: HTTP %d %s", containerID, resp.StatusCode, strings.TrimSpace(string(body)))
}
}
// removeLabelledContainers force-removes every container the runner created.
// Only safe while no run executes.
func (r *Runner) removeLabelledContainers(ctx context.Context) error {
filters, _ := json.Marshal(map[string][]string{"label": {"com.hearthforge.ci"}})
resp, body, err := r.doJSON(ctx, http.MethodGet,
"/containers/json?all=true&filters="+url.QueryEscape(string(filters)), nil)
if err != nil {
return err
}
if resp.StatusCode >= 300 {
return fmt.Errorf("docker returned %d listing containers", resp.StatusCode)
}
var list []struct{ Id string }
if err := json.Unmarshal(body, &list); err != nil {
return err
}
for _, c := range list {
r.removeContainer(ctx, c.Id)
}
return nil
}
// --- Exec ---
type execResult struct {
@@ -500,10 +541,12 @@ func (l *logBuffer) append(text string) {
if room > 0 {
l.b.WriteString(text[:room])
}
fmt.Fprintf(&l.b, "\n[log truncated at %d bytes]\n", ciMaxLogBytes)
l.b.WriteString(logTruncatedNotice)
l.truncated = true
}
var logTruncatedNotice = fmt.Sprintf("\n[log truncated at %d bytes]\n", ciMaxLogBytes)
func (l *logBuffer) String() string { return l.b.String() }
// --- Archive transfer ---
@@ -518,11 +561,9 @@ func (r *Runner) copyFromImage(ctx context.Context, containerID string, spec Cop
if err := r.pullImage(ctx, spec.Image); err != nil {
return err
}
// Unnamed on purpose. The startup sweep deletes by exact name, and a
// retry reuses the run id, so a leaked named container would fail every
// retry with a name conflict.
// Unnamed: one run creates several helpers.
resp, body, err := r.doJSON(ctx, http.MethodPost, "/containers/create",
map[string]any{"Image": spec.Image, "Cmd": []string{"true"}})
map[string]any{"Image": spec.Image, "Cmd": []string{"true"}, "Labels": ciContainerLabels})
if err != nil {
return err
}
@@ -534,8 +575,7 @@ func (r *Runner) copyFromImage(ctx context.Context, containerID string, spec Cop
if err := json.Unmarshal(body, &created); err != nil {
return err
}
// The cleanup must still run when the run is cancelled mid-copy.
defer r.removeContainer(context.WithoutCancel(ctx), created.Id)
defer r.removeContainer(ctx, created.Id)
// Docker rejects the upload unless the destination already exists.
if err := r.mkdirInContainer(ctx, containerID, spec.To); err != nil {
@@ -625,8 +665,7 @@ func writeFirstTarFile(src io.Reader, destPath string, maxBytes int64) (int64, e
}
}
// containerName is the name every run's container carries. The startup sweep
// removes leftovers by this name.
// containerName is the name every run's container carries.
func containerName(runID int64) string {
return fmt.Sprintf("hearthforge-ci-%d", runID)
}
Minternal/ci/docker_test.go
@@ -6,7 +6,11 @@ import (
"net"
"net/http"
"net/http/httptest"
"path"
"path/filepath"
"slices"
"strings"
"sync"
"testing"
"hearthforge/internal/config"
@@ -65,3 +69,38 @@ func TestSocketMissingIsReported(t *testing.T) {
t.Fatalf("got %v", err)
}
}
// A crash can leak [[copy]] helpers, which carry no run name. The startup
// sweep finds them by label.
func TestCancelStaleRunsSweepsLabelledContainers(t *testing.T) {
var mu sync.Mutex
var deleted []string
r := newMockEngine(t, http.HandlerFunc(func(w http.ResponseWriter, req *http.Request) {
switch {
case req.Method == http.MethodGet && req.URL.Path == "/v1.47/containers/json":
if !strings.Contains(req.URL.Query().Get("filters"), "com.hearthforge.ci") {
t.Errorf("filters = %q", req.URL.Query().Get("filters"))
}
w.Write([]byte(`[{"Id":"helper1"}]`))
case req.Method == http.MethodDelete:
mu.Lock()
deleted = append(deleted, path.Base(req.URL.Path))
mu.Unlock()
w.WriteHeader(http.StatusNotFound)
default:
t.Errorf("unexpected request %s %s", req.Method, req.URL.Path)
}
}))
_, d := newRunnerDB(t)
r.db = d
id := insertRun(t, d, "running")
if err := r.CancelStaleRuns(context.Background()); err != nil {
t.Fatal(err)
}
mu.Lock()
defer mu.Unlock()
want := []string{containerName(id), "helper1"}
if !slices.Equal(deleted, want) {
t.Fatalf("deleted %v, want %v", deleted, want)
}
}
Minternal/ci/execute.go
@@ -12,10 +12,16 @@ import (
"hearthforge/internal/db"
)
// setupTimeout bounds everything before the first step: pulls, copies and
// the checkout upload.
const setupTimeout = 30 * time.Minute
// executeRun drives one run from "pending" to a final status. It never
// returns an error: every outcome is written to the run and step rows.
func (r *Runner) executeRun(ctx context.Context, runID int64, t *task) {
var containerID string
// Step results must land even when the run is cancelled mid-write.
dbCtx := context.WithoutCancel(ctx)
// Set once the config is known. The cleanup block cannot see cfg, and
// pruning off another branch is wrong.
var pruneRepo string
@@ -34,6 +40,8 @@ func (r *Runner) executeRun(ctx context.Context, runID int64, t *task) {
err := func() error {
r.execSQL(ctx, `UPDATE ci_runs SET status = 'running', started_at = ? WHERE id = ?`, db.NowISO(), runID)
setupCtx, cancelSetup := context.WithTimeout(ctx, setupTimeout)
defer cancelSetup()
run, err := r.loadRun(ctx, runID)
if err != nil {
@@ -84,20 +92,23 @@ func (r *Runner) executeRun(ctx context.Context, runID int64, t *task) {
stepIDs[i] = r.insertStep(ctx, runID, step.Name, "pending", false)
}
if err := r.pullImage(ctx, cfg.Image); err != nil {
if err := r.pullImage(setupCtx, cfg.Image); err != nil {
return err
}
if ctx.Err() != nil {
return ctx.Err()
}
containerID, err = r.createContainer(ctx, runID, repoName, cfg, envArray)
// Set before the create: a create cut short can still leave it behind.
containerID = containerName(runID)
id, err := r.createContainer(setupCtx, runID, repoName, cfg, envArray)
if err != nil {
return err
}
containerID = id
r.setContainer(t, containerID)
if err := r.startContainer(ctx, containerID); err != nil {
if err := r.startContainer(setupCtx, containerID); err != nil {
return err
}
if ctx.Err() != nil {
@@ -105,7 +116,7 @@ func (r *Runner) executeRun(ctx context.Context, runID int64, t *task) {
}
if cfg.WorkDir != "" {
if err := r.mkdirInContainer(ctx, containerID, cfg.WorkDir); err != nil {
if err := r.mkdirInContainer(setupCtx, containerID, cfg.WorkDir); err != nil {
return fmt.Errorf("create workdir: %w", err)
}
}
@@ -113,7 +124,7 @@ func (r *Runner) executeRun(ctx context.Context, runID int64, t *task) {
// Copies run before the clone, so a step can rely on the tools they
// bring in, and so they can supply a shell the image lacks.
for _, spec := range cfg.Copy {
if err := r.copyFromImage(ctx, containerID, spec); err != nil {
if err := r.copyFromImage(setupCtx, containerID, spec); err != nil {
return err
}
if ctx.Err() != nil {
@@ -122,10 +133,11 @@ func (r *Runner) executeRun(ctx context.Context, runID int64, t *task) {
}
if cfg.CloneProjectTo != "" && run.CommitSha != "" {
if err := r.uploadCheckout(ctx, containerID, repoName, run.CommitSha, cfg.CloneProjectTo); err != nil {
if err := r.uploadCheckout(setupCtx, containerID, repoName, run.CommitSha, cfg.CloneProjectTo); err != nil {
return err
}
}
cancelSetup()
r.execSQL(ctx, `UPDATE ci_steps SET status = 'success', finished_at = ? WHERE id = ?`, db.NowISO(), setupStepID)
@@ -137,9 +149,8 @@ func (r *Runner) executeRun(ctx context.Context, runID int64, t *task) {
sawWarning := false
for i, step := range cfg.Steps {
// Returned, not broken out of: the follow-up SQL below runs on the
// cancelled context and would silently do nothing, leaving every
// remaining step pending. The caller's cleanup context marks them.
// Returned, not broken out of: the caller marks the remaining
// steps cancelled, not skipped.
if ctx.Err() != nil {
return ctx.Err()
}
@@ -173,16 +184,33 @@ func (r *Runner) executeRun(ctx context.Context, runID int64, t *task) {
})
}
timeout := step.Timeout
if timeout == 0 {
timeout = cfg.Timeout
}
if timeout == 0 {
timeout = r.cfg.CIDefaultTimeout
}
if step.RunIf != "" {
res, err := r.exec(ctx, containerID, append(append([]string{}, shell...), step.RunIf),
condCtx, cancel := context.WithTimeout(ctx, time.Duration(timeout)*time.Second)
res, err := r.exec(condCtx, containerID, append(append([]string{}, shell...), step.RunIf),
cfg.WorkDir, stepEnv, nil)
cancel()
if ctx.Err() != nil {
return ctx.Err()
}
if err != nil {
// The engine went away. That is a run failure, not a
// A hung condition or a lost engine is a failure, not a
// condition that did not hold.
return err
r.execSQL(dbCtx,
`UPDATE ci_steps SET status = 'failure', started_at = ?, finished_at = ?, log = ? WHERE id = ?`,
db.NowISO(), db.NowISO(), fmt.Sprintf("run_if failed: %s\n", err), stepID)
runFailed = true
continue
}
if res.exitCode != 0 {
r.execSQL(ctx,
r.execSQL(dbCtx,
`UPDATE ci_steps SET status = 'skipped', started_at = ?, finished_at = ?, log = ? WHERE id = ?`,
db.NowISO(), db.NowISO(), "Skipped: condition not met", stepID)
continue
@@ -192,25 +220,30 @@ func (r *Runner) executeRun(ctx context.Context, runID int64, t *task) {
// clear: drop the directory and re-extract. `git clean` would
// need git in the image, and this also removes untracked files.
if step.Clear && cfg.CloneProjectTo != "" && run.CommitSha != "" {
clearCtx, cancel := context.WithTimeout(ctx, time.Duration(timeout)*time.Second)
clearErr := func() error {
// One exec, not two. `rm -rf` can delete the container's
// WorkingDir, and every later exec then fails to chdir.
// This exec chdirs first.
cmd := append(append([]string{}, shell...),
`rm -rf "$1" && mkdir -p "$1"`, "sh", cfg.CloneProjectTo)
reset, err := r.exec(ctx, containerID, cmd, "", nil, nil)
reset, err := r.exec(clearCtx, containerID, cmd, "", nil, nil)
if err != nil {
return err
}
if reset.exitCode != 0 {
return errors.New(strings.TrimSpace(reset.log))
}
return r.uploadCheckout(ctx, containerID, repoName, run.CommitSha, cfg.CloneProjectTo)
return r.uploadCheckout(clearCtx, containerID, repoName, run.CommitSha, cfg.CloneProjectTo)
}()
cancel()
if ctx.Err() != nil {
return ctx.Err()
}
if clearErr != nil {
// Not returned: the outer handler only records a message
// while setup still owns the run. The step row survives.
r.execSQL(ctx,
r.execSQL(dbCtx,
`UPDATE ci_steps SET status = 'failure', started_at = ?, finished_at = ?, log = ? WHERE id = ?`,
db.NowISO(), db.NowISO(),
fmt.Sprintf("Failed to reset %s: %s\n", cfg.CloneProjectTo, clearErr), stepID)
@@ -234,19 +267,12 @@ func (r *Runner) executeRun(ctx context.Context, runID int64, t *task) {
if cfg.ShellSetup != "" {
command = cfg.ShellSetup + "\n" + step.RunSh
}
timeout := step.Timeout
if timeout == 0 {
timeout = cfg.Timeout
}
if timeout == 0 {
timeout = r.cfg.CIDefaultTimeout
}
stepCtx, cancel := context.WithTimeout(ctx, time.Duration(timeout)*time.Second)
res, execErr := r.exec(stepCtx, containerID,
append(append([]string{}, shell...), command), cfg.WorkDir, stepEnv,
func(partial string) {
r.execSQL(ctx, `UPDATE ci_steps SET log = ? WHERE id = ?`,
maskSecrets(partial, secretValues), stepID)
maskLog(partial, secretValues, true), stepID)
})
timedOut := stepCtx.Err() == context.DeadlineExceeded
cancel()
@@ -264,14 +290,14 @@ func (r *Runner) executeRun(ctx context.Context, runID int64, t *task) {
stepLog = fmt.Sprintf("Step timed out after %ds\n", timeout)
// Kill the container now so the timed-out command stops
// immediately rather than lingering until cleanup.
r.removeContainer(context.WithoutCancel(ctx), containerID)
r.removeContainer(ctx, containerID)
containerID = ""
r.setContainer(t, "")
case execErr != nil:
stepStatus = onFail
stepLog = fmt.Sprintf("Step failed: %s\n", execErr)
default:
stepLog = maskSecrets(res.log, secretValues)
stepLog = maskLog(res.log, secretValues, false)
if res.exitCode != 0 {
stepStatus = onFail
}
@@ -288,11 +314,14 @@ func (r *Runner) executeRun(ctx context.Context, runID int64, t *task) {
r.collectArtifacts(ctx, runID, containerID, step)
}
r.execSQL(ctx, `UPDATE ci_steps SET status = ?, finished_at = ?, log = ? WHERE id = ?`,
r.execSQL(dbCtx, `UPDATE ci_steps SET status = ?, finished_at = ?, log = ? WHERE id = ?`,
stepStatus, db.NowISO(), stepLog, stepID)
}
if ctx.Err() != nil {
return ctx.Err()
}
r.execSQL(ctx,
r.execSQL(dbCtx,
`UPDATE ci_steps SET status = 'skipped', started_at = ?, finished_at = ?, log = 'Skipped: previous step failed'
WHERE run_id = ? AND status = 'pending'`, db.NowISO(), db.NowISO(), runID)
@@ -304,10 +333,6 @@ func (r *Runner) executeRun(ctx context.Context, runID int64, t *task) {
return nil
}()
// The run context may be cancelled from here on, so cleanup and the
// failure rows use a context that outlives it.
cleanupCtx := context.WithoutCancel(ctx)
if err != nil {
cancelled := ctx.Err() != nil
dockerUnavailable := errors.Is(err, errNoSocket)
@@ -322,37 +347,46 @@ func (r *Runner) executeRun(ctx context.Context, runID int64, t *task) {
}
finalStatus = status
setupLog := fmt.Sprintf("Error: %s\n", err)
if dockerUnavailable {
switch {
case dockerUnavailable:
setupLog = skipLog
case !cancelled && errors.Is(err, context.DeadlineExceeded):
setupLog = fmt.Sprintf("Error: pipeline setup timed out after %s\n", setupTimeout)
}
// Only while setup still owns the run. Once it succeeds, a failure
// belongs to a step, and that step carries its own log.
r.execSQL(cleanupCtx,
r.execSQL(dbCtx,
`UPDATE ci_steps SET status = ?, finished_at = ?, log = ? WHERE id = ? AND status = 'running'`,
stepStatus, db.NowISO(), setupLog, setupStepID)
r.execSQL(cleanupCtx,
// The step a cancel interrupted keeps its partial log.
r.execSQL(dbCtx,
`UPDATE ci_steps SET status = ?, finished_at = ? WHERE run_id = ? AND status = 'running'`,
stepStatus, db.NowISO(), runID)
r.execSQL(dbCtx,
`UPDATE ci_steps SET status = 'skipped', finished_at = ?, log = ? WHERE run_id = ? AND status = 'pending'`,
db.NowISO(), skipLog, runID)
}
if containerID != "" {
r.removeContainer(cleanupCtx, containerID)
r.removeContainer(dbCtx, containerID)
}
r.execSQL(cleanupCtx, `UPDATE ci_runs SET status = ?, finished_at = ? WHERE id = ?`,
// CancelRun may have written 'cancelled' after the last ctx check.
r.execSQL(dbCtx, `UPDATE ci_runs SET status = ?, finished_at = ? WHERE id = ? AND status <> 'cancelled'`,
finalStatus, db.NowISO(), runID)
r.finishSlot(runID, t)
var repoID int64
if r.db.QueryRowContext(cleanupCtx, `SELECT repo_id FROM ci_runs WHERE id = ?`, runID).Scan(&repoID) == nil {
r.pruneHistory(cleanupCtx, repoID)
if r.db.QueryRowContext(dbCtx, `SELECT repo_id FROM ci_runs WHERE id = ?`, runID).Scan(&repoID) == nil {
r.pruneHistory(dbCtx, repoID)
}
engineCtx, cancelEngine := context.WithTimeout(dbCtx, cleanupTimeout)
defer cancelEngine()
if pruneRepo != "" {
r.pruneStaleCaches(cleanupCtx, pruneRepo, cacheEntries)
r.pruneStaleCaches(engineCtx, pruneRepo, cacheEntries)
}
// After removeContainer above: the daemon will not delete a volume that
// is still mounted.
if limitsRepo != "" {
if dropped := r.enforceCacheLimits(cleanupCtx, limitsRepo, cacheEntries); len(dropped) > 0 {
if dropped := r.enforceCacheLimits(engineCtx, limitsRepo, cacheEntries); len(dropped) > 0 {
// Recorded on the run. The only other symptom is a slow next build
// with no visible cause.
lines := make([]string, len(dropped))
@@ -360,13 +394,14 @@ func (r *Runner) executeRun(ctx context.Context, runID int64, t *task) {
lines[i] = fmt.Sprintf("Dropped cache %s: %s over the %s limit",
d.path, formatBytes(d.size), formatBytes(d.maxSize))
}
r.execSQL(cleanupCtx,
r.execSQL(dbCtx,
`INSERT INTO ci_steps (run_id, name, status, started_at, finished_at, log)
VALUES (?, 'cache', 'success', ?, ?, ?)`,
runID, db.NowISO(), db.NowISO(), strings.Join(lines, "\n")+"\n")
}
}
// A slot just freed up. Start the next queued run, if any.
// Last, so StopRepo's wait also covers the cleanup writes above.
r.finishSlot(runID, t)
r.pumpQueue()
}
Minternal/ci/run.go
@@ -12,10 +12,12 @@ import (
"os/exec"
"path"
"path/filepath"
"slices"
"sort"
"strconv"
"strings"
"sync"
"time"
"hearthforge/internal/config"
"hearthforge/internal/db"
@@ -250,6 +252,14 @@ func (r *Runner) TriggerRun(ctx context.Context, repoName string, opts TriggerOp
// queued or executing.
var ErrRunNotFinished = errors.New("run is not finished")
// Active reports whether a run still holds its slot. A run frees it only
// after its cleanup, so a finished status alone does not mean idle.
func (r *Runner) Active(runID int64) bool {
r.mu.Lock()
defer r.mu.Unlock()
return r.running[runID] != nil
}
// RetryRun clears a finished run's steps and artifacts and runs it again.
// The reset only applies to a run in a terminal status, so a retry cannot
// hijack a run that is still executing.
@@ -258,6 +268,11 @@ func (r *Runner) RetryRun(ctx context.Context, runID, retriedBy int64) error {
if err := r.db.QueryRowContext(ctx, `SELECT repo_id FROM ci_runs WHERE id = ?`, runID).Scan(&repoID); err != nil {
return errors.New("run not found")
}
// CancelRun marks the row before the execution has cleaned up. Its
// cleanup would clobber the new attempt's rows and container.
if r.Active(runID) {
return ErrRunNotFinished
}
res, err := r.db.ExecContext(ctx,
`UPDATE ci_runs SET status = 'pending', triggered_by = ?, started_at = NULL, finished_at = NULL
WHERE id = ? AND status IN ('success','failure','warning','cancelled','skipped')`,
@@ -323,30 +338,65 @@ func (r *Runner) PurgeRepoCaches(ctx context.Context, repoName string) (int, err
return removed, nil
}
// CancelStaleRuns cleans up runs a restart interrupted. Their containers are
// force-removed and the rows read "cancelled", so they can be retried.
func (r *Runner) CancelStaleRuns(ctx context.Context) error {
rows, err := r.db.QueryContext(ctx,
`SELECT id FROM ci_runs WHERE status IN ('pending','running','queued')`)
// StopRepo cancels the repo's active runs, waits for them to clean up, and
// deletes their artifacts and cache volumes. Call it before the repo row goes:
// the run ids come from the database.
func (r *Runner) StopRepo(ctx context.Context, repoID int64, repoName string) {
ids, err := r.queryIDs(ctx, `SELECT id FROM ci_runs WHERE repo_id = ?`, repoID)
if err != nil {
return err
log.Printf("[ci] stop repo %s: %v", repoName, err)
return
}
for _, id := range ids {
r.CancelRun(ctx, id)
}
// A cancelled run removes its container before it frees its slot, and a
// volume cannot be deleted while a container holds it.
for deadline := time.Now().Add(30 * time.Second); time.Now().Before(deadline); {
if !slices.ContainsFunc(ids, r.Active) {
break
}
time.Sleep(100 * time.Millisecond)
}
for _, id := range ids {
os.RemoveAll(r.artifactDir(id))
}
if _, err := r.PurgeRepoCaches(ctx, repoName); err != nil && !errors.Is(err, errNoSocket) {
log.Printf("[ci] purge caches of %s: %v", repoName, err)
}
}
func (r *Runner) queryIDs(ctx context.Context, q string, args ...any) ([]int64, error) {
rows, err := r.db.QueryContext(ctx, q, args...)
if err != nil {
return nil, err
}
defer rows.Close()
var ids []int64
for rows.Next() {
var id int64
if err := rows.Scan(&id); err != nil {
rows.Close()
return err
return nil, err
}
ids = append(ids, id)
}
rows.Close()
if err := rows.Err(); err != nil {
return ids, rows.Err()
}
// CancelStaleRuns cleans up runs a restart interrupted. Their containers are
// force-removed and the rows read "cancelled", so they can be retried.
func (r *Runner) CancelStaleRuns(ctx context.Context) error {
ids, err := r.queryIDs(ctx, `SELECT id FROM ci_runs WHERE status IN ('pending','running','queued')`)
if err != nil {
return err
}
// By name too: containers from before the label existed carry none.
for _, id := range ids {
r.removeContainer(ctx, containerName(id))
}
if err := r.removeLabelledContainers(ctx); err != nil && !errors.Is(err, errNoSocket) {
log.Printf("[ci] sweep containers: %v", err)
}
now := db.NowISO()
r.execSQL(ctx, `UPDATE ci_runs SET status = 'cancelled', finished_at = ?
WHERE status IN ('pending','running','queued')`, now)
@@ -359,25 +409,15 @@ func (r *Runner) artifactDir(runID int64) string {
return filepath.Join(r.cfg.CIArtifactsDir(), strconv.FormatInt(runID, 10))
}
// pruneHistory keeps only the newest CI_MAX_HISTORY runs of a repository.
// pruneHistory keeps only the CI_MAX_HISTORY most recently finished runs of
// a repository. By finish time, not id: a retried old run counts as recent.
func (r *Runner) pruneHistory(ctx context.Context, repoID int64) {
// Active runs never get pruned: their row is what the queue and the
// finish path update.
rows, err := r.db.QueryContext(ctx,
ids, err := r.queryIDs(ctx,
`SELECT id FROM ci_runs WHERE repo_id = ?
AND status NOT IN ('pending','running','queued') ORDER BY id DESC`, repoID)
if err != nil {
return
}
var ids []int64
for rows.Next() {
var id int64
if rows.Scan(&id) == nil {
ids = append(ids, id)
}
}
rows.Close()
if len(ids) <= r.cfg.CIMaxHistory {
AND status NOT IN ('pending','running','queued') ORDER BY finished_at DESC, id DESC`, repoID)
if err != nil || len(ids) <= r.cfg.CIMaxHistory {
return
}
toDelete := ids[r.cfg.CIMaxHistory:]
@@ -424,7 +464,8 @@ func buildEnvVars(runID int64, repoName, baseURL, host string, run runRow, cfg *
set("CI_COMMIT_BRANCH", run.CommitBranch)
set("CI_COMMIT_TAG", run.CommitTag)
set("CI_COMMIT_REF_NAME", refName)
set("CI_REGISTRY", host+"/"+repoName)
// Image names must be lowercase. The registry lowers the repo name too.
set("CI_REGISTRY", host+"/"+strings.ToLower(repoName))
for _, name := range cfg.VariableOrder {
if def, ok := cfg.Variables[name]; ok && def.Default != "" {
@@ -453,7 +494,10 @@ func buildEnvVars(runID int64, repoName, baseURL, host string, run runRow, cfg *
// maskSecrets replaces secret values in a log. It is a plain text match: a
// secret printed in encoded or split form is not caught.
func maskSecrets(text string, secrets []string) string {
for _, s := range secrets {
// Longest first: a secret that contains a shorter one would otherwise
// be left partly visible.
sorted := slices.SortedFunc(slices.Values(secrets), func(a, b string) int { return len(b) - len(a) })
for _, s := range sorted {
if s != "" {
text = strings.ReplaceAll(text, s, "[MASKED]")
}
@@ -461,6 +505,30 @@ func maskSecrets(text string, secrets []string) string {
return text
}
// maskLog masks a step log. A partial log, or one cut at the size limit, may
// end in the first half of a secret, so a tail that starts one is dropped.
func maskLog(text string, secrets []string, partial bool) string {
body, truncated := strings.CutSuffix(text, logTruncatedNotice)
body = maskSecrets(body, secrets)
if !partial && !truncated {
return body
}
cut := 0
for _, s := range secrets {
for k := min(len(s)-1, len(body)); k > cut; k-- {
if strings.HasSuffix(body, s[:k]) {
cut = k
break
}
}
}
body = body[:len(body)-cut]
if truncated {
body += logTruncatedNotice
}
return body
}
type secret struct{ name, value string }
type runRow struct {
Minternal/ci/run_test.go
@@ -4,6 +4,7 @@ import (
"context"
"errors"
"path/filepath"
"slices"
"testing"
"hearthforge/internal/config"
@@ -104,3 +105,64 @@ func TestFinishSlotKeepsForeignTask(t *testing.T) {
t.Fatal("finishSlot did not release its own slot")
}
}
func TestMaskLog(t *testing.T) {
secrets := []string{"abc", "abcdef"}
if got := maskLog("x abcdef y", secrets, false); got != "x [MASKED] y" {
t.Errorf("longest first: got %q", got)
}
if got := maskLog("x ab", secrets, false); got != "x ab" {
t.Errorf("complete log: got %q", got)
}
if got := maskLog("x abcdefg ab", secrets, true); got != "x [MASKED]g " {
t.Errorf("partial log: got %q", got)
}
if got := maskLog("x ab"+logTruncatedNotice, secrets, false); got != "x "+logTruncatedNotice {
t.Errorf("truncated log: got %q", got)
}
}
func TestRegistryIsLowercase(t *testing.T) {
env, _ := buildEnvVars(1, "MyRepo", "", "git.example.com", runRow{}, &Config{}, nil, nil)
if !slices.Contains(env, "CI_REGISTRY=git.example.com/myrepo") {
t.Fatalf("env = %v", env)
}
}
// A retried old run finishes last, so it must survive the prune.
func TestPruneHistoryKeepsRecentlyFinished(t *testing.T) {
r, d := newRunnerDB(t)
r.cfg.CIMaxHistory = 2
var ids []int64
for i, finished := range []string{"2026-01-04", "2026-01-02", "2026-01-03"} {
ids = append(ids, insertRun(t, d, "success"))
if _, err := d.ExecContext(context.Background(),
`UPDATE ci_runs SET finished_at = ? WHERE id = ?`, finished, ids[i]); err != nil {
t.Fatal(err)
}
}
r.pruneHistory(context.Background(), 1)
var left []int64
rows, err := d.QueryContext(context.Background(), `SELECT id FROM ci_runs ORDER BY id`)
if err != nil {
t.Fatal(err)
}
defer rows.Close()
for rows.Next() {
var id int64
rows.Scan(&id)
left = append(left, id)
}
if !slices.Equal(left, []int64{ids[0], ids[2]}) {
t.Fatalf("kept %v, want %v", left, []int64{ids[0], ids[2]})
}
}
func TestRetryRunRejectsRunStillCleaningUp(t *testing.T) {
r, d := newRunnerDB(t)
id := insertRun(t, d, "cancelled")
r.running[id] = &task{}
if err := r.RetryRun(context.Background(), id, 1); !errors.Is(err, ErrRunNotFinished) {
t.Fatalf("got %v", err)
}
}
Minternal/config/config.go
@@ -9,6 +9,7 @@ import (
"os"
"path/filepath"
"strconv"
"strings"
)
type Config struct {
@@ -143,8 +144,17 @@ func Load() (*Config, error) {
return nil, fmt.Errorf("BASE_URL %q is not a valid URL", c.BaseURL)
}
c.PublicHTTPS = u.Scheme == "https"
c.PublicOrigin = u.Scheme + "://" + u.Host
c.PublicHost = u.Host
// Browsers send Origin in this canonical form. The CSRF check and
// WebAuthn compare it verbatim.
host := strings.ToLower(u.Host)
switch u.Scheme {
case "https":
host = strings.TrimSuffix(host, ":443")
case "http":
host = strings.TrimSuffix(host, ":80")
}
c.PublicOrigin = u.Scheme + "://" + host
c.PublicHost = host
c.CommitterEmail = strEnv("COMMITTER_EMAIL", owner+"@"+u.Hostname())
return c, nil
}
Ainternal/config/config_test.go
@@ -0,0 +1,22 @@
package config
import "testing"
func TestPublicOriginIsCanonical(t *testing.T) {
for base, want := range map[string]string{
"HTTPS://Forge.Example.COM:443/": "https://forge.example.com",
"http://Forge.example.com:80": "http://forge.example.com",
"http://localhost:3000": "http://localhost:3000",
"https://[::1]:443": "https://[::1]",
"https://forge.example.com:8443": "https://forge.example.com:8443",
} {
t.Setenv("BASE_URL", base)
c, err := Load()
if err != nil {
t.Fatal(err)
}
if c.PublicOrigin != want {
t.Errorf("BASE_URL %q: PublicOrigin = %q, want %q", base, c.PublicOrigin, want)
}
}
}
Minternal/db/issues.go
@@ -347,10 +347,14 @@ func scanReactions(rows *sql.Rows) ([]Reaction, error) {
func (d *DB) ToggleIssueReaction(ctx context.Context, issueID int64, commentID *int64,
userID int64, emoji string,
) error {
return d.toggleReaction(ctx, "issue_reactions", "issue_id", issueID, commentID, userID, emoji)
return d.toggleReaction(ctx, "issue_reactions", "issue_comments", "issue_id", issueID, commentID, userID, emoji)
}
func (d *DB) toggleReaction(ctx context.Context, table, parentColumn string, parentID int64,
// ErrCommentNotFound reports a reaction on a comment that is not part of the
// target issue or patch.
var ErrCommentNotFound = errors.New("comment not found")
func (d *DB) toggleReaction(ctx context.Context, table, commentTable, parentColumn string, parentID int64,
commentID *int64, userID int64, emoji string,
) error {
tx, err := d.BeginTx(ctx, nil)
@@ -359,6 +363,19 @@ func (d *DB) toggleReaction(ctx context.Context, table, parentColumn string, par
}
defer tx.Rollback()
if commentID != nil {
var one int
err := tx.QueryRowContext(ctx,
`SELECT 1 FROM `+commentTable+` WHERE id = ? AND `+parentColumn+` = ?`,
*commentID, parentID).Scan(&one)
if errors.Is(err, sql.ErrNoRows) {
return ErrCommentNotFound
}
if err != nil {
return err
}
}
commentCond := `comment_id IS NULL`
args := []any{parentID}
if commentID != nil {
Minternal/db/password.go
@@ -24,13 +24,23 @@ const (
var errBadHash = errors.New("invalid argon2 hash")
// argonSlots caps parallel argon2 runs. Each run allocates 64 MiB, so a
// burst of logins could otherwise exhaust memory.
var argonSlots = make(chan struct{}, 2)
func idKey(pw, salt []byte, time, memory uint32, threads uint8, keyLen uint32) []byte {
argonSlots <- struct{}{}
defer func() { <-argonSlots }()
return argon2.IDKey(pw, salt, time, memory, threads, keyLen)
}
// HashPassword returns a PHC-format argon2id hash string.
func HashPassword(pw string) (string, error) {
salt := make([]byte, argonSaltLen)
if _, err := rand.Read(salt); err != nil {
return "", err
}
key := argon2.IDKey([]byte(pw), salt, argonTime, argonMemory, argonThreads, argonKeyLen)
key := idKey([]byte(pw), salt, argonTime, argonMemory, argonThreads, argonKeyLen)
b64 := base64.RawStdEncoding
return fmt.Sprintf("$argon2id$v=%d$m=%d,t=%d,p=%d$%s$%s",
argon2.Version, argonMemory, argonTime, argonThreads,
@@ -80,6 +90,6 @@ func VerifyPassword(hash, pw string) (bool, error) {
if err != nil {
return false, errBadHash
}
got := argon2.IDKey([]byte(pw), salt, time, memory, threads, uint32(len(want)))
got := idKey([]byte(pw), salt, time, memory, threads, uint32(len(want)))
return subtle.ConstantTimeCompare(got, want) == 1, nil
}
Ainternal/db/password_test.go
@@ -0,0 +1,28 @@
package db
import (
"testing"
"time"
)
func TestVerifyPasswordWaitsForArgonSlot(t *testing.T) {
for range cap(argonSlots) {
argonSlots <- struct{}{}
}
done := make(chan bool)
go func() {
ok, _ := VerifyPassword(npmHash, "testpass123")
done <- ok
}()
select {
case <-done:
t.Fatal("VerifyPassword ran while every slot was taken")
case <-time.After(100 * time.Millisecond):
}
for range cap(argonSlots) {
<-argonSlots
}
if !<-done {
t.Fatal("VerifyPassword rejected the right password")
}
}
Minternal/db/patches.go
@@ -316,7 +316,7 @@ func (d *DB) ListPatchReactions(ctx context.Context, patchID int64) ([]Reaction,
func (d *DB) TogglePatchReaction(ctx context.Context, patchID int64, commentID *int64,
userID int64, emoji string,
) error {
return d.toggleReaction(ctx, "patch_reactions", "patch_id", patchID, commentID, userID, emoji)
return d.toggleReaction(ctx, "patch_reactions", "patch_comments", "patch_id", patchID, commentID, userID, emoji)
}
func (d *DB) PatchLabels(ctx context.Context, patchID int64) ([]Label, error) {
Minternal/db/sessions.go
@@ -12,6 +12,8 @@ type SessionUser struct {
Username string
IsAdmin bool
AvatarVersion int64
// SessionCreatedAt is when this session signed in, in ISOLayout.
SessionCreatedAt string
}
func (d *DB) CreateSession(ctx context.Context, id string, userID int64, expiresAt, createdAt string) error {
@@ -26,6 +28,12 @@ func (d *DB) DeleteSession(ctx context.Context, id string) error {
return err
}
// DeleteOtherSessions signs a user out everywhere except the session keepID.
func (d *DB) DeleteOtherSessions(ctx context.Context, userID int64, keepID string) error {
_, err := d.ExecContext(ctx, `DELETE FROM sessions WHERE user_id = ? AND id != ?`, userID, keepID)
return err
}
func (d *DB) DeleteExpiredSessions(ctx context.Context) error {
// expires_at is stored in the ISO form NowISO writes. datetime('now') uses
// a space instead of the T, which compares wrong for the same day.
@@ -47,7 +55,7 @@ func (d *DB) SessionUserAllowPending(ctx context.Context, sessionID, now string)
}
func (d *DB) sessionUser(ctx context.Context, sessionID, now string, allowPending bool) (*SessionUser, error) {
query := `SELECT users.id, users.username, users.avatar_version
query := `SELECT users.id, users.username, users.avatar_version, sessions.created_at
FROM sessions
JOIN users ON users.id = sessions.user_id
WHERE sessions.id = ? AND sessions.expires_at > ?`
@@ -55,7 +63,7 @@ func (d *DB) sessionUser(ctx context.Context, sessionID, now string, allowPendin
query += ` AND users.is_pending = 0`
}
var u SessionUser
err := d.QueryRowContext(ctx, query, sessionID, now).Scan(&u.ID, &u.Username, &u.AvatarVersion)
err := d.QueryRowContext(ctx, query, sessionID, now).Scan(&u.ID, &u.Username, &u.AvatarVersion, &u.SessionCreatedAt)
if errors.Is(err, sql.ErrNoRows) {
return nil, nil
}
Minternal/gitcmd/gitcmd.go
@@ -41,6 +41,7 @@ var (
ErrBadRef = errors.New("ref does not resolve")
ErrConflict = errors.New("patch does not apply")
ErrRefChanged = errors.New("ref changed concurrently")
ErrTooLarge = errors.New("output too large")
)
var validRepoName = regexp.MustCompile(`^[a-zA-Z0-9._-]+$`)
@@ -144,10 +145,37 @@ func (g *Git) lock(name string) *sync.Mutex {
type runOpts struct {
extraEnv []string // appended to the sanitized env
stdin []byte
maxOut int64 // when > 0, more stdout kills git and returns ErrTooLarge
}
// cappedWriter kills the process on overflow. Without the kill, git blocks
// on a full pipe and Wait never returns.
type cappedWriter struct {
buf *bytes.Buffer
max int64
kill context.CancelFunc
over bool
}
func (w *cappedWriter) Write(p []byte) (int, error) {
if int64(w.buf.Len()+len(p)) > w.max {
w.over = true
w.kill()
return 0, ErrTooLarge
}
return w.buf.Write(p)
}
// run executes git and returns stdout. Stderr goes into the error.
func (g *Git) run(ctx context.Context, opt runOpts, args ...string) ([]byte, error) {
var out, errBuf bytes.Buffer
var capped *cappedWriter
if opt.maxOut > 0 {
var cancel context.CancelFunc
ctx, cancel = context.WithCancel(ctx)
defer cancel()
capped = &cappedWriter{buf: &out, max: opt.maxOut, kill: cancel}
}
cmd := exec.CommandContext(ctx, "git", args...)
cmd.Env = g.env
if len(opt.extraEnv) > 0 {
@@ -156,10 +184,15 @@ func (g *Git) run(ctx context.Context, opt runOpts, args ...string) ([]byte, err
if opt.stdin != nil {
cmd.Stdin = bytes.NewReader(opt.stdin)
}
var out, errBuf bytes.Buffer
cmd.Stdout = &out
if capped != nil {
cmd.Stdout = capped
}
cmd.Stderr = &errBuf
if err := cmd.Run(); err != nil {
if capped != nil && capped.over {
return nil, fmt.Errorf("git %s: %w", args[0], ErrTooLarge)
}
return out.Bytes(), fmt.Errorf("git %s: %w: %s", args[0], err, strings.TrimSpace(errBuf.String()))
}
return out.Bytes(), nil
@@ -189,18 +222,21 @@ func (g *Git) verifyArgs() []string {
type SigStatus string
const (
SigGood SigStatus = "good"
SigBad SigStatus = "bad"
SigNone SigStatus = "none"
SigGood SigStatus = "good"
SigBad SigStatus = "bad"
SigUnverified SigStatus = "unverified"
SigNone SigStatus = "none"
)
// parseSigStatus maps git's %G? codes onto the three badges.
// parseSigStatus maps git's %G? codes onto the badges.
func parseSigStatus(code string) SigStatus {
switch code {
case "G", "X", "Y", "R":
case "G":
return SigGood
case "B", "U", "E":
case "B", "R":
return SigBad
case "U", "X", "Y", "E":
return SigUnverified
}
return SigNone
}
@@ -386,7 +422,7 @@ func (g *Git) LsTree(ctx context.Context, name, ref, subpath string) ([]TreeEntr
}
// A trailing `--` with no pathspec means "match nothing" to ls-tree,
// so only add the separator when there is a path.
args := []string{"-C", p, "ls-tree", "--long", "--end-of-options", ref}
args := []string{"-C", p, "ls-tree", "--long", "-z", "--end-of-options", ref}
if subpath != "" {
if !ValidPath(subpath) {
return nil, fmt.Errorf("%q: %w", subpath, ErrInvalidRef)
@@ -399,7 +435,7 @@ func (g *Git) LsTree(ctx context.Context, name, ref, subpath string) ([]TreeEntr
}
prefix := subpath + "/"
var entries []TreeEntry
for _, line := range splitLines(out) {
for _, line := range strings.Split(out, "\x00") {
// format: <mode> SP <type> SP <object> SP <size> TAB <file>
tab := strings.IndexByte(line, '\t')
if tab < 0 {
@@ -441,8 +477,9 @@ func (g *Git) Show(ctx context.Context, name, ref, filePath string) ([]byte, err
return out, nil
}
// Diff returns the patch text for one commit.
func (g *Git) Diff(ctx context.Context, name, sha string) (string, error) {
// Diff returns the patch text for one commit. Output larger than maxBytes
// stops git and returns ErrTooLarge, so a huge commit is never buffered.
func (g *Git) Diff(ctx context.Context, name, sha string, maxBytes int64) (string, error) {
p, err := g.repoDir(name)
if err != nil {
return "", err
@@ -450,26 +487,41 @@ func (g *Git) Diff(ctx context.Context, name, sha string) (string, error) {
if !ValidRef(sha) {
return "", fmt.Errorf("%q: %w", sha, ErrInvalidRef)
}
return g.text(ctx, "-C", p, "diff-tree", "--no-commit-id", "-r", "-p", "-M", "--root", "--end-of-options", sha, "--")
out, err := g.run(ctx, runOpts{maxOut: maxBytes}, "-C", p, "diff-tree", "--no-commit-id", "-r", "-p", "-M", "--root",
"--end-of-options", sha, "--")
return string(out), err
}
// BlobSize returns the size of a blob object, 0 for the all-zero hash.
func (g *Git) BlobSize(ctx context.Context, name, hash string) (int64, error) {
// BlobSizes returns the sizes of the given blob ids in one git call. Ids that
// do not resolve, like the all-zero id, are missing from the map.
func (g *Git) BlobSizes(ctx context.Context, name string, hashes []string) (map[string]int64, error) {
p, err := g.repoDir(name)
if err != nil {
return 0, err
}
if strings.Trim(hash, "0") == "" {
return 0, nil
return nil, err
}
if !ValidRef(hash) {
return 0, fmt.Errorf("%q: %w", hash, ErrInvalidRef)
var in strings.Builder
for _, h := range hashes {
if !ValidRef(h) {
return nil, fmt.Errorf("%q: %w", h, ErrInvalidRef)
}
in.WriteString(h + "\n")
}
out, err := g.line(ctx, "-C", p, "cat-file", "-s", "--end-of-options", hash)
out, err := g.run(ctx, runOpts{stdin: []byte(in.String())}, "-C", p, "cat-file", "--batch-check")
if err != nil {
return 0, fmt.Errorf("cat-file: %w", err)
return nil, fmt.Errorf("cat-file: %w", err)
}
return strconv.ParseInt(out, 10, 64)
// One output line per input line, in input order.
sizes := map[string]int64{}
for i, line := range strings.Split(strings.TrimSuffix(string(out), "\n"), "\n") {
f := strings.Fields(line)
if i >= len(hashes) || len(f) != 3 {
continue
}
if n, err := strconv.ParseInt(f[2], 10, 64); err == nil {
sizes[hashes[i]] = n
}
}
return sizes, nil
}
// FileSize returns the size of the blob at ref:filePath. A path that is not a
@@ -482,19 +534,36 @@ func (g *Git) FileSize(ctx context.Context, name, ref, filePath string) (int64,
if !ValidRef(ref) || !ValidPath(filePath) {
return 0, ErrInvalidRef
}
// --batch-check reports the type as well as the size in one process.
// Without the type a directory would answer with the tree's size, and
// the streaming readers would then send an empty body.
out, err := g.run(ctx, runOpts{stdin: []byte(ref + ":" + filePath + "\n")},
"-C", p, "cat-file", "--batch-check")
if err != nil {
_, typ, size, err := g.objectInfo(ctx, p, ref, filePath)
if err != nil || typ != "blob" {
return 0, fmt.Errorf("%s:%s: %w", ref, filePath, ErrNotFound)
}
fields := strings.Fields(string(out))
if len(fields) != 3 || fields[1] != "blob" {
return 0, fmt.Errorf("%s:%s: %w", ref, filePath, ErrNotFound)
return size, nil
}
// objectInfo returns the id, type, and size of the object at rev:filePath.
// All are empty when rev is empty or the path does not exist there.
func (g *Git) objectInfo(ctx context.Context, p, rev, filePath string) (id, typ string, size int64, err error) {
if rev == "" {
return "", "", 0, nil
}
out, err := g.run(ctx, runOpts{stdin: []byte(rev + ":" + filePath + "\n")},
"-C", p, "cat-file", "--batch-check=%(objectname) %(objecttype) %(objectsize)")
if err != nil {
return "", "", 0, err
}
line := strings.TrimSpace(string(out))
if strings.HasSuffix(line, " missing") {
return "", "", 0, nil
}
f := strings.Fields(line)
if len(f) != 3 {
return "", "", 0, fmt.Errorf("cat-file: unexpected output %q", line)
}
return strconv.ParseInt(fields[2], 10, 64)
size, err = strconv.ParseInt(f[2], 10, 64)
return f[0], f[1], size, err
}
// cachedRefs serves a ref list from cache, or fills it via load.
Minternal/gitcmd/gitcmd_test.go
@@ -193,7 +193,7 @@ func TestWriteOpsAndSigning(t *testing.T) {
}
who := Ident{Name: "Bot", Email: "bot@example.com"}
sha, err := g.EditFile(ctx, repo, "main", "", "docs/new.txt", []byte("new\n"), "add file", who)
sha, err := g.EditFile(ctx, repo, "main", "", "", "docs/new.txt", []byte("new\n"), "add file", who)
if err != nil {
t.Fatal(err)
}
@@ -205,7 +205,7 @@ func TestWriteOpsAndSigning(t *testing.T) {
t.Fatalf("expected signed commit %s, got %+v", sha, commits[0])
}
if _, err := g.MoveFile(ctx, repo, "main", "docs/new.txt", "docs/moved.txt", "move", who); err != nil {
if _, err := g.EditFile(ctx, repo, "main", "", "docs/new.txt", "docs/moved.txt", []byte("new\n"), "move", who); err != nil {
t.Fatal(err)
}
entries, err := g.LsTree(ctx, repo, "main", "docs")
@@ -216,7 +216,7 @@ func TestWriteOpsAndSigning(t *testing.T) {
t.Fatalf("after move: %+v", entries)
}
if _, err := g.DeleteFile(ctx, repo, "main", "docs/moved.txt", "delete", who); err != nil {
if _, err := g.DeleteFile(ctx, repo, "main", "", "docs/moved.txt", "delete", who); err != nil {
t.Fatal(err)
}
if _, err := g.Show(ctx, repo, "main", "docs/moved.txt"); err == nil {
@@ -349,3 +349,54 @@ func TestExtractPatchMetaFoldedEncodedSubject(t *testing.T) {
t.Fatalf("subject = %q", got)
}
}
func TestParseSigStatus(t *testing.T) {
for code, want := range map[string]SigStatus{
"G": SigGood, "B": SigBad, "R": SigBad,
"U": SigUnverified, "X": SigUnverified, "Y": SigUnverified, "E": SigUnverified,
"N": SigNone, "": SigNone,
} {
if got := parseSigStatus(code); got != want {
t.Errorf("parseSigStatus(%q) = %q, want %q", code, got, want)
}
}
}
func TestLsTreeNonASCIIName(t *testing.T) {
g, repo := newSigningGit(t)
if _, err := g.EditFile(context.Background(), repo, "main", "", "", "dir/ä ö.txt", []byte("x\n"), "add", testWho); err != nil {
t.Fatal(err)
}
entries, err := g.LsTree(context.Background(), repo, "main", "dir")
if err != nil {
t.Fatal(err)
}
if len(entries) != 1 || entries[0].Name != "ä ö.txt" || entries[0].Size != "2" {
t.Fatalf("entries = %+v", entries)
}
}
func TestDiffLimitAndBlobSizes(t *testing.T) {
g, repo := newSigningGit(t)
ctx := context.Background()
sha, err := g.EditFile(ctx, repo, "main", "", "", "big.txt", []byte(strings.Repeat("line\n", 1000)), "add", testWho)
if err != nil {
t.Fatal(err)
}
if _, err := g.Diff(ctx, repo, sha, 1000); !errors.Is(err, ErrTooLarge) {
t.Fatalf("want ErrTooLarge, got %v", err)
}
out, err := g.Diff(ctx, repo, sha, 1<<20)
if err != nil || !strings.Contains(out, "+++ b/big.txt") {
t.Fatalf("diff = %q, %v", out, err)
}
hello, _ := g.run(ctx, runOpts{}, "-C", g.RepoPath(repo), "rev-parse", "main:hello.txt")
blob := strings.TrimSpace(string(hello))
sizes, err := g.BlobSizes(ctx, repo, []string{blob[:7], "deadbeef"})
if err != nil {
t.Fatal(err)
}
if len(sizes) != 1 || sizes[blob[:7]] != int64(len("hello archive\n")) {
t.Fatalf("sizes = %v", sizes)
}
}
Minternal/gitcmd/sync.go
@@ -87,8 +87,18 @@ func (g *Git) ConvertNonBareRepos() {
if !e.IsDir() {
continue
}
if strings.HasPrefix(e.Name(), ".") && strings.HasSuffix(e.Name(), bareTmpSuffix) {
g.recoverBareTmp(e.Name())
continue
}
dir := filepath.Join(g.cfg.ReposDir(), e.Name())
if st, err := os.Stat(filepath.Join(dir, ".git")); err != nil || !st.IsDir() {
// Lstat, because the removal of the work tree must never follow a
// symlinked .git into the real git data.
st, err := os.Lstat(filepath.Join(dir, ".git"))
if err != nil || !st.IsDir() {
if err == nil && st.Mode()&os.ModeSymlink != 0 {
log.Printf("Skipping bare conversion of %s: .git is a symlink", e.Name())
}
continue
}
if err := g.convertNonBareRepo(e.Name(), dir); err != nil {
@@ -109,7 +119,7 @@ func (g *Git) convertNonBareRepo(entryName, entryPath string) error {
target := filepath.Join(g.cfg.ReposDir(), baseName)
if strings.HasSuffix(entryName, ".git") {
tmp := filepath.Join(g.cfg.ReposDir(), "."+entryName+".bare_tmp")
tmp := filepath.Join(g.cfg.ReposDir(), "."+entryName+bareTmpSuffix)
if err := os.Rename(dotGit, tmp); err != nil {
return err
}
@@ -134,6 +144,25 @@ func (g *Git) convertNonBareRepo(entryName, entryPath string) error {
return nil
}
const bareTmpSuffix = ".bare_tmp"
// recoverBareTmp finishes a *.git conversion that stopped after the git data
// was moved aside to the hidden temporary name.
func (g *Git) recoverBareTmp(tmpName string) {
tmp := filepath.Join(g.cfg.ReposDir(), tmpName)
target := filepath.Join(g.cfg.ReposDir(), strings.TrimSuffix(tmpName[1:], bareTmpSuffix))
if _, err := os.Lstat(target); err == nil {
log.Printf("WARNING: %s holds the git data of %s, but %s still exists. Resolve by hand.",
tmp, filepath.Base(target), target)
return
}
if err := os.Rename(tmp, target); err != nil {
log.Printf("WARNING: could not restore %s to %s: %v", tmp, target, err)
return
}
log.Printf("Recovered interrupted bare conversion: %s", filepath.Base(target))
}
// EnsureSigningSetup generates the ssh host key if missing and writes the
// allowed_signers file used to verify commit signatures.
func (g *Git) EnsureSigningSetup() error {
Ainternal/gitcmd/sync_test.go
@@ -0,0 +1,38 @@
package gitcmd
import (
"os"
"path/filepath"
"testing"
)
func TestConvertNonBareSkipsSymlinkedDotGit(t *testing.T) {
g, repo := newTestGit(t)
real := g.RepoPath(repo)
wt := filepath.Join(g.cfg.ReposDir(), "linked")
if err := os.Mkdir(wt, 0o755); err != nil {
t.Fatal(err)
}
if err := os.Symlink(real, filepath.Join(wt, ".git")); err != nil {
t.Fatal(err)
}
g.ConvertNonBareRepos()
if _, err := os.Stat(filepath.Join(real, "HEAD")); err != nil {
t.Fatalf("real git data damaged: %v", err)
}
if _, err := os.Lstat(filepath.Join(wt, ".git")); err != nil {
t.Fatalf("symlinked repo was touched: %v", err)
}
}
func TestConvertNonBareRecoversBareTmp(t *testing.T) {
g, repo := newTestGit(t)
tmp := filepath.Join(g.cfg.ReposDir(), ".x.git"+bareTmpSuffix)
if err := os.Rename(g.RepoPath(repo), tmp); err != nil {
t.Fatal(err)
}
g.ConvertNonBareRepos()
if _, err := os.Stat(filepath.Join(g.cfg.ReposDir(), "x.git", "HEAD")); err != nil {
t.Fatalf("not recovered: %v", err)
}
}
Minternal/gitcmd/write.go
@@ -7,6 +7,7 @@ import (
"mime"
"os"
"os/exec"
"path/filepath"
"strings"
"hearthforge/internal/util"
@@ -29,15 +30,12 @@ func withTempDir(prefix string, fn func(dir string) error) error {
return fn(dir)
}
// gitArgs builds the leading arguments. work is the dummy work tree that
// `update-index --remove` insists on even in a bare repo.
func gitArgs(p, work string, rest ...string) []string {
var args []string
if work != "" {
args = append(args, "--work-tree="+work)
}
args = append(args, "-C", p)
return append(args, rest...)
// idx runs git in repo p against a private index file inside work. work is
// the dummy work tree that `update-index` insists on even in a bare repo.
// Writes never share the repo index, which may hold stale entries.
func (g *Git) idx(ctx context.Context, p, work string, opt runOpts, rest ...string) ([]byte, error) {
opt.extraEnv = append(opt.extraEnv, "GIT_INDEX_FILE="+filepath.Join(work, "index"))
return g.run(ctx, opt, append([]string{"--work-tree=" + work, "-C", p}, rest...)...)
}
// treeFileMode returns the 6-digit octal mode of a path at a ref, or "" when
@@ -60,12 +58,25 @@ func (g *Git) treeFileMode(ctx context.Context, p, ref, filePath string) string
return fields[0]
}
// branchRef returns the full ref of a branch as Branches lists it. Branches
// prints "heads/v1" when a tag v1 exists too, so "refs/heads/"+branch would
// name a different ref.
func (g *Git) branchRef(ctx context.Context, p, branch string) string {
out, err := g.line(ctx, "-C", p, "rev-parse", "--verify", "--quiet", "--symbolic-full-name",
"--end-of-options", branch)
if err == nil && strings.HasPrefix(out, "refs/heads/") {
return out
}
return "refs/heads/" + branch
}
func (g *Git) writeTree(ctx context.Context, p, work string) (string, error) {
return g.line(ctx, gitArgs(p, work, "write-tree")...)
out, err := g.idx(ctx, p, work, runOpts{}, "write-tree")
return strings.TrimSpace(string(out)), err
}
func (g *Git) readTree(ctx context.Context, p, work, ref string) error {
_, err := g.run(ctx, runOpts{}, gitArgs(p, work, "read-tree", "--end-of-options", ref)...)
_, err := g.idx(ctx, p, work, runOpts{}, "read-tree", "--end-of-options", ref)
return err
}
@@ -152,14 +163,12 @@ func (g *Git) CheckPatch(ctx context.Context, name, patch string) (ApplyResult,
}
res := ApplyResult{Status: "conflict"}
err = withTempDir("patch", func(dir string) error {
env := []string{"GIT_INDEX_FILE=" + dir + "/index"}
// A bare repo has no work tree, so seed the index from HEAD and check
// against objects with --cached.
if _, err := g.run(ctx, runOpts{extraEnv: env}, "-C", p, "read-tree", "HEAD"); err != nil {
if err := g.readTree(ctx, p, dir, "HEAD"); err != nil {
return err
}
out, err := g.run(ctx, runOpts{extraEnv: env, stdin: []byte(patch)},
"-C", p, "apply", "--check", "--cached")
out, err := g.idx(ctx, p, dir, runOpts{stdin: []byte(patch)}, "apply", "--check", "--cached")
if err != nil {
res.Output = err.Error()
return nil
@@ -181,17 +190,20 @@ func (g *Git) ApplyPatch(ctx context.Context, name, patch string, author, commit
if err != nil {
return err
}
if err := g.readTree(ctx, p, "", parent); err != nil {
return err
}
if _, err := g.run(ctx, runOpts{stdin: []byte(patch)}, "-C", p, "apply", "--cached"); err != nil {
return fmt.Errorf("%w: %s", ErrConflict, err)
}
tree, err := g.writeTree(ctx, p, "")
if err != nil {
err = withTempDir("patch", func(work string) error {
if err := g.readTree(ctx, p, work, parent); err != nil {
return err
}
if _, err := g.idx(ctx, p, work, runOpts{stdin: []byte(patch)}, "apply", "--cached"); err != nil {
return fmt.Errorf("%w: %s", ErrConflict, err)
}
tree, err := g.writeTree(ctx, p, work)
if err != nil {
return err
}
sha, err = g.commitTree(ctx, p, tree, parent, PatchCommitMessage(patch), author, committer)
return err
}
sha, err = g.commitTree(ctx, p, tree, parent, PatchCommitMessage(patch), author, committer)
})
if err != nil {
return err
}
@@ -204,10 +216,20 @@ func (g *Git) ApplyPatch(ctx context.Context, name, patch string, author, commit
return sha, err
}
// StaleError reports that the edited file changed on the branch after the
// editor loaded it. Tip is the current branch tip, which the editor offers as
// the new base.
type StaleError struct{ Tip string }
func (e *StaleError) Error() string { return "file changed on the branch (tip " + e.Tip + ")" }
// EditFile writes content at newPath on branch and commits it.
// oldPath empty means create. oldPath != newPath means rename.
func (g *Git) EditFile(ctx context.Context, name, branch, oldPath, newPath string, content []byte, message string, who Ident) (string, error) {
if !ValidRef(branch) {
// base is the commit the edit was made on. When the branch moved past base,
// the edit lands on the new tip only if oldPath is unchanged there.
// Otherwise it returns a *StaleError. An empty base skips the check.
func (g *Git) EditFile(ctx context.Context, name, branch, base, oldPath, newPath string, content []byte, message string, who Ident) (string, error) {
if !ValidRef(branch) || (base != "" && !ValidRef(base)) {
return "", fmt.Errorf("%q: %w", branch, ErrInvalidRef)
}
if !ValidPath(newPath) || (oldPath != "" && !ValidPath(oldPath)) {
@@ -215,24 +237,40 @@ func (g *Git) EditFile(ctx context.Context, name, branch, oldPath, newPath strin
}
var sha string
err := g.writeOp(name, func(p string) error {
branchRef := "refs/heads/" + branch
branchRef := g.branchRef(ctx, p, branch)
parent, err := g.branchTip(ctx, p, branchRef)
if err != nil {
return err
}
if parent == "" && oldPath != "" {
return fmt.Errorf("branch %q: %w", branch, ErrNotFound)
}
// A create has no old file. The check below keeps its path free.
if base != "" && base != parent && oldPath != "" {
if err := g.checkUnchanged(ctx, p, base, parent, oldPath); err != nil {
return err
}
}
if newPath != oldPath {
if _, typ, _, err := g.objectInfo(ctx, p, parent, newPath); err != nil {
return err
} else if typ != "" {
return fmt.Errorf("path %q: %w", newPath, ErrExists)
}
}
return withTempDir("edit", func(work string) error {
if parent != "" {
if err := g.readTree(ctx, p, work, branchRef); err != nil {
if err := g.readTree(ctx, p, work, parent); err != nil {
return err
}
}
mode := "100644"
if oldPath != "" {
if m := g.treeFileMode(ctx, p, branchRef, oldPath); m != "" {
if m := g.treeFileMode(ctx, p, parent, oldPath); m != "" {
mode = m
}
if oldPath != newPath {
if _, err := g.run(ctx, runOpts{}, gitArgs(p, work, "update-index", "--remove", "--", oldPath)...); err != nil {
if _, err := g.idx(ctx, p, work, runOpts{}, "update-index", "--force-remove", "--", oldPath); err != nil {
return err
}
}
@@ -241,8 +279,8 @@ func (g *Git) EditFile(ctx context.Context, name, branch, oldPath, newPath strin
if err != nil {
return err
}
if _, err := g.run(ctx, runOpts{}, gitArgs(p, work, "update-index", "--add",
"--cacheinfo", mode+","+blob+","+newPath)...); err != nil {
if _, err := g.idx(ctx, p, work, runOpts{}, "update-index", "--add",
"--cacheinfo", mode+","+blob+","+newPath); err != nil {
return err
}
tree, err := g.writeTree(ctx, p, work)
@@ -259,18 +297,27 @@ func (g *Git) EditFile(ctx context.Context, name, branch, oldPath, newPath strin
return sha, err
}
// MoveFile renames a path on a branch, keeping its contents and mode.
func (g *Git) MoveFile(ctx context.Context, name, branch, oldPath, newPath, message string, who Ident) (string, error) {
content, err := g.Show(ctx, name, "refs/heads/"+branch, oldPath)
// checkUnchanged returns a *StaleError unless filePath is the same object at
// base and tip.
func (g *Git) checkUnchanged(ctx context.Context, p, base, tip, filePath string) error {
was, _, _, err := g.objectInfo(ctx, p, base, filePath)
if err != nil {
return "", err
return err
}
return g.EditFile(ctx, name, branch, oldPath, newPath, content, message, who)
now, _, _, err := g.objectInfo(ctx, p, tip, filePath)
if err != nil {
return err
}
if was == "" || was != now {
return &StaleError{Tip: tip}
}
return nil
}
// DeleteFile removes a path on a branch and commits it.
func (g *Git) DeleteFile(ctx context.Context, name, branch, filePath, message string, who Ident) (string, error) {
if !ValidRef(branch) {
// DeleteFile removes a file on a branch and commits it. base follows the
// EditFile rule for a branch that moved past it.
func (g *Git) DeleteFile(ctx context.Context, name, branch, base, filePath, message string, who Ident) (string, error) {
if !ValidRef(branch) || (base != "" && !ValidRef(base)) {
return "", fmt.Errorf("%q: %w", branch, ErrInvalidRef)
}
if !ValidPath(filePath) {
@@ -278,7 +325,7 @@ func (g *Git) DeleteFile(ctx context.Context, name, branch, filePath, message st
}
var sha string
err := g.writeOp(name, func(p string) error {
branchRef := "refs/heads/" + branch
branchRef := g.branchRef(ctx, p, branch)
// The parent is resolved before the tree is read, so a push that
// lands in between is caught by update-ref instead of being reverted.
parent, err := g.branchTip(ctx, p, branchRef)
@@ -288,11 +335,21 @@ func (g *Git) DeleteFile(ctx context.Context, name, branch, filePath, message st
if parent == "" {
return fmt.Errorf("branch %q: %w", branch, ErrNotFound)
}
if _, typ, _, err := g.objectInfo(ctx, p, parent, filePath); err != nil {
return err
} else if typ != "blob" {
return fmt.Errorf("file %q: %w", filePath, ErrNotFound)
}
if base != "" && base != parent {
if err := g.checkUnchanged(ctx, p, base, parent, filePath); err != nil {
return err
}
}
return withTempDir("del", func(work string) error {
if err := g.readTree(ctx, p, work, parent); err != nil {
return err
}
if _, err := g.run(ctx, runOpts{}, gitArgs(p, work, "update-index", "--remove", "--", filePath)...); err != nil {
if _, err := g.idx(ctx, p, work, runOpts{}, "update-index", "--force-remove", "--", filePath); err != nil {
return err
}
tree, err := g.writeTree(ctx, p, work)
@@ -315,7 +372,8 @@ func (g *Git) CreateBranch(ctx context.Context, name, branch, sourceRef string)
return fmt.Errorf("%q: %w", branch, ErrInvalidRef)
}
return g.writeOp(name, func(p string) error {
sha, err := g.ResolveRef(ctx, name, sourceRef)
// ^{commit} peels an annotated tag.
sha, err := g.ResolveRef(ctx, name, sourceRef+"^{commit}")
if err != nil {
return err
}
@@ -333,11 +391,12 @@ func (g *Git) DeleteBranch(ctx context.Context, name, branch string) error {
return fmt.Errorf("%q: %w", branch, ErrInvalidRef)
}
return g.writeOp(name, func(p string) error {
if _, err := g.ResolveRef(ctx, name, "refs/heads/"+branch); err != nil {
ref := g.branchRef(ctx, p, branch)
if _, err := g.ResolveRef(ctx, name, ref); err != nil {
return fmt.Errorf("branch %q: %w", branch, ErrNotFound)
}
defer g.InvalidateRefCache(name)
_, err := g.run(ctx, runOpts{}, "-C", p, "update-ref", "-d", "refs/heads/"+branch)
_, err := g.run(ctx, runOpts{}, "-C", p, "update-ref", "-d", ref)
return err
})
}
@@ -348,7 +407,8 @@ func (g *Git) RenameBranch(ctx context.Context, name, oldName, newName string) e
return ErrInvalidRef
}
return g.writeOp(name, func(p string) error {
sha, err := g.ResolveRef(ctx, name, "refs/heads/"+oldName)
oldRef := g.branchRef(ctx, p, oldName)
sha, err := g.ResolveRef(ctx, name, oldRef)
if err != nil {
return fmt.Errorf("branch %q: %w", oldName, ErrNotFound)
}
@@ -359,7 +419,7 @@ func (g *Git) RenameBranch(ctx context.Context, name, oldName, newName string) e
if err := g.updateRef(ctx, p, "refs/heads/"+newName, sha, ""); err != nil {
return err
}
_, err = g.run(ctx, runOpts{}, "-C", p, "update-ref", "-d", "refs/heads/"+oldName)
_, err = g.run(ctx, runOpts{}, "-C", p, "update-ref", "-d", oldRef)
return err
})
}
Ainternal/gitcmd/write_test.go
@@ -0,0 +1,180 @@
package gitcmd
import (
"context"
"errors"
"os/exec"
"slices"
"testing"
)
var testWho = Ident{Name: "Bot", Email: "bot@example.com"}
// newSigningGit is newTestGit with commit signing set up.
func newSigningGit(t *testing.T) (*Git, string) {
t.Helper()
if _, err := exec.LookPath("ssh-keygen"); err != nil {
t.Skip("ssh-keygen not installed")
}
g, repo := newTestGit(t)
if err := g.EnsureSigningSetup(); err != nil {
t.Fatal(err)
}
return g, repo
}
func mustShow(t *testing.T, g *Git, repo, ref, path string) string {
t.Helper()
b, err := g.Show(context.Background(), repo, ref, path)
if err != nil {
t.Fatal(err)
}
return string(b)
}
func TestEditFileAtBase(t *testing.T) {
g, repo := newSigningGit(t)
ctx := context.Background()
if _, err := g.EditFile(ctx, repo, "main", "", "", "other.txt", []byte("one\n"), "add", testWho); err != nil {
t.Fatal(err)
}
base, _ := g.ResolveRef(ctx, repo, "main")
_, err := g.EditFile(ctx, repo, "main", "", "other.txt", "other.txt", []byte("two\n"), "push", testWho)
if err != nil {
t.Fatal(err)
}
// The concurrent change did not touch hello.txt, so the edit lands on top.
if _, err := g.EditFile(ctx, repo, "main", base, "hello.txt", "hello.txt", []byte("edited\n"), "e", testWho); err != nil {
t.Fatal(err)
}
if got := mustShow(t, g, repo, "main", "other.txt"); got != "two\n" {
t.Fatalf("pushed change reverted: %q", got)
}
// other.txt changed since base, so the edit must not commit.
tip, _ := g.ResolveRef(ctx, repo, "main")
_, err = g.EditFile(ctx, repo, "main", base, "other.txt", "other.txt", []byte("mine\n"), "e", testWho)
var stale *StaleError
if !errors.As(err, &stale) || stale.Tip != tip {
t.Fatalf("want StaleError at %s, got %v", tip, err)
}
if now, _ := g.ResolveRef(ctx, repo, "main"); now != tip {
t.Fatal("stale edit moved the branch")
}
// Saving again with the new tip as base overwrites on purpose.
if _, err := g.EditFile(ctx, repo, "main", stale.Tip, "other.txt", "other.txt", []byte("mine\n"), "e", testWho); err != nil {
t.Fatal(err)
}
}
func TestEditFileExistenceChecks(t *testing.T) {
g, repo := newSigningGit(t)
ctx := context.Background()
if _, err := g.EditFile(ctx, repo, "main", "", "", "dir/a.txt", []byte("a\n"), "add", testWho); err != nil {
t.Fatal(err)
}
if _, err := g.EditFile(ctx, repo, "main", "", "", "hello.txt", []byte("x\n"), "c", testWho); !errors.Is(err, ErrExists) {
t.Fatalf("create over file: %v", err)
}
if _, err := g.EditFile(ctx, repo, "main", "", "hello.txt", "dir/a.txt", []byte("x\n"), "r", testWho); !errors.Is(err, ErrExists) {
t.Fatalf("rename over file: %v", err)
}
if _, err := g.EditFile(ctx, repo, "main", "", "hello.txt", "dir", []byte("x\n"), "r", testWho); !errors.Is(err, ErrExists) {
t.Fatalf("rename over dir: %v", err)
}
tip, _ := g.ResolveRef(ctx, repo, "main")
for _, p := range []string{"missing.txt", "dir"} {
if _, err := g.DeleteFile(ctx, repo, "main", "", p, "d", testWho); !errors.Is(err, ErrNotFound) {
t.Fatalf("delete %s: %v", p, err)
}
}
if now, _ := g.ResolveRef(ctx, repo, "main"); now != tip {
t.Fatal("rejected write moved the branch")
}
}
func TestEditFileBranchAndTagSameName(t *testing.T) {
g, repo := newSigningGit(t)
ctx := context.Background()
p := g.RepoPath(repo)
if err := g.CreateBranch(ctx, repo, "v1", "main"); err != nil {
t.Fatal(err)
}
if _, err := g.run(ctx, runOpts{}, "-C", p, "tag", "v1", "main"); err != nil {
t.Fatal(err)
}
g.InvalidateRefCache(repo)
if branches, _ := g.Branches(ctx, repo); !slices.Contains(branches, "heads/v1") {
t.Fatalf("branches = %v", branches)
}
if _, err := g.EditFile(ctx, repo, "heads/v1", "", "", "new.txt", []byte("n\n"), "add", testWho); err != nil {
t.Fatal(err)
}
if got := mustShow(t, g, repo, "refs/heads/v1", "new.txt"); got != "n\n" {
t.Fatalf("new.txt on v1 = %q", got)
}
if got := mustShow(t, g, repo, "refs/heads/v1", "hello.txt"); got != "hello archive\n" {
t.Fatalf("hello.txt on v1 = %q", got)
}
if _, err := g.ResolveRef(ctx, repo, "refs/heads/heads/v1"); err == nil {
t.Fatal("write created refs/heads/heads/v1")
}
}
func TestEditFileNewBranchStartsEmpty(t *testing.T) {
g, repo := newSigningGit(t)
ctx := context.Background()
if _, err := g.EditFile(ctx, repo, "main", "", "hello.txt", "hello.txt", []byte("x\n"), "e", testWho); err != nil {
t.Fatal(err)
}
if _, err := g.EditFile(ctx, repo, "orphan", "", "", "only.txt", []byte("o\n"), "add", testWho); err != nil {
t.Fatal(err)
}
entries, err := g.LsTree(ctx, repo, "orphan", "")
if err != nil {
t.Fatal(err)
}
if len(entries) != 1 || entries[0].Name != "only.txt" {
t.Fatalf("orphan tree: %+v", entries)
}
}
func TestCreateBranchFromAnnotatedTag(t *testing.T) {
g, repo := newSigningGit(t)
ctx := context.Background()
if err := g.CreateTag(ctx, repo, "rel", "main", "release", Ident{Name: "T", Email: "t@e"}); err != nil {
t.Fatal(err)
}
if err := g.CreateBranch(ctx, repo, "from-tag", "rel"); err != nil {
t.Fatal(err)
}
want, _ := g.ResolveRef(ctx, repo, "main")
if got, _ := g.ResolveRef(ctx, repo, "refs/heads/from-tag"); got != want {
t.Fatalf("branch at %s, want commit %s", got, want)
}
}
func TestDeleteFileAtBase(t *testing.T) {
g, repo := newSigningGit(t)
ctx := context.Background()
if _, err := g.EditFile(ctx, repo, "main", "", "", "a.txt", []byte("a\n"), "add", testWho); err != nil {
t.Fatal(err)
}
base, _ := g.ResolveRef(ctx, repo, "main")
if _, err := g.EditFile(ctx, repo, "main", "", "a.txt", "a.txt", []byte("pushed\n"), "push", testWho); err != nil {
t.Fatal(err)
}
_, err := g.DeleteFile(ctx, repo, "main", base, "a.txt", "d", testWho)
var stale *StaleError
if !errors.As(err, &stale) {
t.Fatalf("want StaleError, got %v", err)
}
// hello.txt did not change since base, so its delete lands on the tip.
if _, err := g.DeleteFile(ctx, repo, "main", base, "hello.txt", "d", testWho); err != nil {
t.Fatal(err)
}
if got := mustShow(t, g, repo, "main", "a.txt"); got != "pushed\n" {
t.Fatalf("a.txt = %q", got)
}
}
Minternal/highlight/diff.go
@@ -76,6 +76,8 @@ type ParsedFile struct {
BinaryNewSize int64
hasOldSize bool
hasNewSize bool
oldBlob string // set for "Binary files ..." lines, which carry no size
newBlob string
hunks []parsedHunk
}
@@ -85,17 +87,43 @@ var (
hunkRE = regexp.MustCompile(`@@ -(\d+)(?:,\d+)? \+(\d+)(?:,\d+)? @@`)
)
// trimDiffPath strips the "--- " or "+++ " marker and the "a/" or "b/" path
// prefix. It never slices, so a short or truncated line is safe.
// unquotePath undoes git's C-style quoting of paths with non-ASCII or
// special bytes. Go's string escapes are a superset of git's.
func unquotePath(s string) string {
if strings.HasPrefix(s, `"`) {
if u, err := strconv.Unquote(s); err == nil {
return u
}
}
return s
}
// trimDiffPath strips the "--- " or "+++ " marker, the quoting and the "a/"
// or "b/" path prefix. It never slices, so a short or truncated line is safe.
// Git appends a TAB to names that contain a space.
func trimDiffPath(line, marker, prefix string) string {
return strings.TrimPrefix(strings.TrimPrefix(line, marker), prefix)
return strings.TrimPrefix(unquotePath(strings.TrimSuffix(strings.TrimPrefix(line, marker), "\t")), prefix)
}
// diffGitNewPath returns the "b/" path of a "diff --git" line, or "".
func diffGitNewPath(line string) string {
if strings.HasSuffix(line, `"`) {
// A quoted name escapes every inner quote, so ` "` starts the last token.
if i := strings.LastIndex(line, ` "`); i >= 0 {
return strings.TrimPrefix(unquotePath(line[i+1:]), "b/")
}
}
if m := diffGitRE.FindStringSubmatch(line); m != nil {
return m[2]
}
return ""
}
// parseDiff parses `git diff` output into per-file structures.
//
// blobSize resolves a blob SHA to its byte size, for "Binary files ..." lines
// that carry no size. It may be nil.
func parseDiff(raw string, blobSize func(sha string) int64) []ParsedFile {
// blobSizes resolves blob SHAs to byte sizes in one batch, for "Binary files
// ..." lines that carry no size. It may be nil.
func parseDiff(raw string, blobSizes func(shas []string) map[string]int64) []ParsedFile {
var files []ParsedFile
all := strings.Split(raw, "\n")
i := 0
@@ -105,10 +133,7 @@ func parseDiff(raw string, blobSize func(sha string) int64) []ParsedFile {
i++
continue
}
fallback := ""
if m := diffGitRE.FindStringSubmatch(all[i]); m != nil {
fallback = m[2]
}
fallback := diffGitNewPath(all[i])
file := ParsedFile{OldPath: fallback, NewPath: fallback, Status: StatusModified}
i++
@@ -125,14 +150,14 @@ func parseDiff(raw string, blobSize func(sha string) int64) []ParsedFile {
file.Status = StatusDeleted
case strings.HasPrefix(line, "rename from "):
file.Status = StatusRenamed
file.OldPath = line[12:]
file.OldPath = unquotePath(line[12:])
case strings.HasPrefix(line, "rename to "):
file.NewPath = line[10:]
file.NewPath = unquotePath(line[10:])
case strings.HasPrefix(line, "copy from "):
file.Status = StatusCopied
file.OldPath = line[10:]
file.OldPath = unquotePath(line[10:])
case strings.HasPrefix(line, "copy to "):
file.NewPath = line[8:]
file.NewPath = unquotePath(line[8:])
case strings.HasPrefix(line, "--- ") && line != "--- /dev/null":
file.OldPath = trimDiffPath(line, "--- ", "a/")
case strings.HasPrefix(line, "+++ ") && line != "+++ /dev/null":
@@ -143,10 +168,7 @@ func parseDiff(raw string, blobSize func(sha string) int64) []ParsedFile {
}
case strings.HasPrefix(line, "Binary files "):
file.IsBinary = true
if blobSize != nil {
file.BinaryOldSize, file.hasOldSize = blobSize(oldBlob), true
file.BinaryNewSize, file.hasNewSize = blobSize(newBlob), true
}
file.oldBlob, file.newBlob = oldBlob, newBlob
case line == "GIT binary patch":
file.IsBinary = true
case file.IsBinary && strings.HasPrefix(line, "literal "):
@@ -186,9 +208,37 @@ func parseDiff(raw string, blobSize func(sha string) int64) []ParsedFile {
}
files = append(files, file)
}
if blobSizes != nil {
fillBlobSizes(files, blobSizes)
}
return files
}
// fillBlobSizes sets the sizes of "Binary files ..." entries. A blob that
// does not resolve, like the all-zero id of an added file, counts as 0.
func fillBlobSizes(files []ParsedFile, blobSizes func(shas []string) map[string]int64) {
var shas []string
for _, f := range files {
for _, sha := range []string{f.oldBlob, f.newBlob} {
if strings.Trim(sha, "0") != "" {
shas = append(shas, sha)
}
}
}
if len(shas) == 0 {
return
}
sizes := blobSizes(shas)
for i := range files {
f := &files[i]
if f.oldBlob == "" && f.newBlob == "" {
continue
}
f.BinaryOldSize, f.hasOldSize = sizes[f.oldBlob], true
f.BinaryNewSize, f.hasNewSize = sizes[f.newBlob], true
}
}
// escapeHTMLAndCtrl escapes HTML and renders C0 control characters as caret
// notation in a visible span. TAB, LF and DEL are left alone.
func escapeHTMLAndCtrl(s string) string {
@@ -316,11 +366,11 @@ func (h *Highlighter) highlightFile(file ParsedFile) RenderedDiffFile {
// PrepareDiff parses and highlights a whole diff, with caching.
// cacheKey may be empty to skip caching.
func (h *Highlighter) PrepareDiff(raw, cacheKey string, blobSize func(sha string) int64) []RenderedDiffFile {
func (h *Highlighter) PrepareDiff(raw, cacheKey string, blobSizes func(shas []string) map[string]int64) []RenderedDiffFile {
if v, ok := h.diffs.Get(cacheKey); ok {
return v
}
parsed := parseDiff(raw, blobSize)
parsed := parseDiff(raw, blobSizes)
out := make([]RenderedDiffFile, len(parsed))
for i, f := range parsed {
out[i] = h.highlightFile(f)
Ainternal/highlight/diff_test.go
@@ -0,0 +1,48 @@
package highlight
import "testing"
// Git C-quotes non-ASCII names and appends a TAB to names with a space.
func TestParseDiffQuotedPathsAndBatchedBinarySizes(t *testing.T) {
raw := "diff --git \"a/b \\303\\274.bin\" \"b/b \\303\\274.bin\"\n" +
"new file mode 100644\n" +
"index 0000000..88768ef\n" +
"Binary files /dev/null and \"b/b \\303\\274.bin\" differ\n" +
"diff --git a/x.bin b/x.bin\n" +
"index 1111111..2222222 100644\n" +
"Binary files a/x.bin and b/x.bin differ\n" +
"diff --git \"a/d/\\303\\244 \\303\\266.txt\" \"b/d/\\303\\244 \\303\\266.txt\"\n" +
"new file mode 100644\n" +
"index 0000000..587be6b\n" +
"--- /dev/null\n" +
"+++ \"b/d/\\303\\244 \\303\\266.txt\"\t\n" +
"@@ -0,0 +1 @@\n" +
"+x\n" +
"diff --git a/sp ace.txt \"b/r\\303\\251 name.txt\"\n" +
"similarity index 100%\n" +
"rename from sp ace.txt\n" +
"rename to \"r\\303\\251 name.txt\"\n"
calls := 0
files := parseDiff(raw, func(shas []string) map[string]int64 {
calls++
return map[string]int64{"88768ef": 5, "1111111": 7, "2222222": 9}
})
if calls != 1 {
t.Fatalf("blobSizes called %d times", calls)
}
if len(files) != 4 {
t.Fatalf("got %d files", len(files))
}
if f := files[0]; f.NewPath != "b ü.bin" || !f.IsBinary || f.BinaryOldSize != 0 || f.BinaryNewSize != 5 || !f.hasNewSize {
t.Fatalf("added binary: %+v", f)
}
if f := files[1]; f.BinaryOldSize != 7 || f.BinaryNewSize != 9 {
t.Fatalf("modified binary: %+v", f)
}
if f := files[2]; f.NewPath != "d/ä ö.txt" || f.Added != 1 {
t.Fatalf("text file: %+v", f)
}
if f := files[3]; f.OldPath != "sp ace.txt" || f.NewPath != "ré name.txt" || f.Status != StatusRenamed {
t.Fatalf("rename: %+v", f)
}
}
Minternal/highlight/highlight.go
@@ -22,12 +22,14 @@ import (
const BinaryDetectBytes = 8000
const (
maxFileCache = 500
maxDiffCache = 500
maxFileCache = 500
maxDiffCache = 500
maxFileCacheBytes = 64 << 20
maxDiffCacheBytes = 64 << 20
)
// classPrefix keeps chroma token classes out of the app's own class namespace.
const classPrefix = "ch-"
// ClassPrefix keeps chroma token classes out of the app's own class namespace.
const ClassPrefix = "ch-"
// Highlighter holds the bounded render caches. Create one at startup.
type Highlighter struct {
@@ -40,11 +42,28 @@ type Highlighter struct {
func New(inlineMaxBytes int64) *Highlighter {
return &Highlighter{
inlineMaxBytes: inlineMaxBytes,
files: util.NewCache[string, FileView](maxFileCache, 0),
diffs: util.NewCache[string, []RenderedDiffFile](maxDiffCache, 0),
files: util.NewSizedCache[string](maxFileCache, maxFileCacheBytes, 0, fileViewSize),
diffs: util.NewSizedCache[string](maxDiffCache, maxDiffCacheBytes, 0, diffSize),
}
}
func fileViewSize(v FileView) int64 { return int64(len(v.HTML)) }
// diffSize approximates the memory of a rendered diff by its string bytes.
func diffSize(files []RenderedDiffFile) int64 {
var n int
for _, f := range files {
n += len(f.OldPath) + len(f.NewPath)
for _, h := range f.Hunks {
n += len(h.Header)
for _, r := range h.Rows {
n += len(r.HTML) + len(r.Type)
}
}
}
return int64(n)
}
// HasBinaryContent reports whether the first BinaryDetectBytes contain a NUL.
// Git uses the same rule. Other control characters stay text.
func HasBinaryContent(content []byte) bool {
@@ -148,7 +167,7 @@ func tokenClass(t chroma.TokenType) string {
if cls == "" {
return ""
}
return classPrefix + cls
return ClassPrefix + cls
}
t = t.Parent()
}
Minternal/markdown/markdown.go
@@ -23,7 +23,10 @@ import (
hfutil "hearthforge/internal/util"
)
const maxMDCache = 50
const (
maxMDCache = 50
maxMDCacheBytes = 32 << 20
)
// Context points relative links and images at a repo's blob and raw routes.
type Context struct {
@@ -61,7 +64,7 @@ func New() *Renderer {
parser.WithASTTransformers(util.Prioritized(taskListTransformer{}, 100)),
),
)
return &Renderer{md: md, policy: newPolicy(), cache: hfutil.NewCache[string, string](maxMDCache, 0)}
return &Renderer{md: md, policy: newPolicy(), cache: hfutil.NewSizedCache[string](maxMDCache, maxMDCacheBytes, 0, func(s string) int64 { return int64(len(s)) })}
}
// newPolicy allows what the markdown renderer emits and nothing else.
@@ -75,8 +78,11 @@ func newPolicy() *bluemonday.Policy {
p.AllowAttrs("type", "checked", "disabled").OnElements("input")
// Table cell alignment from GFM tables.
p.AllowAttrs("align").OnElements("td", "th")
// Syntax highlighting and task-list class hooks.
p.AllowAttrs("class").Globally()
// Only the classes the renderer emits. Other site classes could fake UI.
p.AllowAttrs("class").Matching(regexp.MustCompile(`^language-\S+$`)).OnElements("code")
p.AllowAttrs("class").Matching(regexp.MustCompile(`^` + highlight.ClassPrefix + `[\w-]+$`)).OnElements("span")
p.AllowAttrs("class").Matching(regexp.MustCompile(`^task-list-item$`)).OnElements("li")
p.AllowAttrs("class").Matching(regexp.MustCompile(`^task-list-item-checkbox$`)).OnElements("input")
return p
}
Minternal/markdown/markdown_test.go
@@ -202,3 +202,11 @@ func TestRenderCacheKeyIncludesRef(t *testing.T) {
t.Errorf("tag render served the cached main entry: %s", tag)
}
}
func TestSiteClassesAreStripped(t *testing.T) {
r := New()
got := r.Render(`<span class="badge badge-open">Merged</span> <div class="ch-k">x</div> <code class="language-go evil">y</code>`, "", nil)
if strings.Contains(got, "class=") {
t.Errorf("spoof class survived: %q", got)
}
}
Minternal/ratelimit/ratelimit.go
@@ -4,6 +4,7 @@ package ratelimit
import (
"net"
"net/http"
"net/netip"
"strings"
"sync"
"time"
@@ -60,28 +61,34 @@ func (l *Limiter) Allow(key string) bool {
return true
}
// Blocked reports whether key is over the limit without recording an event.
// Callers that only want to count failures check this first and call Allow
// after a failure.
func (l *Limiter) Blocked(key string) bool {
l.mu.Lock()
defer l.mu.Unlock()
b := l.buckets[key]
return b != nil && time.Now().Before(b.resetAt) && b.count >= l.max
}
// ClientIP returns the address to rate-limit on.
// With a trusted proxy it takes the first X-Forwarded-For entry, because the
// proxy appends the real client there. Otherwise the header is attacker
// controlled and only the socket address can be believed.
// ClientIP returns the rate-limit key of a request. It is never empty.
// With a trusted proxy it takes the first X-Forwarded-For entry, so the proxy
// must overwrite that header. Otherwise it uses the socket address. IPv6 is
// keyed by /64, because one client usually owns a whole /64.
func ClientIP(r *http.Request, trustedProxy bool) string {
ip := ""
if trustedProxy {
first, _, _ := strings.Cut(r.Header.Get("X-Forwarded-For"), ",")
return strings.TrimSpace(first)
ip = strings.TrimSpace(first)
}
host, _, err := net.SplitHostPort(r.RemoteAddr)
if ip == "" {
host, _, err := net.SplitHostPort(r.RemoteAddr)
if err != nil {
host = r.RemoteAddr
}
ip = host
}
addr, err := netip.ParseAddr(ip)
if err != nil {
return r.RemoteAddr
if ip == "" {
return "unknown"
}
return ip
}
addr = addr.Unmap()
if addr.Is6() {
p, _ := addr.Prefix(64)
return p.String()
}
return host
return addr.String()
}
Minternal/ratelimit/ratelimit_test.go
@@ -80,3 +80,31 @@ func TestClientIP(t *testing.T) {
t.Errorf("untrusted: got %q", got)
}
}
func TestClientIPFallbackAndIPv6Prefix(t *testing.T) {
r, err := http.NewRequest("GET", "http://x/", nil)
if err != nil {
t.Fatal(err)
}
r.RemoteAddr = "10.0.0.1:5555"
if got := ClientIP(r, true); got != "10.0.0.1" {
t.Errorf("trusted proxy without header: got %q", got)
}
r.RemoteAddr = "[2001:db8:1:2:aaaa::1]:5555"
a := ClientIP(r, false)
r.RemoteAddr = "[2001:db8:1:2:bbbb::9]:5555"
if b := ClientIP(r, false); a != b || a != "2001:db8:1:2::/64" {
t.Errorf("same /64 got %q and %q", a, b)
}
r.RemoteAddr = "[::ffff:10.0.0.2]:5555"
if got := ClientIP(r, false); got != "10.0.0.2" {
t.Errorf("mapped IPv4: got %q", got)
}
r.RemoteAddr = ""
if got := ClientIP(r, true); got == "" {
t.Error("key must never be empty")
}
}
Minternal/sshd/sshd.go
@@ -14,6 +14,7 @@ import (
"regexp"
"strings"
"sync"
"time"
"github.com/gliderlabs/ssh"
gossh "golang.org/x/crypto/ssh"
@@ -73,6 +74,9 @@ func (s *Server) ListenAndServe() error {
HostSigners: []ssh.Signer{signer},
PublicKeyHandler: s.publicKey,
Handler: s.session,
// Reset by every read and write. Long, because git is silent while it
// counts objects.
IdleTimeout: 5 * time.Minute,
}
s.mu.Lock()
s.srv = srv
Minternal/util/cache.go
@@ -7,6 +7,7 @@ import (
type cacheEntry[V any] struct {
value V
size int64
// expiresAt is the zero time when the cache has no TTL.
expiresAt time.Time
}
@@ -15,11 +16,14 @@ type cacheEntry[V any] struct {
//
// ponytail: FIFO eviction, swap for LRU if hit rate matters.
type Cache[K comparable, V any] struct {
mu sync.Mutex
max int
ttl time.Duration
items map[K]cacheEntry[V]
order []K
mu sync.Mutex
max int
ttl time.Duration
maxBytes int64
sizeOf func(V) int64
bytes int64
items map[K]cacheEntry[V]
order []K
}
// NewCache returns a cache holding at most max entries. A zero ttl means
@@ -28,6 +32,14 @@ func NewCache[K comparable, V any](max int, ttl time.Duration) *Cache[K, V] {
return &Cache[K, V]{max: max, ttl: ttl, items: make(map[K]cacheEntry[V], max)}
}
// NewSizedCache is NewCache with a second bound: the sum of sizeOf over all
// entries stays at or below maxBytes. Values larger than maxBytes are not cached.
func NewSizedCache[K comparable, V any](max int, maxBytes int64, ttl time.Duration, sizeOf func(V) int64) *Cache[K, V] {
c := NewCache[K, V](max, ttl)
c.maxBytes, c.sizeOf = maxBytes, sizeOf
return c
}
// Get returns the value for key. Expired entries are dropped and report false.
func (c *Cache[K, V]) Get(key K) (V, bool) {
c.mu.Lock()
@@ -44,24 +56,31 @@ func (c *Cache[K, V]) Get(key K) (V, bool) {
}
// Set stores a value, evicting the oldest entries when the cache is full.
func (c *Cache[K, V]) Set(key K, v V) {
func (c *Cache[K, V]) Set(key K, v V) { c.SetTTL(key, v, c.ttl) }
// SetTTL is Set with a per-entry ttl. A zero ttl means the entry never expires.
func (c *Cache[K, V]) SetTTL(key K, v V, ttl time.Duration) {
c.mu.Lock()
defer c.mu.Unlock()
e := cacheEntry[V]{value: v}
if c.ttl > 0 {
e.expiresAt = time.Now().Add(c.ttl)
if ttl > 0 {
e.expiresAt = time.Now().Add(ttl)
}
if _, exists := c.items[key]; exists {
// Refresh in place: the insertion order does not change.
c.items[key] = e
return
c.remove(key)
if c.sizeOf != nil {
e.size = c.sizeOf(v)
if e.size > c.maxBytes {
return
}
}
for len(c.order) >= c.max {
for len(c.order) > 0 && (len(c.order) >= c.max || c.bytes+e.size > c.maxBytes && c.sizeOf != nil) {
c.bytes -= c.items[c.order[0]].size
delete(c.items, c.order[0])
c.order = c.order[1:]
}
c.items[key] = e
c.order = append(c.order, key)
c.bytes += e.size
}
// Delete drops one entry.
@@ -80,9 +99,11 @@ func (c *Cache[K, V]) Len() int {
// remove deletes key from both the map and the order. The caller holds the lock.
func (c *Cache[K, V]) remove(key K) {
if _, ok := c.items[key]; !ok {
e, ok := c.items[key]
if !ok {
return
}
c.bytes -= e.size
delete(c.items, key)
for i, k := range c.order {
if k == key {
Minternal/util/cache_test.go
@@ -41,9 +41,35 @@ func TestCacheTTLAndDelete(t *testing.T) {
if _, ok := c2.Get(1); ok {
t.Error("deleted entry was returned")
}
c2.SetTTL(3, "short", time.Millisecond)
time.Sleep(2 * time.Millisecond)
if _, ok := c2.Get(3); ok {
t.Error("per-entry ttl was ignored")
}
c2.Set(2, "y")
c2.Set(2, "z")
if v, _ := c2.Get(2); v != "z" {
t.Errorf("Set did not refresh the value, got %q", v)
}
}
func TestCacheEvictsByBytes(t *testing.T) {
c := NewSizedCache[string, string](10, 5, 0, func(s string) int64 { return int64(len(s)) })
c.Set("a", "xx")
c.Set("b", "yy")
c.Set("c", "zz")
if _, ok := c.Get("a"); ok {
t.Error("oldest entry was not evicted by the byte limit")
}
if _, ok := c.Get("b"); !ok {
t.Error("entry within the byte limit was evicted")
}
c.Set("big", "123456")
if _, ok := c.Get("big"); ok {
t.Error("entry larger than the byte limit was cached")
}
c.Set("b", "12345")
if c.Len() != 1 {
t.Errorf("cache holds %d entries, want 1", c.Len())
}
}
Minternal/web/auth.go
@@ -79,12 +79,22 @@ func (s *Server) authRoutes(r chi.Router) {
r.Post("/auth/passkey/login/verify", s.passkeyLoginVerify)
}
// localPath returns p when it is a path on this site, else "". It keeps a
// return URL from sending the user to another host.
func localPath(p string) string {
if _, err := neturl.Parse(p); err != nil ||
!strings.HasPrefix(p, "/") || strings.HasPrefix(p, "//") || strings.Contains(p, "\\") {
return ""
}
return p
}
func (s *Server) loginPage(w http.ResponseWriter, r *http.Request) {
views.Render(w, http.StatusOK, views.Login(s.Cfg, ""))
views.Render(w, http.StatusOK, views.Login(s.Cfg, "", localPath(r.URL.Query().Get("next"))))
}
func (s *Server) loginError(w http.ResponseWriter, msg string) {
views.Render(w, http.StatusOK, views.Login(s.Cfg, msg))
func (s *Server) loginError(w http.ResponseWriter, r *http.Request, msg string) {
views.Render(w, http.StatusOK, views.Login(s.Cfg, msg, localPath(r.FormValue("next"))))
}
func (s *Server) login(w http.ResponseWriter, r *http.Request) {
@@ -104,16 +114,16 @@ func (s *Server) login(w http.ResponseWriter, r *http.Request) {
// Spend the same argon2 time as a real check, so the response time
// does not reveal whether the username exists.
db.VerifyDummyPassword(r.FormValue("password"))
s.loginError(w, "Invalid username or password")
s.loginError(w, r, "Invalid username or password")
return
}
ok, err := db.VerifyPassword(*user.PasswordHash, r.FormValue("password"))
if err != nil || !ok {
s.loginError(w, "Invalid username or password")
s.loginError(w, r, "Invalid username or password")
return
}
if user.IsPending {
s.loginError(w, "Your account is awaiting approval.")
s.loginError(w, r, "Your account is awaiting approval.")
return
}
cookie, err := s.newSession(r.Context(), user.ID)
@@ -122,7 +132,11 @@ func (s *Server) login(w http.ResponseWriter, r *http.Request) {
return
}
http.SetCookie(w, cookie)
redirectTo(w, r, "/")
next := localPath(r.FormValue("next"))
if next == "" {
next = "/"
}
redirectTo(w, r, next)
}
func (s *Server) logout(w http.ResponseWriter, r *http.Request) {
@@ -167,7 +181,7 @@ func (s *Server) register(w http.ResponseWriter, r *http.Request) {
s.registerError(w, "Username may only contain letters, numbers, hyphens, and underscores")
return
}
if username == db.AdminUsername {
if strings.EqualFold(username, db.AdminUsername) {
s.registerError(w, "That username is reserved")
return
}
@@ -258,7 +272,7 @@ func (s *Server) passkeyCreateUser(w http.ResponseWriter, r *http.Request) {
jsonError(w, http.StatusBadRequest, "Invalid username")
return
}
if body.Username == db.AdminUsername {
if strings.EqualFold(body.Username, db.AdminUsername) {
jsonError(w, http.StatusBadRequest, "That username is reserved")
return
}
Minternal/web/auth_test.go
@@ -313,7 +313,7 @@ func TestPasskeyLoginOptionsShape(t *testing.T) {
t.Fatalf("status = %d, want 200: %s", rec.Code, rec.Body.String())
}
body := rec.Body.String()
for _, want := range []string{`"challenge"`, `"rpId":"localhost"`} {
for _, want := range []string{`"challenge"`, `"rpId":"localhost"`, `"userVerification":"required"`} {
if !strings.Contains(body, want) {
t.Errorf("body %q is missing %s", body, want)
}
Minternal/web/e2e/ci_mock_test.go
@@ -63,6 +63,11 @@ type mockDocker struct {
volumeUsage map[string]ciVolumeUsage
// lastCreateBody is the body of the last POST /containers/create.
lastCreateBody map[string]any
// requests logs container creates, deletes and archive downloads.
requests []string
// archiveDelay holds every GET /archive open, so a cancel can land
// during artifact collection.
archiveDelay time.Duration
// execMap holds the response of an exec id, assigned at creation.
execMap map[string]execResp
// execQueue is consumed in order as execs are created.
@@ -129,6 +134,14 @@ func (m *mockDocker) reset() {
m.volumesOnHost = nil
m.volumeUsage = map[string]ciVolumeUsage{}
m.lastCreateBody = nil
m.requests = nil
m.archiveDelay = 0
}
func (m *mockDocker) containerRequests() []string {
m.mu.Lock()
defer m.mu.Unlock()
return append([]string(nil), m.requests...)
}
func (m *mockDocker) setVolumesOnHost(names ...string) {
@@ -260,6 +273,9 @@ func (m *mockDocker) ServeHTTP(w http.ResponseWriter, r *http.Request) {
if name == "" {
name = "mock-ctr-001"
}
m.mu.Lock()
m.requests = append(m.requests, "POST "+path.Base(p)+"?name="+qs.Get("name"))
m.mu.Unlock()
// A copy creates its own source container, so the run's container
// must keep its identity.
if !strings.Contains(name, "-copy-") {
@@ -327,6 +343,17 @@ func (m *mockDocker) ServeHTTP(w http.ResponseWriter, r *http.Request) {
// Archive download: artifact collection and [[copy]].
case r.Method == http.MethodGet && reContainerArchive.MatchString(p):
m.mu.Lock()
delay := m.archiveDelay
m.requests = append(m.requests, "GET archive")
m.mu.Unlock()
if delay > 0 {
select {
case <-time.After(delay):
case <-r.Context().Done():
return
}
}
filePath := qs.Get("path")
if filePath == "" {
filePath = "file.txt"
@@ -336,6 +363,9 @@ func (m *mockDocker) ServeHTTP(w http.ResponseWriter, r *http.Request) {
// Delete container.
case r.Method == http.MethodDelete && strings.Contains(p, "/containers/"):
m.mu.Lock()
m.requests = append(m.requests, "DELETE "+path.Base(p))
m.mu.Unlock()
w.WriteHeader(http.StatusNoContent)
// Volume create, used for cache volumes.
Minternal/web/e2e/ci_test.go
@@ -13,6 +13,7 @@ import (
"net/url"
"os"
"path/filepath"
"slices"
"strconv"
"strings"
"testing"
@@ -119,15 +120,12 @@ func ciPushRun(e *env, sha, branch string) int64 {
gitRun(t, e.repoPath("ci-repo"), "push", "--force", e.authURL("ci-repo"),
sha+":refs/heads/"+branch)
e.Srv.Git.InvalidateRefCache("ci-repo")
deadline := time.Now().Add(10 * time.Second)
for time.Now().Before(deadline) {
if id := ciLatestRunID(e); id > before {
return id
}
time.Sleep(50 * time.Millisecond)
}
t.Fatalf("push to %s did not create a run", branch)
return 0
var id int64
ciWaitFor(t, "a run from the push to "+branch, func() bool {
id = ciLatestRunID(e)
return id > before
})
return id
}
// ciWaitForRun polls until the run leaves pending/running/queued.
@@ -138,16 +136,13 @@ func ciWaitForRun(e *env, runID int64, timeout ...time.Duration) string {
if len(timeout) > 0 {
limit = timeout[0]
}
deadline := time.Now().Add(limit)
for time.Now().Before(deadline) {
status := ciRunStatus(e, runID)
if status != "" && status != "pending" && status != "running" && status != "queued" {
return status
}
time.Sleep(50 * time.Millisecond)
}
t.Fatalf("run %d did not complete within %s", runID, limit)
return ""
var status string
ciWaitFor(t, "run "+strconv.FormatInt(runID, 10)+" to complete", func() bool {
status = ciRunStatus(e, runID)
return status != "" && status != "pending" && status != "running" && status != "queued" &&
!e.Srv.CI.Active(runID)
}, limit)
return status
}
func ciRunStatus(e *env, runID int64) string {
@@ -474,6 +469,135 @@ func TestCICancel(t *testing.T) {
})
}
const ciSlowTOML = `
image = "debian:latest"
[on]
manual = true
[[steps]]
name = "slow"
run_sh = "sleep 60"
[[steps]]
name = "after"
run_sh = "echo after"
`
// ciStepStatus is the status of the first step with that name, or "".
func ciStepStatus(e *env, runID int64, name string) string {
if steps := ciSteps(e, runID, name); len(steps) > 0 {
return steps[0].Status
}
return ""
}
// ciWaitFor polls cond for up to ten seconds, or the given timeout.
func ciWaitFor(t *testing.T, what string, cond func() bool, timeout ...time.Duration) {
t.Helper()
limit := 10 * time.Second
if len(timeout) > 0 {
limit = timeout[0]
}
for deadline := time.Now().Add(limit); time.Now().Before(deadline); {
if cond() {
return
}
time.Sleep(20 * time.Millisecond)
}
t.Fatalf("timed out waiting for %s", what)
}
func TestCICancelMidRun(t *testing.T) {
e, m, admin := ciEnv(t)
t.Run("the interrupted step reads cancelled", func(t *testing.T) {
sha := ciSeedToml(e, ciSlowTOML)
m.reset()
m.queueExec(execResp{delay: 30 * time.Second})
runID := ciTrigger(e, admin, sha, nil)
ciWaitFor(t, "slow step to run", func() bool { return ciStepStatus(e, runID, "slow") == "running" })
admin.post(ciRunPath(runID)+"/cancel", url.Values{}).mustRedirect(ciRunPath(runID))
ciWaitFor(t, "slow step to settle", func() bool { return ciStepStatus(e, runID, "slow") != "running" })
if got := ciStep(e, runID, "slow").Status; got != "cancelled" {
t.Errorf("slow status = %q", got)
}
if got := ciStep(e, runID, "after").Status; got != "skipped" {
t.Errorf("after status = %q", got)
}
if got := ciRunStatus(e, runID); got != "cancelled" {
t.Errorf("run status = %q", got)
}
})
t.Run("a cancel during artifact collection is not overwritten", func(t *testing.T) {
sha := ciSeedToml(e, ciArtifactTOML)
m.reset()
m.archiveDelay = 30 * time.Second
runID := ciTrigger(e, admin, sha, nil)
ciWaitFor(t, "artifact download", func() bool {
return slices.Contains(m.containerRequests(), "GET archive")
})
admin.post(ciRunPath(runID)+"/cancel", url.Values{}).mustRedirect(ciRunPath(runID))
ciWaitFor(t, "build step to settle", func() bool { return ciStepStatus(e, runID, "build") != "running" })
// The final status write follows the step write within milliseconds.
time.Sleep(300 * time.Millisecond)
if got := ciRunStatus(e, runID); got != "cancelled" {
t.Errorf("run status = %q", got)
}
})
}
func TestCIContainerLifecycle(t *testing.T) {
e, m, admin := ciEnv(t)
sha := ciSeedToml(e, ciSimpleTOML)
m.reset()
runID := ciTrigger(e, admin, sha, nil)
ciWaitForRun(e, runID)
name := "hearthforge-ci-" + strconv.FormatInt(runID, 10)
reqs := m.containerRequests()
// A leftover from a cancelled create must not block the name.
if len(reqs) < 2 || reqs[0] != "DELETE "+name || reqs[1] != "POST create?name="+name {
t.Errorf("requests = %v", reqs)
}
labels, _ := m.createBody()["Labels"].(map[string]any)
if labels["com.hearthforge.ci"] != "1" {
t.Errorf("labels = %v", labels)
}
}
func TestCIRepoDeleteStopsRuns(t *testing.T) {
e, m, admin := ciEnv(t)
sha := ciSeedToml(e, ciArtifactTOML)
m.reset()
done := ciTrigger(e, admin, sha, nil)
ciWaitForRun(e, done)
artifacts := filepath.Join(e.Cfg.CIArtifactsDir(), strconv.FormatInt(done, 10))
if _, err := os.Stat(artifacts); err != nil {
t.Fatalf("artifacts of run %d: %v", done, err)
}
m.queueExec(execResp{delay: 30 * time.Second})
active := ciTrigger(e, admin, sha, nil)
ciWaitFor(t, "build step to run", func() bool { return ciStepStatus(e, active, "build") == "running" })
m.setVolumesOnHost("hearthforge-ci-cache-old")
admin.post("/ci-repo/settings/delete", nil).mustRedirect("/")
if !slices.Contains(m.containerRequests(), "DELETE hearthforge-ci-"+strconv.FormatInt(active, 10)) {
t.Errorf("active container not removed: %v", m.containerRequests())
}
if _, err := os.Stat(artifacts); !os.IsNotExist(err) {
t.Errorf("artifacts still there: %v", err)
}
if !slices.Contains(m.deletedVolumes(), "hearthforge-ci-cache-old") {
t.Errorf("volumes deleted = %v", m.deletedVolumes())
}
}
// ── artifacts ────────────────────────────────────────────────────────────
func TestCIArtifacts(t *testing.T) {
Minternal/web/e2e/fileediting_test.go
@@ -115,4 +115,81 @@ func TestFileEditing(t *testing.T) {
sha := e.headCommit("edit-repo")
admin.get("/edit-repo/edit/" + sha + "/index.js").mustStatus(404)
})
t.Run("save after a push to another file lands on top of it", func(t *testing.T) {
base := admin.get("/edit-repo/edit/main/index.js").Value("input[name=base]")
if base != e.headCommit("edit-repo") {
t.Fatalf("base = %q", base)
}
e.seedRepo("edit-repo", map[string]string{"other.txt": "pushed\n"}, "Push other")
admin.post("/edit-repo/edit/main/index.js", url.Values{
"content": {"on top\n"}, "new_path": {"index.js"}, "message": {"On top"}, "base": {base},
}).mustRedirect("/edit-repo/commit/")
if got := gitRun(t, e.repoPath("edit-repo"), "show", "main:other.txt"); got != "pushed" {
t.Errorf("other.txt = %q", got)
}
})
t.Run("save after a push to the same file keeps the input and asks again", func(t *testing.T) {
base := admin.get("/edit-repo/edit/main/index.js").Value("input[name=base]")
tip := e.seedRepo("edit-repo", map[string]string{"index.js": "pushed\n"}, "Push index")
form := url.Values{"content": {"mine\n"}, "new_path": {"index.js"}, "message": {"Mine"}, "base": {base}}
r := admin.post("/edit-repo/edit/main/index.js", form).mustStatus(409)
if got := r.Value("textarea[name=content]"); got != "mine\n" {
t.Errorf("content = %q", got)
}
if got := r.Value("input[name=base]"); got != tip {
t.Errorf("base = %q, want %q", got, tip)
}
if !r.Has(`.form-error a[href="/edit-repo/blob/` + tip + `/index.js"]`) {
t.Error("link to the current version missing")
}
if e.headCommit("edit-repo") != tip {
t.Fatal("stale save moved the branch")
}
form.Set("base", tip)
admin.post("/edit-repo/edit/main/index.js", form).mustRedirect("/edit-repo/commit/")
if got := gitRun(t, e.repoPath("edit-repo"), "show", "main:index.js"); got != "mine" {
t.Errorf("index.js = %q", got)
}
})
t.Run("create over an existing file is rejected with the input kept", func(t *testing.T) {
r := admin.post("/edit-repo/new-file/main", url.Values{
"path": {"README.md"}, "content": {"new body"}, "message": {"Clobber"},
}).mustStatus(409)
if got := r.Value("textarea[name=content]"); got != "new body" {
t.Errorf("content = %q", got)
}
if got := r.Value("input[name=path]"); got != "README.md" {
t.Errorf("path = %q", got)
}
})
t.Run("delete after a push to the same file shows the current version", func(t *testing.T) {
base := admin.get("/edit-repo/blob/main/README.md").Value(`form[action*="/delete-file/"] input[name=base]`)
if base != e.headCommit("edit-repo") {
t.Fatalf("base = %q", base)
}
tip := e.seedRepo("edit-repo", map[string]string{"README.md": "pushed\n"}, "Push readme")
r := admin.post("/edit-repo/delete-file/main/README.md", url.Values{"base": {base}}).mustStatus(409)
if !r.Has(".form-error") || !r.Contains("pushed") {
t.Error("error or current content missing")
}
if got := r.Value(`form[action*="/delete-file/"] input[name=base]`); got != tip {
t.Errorf("base = %q, want %q", got, tip)
}
if e.headCommit("edit-repo") != tip {
t.Fatal("stale delete moved the branch")
}
admin.post("/edit-repo/delete-file/main/README.md", url.Values{"base": {tip}}).mustRedirect("/edit-repo/commit/")
})
t.Run("delete of a missing file makes no commit", func(t *testing.T) {
tip := e.headCommit("edit-repo")
admin.post("/edit-repo/delete-file/main/nope.txt", url.Values{}).mustStatus(404)
if e.headCommit("edit-repo") != tip {
t.Error("branch moved")
}
})
}
Minternal/web/e2e/issues_test.go
@@ -260,6 +260,17 @@ func TestIssueEditing(t *testing.T) {
}
})
t.Run("author can delete own issue only while open", func(t *testing.T) {
closed := issuesCreate(alice, "my-repo", "Closed later", "")
admin.post(closed+"/close", nil)
alice.post(closed+"/delete", nil).mustStatus(http.StatusForbidden)
admin.get(closed).mustStatus(http.StatusOK)
open := issuesCreate(alice, "my-repo", "Still open", "")
alice.post(open+"/delete", nil).mustStatus(http.StatusFound)
admin.get(open).mustStatus(http.StatusNotFound)
})
t.Run("admin can delete issue", func(t *testing.T) {
admin.post(issuePath+"/delete", nil).mustStatus(http.StatusFound)
admin.get(issuePath).mustStatus(http.StatusNotFound)
Minternal/web/e2e/patches_test.go
@@ -385,6 +385,16 @@ new file mode 100644
admin.get(conflictURL).mustStatus(http.StatusNotFound)
})
t.Run("author cannot delete own closed patch", func(t *testing.T) {
alice := e.login("alice", "password123")
p := patchCreate(alice, "my-repo", "Alice patch", "", uploadTestPatch).
mustRedirect("/my-repo/patches/")
admin.post(p+"/close", nil)
alice.post(p+"/delete", nil).mustStatus(http.StatusForbidden)
admin.get(p).mustStatus(http.StatusOK)
admin.post(p+"/delete", nil).mustStatus(http.StatusFound)
})
// ── Patch file re-upload & version protection ──────────────────────────
t.Run("create patch for re-upload tests", func(t *testing.T) {
Ainternal/web/e2e/rendercaps_test.go
@@ -0,0 +1,56 @@
package e2e
import (
"strings"
"testing"
)
func TestRenderCapsAndNonASCIIPaths(t *testing.T) {
e := newEnv(t, "MAX_RENDER_BYTES", "2000")
admin := e.admin()
e.createRepo(admin, "caps")
e.seedRepo("caps", map[string]string{
"README.md": "# small\n",
"docs/README.md": strings.Repeat("big readme line\n", 200),
})
small := e.seedRepo("caps", map[string]string{
"dir/ä ö.txt": "umlaut\n",
"bin/bild ü.bin": "\x00\x01\x02binary",
}, "non-ASCII names")
t.Run("non-ASCII name links to a working blob", func(t *testing.T) {
r := admin.get("/caps/tree/main/dir").mustStatus(200)
if got := r.Text(".file-name a[href*='/blob/']"); got != "ä ö.txt" {
t.Fatalf("tree name = %q", got)
}
admin.get(r.Attr(".file-name a[href*='/blob/']", "href")).mustStatus(200)
})
t.Run("commit diff shows unquoted paths and binary sizes", func(t *testing.T) {
r := admin.get("/caps/commit/" + small).mustStatus(200)
if !r.Contains("ä ö.txt") || !r.Contains("bild ü.bin") || r.Contains(`\303`) {
t.Fatal("diff paths are missing or C-quoted")
}
})
t.Run("oversized README shows a download notice", func(t *testing.T) {
r := admin.get("/caps/tree/main/docs").mustStatus(200)
if !r.Contains("README too large to display inline") || r.Contains("big readme line") {
t.Fatal("oversized README was rendered")
}
if href := r.Attr(".readme-section .file-download-notice a", "href"); href != "/caps/raw/main/docs/README.md" {
t.Fatalf("download href = %q", href)
}
if r := admin.get("/caps").mustStatus(200); !r.Has(".readme-section .markdown-body") {
t.Fatal("small root README not rendered")
}
})
t.Run("oversized commit diff is not rendered", func(t *testing.T) {
big := e.seedRepo("caps", map[string]string{"big.txt": strings.Repeat("line\n", 1000)}, "big")
r := admin.get("/caps/commit/" + big).mustStatus(200)
if !r.Contains("Diff is too large to render inline") || r.Has(".diff-file") {
t.Fatal("oversized diff was rendered")
}
})
}
Minternal/web/issues.go
@@ -1,6 +1,7 @@
package web
import (
"errors"
"net/http"
"os"
"slices"
@@ -21,6 +22,30 @@ func allowedReaction(emoji string) bool {
return slices.Contains(views.AllowedReactions, emoji)
}
// toggleReaction reads the optional comment_id and runs toggle. It writes the
// error response and returns false on failure.
func toggleReaction(w http.ResponseWriter, r *http.Request, toggle func(commentID *int64) error) bool {
var commentID *int64
if raw := r.FormValue("comment_id"); raw != "" {
n, ok := leadingInt(raw)
if !ok {
http.Error(w, "Not found", http.StatusNotFound)
return false
}
commentID = &n
}
err := toggle(commentID)
if errors.Is(err, db.ErrCommentNotFound) {
http.Error(w, "Not found", http.StatusNotFound)
return false
}
if err != nil {
http.Error(w, "Internal Server Error", http.StatusInternalServerError)
return false
}
return true
}
// visibleRepo loads the {repo} URL parameter and hides private repos from
// non-admins. It writes a 404 and returns false when the repo is not visible.
func (s *Server) visibleRepo(w http.ResponseWriter, r *http.Request) (*db.Repo, bool) {
@@ -64,12 +89,15 @@ func leadingInt(s string) (int64, bool) {
return n, err == nil
}
// parseLabelIDs turns repeated form or query values into label ids.
// parseLabelIDs turns repeated or comma-separated form and query values
// into label ids.
func parseLabelIDs(values []string) []int64 {
var out []int64
for _, v := range values {
if n, ok := leadingInt(v); ok {
out = append(out, n)
for part := range strings.SplitSeq(v, ",") {
if n, ok := leadingInt(part); ok {
out = append(out, n)
}
}
}
return out
@@ -371,11 +399,7 @@ func (s *Server) editIssueComment(w http.ResponseWriter, r *http.Request) {
return
}
user := User(r)
if (auth.AuthorID == nil || *auth.AuthorID != user.ID) && !user.IsAdmin {
http.Error(w, "Forbidden", http.StatusForbidden)
return
}
if auth.Status != "open" && !user.IsAdmin {
if !views.CanEdit(user, auth.AuthorID, auth.Status) {
http.Error(w, "Forbidden", http.StatusForbidden)
return
}
@@ -412,14 +436,9 @@ func (s *Server) reactIssue(w http.ResponseWriter, r *http.Request) {
if !ok {
return
}
var commentID *int64
if raw := r.FormValue("comment_id"); raw != "" {
if n, ok := leadingInt(raw); ok {
commentID = &n
}
}
if err := s.DB.ToggleIssueReaction(r.Context(), issue.ID, commentID, User(r).ID, emoji); err != nil {
http.Error(w, "Internal Server Error", http.StatusInternalServerError)
if !toggleReaction(w, r, func(commentID *int64) error {
return s.DB.ToggleIssueReaction(r.Context(), issue.ID, commentID, User(r).ID, emoji)
}) {
return
}
http.Redirect(w, r, "/"+repo.Name+"/issues/"+strconv.FormatInt(num, 10), http.StatusSeeOther)
@@ -469,7 +488,7 @@ func (s *Server) deleteIssue(w http.ResponseWriter, r *http.Request) {
return
}
user := User(r)
if (issue.AuthorID == nil || *issue.AuthorID != user.ID) && !user.IsAdmin {
if !views.CanEdit(user, issue.AuthorID, issue.Status) {
http.Error(w, "Forbidden", http.StatusForbidden)
return
}
@@ -494,11 +513,7 @@ func (s *Server) editIssue(w http.ResponseWriter, r *http.Request) {
return
}
user := User(r)
if (issue.AuthorID == nil || *issue.AuthorID != user.ID) && !user.IsAdmin {
http.Error(w, "Forbidden", http.StatusForbidden)
return
}
if issue.Status != "open" && !user.IsAdmin {
if !views.CanEdit(user, issue.AuthorID, issue.Status) {
http.Error(w, "Forbidden", http.StatusForbidden)
return
}
Minternal/web/issues_test.go
@@ -6,6 +6,7 @@ import (
"net/http/httptest"
"net/url"
"path/filepath"
"slices"
"strings"
"testing"
@@ -181,3 +182,28 @@ func TestEditIssueRejectsEmpty(t *testing.T) {
t.Errorf("edit with blank comment status = %d, want 422", res.Code)
}
}
func TestReactionRejectsForeignComment(t *testing.T) {
h, _, admin := issueTestServer(t)
do(t, h, admin, "POST", "/demo/issues", url.Values{"title": {"One"}})
do(t, h, admin, "POST", "/demo/issues", url.Values{"title": {"Two"}})
do(t, h, admin, "POST", "/demo/issues/1/comments", url.Values{"body": {"On one."}})
for _, id := range []string{"1", "999", "x"} {
res := do(t, h, admin, "POST", "/demo/issues/2/react", url.Values{"emoji": {"👍"}, "comment_id": {id}})
if res.Code != http.StatusNotFound {
t.Errorf("comment_id %s: status = %d, want 404", id, res.Code)
}
}
res := do(t, h, admin, "POST", "/demo/issues/1/react", url.Values{"emoji": {"👍"}, "comment_id": {"1"}})
if res.Code != http.StatusSeeOther {
t.Errorf("own comment: status = %d, want 303", res.Code)
}
}
func TestParseLabelIDs(t *testing.T) {
got := parseLabelIDs([]string{"1,2", "3", "x,4"})
if !slices.Equal(got, []int64{1, 2, 3, 4}) {
t.Errorf("parseLabelIDs = %v", got)
}
}
Minternal/web/limits.go
@@ -19,6 +19,7 @@ var (
patchCreateLimiter = ratelimit.New(10, time.Minute)
labelWriteLimiter = ratelimit.New(30, time.Minute)
uploadLimiter = ratelimit.New(10, time.Minute)
passwordLimiter = ratelimit.New(10, time.Minute)
)
// limitKey is "u<id>" for logged-in users, else the client IP.
@@ -40,7 +41,7 @@ func (s *Server) allowed(r *http.Request, l *ratelimit.Limiter, byIP bool) bool
if byIP {
key = ratelimit.ClientIP(r, s.Cfg.TrustedProxy)
}
return key == "" || l.Allow(key)
return l.Allow(key)
}
// limited writes a plain-text 429 when the request exceeded l. Callers return
Minternal/web/patches.go
@@ -10,6 +10,7 @@ import (
"net/http"
"strconv"
"strings"
"time"
"github.com/go-chi/chi/v5"
@@ -78,15 +79,23 @@ func newVersion() string {
// never shown, it only stops the next view from spawning git again.
const patchCheckError = "error"
const (
patchCheckTimeout = time.Minute
patchCheckErrorTTL = time.Minute
)
// runPatchCheck previews the patch and caches the result.
func (s *Server) runPatchCheck(r *http.Request, repoName string, patchID int64,
content string,
) *gitcmd.ApplyResult {
result, err := s.Git.CheckPatch(r.Context(), repoName, content)
// A client disconnect must not cancel the check and cache its error.
ctx, cancel := context.WithTimeout(context.WithoutCancel(r.Context()), patchCheckTimeout)
defer cancel()
result, err := s.Git.CheckPatch(ctx, repoName, content)
if err != nil {
// Cache the failure too. Without it every anonymous view of a broken
// patch spawns git again.
s.Patches.Set(patchID, gitcmd.ApplyResult{Status: patchCheckError})
// Cache the failure briefly, so views of a broken patch do not spawn
// git on every request, and a transient error clears soon.
s.Patches.SetTTL(patchID, gitcmd.ApplyResult{Status: patchCheckError}, patchCheckErrorTTL)
return nil
}
s.Patches.Set(patchID, result)
@@ -505,7 +514,7 @@ func (s *Server) deletePatch(w http.ResponseWriter, r *http.Request) {
return
}
user := User(r)
if (patch.AuthorID == nil || *patch.AuthorID != user.ID) && !user.IsAdmin {
if !views.CanEdit(user, patch.AuthorID, patch.Status) {
http.Error(w, "Forbidden", http.StatusForbidden)
return
}
@@ -572,11 +581,7 @@ func (s *Server) editPatchComment(w http.ResponseWriter, r *http.Request) {
return
}
user := User(r)
if (auth.AuthorID == nil || *auth.AuthorID != user.ID) && !user.IsAdmin {
http.Error(w, "Forbidden", http.StatusForbidden)
return
}
if auth.Status != "open" && !user.IsAdmin {
if !views.CanEdit(user, auth.AuthorID, auth.Status) {
http.Error(w, "Forbidden", http.StatusForbidden)
return
}
@@ -613,14 +618,9 @@ func (s *Server) reactPatch(w http.ResponseWriter, r *http.Request) {
if !ok {
return
}
var commentID *int64
if raw := r.FormValue("comment_id"); raw != "" {
if n, ok := leadingInt(raw); ok {
commentID = &n
}
}
if err := s.DB.TogglePatchReaction(r.Context(), patch.ID, commentID, User(r).ID, emoji); err != nil {
http.Error(w, "Internal Server Error", http.StatusInternalServerError)
if !toggleReaction(w, r, func(commentID *int64) error {
return s.DB.TogglePatchReaction(r.Context(), patch.ID, commentID, User(r).ID, emoji)
}) {
return
}
http.Redirect(w, r, "/"+repo.Name+"/patches/"+strconv.FormatInt(num, 10), http.StatusSeeOther)
@@ -640,11 +640,7 @@ func (s *Server) editPatch(w http.ResponseWriter, r *http.Request) {
return
}
user := User(r)
if (patch.AuthorID == nil || *patch.AuthorID != user.ID) && !user.IsAdmin {
http.Error(w, "Forbidden", http.StatusForbidden)
return
}
if patch.Status != "open" && !user.IsAdmin {
if !views.CanEdit(user, patch.AuthorID, patch.Status) {
http.Error(w, "Forbidden", http.StatusForbidden)
return
}
Minternal/web/patches_test.go
@@ -81,7 +81,7 @@ func patchTestServer(t *testing.T) (http.Handler, *Server, *db.SessionUser) {
t.Fatal(err)
}
who := gitcmd.Ident{Name: "Admin", Email: "admin@example.com"}
if _, err := s.Git.EditFile(ctx, "demo", "main", "", "README.md",
if _, err := s.Git.EditFile(ctx, "demo", "main", "", "", "README.md",
[]byte("hello\n"), "init", who); err != nil {
t.Fatal(err)
}
Minternal/web/registry.go
@@ -18,6 +18,7 @@ import (
"hearthforge/internal/db"
"hearthforge/internal/ratelimit"
"hearthforge/internal/util"
)
// The registry implements the OCI Distribution Spec under /v2/. An image
@@ -41,8 +42,8 @@ var (
imagePathRe = regexp.MustCompile(`^[a-z0-9]+(?:[._-][a-z0-9]+)*$`)
)
// registryAuthLimiter counts failed Basic auth attempts per IP. Successful
// pulls make one request per layer and must not count against it.
// registryAuthLimiter counts argon2 checks per IP. Clients send Basic auth
// on every request, so verified credentials are cached and skip it.
var registryAuthLimiter = ratelimit.New(10, time.Minute)
func (s *Server) registryRoutes(r chi.Router) {
@@ -61,30 +62,38 @@ func registryError(w http.ResponseWriter, status int, code, msg string) {
}
// registryUser resolves Basic auth. ok is false when the header is missing
// or wrong. The limiter blocks an IP after repeated failures.
// or wrong. Each argon2 check is reserved in the limiter before it runs, so
// parallel guesses count too.
func (s *Server) registryUser(r *http.Request) (username string, isAdmin, ok bool) {
username, password, found := r.BasicAuth()
if !found {
if !found || len(password) > s.Cfg.MaxPasswordBytes {
return "", false, false
}
ip := ratelimit.ClientIP(r, s.Cfg.TrustedProxy)
if !s.Cfg.RateLimitDisabled && registryAuthLimiter.Blocked(ip) {
hash, exists, err := s.DB.ActivePasswordHash(r.Context(), username)
if err != nil {
return "", false, false
}
if len(password) <= s.Cfg.MaxPasswordBytes {
hash, exists, err := s.DB.ActivePasswordHash(r.Context(), username)
if err == nil && exists {
if valid, err := db.VerifyPassword(hash, password); err == nil && valid {
return username, username == db.AdminUsername, true
}
}
key := sha256.Sum256([]byte(username + "\x00" + password))
if cached, ok := registryAuth.Get(key); exists && ok && cached == hash {
return username, username == db.AdminUsername, true
}
if !s.Cfg.RateLimitDisabled {
registryAuthLimiter.Allow(ip)
if !s.allowed(r, registryAuthLimiter, true) {
return "", false, false
}
return "", false, false
if !exists {
return "", false, false
}
if valid, err := db.VerifyPassword(hash, password); err != nil || !valid {
return "", false, false
}
registryAuth.Set(key, hash)
return username, username == db.AdminUsername, true
}
// registryAuth remembers verified credentials. The value is the hash it
// matched, so a password change invalidates the entry.
var registryAuth = util.NewCache[[32]byte, string](1000, 5*time.Minute)
// challenge answers 401 with the Basic scheme so clients retry with
// credentials.
func challenge(w http.ResponseWriter) {
Minternal/web/releases_test.go
@@ -68,7 +68,7 @@ func releaseTestServer(t *testing.T) (http.Handler, *Server, *db.SessionUser) {
t.Fatal(err)
}
who := gitcmd.Ident{Name: "Admin", Email: "admin@localhost"}
if _, err := s.Git.EditFile(ctx, "demo", "main", "", "hello.txt",
if _, err := s.Git.EditFile(ctx, "demo", "main", "", "", "hello.txt",
[]byte("hello\n"), "init", who); err != nil {
t.Fatal(err)
}
Minternal/web/repos.go
@@ -1,6 +1,7 @@
package web
import (
"bytes"
"context"
"errors"
"net/http"
@@ -8,6 +9,7 @@ import (
"os"
"regexp"
"slices"
"strconv"
"strings"
"github.com/go-chi/chi/v5"
@@ -260,39 +262,41 @@ func (s *Server) createRepo(w http.ResponseWriter, r *http.Request) {
redirectTo(w, r, "/"+name)
}
// readme finds a README in an already-listed directory and returns its
// contents and its path relative to the repo root.
func (s *Server) readme(r *http.Request, repoName, ref, dir string,
// readme finds and renders the README of an already-listed directory.
// resolved is the commit ref resolves to and keys the render cache.
func (s *Server) readme(r *http.Request, repoName, ref, dir, resolved string,
entries []gitcmd.TreeEntry,
) (string, string) {
names := map[string]bool{}
) views.Readme {
sizes := map[string]string{}
for _, e := range entries {
names[e.Name] = true
sizes[e.Name] = e.Size
}
prefix := ""
if dir != "" {
prefix = dir + "/"
}
for _, name := range readmeNames {
if !names[name] {
size, ok := sizes[name]
if !ok {
continue
}
if n, err := strconv.ParseInt(size, 10, 64); err == nil && n > s.Cfg.MaxRenderBytes {
return views.Readme{Path: prefix + name, Size: n, TooLarge: true}
}
content, err := s.Git.Show(r.Context(), repoName, ref, prefix+name)
if err != nil || len(content) == 0 {
return "", ""
if err != nil || len(bytes.TrimSpace(content)) == 0 {
return views.Readme{}
}
key := ""
if resolved != "" {
key = "readme:" + repoName + ":" + resolved + ":" + dir
}
return views.Readme{
Path: prefix + name,
HTML: s.MD.Render(string(content), key, &markdown.Context{Repo: repoName, Ref: ref, Dir: dir}),
}
return string(content), prefix + name
}
return "", ""
}
// renderReadme renders README markdown with repo-relative link rewriting.
func (s *Server) renderReadme(content, repoName, ref, dir, resolved string) string {
key := ""
if resolved != "" {
key = "readme:" + repoName + ":" + resolved + ":" + dir
}
return s.MD.Render(content, key, &markdown.Context{Repo: repoName, Ref: ref, Dir: dir})
return views.Readme{}
}
func (s *Server) repoHome(w http.ResponseWriter, r *http.Request) {
@@ -301,9 +305,9 @@ func (s *Server) repoHome(w http.ResponseWriter, r *http.Request) {
return
}
var (
entries []gitcmd.TreeEntry
branches, tags []string
readmeHTML, path string
entries []gitcmd.TreeEntry
branches, tags []string
readme views.Readme
)
hasContent := s.Git.HasCommits(r.Context(), repo.Name)
if hasContent {
@@ -311,13 +315,10 @@ func (s *Server) repoHome(w http.ResponseWriter, r *http.Request) {
branches, _ = s.Git.Branches(r.Context(), repo.Name)
tags, _ = s.Git.Tags(r.Context(), repo.Name)
resolved, _ := s.Git.ResolveRef(r.Context(), repo.Name, repo.DefaultBranch)
if content, p := s.readme(r, repo.Name, repo.DefaultBranch, "", entries); p != "" {
readmeHTML = s.renderReadme(content, repo.Name, repo.DefaultBranch, "", resolved)
path = p
}
readme = s.readme(r, repo.Name, repo.DefaultBranch, "", resolved, entries)
}
views.Render(w, http.StatusOK, views.RepoHome(s.Cfg, User(r), repo, entries,
readmeHTML, path, hasContent, branches, tags))
readme, hasContent, branches, tags))
}
// branchSwitch turns the ref selector's GET form into a redirect.
@@ -422,13 +423,14 @@ func (s *Server) deleteRepo(w http.ResponseWriter, r *http.Request) {
http.Error(w, "Failed to delete repository", http.StatusInternalServerError)
return
}
if s.CI != nil {
s.CI.StopRepo(r.Context(), repo.ID, repo.Name)
}
if err := s.deleteRepoRow(r, repo.ID); err != nil {
http.Error(w, "Database error", http.StatusInternalServerError)
return
}
s.Git.InvalidateRefCache(repo.Name)
// CI cache volumes are labelled by repo name and survive the DB cascade.
s.purgeCaches(repo.Name)
redirectTo(w, r, "/")
}
Minternal/web/repos_files.go
@@ -13,6 +13,7 @@ import (
"github.com/gabriel-vasile/mimetype"
"hearthforge/internal/db"
"hearthforge/internal/gitcmd"
"hearthforge/internal/highlight"
"hearthforge/internal/markdown"
@@ -75,13 +76,9 @@ func (s *Server) renderTree(w http.ResponseWriter, r *http.Request, subpath stri
branches, _ := s.Git.Branches(r.Context(), repo.Name)
tags, _ := s.Git.Tags(r.Context(), repo.Name)
readmeHTML, readmePath := "", ""
if content, p := s.readme(r, repo.Name, ref, subpath, entries); p != "" {
readmeHTML = s.renderReadme(content, repo.Name, ref, subpath, resolved)
readmePath = p
}
readme := s.readme(r, repo.Name, ref, subpath, resolved, entries)
views.Render(w, http.StatusOK, views.FileTree(s.Cfg, User(r), repo, ref, subpath,
entries, branches, tags, readmeHTML, readmePath))
entries, branches, tags, readme))
}
func (s *Server) blobView(w http.ResponseWriter, r *http.Request) {
@@ -89,28 +86,33 @@ func (s *Server) blobView(w http.ResponseWriter, r *http.Request) {
if !ok {
return
}
s.renderBlob(w, r, repo, http.StatusOK, r.URL.Query().Get("error"))
}
// renderBlob shows the file at the ref and path of the request URL.
func (s *Server) renderBlob(w http.ResponseWriter, r *http.Request, repo *db.Repo, status int, blobError string) {
ref := refParam(r, "ref")
filePath := refParam(r, "*")
filename := path.Base(filePath)
blobError := r.URL.Query().Get("error")
// Everything below reads at one commit, which the delete form sends as its base.
commitSHA, err := s.Git.ResolveRef(r.Context(), repo.Name, ref)
if err != nil {
http.Error(w, "Not found", http.StatusNotFound)
return
}
// Check the size before reading the blob. Holding a huge buffer and then
// highlighting it is the cheapest denial-of-service against a public repo.
size, sizeErr := s.Git.FileSize(r.Context(), repo.Name, ref, filePath)
size, sizeErr := s.Git.FileSize(r.Context(), repo.Name, commitSHA, filePath)
if sizeErr == nil && size > s.Cfg.MaxRenderBytes {
branches, _ := s.Git.Branches(r.Context(), repo.Name)
tags, _ := s.Git.Tags(r.Context(), repo.Name)
views.Render(w, http.StatusOK, views.FileBlob(s.Cfg, User(r), repo, ref, filePath,
views.Render(w, status, views.FileBlob(s.Cfg, User(r), repo, ref, filePath, commitSHA,
highlight.FileView{Type: "download", Size: size}, branches, tags, "", blobError))
return
}
content, err := s.Git.Show(r.Context(), repo.Name, ref, filePath)
if err != nil {
http.Error(w, "Not found", http.StatusNotFound)
return
}
commitSHA, err := s.Git.ResolveRef(r.Context(), repo.Name, ref)
content, err := s.Git.Show(r.Context(), repo.Name, commitSHA, filePath)
if err != nil {
http.Error(w, "Not found", http.StatusNotFound)
return
@@ -130,7 +132,7 @@ func (s *Server) blobView(w http.ResponseWriter, r *http.Request) {
markdownHTML = s.MD.Render(string(content), cacheKey,
&markdown.Context{Repo: repo.Name, Ref: ref, Dir: dir})
}
views.Render(w, http.StatusOK, views.FileBlob(s.Cfg, User(r), repo, ref, filePath,
views.Render(w, status, views.FileBlob(s.Cfg, User(r), repo, ref, filePath, commitSHA,
view, branches, tags, markdownHTML, blobError))
}
@@ -250,8 +252,8 @@ func sniffContentType(filename string, head []byte) string {
return "text/plain; charset=utf-8"
}
// parseRange reads a single `bytes=a-b` range. It reports partial=false when
// the header is absent or unusable, which serves the whole file.
// parseRange reads one `bytes=a-b` or `bytes=-n` range. partial is false
// when the header is absent or unusable.
func parseRange(header string, total int64) (start, length int64, partial bool) {
spec, ok := strings.CutPrefix(header, "bytes=")
if !ok || total == 0 {
@@ -261,13 +263,17 @@ func parseRange(header string, total int64) (start, length int64, partial bool)
if !ok {
return 0, 0, false
}
start = 0
if from != "" {
n, err := strconv.ParseInt(from, 10, 64)
if err != nil || n < 0 || n >= total {
if from == "" {
n, err := strconv.ParseInt(to, 10, 64)
if err != nil || n <= 0 {
return 0, 0, false
}
start = n
n = min(n, total)
return total - n, n, true
}
start, err := strconv.ParseInt(from, 10, 64)
if err != nil || start < 0 || start >= total {
return 0, 0, false
}
end := total - 1
if to != "" {
@@ -295,7 +301,12 @@ func (s *Server) editFilePage(w http.ResponseWriter, r *http.Request) {
http.Error(w, "Not found", http.StatusNotFound)
return
}
content, err := s.Git.Show(r.Context(), repo.Name, ref, filePath)
base, err := s.Git.ResolveRef(r.Context(), repo.Name, ref)
if err != nil {
http.Error(w, "Not found", http.StatusNotFound)
return
}
content, err := s.Git.Show(r.Context(), repo.Name, base, filePath)
if err != nil || len(content) == 0 {
http.Error(w, "Not found", http.StatusNotFound)
return
@@ -304,8 +315,9 @@ func (s *Server) editFilePage(w http.ResponseWriter, r *http.Request) {
http.Error(w, "Not found", http.StatusNotFound)
return
}
views.Render(w, http.StatusOK, views.FileEdit(s.Cfg, User(r), repo, ref, filePath,
string(content), r.URL.Query().Get("error")))
views.Render(w, http.StatusOK, views.FileEdit(s.Cfg, User(r), repo, ref, filePath, views.FileForm{
Content: string(content), Base: base, Error: r.URL.Query().Get("error"),
}))
}
func (s *Server) editFile(w http.ResponseWriter, r *http.Request) {
@@ -342,10 +354,24 @@ func (s *Server) editFile(w http.ResponseWriter, r *http.Request) {
}
content := strings.ReplaceAll(r.FormValue("content"), "\r\n", "\n")
commit, err := s.Git.EditFile(r.Context(), repo.Name, ref, filePath, targetPath,
base := r.FormValue("base")
commit, err := s.Git.EditFile(r.Context(), repo.Name, ref, base, filePath, targetPath,
[]byte(content), message, s.committer())
if err != nil {
http.Error(w, "Failed to save file", gitStatusCode(err))
form := views.FileForm{Path: targetPath, Content: content, Message: message, Base: base}
var stale *gitcmd.StaleError
switch {
case errors.As(err, &stale):
form.Base, form.Stale = stale.Tip, true
form.Error = "This file changed on " + ref + " since you opened it. " +
"Saving again overwrites that change."
case errors.Is(err, gitcmd.ErrExists):
form.Error = "A file already exists at " + targetPath + "."
default:
http.Error(w, "Failed to save file", gitStatusCode(err))
return
}
views.Render(w, http.StatusConflict, views.FileEdit(s.Cfg, User(r), repo, ref, filePath, form))
return
}
redirectTo(w, r, "/"+repo.Name+"/commit/"+commit)
@@ -358,7 +384,7 @@ func (s *Server) newFilePage(w http.ResponseWriter, r *http.Request) {
}
q := r.URL.Query()
views.Render(w, http.StatusOK, views.NewFileForm(s.Cfg, User(r), repo,
refParam(r, "ref"), q.Get("dir"), q.Get("error")))
refParam(r, "ref"), q.Get("dir"), views.FileForm{Error: q.Get("error")}))
}
func (s *Server) createFile(w http.ResponseWriter, r *http.Request) {
@@ -374,7 +400,8 @@ func (s *Server) createFile(w http.ResponseWriter, r *http.Request) {
s.backTo(w, r, back, "error", "Invalid file path.")
return
}
message := strings.TrimSpace(r.FormValue("message"))
userMessage := strings.TrimSpace(r.FormValue("message"))
message := userMessage
if message == "" {
message = "Add " + filePath
}
@@ -384,8 +411,16 @@ func (s *Server) createFile(w http.ResponseWriter, r *http.Request) {
s.backTo(w, r, back, "error", "Can only create files on a branch.")
return
}
commit, err := s.Git.EditFile(r.Context(), repo.Name, ref, "", filePath,
[]byte(r.FormValue("content")), message, s.committer())
content := r.FormValue("content")
commit, err := s.Git.EditFile(r.Context(), repo.Name, ref, "", "", filePath,
[]byte(content), message, s.committer())
if errors.Is(err, gitcmd.ErrExists) {
views.Render(w, http.StatusConflict, views.NewFileForm(s.Cfg, User(r), repo, ref, "", views.FileForm{
Path: filePath, Content: content, Message: userMessage,
Error: "A file already exists at " + filePath + ".",
}))
return
}
if err != nil {
s.backTo(w, r, back, "error", writeFailMessage(err, "Failed to create file."))
return
@@ -404,7 +439,18 @@ func (s *Server) deleteFile(w http.ResponseWriter, r *http.Request) {
if message == "" {
message = "Delete " + filePath
}
commit, err := s.Git.DeleteFile(r.Context(), repo.Name, ref, filePath, message, s.committer())
commit, err := s.Git.DeleteFile(r.Context(), repo.Name, ref, r.FormValue("base"), filePath,
message, s.committer())
if errors.Is(err, gitcmd.ErrNotFound) {
http.Error(w, "No such file on this branch", http.StatusNotFound)
return
}
var stale *gitcmd.StaleError
if errors.As(err, &stale) {
s.renderBlob(w, r, repo, http.StatusConflict, "This file changed on "+ref+
" since you opened it. The current version is shown below. Delete again to remove it.")
return
}
if err != nil {
s.backTo(w, r, "/"+repo.Name+"/blob/"+views.EscapePath(ref)+"/"+views.EscapePath(filePath), "error",
writeFailMessage(err, "Failed to delete file."))
Ainternal/web/repos_files_test.go
@@ -0,0 +1,30 @@
package web
import "testing"
func TestParseRange(t *testing.T) {
tests := []struct {
header string
start, length int64
partial bool
}{
{"", 0, 0, false},
{"bytes=0-99", 0, 100, true},
{"bytes=100-", 100, 900, true},
{"bytes=900-5000", 900, 100, true},
{"bytes=-500", 500, 500, true},
{"bytes=-5000", 0, 1000, true},
{"bytes=-0", 0, 0, false},
{"bytes=-", 0, 0, false},
{"bytes=--5", 0, 0, false},
{"bytes=1000-", 0, 0, false},
{"bytes=5-2", 0, 0, false},
}
for _, tt := range tests {
start, length, partial := parseRange(tt.header, 1000)
if start != tt.start || length != tt.length || partial != tt.partial {
t.Errorf("%q: got %d,%d,%v want %d,%d,%v", tt.header,
start, length, partial, tt.start, tt.length, tt.partial)
}
}
}
Minternal/web/repos_refs.go
@@ -73,28 +73,24 @@ func (s *Server) commitDetail(w http.ResponseWriter, r *http.Request) {
http.Error(w, "Commit not found", http.StatusNotFound)
return
}
rawDiff, err := s.Git.Diff(r.Context(), repo.Name, sha)
if err != nil {
http.Error(w, "Commit not found", gitStatusCode(err))
rawDiff, err := s.Git.Diff(r.Context(), repo.Name, sha, s.Cfg.MaxRenderBytes)
if errors.Is(err, gitcmd.ErrTooLarge) {
views.Render(w, http.StatusOK, views.CommitDetail(s.Cfg, User(r), repo, sha, meta,
nil, true))
return
}
// Reject an oversized diff after generating it but before highlighting.
if int64(len(rawDiff)) > s.Cfg.MaxRenderBytes {
views.Render(w, http.StatusOK, views.CommitDetail(s.Cfg, User(r), repo, sha, meta,
nil, len(rawDiff)))
if err != nil {
http.Error(w, "Commit not found", gitStatusCode(err))
return
}
blobSize := func(hash string) int64 {
n, err := s.Git.BlobSize(r.Context(), repo.Name, hash)
if err != nil {
return 0
}
return n
blobSizes := func(hashes []string) map[string]int64 {
sizes, _ := s.Git.BlobSizes(r.Context(), repo.Name, hashes)
return sizes
}
// Keyed on the resolved hash: "sha" may be a ref name, and the same name
// points at different commits over time.
files := s.HL.PrepareDiff(rawDiff, "commit:"+repo.Name+":"+meta.Hash, blobSize)
views.Render(w, http.StatusOK, views.CommitDetail(s.Cfg, User(r), repo, sha, meta, files, 0))
files := s.HL.PrepareDiff(rawDiff, "commit:"+repo.Name+":"+meta.Hash, blobSizes)
views.Render(w, http.StatusOK, views.CommitDetail(s.Cfg, User(r), repo, sha, meta, files, false))
}
func (s *Server) branchList(w http.ResponseWriter, r *http.Request) {
Minternal/web/session.go
@@ -14,6 +14,9 @@ const (
sessionCookie = "session"
sessionIDBytes = 32
sessionDuration = 30 * 24 * time.Hour
// reauthAfter is how old a session may be for changes that add a way
// into the account, so a stolen old cookie cannot make itself permanent.
reauthAfter = 15 * time.Minute
)
type ctxKey int
@@ -105,3 +108,24 @@ func (s *Server) clearSession(w http.ResponseWriter, r *http.Request) {
}
http.SetCookie(w, s.sessionCookie("", -1))
}
// recentLogin reports whether the session signed in within reauthAfter.
func recentLogin(u *db.SessionUser) bool {
t, err := time.Parse(db.ISOLayout, u.SessionCreatedAt)
return err == nil && time.Since(t) < reauthAfter
}
// reauthURL sends the user to sign in again and come back to next.
func reauthURL(next string) string {
return "/login?next=" + queryEscape(next)
}
// dropOtherSessions signs the user out on every other device. Call it after
// a change to the account's login methods.
func (s *Server) dropOtherSessions(r *http.Request, userID int64) error {
keep := ""
if c, err := r.Cookie(sessionCookie); err == nil {
keep = c.Value
}
return s.DB.DeleteOtherSessions(r.Context(), userID, keep)
}
Ainternal/web/session_test.go
@@ -0,0 +1,174 @@
package web
import (
"context"
"net/http"
"net/http/httptest"
"net/url"
"strconv"
"strings"
"sync"
"testing"
"time"
"github.com/go-webauthn/webauthn/webauthn"
"hearthforge/internal/db"
)
func getWithCookie(h http.Handler, path string, c *http.Cookie) *httptest.ResponseRecorder {
req := httptest.NewRequest(http.MethodGet, path, nil)
req.AddCookie(c)
rec := httptest.NewRecorder()
h.ServeHTTP(rec, req)
return rec
}
func TestPasswordChangeDropsOtherSessions(t *testing.T) {
_, h := newTestServer(t)
current, other := loginAsAdmin(t, h), loginAsAdmin(t, h)
rec := postForm(t, h, "/settings/password", url.Values{
"current_password": {testAdminPassword},
"new_password": {"new-password-1"}, "confirm_password": {"new-password-1"},
}, current)
if loc := rec.Header().Get("Location"); loc != "/settings?success=password" {
t.Fatalf("Location = %q", loc)
}
if rec := getWithCookie(h, "/settings", other); rec.Code != http.StatusFound {
t.Errorf("other session still works: status %d", rec.Code)
}
if rec := getWithCookie(h, "/settings", current); rec.Code != http.StatusOK {
t.Errorf("current session was dropped: status %d", rec.Code)
}
}
func TestStaleSessionMustSignInAgain(t *testing.T) {
s, h := newTestServer(t)
cookie := loginAsAdmin(t, h)
const note = "needs a recent sign-in"
if strings.Contains(getWithCookie(h, "/settings", cookie).Body.String(), note) {
t.Error("fresh session shows the sign-in note")
}
old := time.Now().UTC().Add(-time.Hour).Format(db.ISOLayout)
if _, err := s.DB.ExecContext(context.Background(), `UPDATE sessions SET created_at = ?`, old); err != nil {
t.Fatal(err)
}
if page := getWithCookie(h, "/settings", cookie).Body.String(); !strings.Contains(page, note) ||
!strings.Contains(page, `href="/login?next=%2Fsettings"`) {
t.Error("stale session lacks the sign-in note")
}
rec := postForm(t, h, "/settings/password/remove", nil, cookie)
if loc := rec.Header().Get("Location"); loc != "/login?next=%2Fsettings" {
t.Fatalf("remove password Location = %q", loc)
}
rec = postForm(t, h, "/settings/passkey/revoke", url.Values{"id": {"1"}}, cookie)
if loc := rec.Header().Get("Location"); loc != "/login?next=%2Fsettings" {
t.Fatalf("revoke passkey Location = %q", loc)
}
rec = postForm(t, h, "/auth/passkey/register/options", nil, cookie)
if rec.Code != http.StatusForbidden || !strings.Contains(rec.Body.String(), `"reauth"`) {
t.Fatalf("passkey options: %d %s", rec.Code, rec.Body.String())
}
page := getWithCookie(h, "/login?next=%2Fsettings", cookie).Body.String()
if !strings.Contains(page, `name="next" value="/settings"`) || !strings.Contains(page, "sign in again") {
t.Error("login page lacks the return path or the notice")
}
for next, want := range map[string]string{"/settings": "/settings", "//evil.example": "/", "/\\evil.example": "/"} {
rec = postForm(t, h, "/login", url.Values{
"username": {db.AdminUsername}, "password": {testAdminPassword}, "next": {next},
})
if loc := rec.Header().Get("Location"); loc != want {
t.Errorf("next %q: Location = %q, want %q", next, loc, want)
}
}
}
func TestPasswordChangeIsRateLimited(t *testing.T) {
s, h := newTestServer(t)
s.Cfg.RateLimitDisabled = false
cookie := loginAsAdmin(t, h)
var loc string
for range 11 {
rec := postForm(t, h, "/settings/password", url.Values{
"new_password": {"new-password-1"}, "confirm_password": {"mismatch-1"},
}, cookie)
loc, _ = url.QueryUnescape(rec.Header().Get("Location"))
}
if !strings.Contains(loc, "Too many attempts") {
t.Errorf("Location = %q", loc)
}
}
func TestReservedUsernameIgnoresCase(t *testing.T) {
_, h := newTestServer(t)
rec := postForm(t, h, "/register", url.Values{
"username": {"Admin"}, "password": {"password123"}, "password2": {"password123"},
})
if !strings.Contains(rec.Body.String(), "reserved") {
t.Errorf("body = %q", rec.Body.String())
}
}
func registryRequest(user, pass, remote string) *http.Request {
req := httptest.NewRequest(http.MethodGet, "/v2/", nil)
req.SetBasicAuth(user, pass)
req.RemoteAddr = remote
return req
}
func TestRegistryUserCachesVerifiedCredentials(t *testing.T) {
s, _ := newTestServer(t)
s.Cfg.RateLimitDisabled = false
const ip = "198.51.100.7:1"
// More requests than the limiter allows: only the first runs argon2.
for i := range 15 {
if _, _, ok := s.registryUser(registryRequest(db.AdminUsername, testAdminPassword, ip)); !ok {
t.Fatalf("request %d rejected", i)
}
}
hash, err := db.HashPassword("changed-password")
if err != nil {
t.Fatal(err)
}
if err := s.DB.SetPasswordHash(context.Background(), 1, &hash); err != nil {
t.Fatal(err)
}
if _, _, ok := s.registryUser(registryRequest(db.AdminUsername, testAdminPassword, ip)); ok {
t.Error("cached credentials survived a password change")
}
}
func TestRegistryUserCountsParallelGuesses(t *testing.T) {
s, _ := newTestServer(t)
s.Cfg.RateLimitDisabled = false
const ip = "198.51.100.8:1"
var wg sync.WaitGroup
for range 10 {
wg.Go(func() { s.registryUser(registryRequest(db.AdminUsername, "wrong", ip)) })
}
wg.Wait()
if _, _, ok := s.registryUser(registryRequest(db.AdminUsername, testAdminPassword, ip)); ok {
t.Error("correct password accepted after the limit was used up")
}
}
func TestChallengeStoreEvictsOldestWhenFull(t *testing.T) {
c := challengeStore{entries: map[string]challengeEntry{}}
c.put("first", webauthn.SessionData{})
later := time.Now().Add(time.Hour)
for i := range maxChallenges - 1 {
c.entries[strconv.Itoa(i)] = challengeEntry{expires: later}
}
c.put("new", webauthn.SessionData{})
if _, ok := c.peek("new"); !ok {
t.Error("new entry was refused")
}
if _, ok := c.peek("first"); ok {
t.Error("oldest entry was not evicted")
}
if len(c.entries) != maxChallenges {
t.Errorf("entries = %d, want %d", len(c.entries), maxChallenges)
}
}
Minternal/web/settings.go
@@ -115,13 +115,21 @@ func (s *Server) settingsPage(w http.ResponseWriter, r *http.Request) {
if c, err := r.Cookie("theme"); err == nil && c.Value != "" {
theme = c.Value
}
reauth := ""
if !recentLogin(u) {
reauth = reauthURL("/settings")
}
q := r.URL.Query()
views.Render(w, http.StatusOK, views.Settings(s.Cfg, u, row.PasswordHash != nil,
passkeys, sshKeys, theme, q.Get("success"), q.Get("error"), pending, missing))
passkeys, sshKeys, theme, q.Get("success"), q.Get("error"), reauth, pending, missing))
}
func (s *Server) changePassword(w http.ResponseWriter, r *http.Request) {
u := User(r)
if !s.allowed(r, passwordLimiter, false) {
settingsError(w, r, "Too many attempts. Please try again later.")
return
}
if err := r.ParseForm(); err != nil {
http.Error(w, "Bad request", http.StatusBadRequest)
return
@@ -156,6 +164,9 @@ func (s *Server) changePassword(w http.ResponseWriter, r *http.Request) {
settingsError(w, r, "Current password is incorrect")
return
}
} else if !recentLogin(u) {
redirectTo(w, r, reauthURL("/settings"))
return
}
hash, err := db.HashPassword(newPassword)
@@ -167,11 +178,19 @@ func (s *Server) changePassword(w http.ResponseWriter, r *http.Request) {
http.Error(w, "Database error", http.StatusInternalServerError)
return
}
if err := s.dropOtherSessions(r, u.ID); err != nil {
http.Error(w, "Database error", http.StatusInternalServerError)
return
}
settingsSuccess(w, r, "password")
}
func (s *Server) removePassword(w http.ResponseWriter, r *http.Request) {
u := User(r)
if !recentLogin(u) {
redirectTo(w, r, reauthURL("/settings"))
return
}
removed, err := s.DB.ClearPasswordHash(r.Context(), u.ID)
if err != nil {
http.Error(w, "Database error", http.StatusInternalServerError)
@@ -181,11 +200,19 @@ func (s *Server) removePassword(w http.ResponseWriter, r *http.Request) {
settingsError(w, r, "Cannot remove password without a passkey")
return
}
if err := s.dropOtherSessions(r, u.ID); err != nil {
http.Error(w, "Database error", http.StatusInternalServerError)
return
}
settingsSuccess(w, r, "password_removed")
}
func (s *Server) revokePasskey(w http.ResponseWriter, r *http.Request) {
u := User(r)
if !recentLogin(u) {
redirectTo(w, r, reauthURL("/settings"))
return
}
id, err := formID(r, "id")
if err != nil {
settingsError(w, r, "Passkey not found")
@@ -209,6 +236,10 @@ func (s *Server) revokePasskey(w http.ResponseWriter, r *http.Request) {
settingsError(w, r, "Cannot revoke last auth method")
return
}
if err := s.dropOtherSessions(r, u.ID); err != nil {
http.Error(w, "Database error", http.StatusInternalServerError)
return
}
settingsSuccess(w, r, "passkey_revoked")
}
@@ -357,7 +388,7 @@ func (s *Server) adminCreateUser(w http.ResponseWriter, r *http.Request) {
settingsError(w, r, "Username may only contain letters, numbers, hyphens, and underscores")
return
}
if username == db.AdminUsername {
if strings.EqualFold(username, db.AdminUsername) {
settingsError(w, r, "That username is reserved")
return
}
Minternal/web/views/auth.go
@@ -9,13 +9,17 @@ import (
. "maragu.dev/gomponents/html"
)
// Login renders the sign-in page. error is empty when there is none.
func Login(cfg *config.Config, errMsg string) g.Node {
// Login renders the sign-in page. errMsg and next may be empty. next is a
// local path to return to after sign-in.
func Login(cfg *config.Config, errMsg, next string) g.Node {
return Layout(Page{Title: "Sign in", Cfg: cfg},
Div(Class("auth-container"),
H1(Class("page-title"), g.Text("Sign in")),
g.If(errMsg != "", P(Class("form-error"), g.Text(errMsg))),
g.If(errMsg == "" && next != "", P(Class("form-success"),
g.Text("Please sign in again to continue."))),
Form(Method("POST"), Action("/login"), Class("auth-form"),
g.If(next != "", Input(Type("hidden"), Name("next"), Value(next))),
Div(Class("form-group"),
Label(For("username"), g.Text("Username")),
Input(ID("username"), Name("username"), Type("text"), Required(),
Minternal/web/views/commentthread.go
@@ -29,12 +29,7 @@ func CommentThread(cfg *config.Config, user *db.SessionUser, comments []ThreadCo
commentReactions map[int64][]ReactionCount, baseURL, parentStatus string,
) g.Node {
reactURL := baseURL + "/react"
canEdit := func(c ThreadComment) bool {
if user == nil {
return false
}
return user.IsAdmin || (c.AuthorID != nil && user.ID == *c.AuthorID && parentStatus == "open")
}
canEdit := func(c ThreadComment) bool { return CanEdit(user, c.AuthorID, parentStatus) }
return g.Group{
g.Map(comments, func(comment ThreadComment) g.Node {
return Div(Class("timeline-item"),
Minternal/web/views/issues.go
@@ -24,6 +24,15 @@ func LabelsQueryParam(ids []int64) string {
return "&labels=" + strings.Join(parts, ",")
}
// CanEdit reports whether user may edit or delete an issue, a patch or one
// of their comments. Admins always may. Authors may only while it is open.
func CanEdit(user *db.SessionUser, authorID *int64, status string) bool {
if user == nil {
return false
}
return user.IsAdmin || (authorID != nil && *authorID == user.ID && status == "open")
}
func labelBadge(label db.Label) g.Node {
return Span(Class("label-badge"),
Style("background:"+label.Color+";color:"+util.LabelTextColor(label.Color)),
@@ -173,7 +182,7 @@ func IssueDetail(cfg *config.Config, user *db.SessionUser, repo *db.Repo, issue
) g.Node {
baseURL := "/" + repo.Name + "/issues/" + strconv.FormatInt(issue.Number, 10)
isAuthor := user != nil && issue.AuthorID != nil && user.ID == *issue.AuthorID
canEditIssue := user != nil && (user.IsAdmin || (isAuthor && issue.Status == "open"))
canEditIssue := CanEdit(user, issue.AuthorID, issue.Status)
canManageLabels := user != nil && (user.IsAdmin || (repo.AllowUserLabels && isAuthor))
isAdmin := user != nil && user.IsAdmin
closeLabel := "Reopen issue"
Minternal/web/views/patches.go
@@ -162,7 +162,7 @@ func PatchDetail(cfg *config.Config, user *db.SessionUser, repo *db.Repo, patch
commentReactions map[int64][]ReactionCount, patchLabels, repoLabels []db.Label,
) g.Node {
isAuthor := user != nil && patch.AuthorID != nil && user.ID == *patch.AuthorID
canEdit := user != nil && (user.IsAdmin || (isAuthor && patch.Status == "open"))
canEdit := CanEdit(user, patch.AuthorID, patch.Status)
canManageLabels := user != nil && (user.IsAdmin || (repo.AllowUserLabels && isAuthor))
isAdmin := user != nil && user.IsAdmin
Minternal/web/views/repos.go
@@ -26,7 +26,11 @@ func sigBadge(status gitcmd.SigStatus) g.Node {
case gitcmd.SigGood:
return Span(Class("sig-badge verified"), g.Text("verified"))
case gitcmd.SigBad:
return Span(Class("sig-badge unverified"), g.Text("unverified"))
return Span(Class("sig-badge bad"), Title("Signature is invalid or its key is revoked"),
g.Text("bad signature"))
case gitcmd.SigUnverified:
return Span(Class("sig-badge unverified"), Title("Signature could not be verified"),
g.Text("unverified"))
}
return nil
}
@@ -91,10 +95,10 @@ func CommitLog(cfg *config.Config, user *db.SessionUser, repo *db.Repo, logRef s
)
}
// CommitDetail renders one commit with its diff. tooLarge is the raw diff size
// when the diff was skipped, and 0 when the diff was rendered.
// CommitDetail renders one commit with its diff. tooLarge means the diff
// exceeded cfg.MaxRenderBytes and was skipped.
func CommitDetail(cfg *config.Config, user *db.SessionUser, repo *db.Repo, sha string,
meta *gitcmd.CommitMeta, files []highlight.RenderedDiffFile, tooLarge int,
meta *gitcmd.CommitMeta, files []highlight.RenderedDiffFile, tooLarge bool,
) g.Node {
short := sha
if len(short) > 7 {
@@ -148,11 +152,11 @@ func CommitDetail(cfg *config.Config, user *db.SessionUser, repo *db.Repo, sha s
Span(Class("commit-meta-value"), sigBadge(meta.SigStatus)))),
),
),
g.If(tooLarge > 0, Div(Class("file-download-notice"),
P(g.Text("Diff is too large to render inline ("+
formatMB(tooLarge)+" MB). Browse individual files at the tree below.")),
g.If(tooLarge, Div(Class("file-download-notice"),
P(g.Text("Diff is too large to render inline (over "+
util.FormatBytes(cfg.MaxRenderBytes)+"). Browse individual files in the tree.")),
)),
g.If(tooLarge == 0, DiffView(files, repo, sha)),
g.If(!tooLarge, DiffView(files, repo, sha)),
),
)
}
@@ -171,11 +175,6 @@ func parentLabel(n int) string {
return "Parent"
}
// formatMB renders a byte count as megabytes with one decimal.
func formatMB(bytes int) string {
return strconv.FormatFloat(float64(bytes)/1024/1024, 'f', 1, 64)
}
// BranchList renders the branch overview with admin actions.
func BranchList(cfg *config.Config, user *db.SessionUser, repo *db.Repo, branches []gitcmd.BranchInfo,
page PageInfo, success, errMsg string,
Minternal/web/views/repos_files.go
@@ -137,18 +137,30 @@ func breadcrumb(repoName, ref, p string, currentLast bool) g.Node {
)
}
// readmeSection renders the rendered README under a tree listing.
func readmeSection(repoName, ref, readmeHTML, readmePath string) g.Node {
if readmeHTML == "" {
// Readme is a directory's README. An empty Path means there is none.
type Readme struct {
Path string
HTML string
Size int64
TooLarge bool
}
// readmeSection renders the README under a tree listing.
func readmeSection(repoName, ref string, readme Readme) g.Node {
if readme.Path == "" {
return nil
}
rawHref := "/" + repoName + "/raw/" + EscapePath(ref) + "/" + EscapePath(readme.Path)
body := g.Node(Div(Class("markdown-body"), g.Raw(readme.HTML)))
if readme.TooLarge {
body = downloadNotice("README too large to display inline ("+util.FormatBytes(readme.Size)+")", rawHref)
}
return Div(Class("readme-section"),
Div(Class("readme-header"),
Span(g.Text("README")),
g.If(readmePath != "",
A(Href("/"+repoName+"/raw/"+EscapePath(ref)+"/"+EscapePath(readmePath)), Class("btn btn-sm"), g.Text("Raw"))),
A(Href(rawHref), Class("btn btn-sm"), g.Text("Raw")),
),
Div(Class("markdown-body"), g.Raw(readmeHTML)),
body,
)
}
@@ -165,7 +177,7 @@ func cloneURLs(cfg *config.Config, name string) (string, string) {
// RepoHome renders the repository landing page.
func RepoHome(cfg *config.Config, user *db.SessionUser, repo *db.Repo, entries []gitcmd.TreeEntry,
readmeHTML, readmePath string, hasContent bool, branches, tags []string,
readme Readme, hasContent bool, branches, tags []string,
) g.Node {
httpURL, sshURL := cloneURLs(cfg, repo.Name)
sshLine := ""
@@ -209,7 +221,7 @@ func RepoHome(cfg *config.Config, user *db.SessionUser, repo *db.Repo, entries [
),
),
FileTreeTable(repo.Name, repo.DefaultBranch, "", entries),
readmeSection(repo.Name, repo.DefaultBranch, readmeHTML, readmePath),
readmeSection(repo.Name, repo.DefaultBranch, readme),
}),
),
)
@@ -217,7 +229,7 @@ func RepoHome(cfg *config.Config, user *db.SessionUser, repo *db.Repo, entries [
// FileTree renders a directory listing at any ref and subpath.
func FileTree(cfg *config.Config, user *db.SessionUser, repo *db.Repo, treeRef, subpath string,
entries []gitcmd.TreeEntry, branches, tags []string, readmeHTML, readmePath string,
entries []gitcmd.TreeEntry, branches, tags []string, readme Readme,
) g.Node {
title := repo.Name
if subpath != "" {
@@ -238,14 +250,14 @@ func FileTree(cfg *config.Config, user *db.SessionUser, repo *db.Repo, treeRef,
),
g.If(subpath != "", breadcrumb(repo.Name, treeRef, subpath, false)),
FileTreeTable(repo.Name, treeRef, subpath, entries),
readmeSection(repo.Name, treeRef, readmeHTML, readmePath),
readmeSection(repo.Name, treeRef, readme),
),
)
}
// FileBlob renders one file: highlighted source, rendered markdown, a media
// preview or a download notice.
func FileBlob(cfg *config.Config, user *db.SessionUser, repo *db.Repo, blobRef, filePath string,
func FileBlob(cfg *config.Config, user *db.SessionUser, repo *db.Repo, blobRef, filePath, base string,
view highlight.FileView, branches, tags []string, markdownHTML, errMsg string,
) g.Node {
parts := splitPath(filePath)
@@ -277,6 +289,7 @@ func FileBlob(cfg *config.Config, user *db.SessionUser, repo *db.Repo, blobRef,
Form(Method("POST"),
Action("/"+repo.Name+"/delete-file/"+EscapePath(blobRef)+"/"+EscapePath(filePath)),
Class("popup-form"), Style("margin-top: var(--space-2);"),
Input(Type("hidden"), Name("base"), Value(base)),
Div(Class("form-group"),
Input(Name("message"), Type("text"),
Placeholder("Delete "+filename), MaxLength("500")),
@@ -301,18 +314,20 @@ func blobBody(repoName, filename, rawHref string, view highlight.FileView, markd
case view.Type == "media":
return Div(Class("file-media"), mediaElement(filename, rawHref, view.MimeType))
case view.Type == "binary":
return Div(Class("file-download-notice"),
P(g.Text("Binary file ("+util.FormatBytes(view.Size)+")")),
A(Href(rawHref), Class("btn btn-primary"), g.Text("Download")),
)
return downloadNotice("Binary file ("+util.FormatBytes(view.Size)+")", rawHref)
default:
return Div(Class("file-download-notice"),
P(g.Text("File too large to display inline ("+util.FormatBytes(view.Size)+")")),
A(Href(rawHref), Class("btn btn-primary"), g.Text("Download")),
)
return downloadNotice("File too large to display inline ("+util.FormatBytes(view.Size)+")", rawHref)
}
}
// downloadNotice stands in for content that is not rendered inline.
func downloadNotice(text, rawHref string) g.Node {
return Div(Class("file-download-notice"),
P(g.Text(text)),
A(Href(rawHref), Class("btn btn-primary"), g.Text("Download")),
)
}
func mediaElement(filename, rawHref, mimeType string) g.Node {
switch {
case strings.HasPrefix(mimeType, "image/"):
@@ -324,15 +339,28 @@ func mediaElement(filename, rawHref, mimeType string) g.Node {
}
}
// FileForm holds the editor fields, so a rejected save shows the input again.
type FileForm struct {
Path, Content, Message, Error string
// Base is the branch tip the content was loaded from.
Base string
// Stale means the file changed on the branch after Base was loaded.
Stale bool
}
// FileEdit renders the in-browser editor for one file.
func FileEdit(cfg *config.Config, user *db.SessionUser, repo *db.Repo, editRef, filePath,
content, errMsg string,
) g.Node {
func FileEdit(cfg *config.Config, user *db.SessionUser, repo *db.Repo, editRef, filePath string, f FileForm) g.Node {
parts := splitPath(filePath)
filename := filePath
if len(parts) > 0 {
filename = parts[len(parts)-1]
}
if f.Path == "" {
f.Path = filePath
}
if f.Message == "" {
f.Message = "Edited " + filename
}
cancelHref := "/" + repo.Name + "/blob/" + EscapePath(editRef) + "/" + EscapePath(filePath)
return Layout(Page{Title: "Edit " + repo.Name + "/" + filePath, User: user, Cfg: cfg},
Div(Class("container"),
@@ -341,8 +369,13 @@ func FileEdit(cfg *config.Config, user *db.SessionUser, repo *db.Repo, editRef,
breadcrumb(repo.Name, editRef, filePath, true),
P(Class("form-hint"),
g.Text(`WARNING: Line endings are normalized to LF (\n) on save.`)),
g.If(errMsg != "", P(Class("form-error"), g.Text(errMsg))),
g.If(f.Error != "", P(Class("form-error"), g.Text(f.Error),
g.If(f.Stale, g.Group{g.Text(" "),
A(Href("/"+repo.Name+"/blob/"+EscapePath(f.Base)+"/"+EscapePath(filePath)),
g.Text("View the current version.")),
}))),
Form(Method("POST"), Action("/"+repo.Name+"/edit/"+EscapePath(editRef)+"/"+EscapePath(filePath)),
Input(Type("hidden"), Name("base"), Value(f.Base)),
Div(Class("file-blob-header"),
Span(Class("file-blob-name"), g.Text(filename)),
Div(Class("file-blob-actions"),
@@ -353,20 +386,20 @@ func FileEdit(cfg *config.Config, user *db.SessionUser, repo *db.Repo, editRef,
Textarea(Name("content"), Class("file-edit-textarea"), Rows("30"),
SpellCheck("false"), g.Attr("autocorrect", "off"),
g.Attr("autocapitalize", "off"), AutoComplete("off"),
g.Text(content)),
g.Text(f.Content)),
),
Div(Class("form-card"),
P(Class("form-hint"), Style("margin-bottom: var(--space-4);"),
g.Text("Committing directly to "), Strong(g.Text(editRef))),
Div(Class("form-group"),
Label(For("file-path"), g.Text("File path")),
Input(ID("file-path"), Name("new_path"), Type("text"), Value(filePath),
Input(ID("file-path"), Name("new_path"), Type("text"), Value(f.Path),
Class("mono"), MaxLength(strconv.Itoa(maxFilePathLength))),
),
Div(Class("form-group"),
Label(For("message"), g.Text("Commit message")),
Textarea(ID("message"), Name("message"), Rows("3"), Required(),
g.Text("Edited "+filename)),
g.Text(f.Message)),
),
Div(Class("form-actions"),
Button(Type("submit"), Class("btn btn-primary"), g.Text("Commit changes")),
@@ -379,18 +412,19 @@ func FileEdit(cfg *config.Config, user *db.SessionUser, repo *db.Repo, editRef,
}
// NewFileForm renders the create-a-file page.
func NewFileForm(cfg *config.Config, user *db.SessionUser, repo *db.Repo, treeRef, dir, errMsg string) g.Node {
defaultPath := ""
func NewFileForm(cfg *config.Config, user *db.SessionUser, repo *db.Repo, treeRef, dir string, f FileForm) g.Node {
cancelHref := "/" + repo.Name + "/tree/" + EscapePath(treeRef)
if dir != "" {
defaultPath = dir + "/"
if f.Path == "" {
f.Path = dir + "/"
}
cancelHref += "/" + dir
}
return Layout(Page{Title: "New file — " + repo.Name, User: user, Cfg: cfg},
Div(Class("container"),
RepoHeader(repo),
RepoNav(cfg, repo, "code", user),
g.If(errMsg != "", P(Class("form-error"), g.Text(errMsg))),
g.If(f.Error != "", P(Class("form-error"), g.Text(f.Error))),
Form(Method("POST"), Action("/"+repo.Name+"/new-file/"+EscapePath(treeRef)),
Div(Class("file-blob-header"),
Span(Class("file-blob-name"), g.Text("New file")),
@@ -402,7 +436,7 @@ func NewFileForm(cfg *config.Config, user *db.SessionUser, repo *db.Repo, treeRe
Textarea(Name("content"), Class("file-edit-textarea"), Rows("20"),
SpellCheck("false"), g.Attr("autocorrect", "off"),
g.Attr("autocapitalize", "off"), AutoComplete("off"),
Placeholder("File contents...")),
Placeholder("File contents..."), g.Text(f.Content)),
),
Div(Class("form-card"),
P(Class("form-hint"), Style("margin-bottom: var(--space-4);"),
@@ -410,12 +444,13 @@ func NewFileForm(cfg *config.Config, user *db.SessionUser, repo *db.Repo, treeRe
Div(Class("form-group"),
Label(For("file-path"), g.Text("File path")),
Input(ID("file-path"), Name("path"), Type("text"), Required(),
Value(defaultPath), Placeholder("path/to/file.txt"), Class("mono"),
Value(f.Path), Placeholder("path/to/file.txt"), Class("mono"),
MaxLength(strconv.Itoa(maxFilePathLength))),
),
Div(Class("form-group"),
Label(For("message"), g.Text("Commit message")),
Textarea(ID("message"), Name("message"), Rows("3"), Placeholder("Add new file")),
Textarea(ID("message"), Name("message"), Rows("3"), Placeholder("Add new file"),
g.Text(f.Message)),
),
Div(Class("form-actions"),
Button(Type("submit"), Class("btn btn-primary"), g.Text("Create file")),
Minternal/web/views/settings.go
@@ -30,8 +30,10 @@ var successMessages = map[string]string{
}
// Settings renders the user settings page, including the admin sections.
// reauth is the sign-in link for a session too old to change auth methods,
// or empty.
func Settings(cfg *config.Config, user *db.SessionUser, hasPassword bool,
passkeys []db.Passkey, sshKeys []db.SSHKey, theme, success, errMsg string,
passkeys []db.Passkey, sshKeys []db.SSHKey, theme, success, errMsg, reauth string,
pendingUsers []db.PendingUser, missingRepos []db.RepoCounts,
) g.Node {
successMsg := successMessages[success]
@@ -82,6 +84,10 @@ func Settings(cfg *config.Config, user *db.SessionUser, hasPassword bool,
),
),
g.If(reauth != "", P(Class("text-muted"),
g.Text("Changing passwords or passkeys needs a recent sign-in. "),
A(Href(reauth), g.Text("Sign in again")))),
Div(Class("form-card"),
H2(Class("section-title"), g.Text("Password")),
P(Class("text-muted"), g.Text(passwordState(hasPassword))),
Minternal/web/webauthn.go
@@ -49,22 +49,24 @@ var challenges = challengeStore{entries: map[string]challengeEntry{}}
// unauthenticated, so without a cap an attacker can grow memory without bound.
const maxChallenges = 10000
// put stores a session and drops every expired entry. It reports false when
// the store is full, so the caller can refuse the ceremony.
func (c *challengeStore) put(key string, session webauthn.SessionData) bool {
// put stores a session and drops every expired entry. When full it evicts
// the oldest entry instead of refusing new ceremonies.
func (c *challengeStore) put(key string, session webauthn.SessionData) {
c.mu.Lock()
defer c.mu.Unlock()
now := time.Now()
oldest := ""
for k, e := range c.entries {
if now.After(e.expires) {
delete(c.entries, k)
} else if oldest == "" || e.expires.Before(c.entries[oldest].expires) {
oldest = k
}
}
if _, replacing := c.entries[key]; !replacing && len(c.entries) >= maxChallenges {
return false
delete(c.entries, oldest)
}
c.entries[key] = challengeEntry{session: session, expires: now.Add(challengeTTL)}
return true
}
// take returns a session and removes it, so a challenge is used once.
@@ -165,6 +167,13 @@ func (s *Server) passkeyRegisterOptions(w http.ResponseWriter, r *http.Request)
jsonError(w, http.StatusUnauthorized, "Not authenticated")
return
}
if !recentLogin(su) {
writeJSON(w, http.StatusForbidden, map[string]string{
"error": "Please sign in again to add a passkey.",
"reauth": reauthURL("/settings"),
})
return
}
wa, err := s.webAuthn()
if err != nil {
jsonError(w, http.StatusInternalServerError, "WebAuthn is misconfigured")
@@ -193,7 +202,7 @@ func (s *Server) passkeyRegisterOptions(w http.ResponseWriter, r *http.Request)
webauthn.WithConveyancePreference(protocol.PreferNoAttestation),
webauthn.WithAuthenticatorSelection(protocol.AuthenticatorSelection{
ResidentKey: protocol.ResidentKeyRequirementPreferred,
UserVerification: protocol.VerificationPreferred,
UserVerification: protocol.VerificationRequired,
}),
)
if err != nil {
@@ -201,10 +210,7 @@ func (s *Server) passkeyRegisterOptions(w http.ResponseWriter, r *http.Request)
return
}
if !challenges.put("reg:"+su.Username, *session) {
jsonError(w, http.StatusServiceUnavailable, "Too many sign-in attempts in flight. Please try again later.")
return
}
challenges.put("reg:"+su.Username, *session)
// @simplewebauthn/browser expects the bare creation options, not the
// { publicKey: ... } wrapper the library's top-level type marshals to.
writeJSON(w, http.StatusOK, creation.Response)
@@ -252,6 +258,10 @@ func (s *Server) passkeyRegisterVerify(w http.ResponseWriter, r *http.Request) {
return
}
challenges.take("reg:" + su.Username)
if err := s.dropOtherSessions(r, su.ID); err != nil {
jsonError(w, http.StatusInternalServerError, "Database error")
return
}
writeJSON(w, http.StatusOK, map[string]bool{"ok": true})
}
@@ -269,15 +279,13 @@ func (s *Server) passkeyLoginOptions(w http.ResponseWriter, r *http.Request) {
}
// No allowCredentials: the user picks a discoverable credential, so the
// sign-in page needs no username.
assertion, session, err := wa.BeginDiscoverableLogin()
assertion, session, err := wa.BeginDiscoverableLogin(
webauthn.WithUserVerification(protocol.VerificationRequired))
if err != nil {
jsonError(w, http.StatusInternalServerError, err.Error())
return
}
if !challenges.put("login:"+session.Challenge, *session) {
jsonError(w, http.StatusServiceUnavailable, "Too many sign-in attempts in flight. Please try again later.")
return
}
challenges.put("login:"+session.Challenge, *session)
writeJSON(w, http.StatusOK, assertion.Response)
}
Mweb/static/assets/css/components.css
@@ -525,10 +525,15 @@
}
.issue-badge.closed,
.patch-badge.closed,
.sig-badge.unverified {
.sig-badge.bad {
background: var(--color-danger-bg);
color: var(--color-danger);
}
.sig-badge.unverified {
background: var(--color-bg-subtle);
color: var(--color-text-muted);
cursor: help;
}
.issue-badge.completed,
.patch-badge.merged {
background: var(--color-merged-bg);
Mweb/static/assets/passkey-login.js
@@ -32,7 +32,9 @@ if (btn) {
body: JSON.stringify(result),
});
if (verResp.ok) {
window.location.href = "/";
// The server validated next when it rendered the form.
const next = document.querySelector('input[name="next"]');
window.location.href = next ? next.value : "/";
} else {
const err = await verResp.json().catch(() => ({}));
showError(err.error ?? "Passkey sign in failed");
Mweb/static/assets/passkey-settings.js
@@ -16,7 +16,14 @@ if (btn) {
const optsResp = await fetch("/auth/passkey/register/options", {
method: "POST",
});
if (!optsResp.ok) throw new Error("Failed to get options");
if (!optsResp.ok) {
const err = await optsResp.json().catch(() => ({}));
if (err.reauth) {
window.location.href = err.reauth;
return;
}
throw new Error(err.error ?? "Failed to get options");
}
const opts = await optsResp.json();
const result = await startRegistration({ optionsJSON: opts });
const verResp = await fetch("/auth/passkey/register/verify", {