CI/CD pipeline (DevOps)

How Bloom is validated, built, and shipped through GitHub Actions. This is the pipeline overview; for deployment mechanics in depth (immutable images, deploy-by-tag, rollback, one-time setup) see deployment.md, and for the contributor-facing merge gate see the "Quality gates" section of AGENTS.md.

At a glance

PR to main ─┬─ CI: security ─▶ test ─▶ build (image ▶ GHCR)      [must pass to merge]
            └─ e2e-preview: full stack in the runner, isolated    [pre-merge gate]

push to main ─ CI (same) ─▶ Deploy (dev, k3s): helm by tag ─▶ e2e (dev): post-deploy
                                                               └─ fail ▶ auto-rollback (helm)

Two rules shape everything:

• main is the dev release branch - a push to it auto-deploys dev. So the bar to land on main is high (full CI + e2e-preview), and the m<N> -> main milestone PR is human-gated.
• Build-once, deploy-many - the API is built once as a SHA-tagged image; every deploy and rollback runs that exact image, never a rebuild.

Continuous integration

ci.yml orchestrates reusable stage workflows; each lives in its own file. It runs on every PR and on pushes to main.

Stage	Workflow	What it does
preflight	ci.yml	Two cheap classifications in one job: the tree lookup (has this exact tree already passed a full run? see below) and the path filter (.github/actions/changes, #788: which gates can this diff affect).
security	ci-security.yml	Dependency vuln scan (OWASP: pnpm audit --prod, straight from the lockfile - no install). Vulnerable production deps block; dev/tooling vulns are reported non-blocking.
test	ci-test.yml	Three jobs. Test (web): lint (eslint), format check (prettier), type-check (tsc), the unit/integration suites (vitest, incl. the @bloom/ui story tests), the admin E2E, and the OpenAPI drift guard. Infra: the Helm chart, OpenTofu module and deploy-script gates. Sites: the docs and landing site builds.
storybook	ci-storybook.yml	The @bloom/ui Storybook static build, the Chromatic visual-regression check (opt-in via CHROMATIC_PROJECT_TOKEN), and the hosted Storybook on Cloudflare Pages (per-PR alias + PR comment; the stable alias on main). Gated on the ui filter; never skipped by the tree lookup.
build	ci-build.yml	Builds the API Docker image and, on a push to main, pushes it to GHCR tagged sha-<12> + latest (PRs build to validate, no push). Also builds the web, admin and server bundles.
changelog	ci-changelog.yml	On PRs into main only: towncrier check fails a PR that adds no changelog.d/ fragment (#518). Assembling CHANGELOG.md from per-PR fragments kills the top-insert merge-conflict class; opt out of the gate with an empty +<slug>.misc.md fragment. See AGENTS.md → "Changelog entries".

Build gating. build runs only when the work is bound for main - a push to main or a PR whose base is main. Issue PRs into a milestone branch (m<N>) skip build and get fast security + test feedback; nothing reaches main without a green build.

Where the full proof runs (the tree lookup). A PR into main runs every gate with the path filter forced open, exactly like a push to main - the milestone PR is the full proof. Its squash merge is byte-for-byte the tree that PR's run tested, so the push to main does not repeat the test stage: the preflight job's tree lookup (scripts/ci/tested-twin.mjs) looks the tree up and, when a full run has already passed it, the test stage is skipped and the run goes straight to build + deploy. How it knows: every full run records the tree it tests as the private R2 artifact tested-tree-<tree> before anything else runs; a path-filtered run (a PR into m<N>) records nothing, because it did not test everything. Only an identical tree qualifies (a docs edit is a real change - the docs site build reads docs/), only a run that concluded success counts, and any doubt (an API error) runs everything. security runs on every push regardless: advisories change under an unchanged tree.

Runner budget: few jobs, many steps. Every job waits for one of the two bloom-arc runner slots (shared with the other repos' CI) and pays ~15-40 s of set-up, so related checks share a job instead of each taking a dispatch of its own: a full PR into main runs 8 CI jobs (preflight, security, the three test jobs, storybook, changelog, build), and a push to main whose tree already passed runs 4 (preflight, security, storybook, build). Inside a shared job every check still runs and reports when an earlier one failed (!cancelled() plus the outcome of the set-up step it needs), so one red run shows every failure, as separate jobs did.

Docs site build. The docs site (apps/docs, Docusaurus) is validated by the docs step of the Sites job in the test stage (ci-test.yml), gated on the docs path filter - it runs on every PR whose diff can affect the site (apps/docs/, docs/ content, the TypeDoc-documented packages, shared config) and on every push to main, where the filters are forced open. Main-bound work therefore always carries a green docs build before the docs deploy fires; the build is deliberately not duplicated in ci-build.yml - the deploy workflow rebuilds the site itself at publish time (static output only, nothing to hand between stages).

Landing site build. The end-user landing site (apps/landing) is validated the same way by the landing steps of the Sites job, gated on the landing path filter (apps/landing/**, the @bloom/ui sources it compiles into its bundle, and shared config). The job runs the full prerender (pnpm --filter bloom-landing build) and asserts the SEO contract on the output - a single <h1> and no <!--app-html--> placeholder left in dist/index.html. Because apps/landing is also a workspace package, an apps/landing/** change additionally opens the web filter so the root lint/format/typecheck/test fan-out covers it. As with the docs site, the build is not duplicated in ci-build.yml; deploy-landing-dev.yml rebuilds at publish time.

Changelog gating. changelog runs only on pull_request events whose base is main - not on the push/merge to main (which carries no fragment) and not on release PRs that touch only CHANGELOG.md. It is orthogonal to the security -> test -> build chain and does not gate deploys.

End-to-end tests (two lanes)

The same Playwright suite (apps/web/e2e) runs in two places:

Pre-merge, ephemeral - e2e-preview.yml (E2E-6)

On every PR to main, stands up the whole stack inside the runner - the API (fake LLM provider, in-memory store, non-prod test-login) and the Vite dev server proxying /api to it - then runs the suite against it. It uses the PR's own API and web source, so API changes are tested pre-merge. Fully isolated: no shared environment, no durable DB, torn down with the job. Make it a required status check to block merges on regressions.

Post-deploy, dev + staging - e2e.yml / e2e-staging.yml (M24-9)

After a dev deploy, runs the suite against the deployed dev dashboard (E2E_BASE_URL; inert until set); the staging twin is dispatch-only (E2E_STAGING_BASE_URL) until M25 ships a staging deploy to hook. Both call the reusable e2e-post-deploy.yml stage, which authenticates via the environment's password-gated test-login secret. Non-blocking to the deploy that already happened. It also closes the loop on bad deploys (E2E-7): a failure after the k3s API deploy auto-rolls the dev API back to the previous helm revision (a pass needs no bookkeeping - helm's history is the rollback target) - decision matrix in scripts/deploy/e2e-promote-rollback.sh, unit-tested offline (see deployment.md).

The pre-merge lane is the isolated safety net; the post-deploy lane guards the live dev environment. See apps/web/e2e/README.md for the flake policy and how to read a failure.

Deploy & rollback

Deploys are gated on CI success and run only from main (production is manual). Depth is in deployment.md; in brief:

Workflow	Trigger	Role
deploy-k3s-dev.yml	CI success on main (+ dispatch)	Deploy the SHA-tagged API image to the dev k3s cluster via Helm (scripts/deploy/helm-dev.sh): sealed-secret sync, helm upgrade --install --set image.tag=sha-<12> --wait, then a /health endpoint gate through the ingress. The sole dev API deploy path since the M24-5 compose→k3s cutover (runbook).
deploy-web-dev.yml	CI success on main	Deploy the web bundle to Cloudflare Pages (dev project bloom-dashboard -> app.dev.bloom.hiddenclaw.org, M48-1).
deploy-docs-dev.yml	CI success on main (+ dispatch)	Build the docs site (apps/docs, Docusaurus) and deploy it to Cloudflare Pages (bloom-docs -> docs.dev.bloom.hiddenclaw.org, M49-1). Fails loud when Cloudflare is unconfigured - never a silent skip (M45-5).
deploy-landing-dev.yml	CI success on main (+ dispatch)	Build the landing site (apps/landing, prerendered / and /adopt) and deploy it to Cloudflare Pages (bloom-landing -> dev.bloom.hiddenclaw.org, M48-1). Same fail-loud preflight as the docs lane (M46-3).
deploy-rollback-k3s-dev.yml	workflow_dispatch	helm rollback the dev k3s release to the previous (or a chosen) revision, health-gated like the deploy.
deploy-production.yml	workflow_dispatch	Deploy/roll back production by tag (mode: deploy/rollback); inert until the production environment is configured.
deploy-web-production.yml	workflow_dispatch	Deploy the web bundle to the production Pages project.
deploy-docs-production.yml	workflow_dispatch	Deploy the docs site to the production Pages project (bloom-docs-prod); same fail-loud contract as the dev docs deploy.

The VM keeps the current + previous (+ last-e2e-green) images so any rollback is local and never depends on the registry. App rollback is not data rollback - image rollback does not revert Supabase migrations; keep schema changes backward-compatible.

Configuration & secrets

GitHub is the source of truth: each deploy renders .env from Actions secrets + variables (scripts/deploy/render-env.sh) - never hand-edit .env on a server. Shared app config lives at the repository level; per-environment overrides live on the dev / production GitHub Environments (they win), so prod uses a distinct bot/DB/domain. Deploy-only infra (DEPLOY_*) is environment-scoped and excluded from the rendered .env. The VM pulls the private image with the workflow's GITHUB_TOKEN (packages: read) - no long-lived registry secret on the VM.

Merge gating summary

• Issue PR -> m<N>: security + test (build skipped). The agent may self-merge after green CI.
• PR -> main (milestone PR or standalone): full security + test + build + e2e-preview. Human-gated - a person reviews and merges (merging main auto-deploys dev). Mark e2e-preview (and, once stable, the deployed e2e) required status checks in branch protection.
• Never merge red CI.

Running the checks locally

From apps/server (the test stage; same scripts exist in every workspace package):

pnpm run lint            # eslint
pnpm run format:check    # prettier
pnpm run typecheck       # strict tsc
pnpm run test            # vitest unit + integration tests

For apps/web: pnpm run lint, pnpm run format:check, pnpm run typecheck, pnpm run test. Pre-commit runs ruff, ruff-format, mypy, pytest, and the commitizen check on every commit. To run the e2e suite locally against an ephemeral stack, see apps/web/e2e/README.md.

Artifact storage and authenticated retrieval: CI artifacts.