The pipeline

Annotated. Copy it, change the DEPLOY_HOOK values, and it works.

Stages and default tags

stages:
  - test
  - build
  - deploy
  - promote

# Pin every job to the self-hosted runner. Without a tag, GitLab offers the job
# to the SaaS shared fleet as well, and those runners are always warm, so they
# win the race every time and our runner never picks anything up. A runner may
# only take a job whose tags it all carries, and the shared runners are
# untagged, so one tag is enough to exclude them.
default:
  tags:
    - self-hosted

Order matters: a failing test stops the build, a failing build stops the deploy, and the promote button only appears after a successful deploy.

Jobs that need Docker

test and build_image drive a Docker daemon, so they declare an image, a dind service, and the TLS wiring:

build_image:
  stage: build
  tags:
    - self-hosted
    - dind
  image: docker:28-cli
  services:
    - docker:28-dind
  variables:
    # Talk to the dind service over TLS. DOCKER_TLS_CERTDIR must match the
    # /certs/client volume the runner is registered with, or client and daemon
    # look at different cert directories.
    DOCKER_HOST: tcp://docker:2376
    DOCKER_TLS_CERTDIR: "/certs"
    DOCKER_TLS_VERIFY: 1
    DOCKER_CERT_PATH: "/certs/client"
    DOCKER_BUILDKIT: 1
  before_script:
    # dind takes a moment to accept connections. Wait explicitly so a slow start
    # fails with this message rather than a confusing "cannot connect to the
    # Docker daemon" from the middle of the build.
    - |
      for i in $(seq 1 30); do
        docker info >/dev/null 2>&1 && break
        if [ "$i" = 30 ]; then echo "dind did not become ready after 30s"; exit 1; fi
        sleep 1
      done
  script:
    - docker login -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" "$CI_REGISTRY"
    - ENV_FILE=".env.$CI_COMMIT_REF_SLUG"
    - test -f "$ENV_FILE" || { echo "$ENV_FILE not found - every deployed branch needs one"; exit 1; }
    - SITE_URL="$(grep -E '^NEXT_PUBLIC_SITE_URL=' "$ENV_FILE" | cut -d= -f2-)"
    - SHA_TAG="$CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA"
    - BRANCH_TAG="$CI_REGISTRY_IMAGE:$CI_COMMIT_REF_SLUG"
    # dind's image store starts empty, so warm the cache from the last build.
    - docker pull "$BRANCH_TAG" || echo "no previous $BRANCH_TAG - building cold"
    - |
      docker build --pull \
        --cache-from "$BRANCH_TAG" \
        --build-arg BUILDKIT_INLINE_CACHE=1 \
        --build-arg NEXT_PUBLIC_SITE_URL="$SITE_URL" \
        -t "$SHA_TAG" -t "$BRANCH_TAG" .
    - docker push "$SHA_TAG"
    - docker push "$BRANCH_TAG"
  rules:
    - if: '$CI_COMMIT_BRANCH == "dev"'
    - if: '$CI_COMMIT_BRANCH == "beta"'
    - if: '$CI_COMMIT_BRANCH == "prod"'

Points worth keeping:

• Two tags every build. :<short-sha> is immutable and answers "what

shipped"; :<branch> is what compose pulls.

• The cache warm. dind starts with an empty image store, so without

docker pull + --cache-from every build reinstalls dependencies cold.

• The .env.<branch> guard. A missing file fails immediately with a clear

message instead of building an image with an empty build-arg.

Jobs that do not need Docker

Give them the smallest image that has the one tool they need.

deploy:
  stage: deploy
  # This job needs curl and nothing else. The docker images ship wget but not
  # curl, and wget cannot report the status code the check below needs.
  image: alpine:3.20
  before_script:
    - apk add --no-cache curl

.promote:
  stage: promote
  # git only. docker:28-cli ships it, and the image is already pulled.
  image: docker:28-cli

The curl point is not pedantry. docker:28-cli has docker, git, and wget, but no curl, and the deploy check depends on curl -w "%{http_code}".

Docker executor, not shell

Jobs run in containers. A shell executor is simpler on day one and worse afterwards:

• Jobs share the host's Docker daemon, so concurrent pipelines collide on image

tags, compose project names, and one host-level ~/.docker/config.json

• Build junk accumulates in the runner's working directory
• The host needs every tool any job might use

With the docker executor each job gets a clean container, and dind gives it a clean daemon. Nothing a build does outlives the job.

The dind gotcha worth knowing: a compose file that bind-mounts the workspace (- .:/app) breaks. The daemon runs in a separate container, so the host path resolves there and mounts an empty directory over your code. Tests then fail in a way that looks like a code bug. Build the code into the image and let CI test that instead, which is what CI should be testing anyway.

The test job

Runs against real dependencies, in Docker, same engine and version as production:

test:
  stage: test
  tags: [self-hosted, dind]
  image: docker:28-cli
  services: [docker:28-dind]
  resource_group: test-stack
  variables:
    DOCKER_HOST: tcp://docker:2376
    DOCKER_TLS_CERTDIR: "/certs"
    DOCKER_TLS_VERIFY: 1
    DOCKER_CERT_PATH: "/certs/client"
    COMPOSE_FILE: docker-compose.test.yml
  script:
    - docker compose up -d <db-service>
    - docker compose run --rm <test-service> pytest --maxfail=3 -q
  after_script:
    - docker compose down -v --remove-orphans || true

In the compose file: tmpfs for the database data directory, a healthcheck with condition: service_healthy so the tests do not race the database, and a raised nofile limit if the engine opens a descriptor per collection.

If you run compose directly rather than through a test service, remember --abort-on-container-exit --exit-code-from <service>. Without them compose exits 0 and a red suite goes green.