Docker, deployment, observability, security

Dockerfile

Multi-stage, non-root, dev dependencies excluded from the runtime image:

FROM python:3.12-slim AS builder
WORKDIR /app
ENV PIP_NO_CACHE_DIR=1 PYTHONDONTWRITEBYTECODE=1
COPY requirements.txt .
RUN pip install --prefix=/install -r requirements.txt

FROM python:3.12-slim AS runtime
WORKDIR /app
ENV PYTHONUNBUFFERED=1 PYTHONDONTWRITEBYTECODE=1
COPY --from=builder /install /usr/local
COPY app ./app
RUN useradd -r -u 1001 appuser && chown -R appuser /app
USER appuser
EXPOSE 8000
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000"]

What each line is defending against:

• Two stages. A single-stage build that installs requirements-dev.txt

ships pytest, linters, and their transitive dependencies into production. That is a larger image and a larger attack surface, for zero benefit.

• COPY app, not COPY .. COPY . drags in .env files, .git, test

fixtures, and local junk. Pair with a real .dockerignore.

• Non-root. A container escape starting as root is a much shorter path to a

bad day.

• PYTHONUNBUFFERED. Without it, logs sit in a buffer and your container

looks silent during the exact incident you are trying to debug.

• Pin the base image to a minor version. python:3-slim will upgrade

underneath you.

Processes and workers

One process per container, scaled by replicas, is the right default under Kubernetes or any orchestrator: the scheduler already handles restarts, rolling updates, and distribution, and per-process metrics stay legible.

Use multiple workers (gunicorn with uvicorn workers) only when running on a fixed VM without an orchestrator.

Worker count is not "cores times two" for an async service. A correctly async FastAPI process saturates a core on its own; oversubscribing adds context switching and memory without throughput. Measure before tuning.

Raise the AnyIO threadpool limit if you deliberately rely on sync endpoints:

import anyio
limiter = anyio.to_thread.current_default_thread_limiter()
limiter.total_tokens = 100      # default 40

Raising this without knowing why is cargo culting. Do it when sync handlers are measurably queueing.

Health checks

Two endpoints, answering two different questions:

@app.get("/healthz")            # liveness: is this process wedged?
async def healthz():
    return {"ok": True}         # NO dependency checks

@app.get("/readyz")             # readiness: can it serve traffic?
async def readyz(db = Depends(get_db)):
    await db.command("ping")
    return {"ok": True}

Pointing liveness at the database is a common and costly mistake: a five second database blip restarts every pod, turning a brief degradation into an outage, and the restarts stampede the recovering database.

Migrations

Never run migrations from application startup. With more than one replica, every pod races to apply the same migration on every rollout. Depending on the engine you get duplicate-key errors, partial application, or a deadlock, and it happens exactly when you are deploying.

Run them as a separate step that executes once:

• Kubernetes: an init job, or a Helm pre-upgrade hook
• Compose: a one-shot service the API depends_on
• CI/CD: a pipeline stage before the rolling update

The same applies to index creation and seeding. Index creation is often idempotent and survives the race, which is worse, because it teaches you the pattern is safe right up until the migration that is not.

Design migrations to be backward compatible for one release: add a column, deploy code that writes both, backfill, then drop. A migration that breaks the currently-running version makes a rolling deploy an outage and a rollback impossible.

Logging

JSON in production, human-readable in development, one request id threaded through everything:

@app.middleware("http")
async def request_id_middleware(request: Request, call_next):
    rid = request.headers.get("x-request-id") or str(uuid4())
    request_id_ctx.set(rid)                 # contextvar, picked up by the formatter
    response = await call_next(request)
    response.headers["x-request-id"] = rid
    return response

Accepting an inbound x-request-id is what lets you follow one user action across services. Returning it lets support paste an id from a screenshot and find the exact request.

• Log at the edges: request in, response out with status and duration, plus

every unhandled exception. Not on every line of business logic.

• Never log passwords, tokens, full card numbers, or entire request bodies

on auth routes. Redact at the formatter, not at each call site, because the call site is where it gets forgotten.

• Use logger.exception inside handlers so the stack trace survives.

Metrics and tracing

Adopt in this order; each step is useful alone, and skipping to the last one usually produces dashboards nobody reads:

1. Request id, JSON logs, health checks. Enough to debug most incidents.
2. RED metrics per route: Rate, Errors, Duration. Alert on error rate and

p95 latency, not on CPU.

1. OpenTelemetry tracing once more than one service is involved and "which

hop was slow" stops being obvious.

Inject trace and span ids into log records, so a slow trace links to its logs without timestamp archaeology.

Security

Tokens. Short-lived access tokens (10 to 15 minutes) plus refresh tokens. Pin the algorithm when decoding; a decoder that trusts the header's alg is how alg: none forgery happens.

jwt.decode(token, settings.jwt_secret, algorithms=["HS256"])   # explicit list

Passwords. bcrypt or argon2, never a bare SHA. Hashing is CPU-bound, so keep it off the event loop (see the async section in SKILL.md).

CORS. Enumerate origins. allow_origins=[""] together with allow_credentials=True is rejected by browsers, and reaching for to make an error go away is how a public API ends up trusting every site.

Rate limiting on auth endpoints specifically. Login, password reset, and token refresh are where credential stuffing lands.

Request size limits at the proxy. FastAPI will happily buffer a 2GB upload into memory if nothing upstream says otherwise.

Secrets from the environment or a secret manager, never committed. Files like .env.local and .env.test belong in .gitignore, and committed .env.<environment> files should hold only non-secret build-time values.

OpenAPI in production. Serving /docs publicly advertises every endpoint and shape. Gate it behind auth or disable it outside development:

app = FastAPI(docs_url=None if settings.environment == "production" else "/docs")