skills/zitadel-production/

references/untested.md

Researched but not yet verified

Everything in this file is from documentation only. Unlike the rest of this skill, none of it has been exercised against a live instance. Treat it as a well-informed starting point, not as established fact: exact field names, console paths and behaviours may differ, and the surrounding pages have already proved that ZITADEL's defaults and error messages surprise you. When you do run one of these, verify it the way the rest was verified (see testing.md), then move it into its own reference file and delete it from here.

Four areas, in the order they are likely to be needed.

1. Service accounts (machine to machine)

Needed first, because your own Terraform provisioning has to authenticate as something, and every client project eventually grows a background job.

Terminology note: "Service Accounts" is the current name. Older material says service user, machine user or technical account, all the same thing.

Three authentication methods

Method	Trade-off
Private Key JWT (recommended)	Decentralised verification, better performance. More complex to implement, and hard to revoke before expiry. The private key must be stored with the highest care.
Client Credentials	Simpler. Documented as less secure: requires server verification per request, and a leaked secret allows impersonation.
Personal Access Token (PAT)	Easiest by far. Long-lived, so trivially exploitable if leaked. Available for service accounts only, not human users.

Private Key JWT, expected flow

Console, Service Accounts, New, set username and display name
Open the account, Keys tab, New, optional expiry, Download the JSON

The key JSON is documented to contain:

{ "type": "serviceaccount", "keyId": "...", "key": "<PRIVATE KEY>", "userId": "..." }

Build and sign an assertion with RS256:

header  { "alg": "RS256", "kid": "<keyId>" }
payload { "iss": "<userId>",
          "sub": "<userId>",
          "aud": "https://<your-domain>",
          "iat": <now, no more than 1 hour old>,
          "exp": <expiry> }

Exchange it:

POST https://<domain>/oauth/v2/token
  grant_type = urn:ietf:params:oauth:grant-type:jwt-bearer
  assertion  = <signed JWT>
  scope      = openid ...

Client Credentials, expected flow

Service account, Actions menu, Generate Client Secret. Shown once, same as an application secret.

POST https://<domain>/oauth/v2/token
  grant_type    = client_credentials
  scope         = openid ...          # openid is required
  Authorization = Basic base64(client_id:client_secret)

The scope everyone misses

To call ZITADEL's own APIs (management, admin, auth), the token request must include:

urn:zitadel:iam:org:project:id:zitadel:aud

This adds the ZITADEL project to the token's audience. Without it, ZITADEL's APIs check their own presence in aud and reject the token. That is the likely cause of a confusing 403 when a service account "obviously has permission".

For your own APIs this is not needed: audience works as documented in integration.md, against the project ID.

Permissions

Getting a token is not authorisation. The account must also be granted a manager role, assigned in the console by adding the service account to an organization with a role:

IAM administrators (instance-wide, the highest level)
Organization administrators
Project administrators
Project grant administrators

To verify: create a service account, grant it an org-level role, obtain a token by private key JWT, confirm it verifies against JWKS like any other, then confirm it is rejected by the ZITADEL API without the audience scope and accepted with it. That last contrast is the whole lesson.

2. Authorization: what ZITADEL does and where it stops

The model

Roles are defined per project, each with a Key (the programmatic

identifier that appears in tokens), a display name, and an optional Group for bulk assignment.

Role Assignments (previously User Grants or Authorizations) bind a user

to one or more roles within an organization. Assignable from the organization, the project, or the user page.

Project Grants share selected roles of your project with **another

organization, and the receiving organization then assigns those roles to its own users self-service**, without asking you.

Project grants are the built-in answer to a client's reseller, partner or franchise scenarios, and another reason to keep the organization layer free.

Where it stops

ZITADEL's model is role-based and project-scoped. The documentation does not address resource-level or relationship-based permissions, and there is no evidence of a Zanzibar-style capability.

So "can this user edit this document" is your problem, not ZITADEL's. The roles claim is an input to that decision, never the decision itself.

Practical ladder:

Roles in the token cover coarse checks: is this an admin, an editor, a

viewer. Free, already there.

Your own tables cover ownership and membership: joins against app_user,

which is exactly why the local user row exists.

OpenFGA or SpiceDB only when relationships get genuinely hard: nested

hierarchies, shared-with-me, inherited permissions across resource trees.

Reaching for step 3 early is a common and expensive mistake. Most client work never leaves step 2.

3. External identity providers and enterprise SSO

ZITADEL acts as an identity broker, so your application speaks OIDC to ZITADEL and never learns that the user actually came from Okta or Google.

Documented providers: Google, GitHub, GitLab, Apple, Entra ID (OIDC and SAML), Okta, Keycloak, LDAP and OpenLDAP, plus generic OIDC, generic SAML, and a JWT IdP.

The part that matters architecturally

IdPs can be configured at instance level (consistent across every organization, centrally managed) or at organization level (per tenant, with delegated administration so an organization manages its own).

Organization-level IdPs are the concrete payoff of putting your client's tenants in ZITADEL organizations. It is the difference between "Acme's staff log in with Acme's Okta" being configuration versus being impossible. If a client's product will ever sell to enterprises, this is the reason organizations belong in ZITADEL rather than in the application database, and it is worth re-reading decision 4 in SKILL.md with that in mind.

Provisioning and linking

Auto-provisioning creates a ZITADEL account on first external login, with the

profile pre-filled from the external IdP.

Account linking associates several external identities with one ZITADEL user,

automatic or manual.

Both have direct consequences for your app_user table: a user may appear with a sub you have never seen, having never registered in your application. JIT provisioning handles this correctly, which is a point in its favour over any registration-time hook.

SAML specifics

Trust is established by exchanging a signing certificate and an entity ID. Rather than exchanging those by hand, both sides can consume metadata: paste an XML document or point at a metadata URL and each side self-configures.

ZITADEL publishes its signing certificate separately for service providers that want it outside the metadata:

https://<domain>/saml/v2/certificate

To verify: MockSAML is the documented way to exercise a SAML federation without an enterprise IdP, and Google is the cheapest social login to prove the broker model end to end. Confirm specifically what sub and amr look like for a federated user, since that is what your API will receive.

4. Actions v2

Custom logic ZITADEL invokes at defined points. Actions v1 ran JavaScript inside ZITADEL; v2 does not. v2 calls out to an HTTP endpoint you host, described as a shift from embedded extensions to programmable, event-driven webhooks.

Three resources:

Resource	What it is
Endpoint	Your external HTTP service containing the logic
Target	A ZITADEL resource describing how to reach that endpoint
Execution	A ZITADEL resource describing when to invoke which target

Four execution types:

Request - intercept or modify an incoming API request before processing
Response - react to or transform an outgoing response
Function - the replacement for Actions v1 internal hooks
Event - react to system events, for example a user lockout

Principal use is token enrichment: adding custom claims for systems that need them, or running external logic during an OIDC or SAML flow.

Cautions worth carrying in

Your endpoint becomes part of ZITADEL's critical path. Its availability

affects ZITADEL operations, which means a webhook you wrote can degrade logins.

Verify payload integrity with signature checks. This is an inbound call from

a service to yours, and it must be authenticated.

Executions run even when the feature flag is disabled. Removing an action

means deleting the Execution, not toggling a flag. That is exactly the kind of behaviour that produces a baffling incident.

Given the critical-path risk, the default should be not to use Actions unless something genuinely cannot be done in your own application after token validation. Enriching a token is rarely worth coupling your login availability to a service you wrote.

Suggested order to verify

Service accounts, private key JWT plus the ZITADEL audience scope. Blocks

Terraform, needed immediately.

Roles end to end: define a role, assign it, enable *Assert Roles on

Authentication*, and confirm the claim finally appears in a token. This closes the one loose end left from the verified sections.

Google as a social IdP, then the same at organization level rather than

instance level, to see the delegation model concretely.

Actions v2, last and only if a real requirement appears.