---
title: "Applicant Portal Identity Infrastructure (Keycloak)"
slug: applicant-portal-identity-infrastructure
section: Portals
group: Applicant Portal
status: stable
description: "The Applicant Portal's Keycloak realm (`applicant-portal`) is provisioned by Terraform in `PPPToo/terraform/keycloak`, and its branded Keycloak image is built and deployed from `PPPToo/keycloak-image`."
last-updated: 2026-10-07
---

# Applicant Portal Identity Infrastructure (Keycloak)

The Applicant Portal's Keycloak realm (`applicant-portal`) is provisioned by Terraform in `PPPToo/terraform/keycloak`, and its branded Keycloak image is built and deployed from `PPPToo/keycloak-image`. [1] [2]

## Overview

PPPToo authenticates applicants against a Keycloak realm named `applicant-portal` in each environment (development, QA, prod). The portal-side session model is described in [PPPToo — Participant Portal v2](/docs/pppto-architecture); this page covers the identity infrastructure underneath it: how the realm is defined, how its secrets and email are handled, and how the Keycloak container image is built and shipped.

| Concern | Where it lives | Mechanism |
| --- | --- | --- |
| Realm, clients, mappers, SMTP | `terraform/keycloak/` | Terraform module applied per environment with `envs/*.tfvars` |
| Runtime drift guard | PPPToo server (`realm-invariants.ts`, `auth.config.ts`) | Startup check that fails closed when the live realm drifts |
| Branded login theme and password-reset SPI | `keycloak-image/` | Docker image deployed to the `applicant-keycloak` Cloud Run service |
| Backend admin client secret | Secret Manager `ppp2-keycloak-admin-client-secret` | Seeded by `scripts/seed-admin-client-secret.sh` |

The PPPToo server reads these Keycloak environment variables (`auth.config.ts`, `auth.controller.ts`, `keycloak-admin.service.ts`):

| Variable | Purpose | Default and behavior |
| --- | --- | --- |
| `KEYCLOAK_ISSUER` | Realm issuer URL; also the base for token requests | Required: startup fails when empty |
| `KEYCLOAK_JWKS_URI` | JWKS endpoint for JWT validation | Optional; defaults to the issuer plus `/protocol/openid-connect/certs` |
| `KEYCLOAK_AUDIENCE` | Expected token audience | Optional; defaults to `back-node-applicant-portal-app`, and startup fails if set to anything else |
| `KEYCLOAK_LOGIN_CLIENT_ID` | Client used for the password grant in `POST /api/v1/auth/login` | Optional; defaults to `front-react-applicant-portal-app` |
| `KEYCLOAK_LOGIN_CLIENT_SECRET` | Secret for the login client | Optional; sent as `client_secret` only when set (the public front client needs none) |
| `KEYCLOAK_CLIENT_ID` | Admin client for `client_credentials` calls (create user, reset password, action emails) | Optional; defaults to `back-node-applicant-portal-app` |
| `KEYCLOAK_ADMIN_CLIENT_SECRET` | Secret of the admin client | No default (empty string); admin calls fail with `keycloak admin auth failed` when wrong |

## Realm as Code (Terraform)

`main.tf` is documented as the source of truth for the realm's security posture across development, QA and prod. It provisions the realm, the two OIDC clients, the audience mapper, the applicant identity claim mapper and the backend service account's realm-management roles. [3]

Apply is per environment, with state prefixed per environment:

```
terraform init -backend-config="bucket=<state-bucket>" -backend-config="prefix=keycloak/<env>"
terraform plan  -var-file=envs/<env>.tfvars
terraform apply -var-file=envs/<env>.tfvars
```

The provider authenticates with `client_credentials` as the confidential backend client `back-node-applicant-portal-app`, so its secret must be exported as `TF_VAR_kc_admin_client_secret` (never placed in a tfvars file). [3] [4]

Dev and QA realms were hand-provisioned before the module existed, so their first apply imported existing objects (procedure in `REALM-IMPORT.md`). Per the README, both dev and QA completed the import on 2026-08-04. After that, `REALM_ASSERT: warn` was removed from the deploy env files so the server enforces the invariants. [4] [5]

### Realm Invariants

The NestJS server re-verifies these at startup and refuses to boot if the live realm has drifted. The Terraform values and the server guard must agree. [3] [4]

| Invariant | Terraform | Server guard |
| --- | --- | --- |
| Access-token lifespan | `access_token_lifespan_seconds` | `REALM_MAX_ACCESS_TOKEN_LIFESPAN_SECONDS` cap |
| Brute-force detection | `security_defenses.brute_force_detection` | `bruteForceProtected` must be true |
| Token audience | audience protocol mapper `front_to_back` | Not a live-realm check: `assertAuthConfig()` in `auth.config.ts` only verifies that `KEYCLOAK_AUDIENCE` equals `EXPECTED_AUDIENCE` |

Other realm settings pinned in `main.tf`: `ssl_required = "external"`, a password policy (default `length(10) and notUsername(undefined) and passwordHistory(3)`), security headers (X-Frame-Options DENY, CSP, nosniff, HSTS), and explicit values for `reset_password_allowed`, `registration_allowed` and `registration_email_as_username`. These are pinned because the Keycloak Terraform provider defaults each to false when omitted, which would silently disable self-service reset and registration. [3] [6]

### Clients and Protocol Mappers

| Resource | Setting | Notes |
| --- | --- | --- |
| `front` (public SPA client) | Direct access grants (ROPC) enabled, standard flow enabled, implicit off | The portal collects credentials and exchanges them; standard flow is used only for hosted execute-actions email flows. PKCE `S256` is pinned. |
| `front` | `login_theme` set at client level | The client-level theme overrides the realm's and is what brands the hosted reset page. |
| `back` (confidential client) | Service account enabled, all interactive flows off, `use_refresh_tokens = false`, redirect URIs and web origins pinned to empty | Used for admin operations (create user, reset password, logout, action emails). |
| Audience mapper on `front` | Adds the `back` client audience to the access token | Name comes from `audience_mapper_name`. |
| User attribute mapper on `front` | `globusApplicantId` into access and ID tokens | Name comes from `applicant_id_mapper_name`. |

`keycloak_realm_user_profile` is deliberately not managed. The resource replaces the whole attribute list on apply and would wipe the live custom attribute schema, so the user-profile config stays hand-managed (and is applied by the image entrypoint, see below). [3]

### Backend Service Account Roles

The backend client's service account is granted these `realm-management` roles through Terraform: `manage-users`, `view-users`, `query-users`, `view-realm`, `view-clients`, `query-clients`. `view-realm`, `view-clients` and `query-clients` were originally granted by hand; after an unrelated SMTP apply destroyed and recreated the role set, dev and QA lost them and the server's startup realm check crash-looped, so they are now declared in code. [3]

### Per-Environment Values

| Setting | development | QA | prod |
| --- | --- | --- | --- |
| Token and SSO session lifespan | 259200 s (72 h) | 259200 s (72 h) | 86400 s (24 h) |
| `login_theme` | module default (`applicant-portal`) | `keycloak` (stock) | `keycloak` (stock) |
| Audience mapper name | `back-node-applicant-portal-app` | `back-node-audience` | module default (`aud-back-node`) |
| Applicant-id mapper name | `globus-applicant-id` | `globus-applicant-id` | module default (`globusApplicantId`) |

Mapper names differ per environment because each hand-provisioned realm drifted independently and a rename through the admin API was observed not to persist, so tfvars pin the live names instead of re-attempting a rename. [5] [7]

Prod `kc_url` and front redirect URIs in `prod.tfvars` hold the real hosts taken from applied state; the README's "Known gaps" list still describes them as placeholders, so trust the tfvars. [4] [8]

## Realm SMTP

The realm's SMTP server powers `execute-actions-email`, the portal's forgot-password flow. [3] [4]

- Dev, QA and prod tfvars all set `smtp_host = "smtp.sendgrid.net"`, port `587`, display name `WildPacks`, a WildPacks no-reply sender address and an Alliance support reply-to address (see `smtp_from` and `smtp_reply_to` in each `envs/*.tfvars`). The realm is shared across brands, so every applicant reset or action email shows the Wild Packs sender. The prod sender change landed through PPPToo PR #342 (2026-09-08) as configuration only and was deliberately not applied at that time. The stale placeholder hosts that made an apply unsafe were corrected in `prod.tfvars` the same day (PR #343) and the stock login theme was pinned (PR #344), but the repo records no prod apply of the sender change, so check the live prod realm's SMTP settings before assuming the Wild Packs sender is in effect there. [5] [8]
- The SMTP password is write-only (Keycloak masks it on read). It is supplied at apply time through `TF_VAR_smtp_auth_password`, and a Terraform precondition fails the plan if SMTP auth is enabled and the password is empty, so an apply cannot blank the credential. [3]
- Leaving `smtp_host` empty removes the realm's SMTP configuration, so do not blank it in any environment.
- Breakage is silent for users: the BFF returns `{ok:true}` whether or not Keycloak could send. After any realm apply, run the forgot-password flow once and confirm the email arrives. [4]

## Backend Client Secret Handling

Keycloak's admin API returns the client secret as the literal mask `**********` on a plain client read. Piping `terraform output` into Secret Manager could therefore write the mask over the real secret, which happened on dev on 2026-08-12. Two safeguards now exist. [3] [4] [9]

1. `keycloak_openid_client.back` has `lifecycle { ignore_changes = [client_secret] }` so a plan or refresh cannot replace state with the masked value.
2. `scripts/seed-admin-client-secret.sh <development|QA|prod>` reads the secret from the dedicated `client-secret` endpoint (not masked), refuses empty or all-asterisk values, and writes a new version of `ppp2-keycloak-admin-client-secret`. It performs only GET requests against Keycloak and needs the current working secret in `TF_VAR_kc_admin_client_secret` to obtain a token. The script has no prod host configured and exits for `prod`.

## Branded Keycloak Image

The Terraform `login_theme` only selects a theme by name. The theme itself and a custom password-reset action ship in the Keycloak image built from `keycloak-image/`. If the deployed image lacks the `applicant-portal` theme, Keycloak falls back to the stock `keycloak` theme. The image was migrated into PPPToo from the standalone `app-portal-server-keycloak-flows` repository. [4] [10]

### Image Contents

| Part | Detail |
| --- | --- |
| Base | `quay.io/keycloak/keycloak:26.5.4`, three-stage Dockerfile (Maven SPI build, optimized `kc.sh build`, runtime) [10] |
| SPI | `BrandedUpdatePassword` extends Keycloak's `UpdatePassword` required action and injects the user attributes `brandLogoUrl`, `brandPrimaryColor` and `brandName` into the page; password behavior is inherited unchanged [11] |
| Theme | `themes/applicant-portal/login` (including `login-update-password.ftl`), copied to `/opt/keycloak/themes` [10] |
| Realm import | `docker/keycloak` copied to `/opt/keycloak/data/import` and loaded with `--import-realm` at start [10] [12] |
| User profile | `docker/user-profile.json`, applied with `kcadm update users/profile` on every start so the `brand*` attributes stay declared [12] |
| Entrypoint | Starts Keycloak in the background, waits for port 8080, then optionally configures Google and Facebook identity providers from env vars and enables the service account on the admin client with the `manage-users` role [12] |

Realm-level settings (SSL requirement, login theme, events) are intentionally not set by the entrypoint, because a `kcadm` read-modify-write on the dev realm was observed to null fields; Terraform owns them. [12]

### Deploy and Promote Pipeline

Four GitHub Actions workflows in PPPToo handle the image. Deploys never take traffic automatically. [13] [14] [15] [16]

| Workflow | Trigger | What it does |
| --- | --- | --- |
| `deploy-dev-keycloak-image.yml` | Push to `development` touching `keycloak-image/**`, or manual — `development` has been a frozen legacy branch since PPPToo moved to the sandbox branching model on 2026-10-04, so in practice manual only | Builds and pushes `keycloak:pilot-<sha7>` to Artifact Registry, deploys to the dev `applicant-keycloak` service with `--no-traffic` and a `pilot-<sha7>` revision tag |
| `deploy-qa-keycloak-image.yml` | Push to `qa` touching `keycloak-image/**`, or manual — `qa` is likewise frozen since 2026-10-04, so in practice manual only | Same flow against the QA project; also sets `--command=/opt/keycloak/entrypoint.sh` so the image's entrypoint runs, and sets the admin bootstrap env/secret |
| `promote-dev-keycloak-image.yml` | Manual only | Takes a revision tag input and shifts 100 percent of traffic to it |
| `promote-qa-keycloak-image.yml` | Manual only | Same for QA |

Both deploy workflows use Docker Buildx with the GitHub Actions layer cache (`cache-from` and `cache-to` of type `gha`, mode `max`) so the Maven and `kc.sh build` layers are reused. The dev deploy mounts the three Postgres client-certificate files via `--update-secrets` (not `--add-volume`), because those mounts did not carry over between deploys. This page covers only the dev and QA image workflows. There is no prod Keycloak image workflow in PPPToo, so a prod image deploy is still a manual process. The Terraform README (`terraform/keycloak/README.md`) still describes the CI path as "pilot only as of 2026-09-14" with QA not yet wired, although the QA workflows now exist. [13] [14]

## Operating Rules and Gotchas

> **Never edit the realm by hand**
>
> Changes through the admin console or `kcadm` are drift the server refuses to boot on. Change `main.tf` or the tfvars and apply. On dev, `kcadm` realm writes were also observed to null values. [4] [12]

- Apply Terraform realm changes before deploying a server revision that depends on them; the server fails closed at boot on an un-hardened realm. [4]
- When importing a realm, grant `manage-realm`, `view-clients` and `manage-clients` to the admin client in a single role-mapping call; sequential single-role calls were observed to drop earlier grants. [5]
- Do not re-add `REALM_ASSERT: warn` to mask a boot failure; the error names the invariant that regressed. [7]
- The Hub routes that link, activate and re-email portal identities refuse with these codes (routes, bodies and envelopes are on [Globus Portal API](/docs/globus-portal-api)): `AMBIGUOUS_APPLICANT_MATCH` (name plus date of birth match several applicants), `PORTAL_USER_ALREADY_LINKED` (the `portalUserId` already belongs to a different applicant), `APPLICANT_ALREADY_ACTIVATED` (resend-invite after the portal account is activated), `PORTAL_IDENTITY_LINK_REQUIRED` (change-email for an activated applicant with no linked portal identity), `APPLICANT_EMAIL_CHANGED_CONCURRENTLY` (the email changed while a correction ran), `PORTAL_IDENTITY_UPDATE_FAILED` (the Applicant Portal login-email update failed with a non-conflict upstream error, 502; a portal-side conflict is a 409 `PORTAL_IDENTITY_CONFLICT` instead) and `PORTAL_IDENTITY_COMPENSATION_FAILED` (the compensating revert of the portal login email, run when anything fails after the portal update was attempted, itself failed, 502; the operation needs reconciliation).
- The image must contain the theme before `login_theme` is set to `applicant-portal`; otherwise the apply fails (QA) or the page falls back to stock. [5] [4]
- `SECRET_BOOTSTRAP=off` (local dev) turns off the server's GCP Secret Manager bootstrap and also skips the realm-invariant assertion at startup, the same as `REALM_ASSERT=off`. When it is not `off`, the loader requires `GCP_PROJECT_ID` and the assertion runs in the mode set by `REALM_ASSERT` (default `enforce`), so setting it in a deployed environment also disables the drift guard above.

## Sources

PRs: PPPToo#178, PPPToo#179, PPPToo#209, PPPToo#340, PPPToo#341, PPPToo#342, PPPToo#343, PPPToo#344, PPPToo#354, PPPToo#355, PPPToo#356, PPPToo#357, PPPToo#367.

[1]: https://github.com/Alliance-Strategies/PPPToo/tree/development/terraform/keycloak "PPPToo terraform/keycloak"
[2]: https://github.com/Alliance-Strategies/PPPToo/tree/development/keycloak-image "PPPToo keycloak-image"
[3]: https://github.com/Alliance-Strategies/PPPToo/blob/development/terraform/keycloak/main.tf "Keycloak realm Terraform module"
[4]: https://github.com/Alliance-Strategies/PPPToo/blob/development/terraform/keycloak/README.md "Keycloak realm as code README"
[5]: https://github.com/Alliance-Strategies/PPPToo/blob/development/terraform/keycloak/REALM-IMPORT.md "Realm import runbook"
[6]: https://github.com/Alliance-Strategies/PPPToo/blob/development/terraform/keycloak/variables.tf "Keycloak module variables"
[7]: https://github.com/Alliance-Strategies/PPPToo/blob/development/deploy/env/development.yaml "PPPToo dev deploy env"
[8]: https://github.com/Alliance-Strategies/PPPToo/blob/development/terraform/keycloak/envs/prod.tfvars "Prod tfvars"
[9]: https://github.com/Alliance-Strategies/PPPToo/blob/development/terraform/keycloak/scripts/seed-admin-client-secret.sh "Seed admin client secret script"
[10]: https://github.com/Alliance-Strategies/PPPToo/blob/development/keycloak-image/Dockerfile "Keycloak image Dockerfile"
[11]: https://github.com/Alliance-Strategies/PPPToo/blob/development/keycloak-image/keycloak-spi/src/main/java/com/allianceabroad/keycloak/BrandedUpdatePassword.java "BrandedUpdatePassword SPI"
[12]: https://github.com/Alliance-Strategies/PPPToo/blob/development/keycloak-image/entrypoint.sh "Keycloak image entrypoint"
[13]: https://github.com/Alliance-Strategies/PPPToo/blob/development/.github/workflows/deploy-dev-keycloak-image.yml "Dev image deploy workflow"
[14]: https://github.com/Alliance-Strategies/PPPToo/blob/development/.github/workflows/deploy-qa-keycloak-image.yml "QA image deploy workflow"
[15]: https://github.com/Alliance-Strategies/PPPToo/blob/development/.github/workflows/promote-dev-keycloak-image.yml "Dev image promote workflow"
[16]: https://github.com/Alliance-Strategies/PPPToo/blob/development/.github/workflows/promote-qa-keycloak-image.yml "QA image promote workflow"