Architecture Overview
The big picture
This guide walks you through building an enterprise-grade identity and access management system. The architecture connects Microsoft's cloud identity platform with an open-source access management broker, giving you fine-grained control over who can access what in your applications.
Azure AD (Entra ID)
Primary Identity Provider β authenticates users against Microsoft's cloud directory. Stores user accounts, group memberships, and organizational data.
Keycloak
Identity Broker & Policy Enforcement β sits between Azure AD and your applications. Manages sessions, maps roles, and enforces fine-grained authorization.
OIDC / OAuth 2.0
Federation Protocol β the industry-standard protocol connecting Azure AD to Keycloak. Handles token exchange, scopes, and secure redirects.
RBAC
Role-Based Access Control β users are assigned roles (employee, manager, director, executive) that determine which actions they can perform.
Cell-Level Security
Fine-grained data access via classification levels and need-to-know approval tied to project codes. Role alone does not grant access β an executive without clearance and approved need-to-know is denied just like any other user.
Classification & Need-to-Know
Data is tagged with a classification level and a project code. Users must hold the matching classification and an approved need-to-know for that project β with CRUD permissions granted individually per approval.
SSO
Single Sign-On β users authenticate once and gain access to all connected applications without re-entering credentials.
π Authentication Flow Step-by-Step
User visits your application β The app detects no active session and redirects the user to Keycloak's login page.
Keycloak presents login options β The user sees a "Sign in with Microsoft Azure AD" button (configured as an Identity Provider in Keycloak).
Redirect to Azure AD β Keycloak initiates an OIDC Authorization Code flow, redirecting the browser to Microsoft's login page with a client_id and redirect_uri.
User authenticates at Azure AD β The user enters their Microsoft credentials (and MFA if configured). Azure AD validates the credentials.
Azure AD returns an authorization code β On success, Azure AD redirects back to Keycloak's broker endpoint with an authorization code.
Keycloak exchanges the code for tokens β Keycloak calls Azure AD's token endpoint using the code + client_secret, receiving an ID token, access token, and refresh token.
Keycloak processes claims and maps roles β The ID token contains claims (email, name, groups). Keycloak's mappers extract these, create/update the local user, and assign Keycloak roles based on Azure AD group memberships.
Keycloak creates its own session & tokens β Keycloak issues its own JWT access token enriched with realm roles, department attributes, and custom claims. This token is what your application actually validates.
Application enforces RBAC & cell-level security β Your app reads the JWT, checks the user's roles and attributes, and grants/denies access to resources accordingly.
Key concept: Keycloak acts as a broker β it federates identity from Azure AD but maintains its own sessions and tokens. This decouples your applications from any single identity provider, so you could add Google, Okta, or LDAP without changing application code.
Azure AD (Entra ID) Setup
Steps 1β6 Β· Configure your cloud identity provider
Step 1 Create Azure AD Tenant
Your Azure AD tenant is your organization's dedicated instance where all users, groups, and app registrations live. Navigate to portal.azure.com, go to Azure Active Directory β Overview, and note your Tenant ID and Primary Domain (e.g., yourcompany.onmicrosoft.com).
Step 2 Create Users and Groups
Groups are the foundation of RBAC. Create groups that mirror your organizational roles and departments. Each group will later map to a Keycloak role.
Step 3 Register Application in Azure AD
This creates the OAuth2/OIDC endpoint Keycloak uses for federation. Go to App Registrations β New Registration, name it Keycloak-Federation, set the redirect URI to your Keycloak broker endpoint, and capture the Application (client) ID and Directory (tenant) ID.
Redirect URI (Web): http://localhost:8080/realms/master/broker/azuread/endpoint Save these values: βββ Application (client) ID: [GUID] βββ Directory (tenant) ID: [GUID] βββ Primary Domain: yourcompany.onmicrosoft.com
Step 4 Create Client Secret
Keycloak needs credentials to authenticate to Azure AD. Go to Certificates & Secrets β New Client Secret. Copy the secret value immediately β you cannot retrieve it later. Set expiry to 24 months or per your security policy.
Step 5 Configure API Permissions
Add Microsoft Graph delegated permissions so Keycloak can read user profiles and group memberships. Then grant admin consent for your organization.
Step 6 Configure Token Claims
Add optional claims (email, family_name, given_name) to the ID token, and enable group claims so Azure AD includes group memberships as Group IDs in tokens sent to Keycloak.
What is a Tenant?
A tenant is a dedicated, isolated instance of Azure AD that an organization receives when it signs up for a Microsoft cloud service. Think of it as your organization's private directory in the cloud. Every user, group, and app registration lives within a specific tenant. The Tenant ID (a GUID) uniquely identifies it across all of Microsoft's infrastructure.
App Registrations vs. Enterprise Applications
App Registration defines the identity configuration β client ID, secrets, redirect URIs, and permissions. It's the "blueprint." An Enterprise Application is the instantiation of that blueprint within a tenant β it controls who can use the app, conditional access policies, and sign-in logs. When you register an app, both are created automatically.
Delegated vs. Application Permissions
Delegated permissions are used when a signed-in user is present β the app acts on behalf of the user and can only access what the user themselves can access. Application permissions are for background services with no signed-in user β the app acts as itself with its own identity. For Keycloak federation, we use delegated permissions because users are actively signing in.
Why Group Claims Matter
By including group Object IDs in the ID token, we avoid extra API calls to Microsoft Graph during login. Keycloak receives the group memberships directly in the token and can immediately map them to local roles. Without this, Keycloak would need to separately query the Graph API for each user's groups β adding latency and complexity.
Client Secret Security
The client secret is essentially the "password" for your app registration. In production, consider using certificates instead of secrets for stronger security. Store secrets in Azure Key Vault or your preferred secret management solution. Never commit them to source control. Set calendar reminders before expiry to rotate them.
Keycloak Installation & Setup
Steps 7β8 Β· Install and configure your identity broker
Step 7 Install Keycloak
Download and extract Keycloak, set admin credentials via environment variables, then start in development mode. Access the admin console at http://localhost:8080.
# Download & extract wget https://github.com/keycloak/keycloak/releases/download/23.0.0/keycloak-23.0.0.zip unzip keycloak-23.0.0.zip && cd keycloak-23.0.0 # Set admin credentials export KEYCLOAK_ADMIN=admin export KEYCLOAK_ADMIN_PASSWORD=admin123! # Start (dev mode) ./bin/kc.sh start-dev # Or production mode ./bin/kc.sh build ./bin/kc.sh start --hostname=localhost
Step 8 Create Keycloak Realm
Realms are isolated tenants within Keycloak. Create a realm called enterprise-apps. Disable user registration (since users come from Azure AD) and enable "Remember me."
Realms: Multi-Tenancy in Keycloak
A Realm is a space where you manage a set of users, credentials, roles, and groups. Realms are completely isolated from each other. The master realm is a special admin realm β you should create separate realms for your applications. Think of each realm like a separate Keycloak installation.
Dev Mode vs. Production Mode
Dev mode (start-dev) uses an embedded H2 database, disables HTTPS requirement, and enables hot-reload. Production mode requires you to configure an external database (PostgreSQL recommended), set up TLS certificates, configure hostname, and build an optimized server. Never use dev mode in production β the H2 database is not suitable for concurrent access.
Why Keycloak as a Broker?
Using Keycloak as an identity broker gives you vendor independence. Your applications only need to know about Keycloak β they never directly communicate with Azure AD. If you later need to switch identity providers, add LDAP federation, or support social logins, you only change Keycloak's configuration, not your application code. Keycloak also adds capabilities Azure AD alone doesn't provide, like fine-grained authorization services.
Identity Provider vs. Identity Broker
An Identity Provider (IdP) authenticates users β it's where credentials are verified (Azure AD in our case). An Identity Broker delegates authentication to external IdPs while managing its own sessions and token issuance. Keycloak is the broker: it doesn't verify passwords, but it manages the full lifecycle of user sessions, role mappings, and access tokens for your applications.
Federation Configuration
Steps 9β10 Β· Connect Azure AD to Keycloak via OIDC
Step 9 Add Azure AD as Identity Provider
In Keycloak, add an OpenID Connect v1.0 identity provider with alias azuread. Point the discovery endpoint to Azure AD's well-known configuration URL, provide the Client ID and Secret from your app registration, and configure attribute mappers.
# Discovery endpoint template https://login.microsoftonline.com/{TENANT_ID}/v2.0/.well-known/openid-configuration # Key settings Alias: azuread Client ID: [from Azure AD App Registration] Client Secret: [from Azure AD] Default Scopes: openid profile email Sync Mode: IMPORT Trust Email: ON
Configure four attribute mappers: email, firstName (from given_name), lastName (from family_name), and azure-groups (from the groups claim).
Step 10 Update Azure AD Redirect URI
Copy the Redirect URI from Keycloak's identity provider settings and add it back in Azure AD's app registration under Authentication β Redirect URIs. This completes the OAuth2 redirect flow loop.
OIDC Discovery Endpoint
The .well-known/openid-configuration URL returns a JSON document describing all the endpoints (authorization, token, userinfo, JWKS), supported scopes, claims, and algorithms. Keycloak reads this once during setup, so it automatically knows where to send users for login, where to exchange codes for tokens, and where to find the public keys for JWT validation.
Mappers: The Bridge Between Systems
Claim mappers are the critical translation layer. Azure AD tokens use specific claim names (e.g., given_name) while Keycloak uses different attribute names (e.g., firstName). Mappers extract values from incoming Azure AD tokens and write them to the correct Keycloak user attributes. Without proper mappers, user profiles would be incomplete and group-to-role mapping wouldn't work.
Sync Mode: IMPORT vs. FORCE
IMPORT creates the user on first login and doesn't update attributes on subsequent logins. FORCE updates user attributes from Azure AD on every login. Use FORCE for attributes you want to stay in sync (like groups and department), and IMPORT for attributes users might customize locally in Keycloak.
The Redirect URI Dance
OAuth2 security relies on the redirect URI being exactly registered. The flow is: your app β Keycloak login β Azure AD login β back to Keycloak (at the broker endpoint) β back to your app. Both Keycloak and Azure AD must have each other's redirect URIs registered. A mismatch of even one character (trailing slash, http vs https) will cause a "redirect_uri_mismatch" error.
RBAC Setup
Steps 11β12 Β· Roles, composite roles, and group mapping
Create Keycloak roles that define what users can do, then map Azure AD groups to those roles for automatic assignment.
β οΈ Important: Roles Are Necessary But Not Sufficient
A role (employee, manager, director, executive) determines the maximum capability a user may exercise β but it does not grant access to any classified data by itself. To see or modify classified data, a user must also hold the required classification level and an approved need-to-know for the specific project code. An executive who lacks classification clearance or need-to-know approval for a project is denied access to that projectβs data, regardless of their role.
Role Hierarchy (Composite Roles)
admin (full access)
βββ executive
βββ director
βββ manager
βββ employee (base level)
Composite = inherits all child roles
A "director" automatically has manager + employee permissions
β Role β Access to classified data
Access = Role + Classification Level + Need-to-Know Approval
An executive without NTK approval for Project-X cannot see Project-X data
Use Advanced Claim to Role mappers to automatically assign Keycloak roles based on Azure AD group Object IDs. For example, membership in the Manager-ReadWrite Azure AD group grants the manager Keycloak role.
Why Composite Roles?
Composite roles create an inheritance chain. Instead of assigning 15 individual permissions to a director, you assign one "director" role that automatically includes manager and employee permissions. This reduces administrative overhead and ensures consistency β when you add a new base permission, every composite role above it inherits it automatically.
Realm Roles vs. Client Roles
Realm roles are global within a Keycloak realm β they apply across all applications. Client roles are specific to a single application (client). For enterprise setups, use realm roles for organizational hierarchy (employee, manager, director) and client roles for application-specific permissions (can_edit_invoice, can_approve_order).
Role Composition in Detail
employee: document:read, document:create, profile:read-own,
profile:update-own, access-public, access-internal
manager: β employee + document:update, document:share,
profile:read-team, access-confidential
director: β manager + document:delete, profile:read-dept,
access-restricted, admin:audit-logs
executive: β director + profile:read-all, access-top-secret,
admin:user-management
Group-to-Role Mapping Strategy
The "Advanced Claim to Role" mapper watches for a specific Azure AD Group Object ID in the groups claim of the incoming token. When it finds a match, it assigns the corresponding Keycloak role. Set Sync Mode to FORCE so that if a user is removed from an Azure AD group, their Keycloak role is also removed on next login.
App Integration & Cell-Level Security
Steps 13β15 Β· Client config, scopes, and fine-grained access
Register your application as a Keycloak client, configure client scopes to include roles in JWT tokens, then implement cell-level security using user attributes and Keycloak's Authorization Services.
Layer 1: User Attributes & Classification
Add classification_level (0β4) and department to each user. Map into access tokens via client scope mappers. The app compares the userβs level against the dataβs classification before any access.
Layer 2: Need-to-Know + Project Code
Even with sufficient clearance, users must have an approved need-to-know entry for a data itemβs project code. Each NTK approval specifies exactly which CRUD operations are permitted. The app checks the user_need_to_know table for a matching project + allowed actions.
Layer 3: Authorization Services
Define Resources, Scopes, Policies, and Permissions in Keycloak. Policies combine role checks, classification-level checks, and NTK lookups. Keycloak evaluates all policies at runtime and returns authorization decisions.
What Is Cell-Level Security?
Unlike RBAC (which controls what actions you can perform), cell-level security controls which specific data items you can see. A "manager" might have read access to documents, but cell-level security ensures they only see documents from their own department, not every department's documents. It's the difference between "can you read documents?" (RBAC) and "which documents can you read?" (cell-level).
Data Classification Levels & Need-to-Know
Documents are classified as Public (L0), Internal (L1), Confidential (L2), Restricted (L3), or Top-Secret (L4). Classification defines the minimum clearance a user must hold, but clearance alone is not enough. Access requires:
Classification Level: Userβs clearance level β₯ dataβs classification level.
Need-to-Know Approval: User has an approved NTK entry for the dataβs project code, granted by a project owner or security officer.
CRUD Permission: The NTK approval specifies exactly which operations (Create, Read, Update, Delete) the user may perform. A user may be approved to Read but not Update or Delete.
Example: An executive (highest role) tries to access a Restricted financial forecast for PROJECT-ATLAS. They hold Top-Secret clearance (L4 β₯ L3 β) but have no NTK approval for PROJECT-ATLAS β access denied. Meanwhile, an employee who holds L3 clearance and has an approved NTK for PROJECT-ATLAS with Read permission can view it.
PostgreSQL Row-Level Security
For the deepest level of protection, use database-level row security. PostgreSQL's RLS policies automatically filter query results based on session variables set from the JWT token. Even if application code has a bug that skips a permission check, the database itself enforces the access rules β it's defense in depth.
-- Enable RLS on the table ALTER TABLE financial_records ENABLE ROW LEVEL SECURITY; -- Policy: users only see their department's rows CREATE POLICY dept_isolation ON financial_records USING (department = current_setting('app.user_department')); -- In app code: set the session variable from JWT await db.query(`SET app.user_department = $1`, [token.department]);
Keycloak Authorization Services: Resources β Policies β Permissions
Resources represent what you're protecting (e.g., /api/finance/*). Scopes define actions (view, edit, delete). Policies define the rules (must have role X, must be in department Y). Permissions tie them together: "Resource A + Scope B requires Policy C." The Decision Strategy (Affirmative = any policy passes, Unanimous = all must pass) controls how multiple policies combine.
Single Sign-On (SSO)
Steps 16β17 Β· Session settings and testing the flow
Configure SSO session lifespans in Keycloak, set appropriate token lifetimes, then test the complete authentication flow end-to-end.
Session Settings
Token Lifespans
How SSO Works Across Applications
When a user logs in to App A, Keycloak creates a session cookie in the browser. When the user then visits App B, that app redirects to Keycloak, but Keycloak sees the existing session cookie and immediately redirects back with new tokens β no login prompt. The user perceives instant access across all connected apps.
Why Short Access Token Lifespans?
Access tokens are "bearer tokens" β anyone who has one can use it. Short lifespans (5 minutes) limit the damage if a token is leaked. Applications use refresh tokens (longer-lived, stored securely server-side) to silently get new access tokens without making the user log in again. This pattern is called "silent renewal."
Testing the Token
Use jwt.io to decode your access token and verify it contains the expected claims: user email, name, realm_access.roles array, department, and any custom attributes. The token header shows the signing algorithm (RS256), and you can validate the signature using Keycloak's public key from its JWKS endpoint.
Application Implementation
Step 18 Β· Node.js integration example
A sample Express.js application showing public routes, protected routes, role-based endpoints, and cell-level data filtering using the Keycloak Node.js adapter.
// Key endpoints pattern app.get('/', /* public */); app.get('/secure', keycloak.protect(), /* any authenticated user */); app.get('/admin', keycloak.protect('admin'), /* admin role required */); app.get('/manager', keycloak.protect('manager'), /* manager role required */); // Cell-level security: classification + need-to-know check app.get('/api/docs/:id', keycloak.protect(), // Gate 1: Role requireClassifiedAccess('read'), // Gate 2+3: Cls + NTK getDocument ); app.put('/api/docs/:id', keycloak.protect('manager'), // Gate 1: Manager+ requireClassifiedAccess('update'), // Gate 2+3: Cls + NTK updateDocument ); // Executive without NTK = denied // Employee with read-only NTK = can read
The keycloak-connect Middleware
The keycloak-connect package handles the entire OIDC flow for your Express app. keycloak.protect() checks for a valid session/token and redirects to login if missing. keycloak.protect('admin') additionally verifies the user has the specified realm role. The token content is available at req.kauth.grant.access_token.content.
Layered Security Pattern
Good security is layered: (1) the Keycloak middleware verifies authentication, (2) role checks verify authorization at the action level, (3) cell-level checks filter data by department/ownership, (4) PostgreSQL RLS provides database-level enforcement. Even if one layer has a bug, the others still protect your data.
UMA Ticket Authorization
For Keycloak's Authorization Services, your app requests a "UMA ticket" from Keycloak's token endpoint, specifying which resource and scope it wants to access. Keycloak evaluates all applicable policies and returns a decision. This moves complex authorization logic out of your application code and into Keycloak's centrally managed policies.
Advanced Cell-Level Security
Step 19 Β· Database-level row security with PostgreSQL
Implement defense-in-depth by enabling PostgreSQL Row-Level Security. Even if your application code has a flaw, the database enforces access rules automatically based on session variables set from the userβs JWT token. The RLS policies enforce both classification clearance and need-to-know approval at the database layer.
-- Enable RLS on classified data table ALTER TABLE documents ENABLE ROW LEVEL SECURITY; -- Policy 1: User classification must meet or exceed data classification CREATE POLICY classification_check ON documents FOR ALL USING ( classification_level <= current_setting('app.user_classification', true)::int ); -- Policy 2: User must have NTK approval for the project code CREATE POLICY ntk_check ON documents FOR SELECT USING ( project_code IS NULL -- unclassified data is visible OR EXISTS ( SELECT 1 FROM user_need_to_know untk WHERE untk.user_id = current_setting('app.user_id', true) AND untk.project_code = documents.project_code AND untk.status = 'approved' AND untk.can_read = true AND (untk.expires_at IS NULL OR untk.expires_at > NOW()) ) ); -- In app: set session vars from JWT before any query await db.query(`SET app.user_id = $1`, [token.sub]); await db.query(`SET app.user_classification = $1`, [token.classification_level]); await db.query(`SET app.user_department = $1`, [token.department]); const results = await db.query('SELECT * FROM documents'); -- β Filtered by classification + NTK automatically!
π‘οΈ Key Principle: Three-Gate Access Check
All three gates must pass. Failing any one gate = access denied.
Monitoring & Troubleshooting
Step 20 Β· Logging, events, and common issues
Enable event logging in both Keycloak and Azure AD to monitor authentication flows, audit access, and diagnose problems.
Keycloak Events
Realm Settings β Events: enable Login Events and Admin Events with 7-day expiration. Monitor successful/failed logins, track IdP usage, and review token issuance.
Azure AD Sign-in Logs
Azure Portal β Azure AD β Sign-in logs: monitor authentication requests from Keycloak, review consent grants, and check for errors.
Common Issues & Fixes
Users can't log in via Azure AD
./bin/kc.sh start-dev --log-level=DEBUG
Roles not appearing in tokens
Cell-level security not working
SSO session timeout issues
Hands-On Demo Setup
Personal Azure + local Docker Compose
This section walks you through building a fully working demo environment on your personal Azure account (free tier) and a local Docker Compose stack. You'll use Microsoft Entra External ID β Microsoft's modern CIAM platform β as the upstream identity provider, Keycloak as the broker, a Node.js API, and two demo client apps that prove SSO and RBAC end-to-end.
What is Microsoft Entra External ID?
Entra External ID is Microsoft's next-generation Customer Identity & Access Management (CIAM) solution, replacing Azure AD B2C. It creates a separate external tenant β isolated from your workforce directory β designed specifically for consumer and partner-facing apps.
Prerequisites
Free account at azure.microsoft.com. Any personal Microsoft account works.
Docker + Docker Compose. WSL2 on Windows or native on macOS/Linux.
For optional local development. Docker images include Node runtime.
Entra External ID β Azure Portal Setup
Step A1 β Create an External Tenant
Sign in to https://entra.microsoft.com. Navigate to Entra ID β Overview β Manage tenants β + Create. Select "External" and continue. Enter a Tenant Name (e.g., MyDemo Customers) and a unique domain prefix (e.g., mydemocustomers). Click Review + Create. Provisioning takes up to 30 minutes.
Step A2 β Switch to the External Tenant & Run the Guide
Once created, use the Settings β Directories + Subscriptions menu to switch to your new external tenant. Browse to Entra ID β Overview β Get started β Start the guide. The wizard walks you through choosing sign-in methods (Email + password or Email + OTP), adding custom attributes, and branding. Choose Email and password for simplicity. Complete a test sign-up with a different email from your admin account.
Step A3 β Register Two App Registrations
Navigate to App registrations β + New registration. Create two applications:
Redirect URI (Web):
http://localhost:8080/realms/demo/broker/entra-external/endpointNote the Application (client) ID and Directory (tenant) ID.
Redirect URI (SPA):
http://localhost:3001/callbackEnable Access tokens and ID tokens under Authentication β Implicit grant.
Step A4 β Create Client Secret for Keycloak-Broker
Open the Keycloak-Broker app β Certificates & secrets β + New client secret. Description: keycloak-demo-secret, expiry: 6 months. Copy the Value immediately β you won't see it again. Save it to your .env file.
Step A5 β Configure API Permissions
On the Keycloak-Broker app β API permissions β + Add a permission β Microsoft Graph (Delegated). Add: openid, profile, email, offline_access. Click "Grant admin consent".
Step A6 β Create a User Flow
Go to External Identities β User flows β + New user flow. Name: demo-signup-signin. Select Email with password. Optionally collect Name and City attributes. Click Create.
Then open the flow β Applications β + Add application β select both Keycloak-Broker and Demo-SPA-Client. This binds the sign-in experience to your registered apps.
Step A7 β Note Your Discovery Endpoint
Your External ID OIDC discovery URL follows this pattern:
https://{your-domain}.ciamlogin.com/{tenant-id}/v2.0/.well-known/openid-configuration # Example: https://mydemocustomers.ciamlogin.com/aaaabbbb-1111-2222-3333-ccccddddeeee/v2.0/.well-known/openid-configuration
This is the URL you'll give Keycloak to auto-discover all token, authorization, and JWKS endpoints.
Local Docker Compose Stack
Create the following files in a project directory. The stack runs Keycloak (identity broker), a Node.js Express API (resource server), and two demo client apps (SPA frontends), plus PostgreSQL for Keycloak storage.
β .env
# === Entra External ID (from Azure Portal) === ENTRA_TENANT_ID=aaaabbbb-1111-2222-3333-ccccddddeeee ENTRA_CLIENT_ID=11112222-aaaa-bbbb-cccc-333344445555 ENTRA_CLIENT_SECRET=your-client-secret-value-here ENTRA_DOMAIN=mydemocustomers # === Keycloak === KC_DB_PASSWORD=supersecretdb KC_ADMIN=admin KC_ADMIN_PASSWORD=admin123!
β‘ docker-compose.yml
version: "3.9" services: postgres: image: postgres:16-alpine environment: POSTGRES_DB: keycloak POSTGRES_USER: keycloak POSTGRES_PASSWORD: ${KC_DB_PASSWORD} volumes: - pgdata:/var/lib/postgresql/data healthcheck: test: ["CMD-SHELL", "pg_isready -U keycloak"] interval: 5s retries: 5 keycloak: image: quay.io/keycloak/keycloak:25.0 command: start-dev environment: KC_DB: postgres KC_DB_URL: jdbc:postgresql://postgres:5432/keycloak KC_DB_USERNAME: keycloak KC_DB_PASSWORD: ${KC_DB_PASSWORD} KC_HEALTH_ENABLED: "true" KEYCLOAK_ADMIN: ${KC_ADMIN} KEYCLOAK_ADMIN_PASSWORD: ${KC_ADMIN_PASSWORD} ports: - "8080:8080" depends_on: postgres: condition: service_healthy api: build: ./api ports: - "4000:4000" environment: KEYCLOAK_URL: http://keycloak:8080 KEYCLOAK_REALM: demo PORT: 4000 depends_on: - keycloak client-a: build: ./client ports: - "3001:80" environment: VITE_KC_URL: http://localhost:8080 VITE_KC_REALM: demo VITE_KC_CLIENT: client-a VITE_API_URL: http://localhost:4000 client-b: build: ./client ports: - "3002:80" environment: VITE_KC_URL: http://localhost:8080 VITE_KC_REALM: demo VITE_KC_CLIENT: client-b VITE_API_URL: http://localhost:4000 volumes: pgdata:
β’ api/Dockerfile + server.js
# api/Dockerfile
FROM node:20-alpine
WORKDIR /app
COPY package.json ./
RUN npm install
COPY . .
EXPOSE 4000
CMD ["node", "server.js"]
// api/server.js β Resource server with JWT validation const express = require("express"); const jwt = require("jsonwebtoken"); const jwksClient = require("jwks-rsa"); const cors = require("cors"); const app = express(); app.use(cors()); const client = jwksClient({ jwksUri: `${process.env.KEYCLOAK_URL}/realms/${process.env.KEYCLOAK_REALM}/protocol/openid-connect/certs` }); function getKey(header, cb) { client.getSigningKey(header.kid, (err, key) => { cb(err, key?.publicKey || key?.rsaPublicKey); }); } function protect(...roles) { return (req, res, next) => { const token = req.headers.authorization?.split(" ")[1]; if (!token) return res.status(401).json({ error: "No token" }); jwt.verify(token, getKey, (err, decoded) => { if (err) return res.status(403).json({ error: "Invalid token" }); if (roles.length > 0) { const userRoles = decoded.realm_access?.roles || []; if (!roles.some(r => userRoles.includes(r))) return res.status(403).json({ error: "Insufficient role" }); } req.user = decoded; next(); }); }; } app.get("/api/public", (req, res) => res.json({ message: "Public β no auth needed" })); app.get("/api/profile", protect(), (req, res) => res.json({ user: req.user.preferred_username, email: req.user.email, roles: req.user.realm_access?.roles })); app.get("/api/admin", protect("admin"), (req, res) => res.json({ message: "Admin-only data", user: req.user.preferred_username })); app.listen(4000, () => console.log("API on :4000"));
β£ client/ β Demo SPA (keycloak-js)
<!-- client/index.html β Minimal SPA that proves SSO --> <script src="https://cdn.jsdelivr.net/npm/keycloak-js@25/dist/keycloak.min.js"></script> <script> const kc = new Keycloak({ url: "http://localhost:8080", realm: "demo", clientId: "client-a" // or "client-b" for second app }); kc.init({ onLoad: "login-required", pkceMethod: "S256" }) .then(auth => { if (!auth) window.location.reload(); document.getElementById("user").textContent = kc.tokenParsed.preferred_username; document.getElementById("roles").textContent = JSON.stringify(kc.tokenParsed.realm_access?.roles); // Call protected API fetch("http://localhost:4000/api/profile", { headers: { Authorization: `Bearer ${kc.token}` } }).then(r => r.json()).then(d => { document.getElementById("api").textContent = JSON.stringify(d, null, 2); }); }); </script> <h1>Demo Client</h1> <p>User: <b id="user"></b></p> <p>Roles: <code id="roles"></code></p> <pre id="api"></pre>
β€ Launch the Stack
# Start everything docker compose up -d --build # Wait for Keycloak health check docker compose logs -f keycloak | grep "Listening on" # Endpoints: Keycloak Admin β http://localhost:8080 (admin / admin123!) Demo Client A β http://localhost:3001 Demo Client B β http://localhost:3002 API β http://localhost:4000/api/public
Keycloak β Wire Up Federation
demo. Enable it.entra-external. Paste the OIDC discovery URL from Step A7. Enter the Client ID and Secret from Step A3/A4. Set Client Authentication: Client secret sent as post. Set Default Scopes: openid profile email.email β email, given_name β firstName, family_name β lastName. Set Sync Mode: FORCE so attributes update on every login.client-a (Root URL: http://localhost:3001) and client-b (Root URL: http://localhost:3002). Both: Access Type = public, Standard Flow = ON, Valid Redirect URIs = http://localhost:3001/* (or 3002). Web Origins = +.employee, manager, admin. Create a client scope "roles" with a User Realm Role mapper to embed roles in the access token. Assign default role to employee.http://localhost:3001. Keycloak login page appears with a "Sign in with Entra External ID" button. Click it β you're redirected to the Entra External ID branded sign-in page β log in with your test user β redirected back to Keycloak β then back to Client A with a valid JWT. Open http://localhost:3002 in another tab β you should be logged in automatically via SSO (no password prompt).Workforce Tenant vs. External Tenant
Traditional Azure AD (now "Entra ID") creates a workforce tenant β a directory for your employees. Entra External ID creates an external tenant, a completely separate directory purpose-built for consumers, partners, and external users. The external tenant has its own set of features: customizable sign-up pages, self-service password reset, social login federation (Google, Facebook, Apple), custom OIDC providers, and user-flow-based policies. Think of it as Azure AD B2C's successor β simpler, more integrated, and built into the Entra admin center.
The OIDC Discovery Endpoint
External tenants use a special domain: *.ciamlogin.com (Customer IAM Login). The discovery URL returns a JSON document containing the authorization endpoint, token endpoint, JWKS URI, supported scopes, and claims. When you paste this into Keycloak, it auto-populates all the OIDC configuration fields β no manual endpoint entry needed.
{
"issuer": "https://mydemocustomers.ciamlogin.com/{tid}/v2.0",
"authorization_endpoint": ".../{tid}/oauth2/v2.0/authorize",
"token_endpoint": ".../{tid}/oauth2/v2.0/token",
"jwks_uri": ".../{tid}/discovery/v2.0/keys",
"scopes_supported": ["openid", "profile", "email", "offline_access"],
"response_types_supported": ["code", "id_token", "code id_token"]
}
User Flows vs. Custom Auth Policies
User Flows are the declarative, no-code way to define sign-up/sign-in experiences. You select the sign-in method (email+password, email+OTP), choose which attributes to collect, and bind the flow to specific app registrations. Under the hood, a user flow creates an Authorization endpoint that triggers the correct authentication experience. For more complex scenarios, you can use Custom Authentication Extensions (REST API callbacks during the flow) β but user flows cover most demo and production cases.
How the Demo Authentication Flow Works
localhost:3001. The keycloak-js adapter detects no session and redirects to Keycloak /auth.authorize endpoint, including its client ID, redirect URI (the broker endpoint), and scopes.email, given_name, family_name, sub.localhost:3001 with the auth code. keycloak-js exchanges it for the Keycloak JWT. The SPA calls the API with Authorization: Bearer {kc-token}.localhost:3002 (Client B), keycloak-js redirects to Keycloak, which sees the existing SSO cookie and immediately issues new tokens β zero login prompts. That's SSO in action.Why Broker Through Keycloak Instead of Directly Using Entra?
You could point your apps directly at Entra External ID β it supports OIDC natively. But brokering through Keycloak gives you: role management (Entra External ID doesn't have Keycloak-style realm roles or composite roles), multi-IdP support (add Google, SAML, LDAP alongside Entra without changing app code), protocol translation (serve SAML apps behind OIDC IdPs), fine-grained authorization (Keycloak Authorization Services, UMA), and portability β if you ever switch from Entra to Okta or another IdP, only the Keycloak broker config changes; your apps are untouched.
Pricing & Limits
Entra External ID is free for the first 50,000 monthly active users (MAU). An MAU is a unique user who authenticates at least once in a calendar month. Beyond that, pricing is per-MAU. For development and small-scale demos, the free tier is more than sufficient. The 30-day free trial includes all features; after that, link an Azure subscription to continue.
Access Control Matrix
Who can do what
Role Capability Matrix (what the role permits)
| Role | Create | Read Own | Public | Team | Dept | All | Edit Own | Edit Team | Delete | Share | Audit |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Employee | β | β | β | β | β | β | β | β | β | β | β |
| Manager | β | β | β | β | β | β | β | β | β | β | β |
| Director | β | β | β | β | β | β | β | β | β | β | β |
| Executive | β | β | β | β | β | β | β | β | β | β | β |
Classification & Need-to-Know Gate (overrides role)
Even if the role matrix above shows β for an action, the user is blocked unless they also pass both classification and need-to-know checks for the specific data item. CRUD permissions are granted individually per NTK approval.
| Data Classification | Min. Clearance | NTK Required? | NTK CRUD Scoped? | Example |
|---|---|---|---|---|
| L0 β Public | None | β No | β | Company blog posts |
| L1 β Internal | L1+ | β No | β | Team wikis, org charts |
| L2 β Confidential | L2+ | β Yes | β Yes | Financial forecasts, HR reports |
| L3 β Restricted | L3+ | β Yes | β Yes | M&A plans, security audits |
| L4 β Top-Secret | L4 | β Yes | β Yes | Board strategies, legal matters |
Real-World Scenario: Who Gets Access?
| User | Role | Clearance | NTK: ATLAS | L3 Doc in ATLAS | Allowed CRUD |
|---|---|---|---|---|---|
| Alice | Executive | L4 | β None | β DENIED | β |
| Bob | Employee | L3 | β R | β Read | Read only |
| Carol | Manager | L2 | β CRUD | β DENIED | Clearance L2 < L3 |
| Dave | Director | L3 | β CR | β Create+Read | Create, Read only |
β οΈ Role capability + Classification clearance + Need-to-Know approval with CRUD scope β all three must align for access.
Classification & Need-to-Know
Cell-level security independent of role hierarchy
Traditional RBAC answers "what actions can this role perform?" but it does not answer "should this specific user see this specific piece of data?" The Classification & Need-to-Know model adds two independent security dimensions that override role permissions. A user must satisfy all three gates β role, classification, and need-to-know β to access any classified data.
Classification Level
Every data record is tagged with a level: L0 (Public), L1 (Internal), L2 (Confidential), L3 (Restricted), L4 (Top-Secret). Every user is granted a clearance level by a security officer. User clearance must β₯ data level.
Need-to-Know (NTK)
Data at L2+ is grouped under a project code (e.g., PROJECT-ATLAS). A user must have an approved NTK entry for that project β granted by the project owner or a security officer β regardless of their role or clearance level.
CRUD-Scoped Permissions
Each NTK approval specifies exactly which operations the user may perform: can_create, can_read, can_update, can_delete. An analyst might get Read-only; a project lead might get full CRUD.
The Three-Gate Access Model
this type of action?
data classification?
this project + CRUD?
approved in NTK
Any gate failure = β Access Denied (regardless of other gates passing)
NTK Approval Workflow
can_read = true, all others false.expires_at date. Expired approvals are automatically denied. Regular review cycles ensure access doesnβt persist beyond need.Implementation: Express.js Middleware
// Three-gate access check middleware function requireClassifiedAccess(crudAction) { return async (req, res, next) => { const token = req.user; // from JWT middleware const docId = req.params.id; // Fetch the document's classification & project code const doc = await db.query( 'SELECT classification_level, project_code FROM documents WHERE id = $1', [docId] ); // Gate 1: Role check (already handled by keycloak.protect()) // Gate 2: Classification check const userLevel = token.classification_level || 0; if (userLevel < doc.classification_level) { await auditLog(docId, token, 'DENIED', 'Classification insufficient'); return res.status(403).json({ error: 'Classification level insufficient' }); } // Gate 3: Need-to-Know check (for L2+ data with a project code) if (doc.project_code && doc.classification_level >= 2) { const ntk = await db.query( `SELECT * FROM user_need_to_know WHERE user_id = $1 AND project_code = $2 AND status = 'approved' AND (expires_at IS NULL OR expires_at > NOW())`, [token.sub, doc.project_code] ); if (!ntk.rows.length) { await auditLog(docId, token, 'DENIED', 'No NTK approval'); return res.status(403).json({ error: 'Need-to-know approval required' }); } // Check specific CRUD permission const crudField = `can_${crudAction}`; // e.g., can_read if (!ntk.rows[0][crudField]) { await auditLog(docId, token, 'DENIED', `NTK lacks ${crudAction}`); return res.status(403).json({ error: `NTK approved but ${crudAction} not permitted` }); } } await auditLog(docId, token, 'GRANTED', crudAction); next(); }; } // Usage in routes app.get('/api/docs/:id', keycloak.protect(), // Gate 1: Role requireClassifiedAccess('read'), // Gate 2+3: Classification + NTK getDocument ); app.put('/api/docs/:id', keycloak.protect('manager'), // Gate 1: Manager+ role requireClassifiedAccess('update'), // Gate 2+3: Classification + NTK updateDocument );
π‘ Why This Matters
In traditional RBAC, an executive who should only see marketing data can technically access engineering secrets because their role inherits all lower permissions. The Classification + NTK model ensures that:
Security Best Practices
Production hardening
π Use HTTPS Everywhere
Never use HTTP for identity flows in production. Configure proper SSL/TLS certificates for both Keycloak and your applications.
π Secure Client Secrets
Store in environment variables or secret managers (Azure Key Vault, HashiCorp Vault). Rotate regularly and never commit to source control.
β Validate JWT Tokens
Always verify JWT signatures, check expiration times, and validate the issuer and audience claims. Never trust tokens without full validation.
β± Short Token Lifespans
Access tokens: 5β15 minutes. Use refresh tokens for extended sessions. This limits exposure if a token is intercepted.
π± Enable MFA in Azure AD
Add an extra security layer. Required for all admin accounts. Consider Conditional Access policies for risk-based MFA.
π Regular Audits
Review access logs monthly. Check for unused accounts. Audit role assignments. Apply the principle of least privilege everywhere.
Comprehensive Auditing
Full CRUD & role action audit trail
Every action in a classified enterprise system must be recorded, attributable, and reviewable. This section covers a complete audit framework that logs all Create, Read, Update, and Delete operations across every role, together with authentication events, role changes, classification gate decisions, and NTK approvals.
⚠️ Why Every Operation Must Be Audited
Regulatory Compliance — SOX, HIPAA, GDPR, and FedRAMP all mandate immutable audit trails for data access. Without logging, your organization fails compliance audits.
Breach Detection — Audit logs are the first artifact forensic teams analyze. A missing or incomplete trail means an incident cannot be reconstructed.
Insider Threat Deterrence — When users know every action is recorded and reviewed, unauthorized access attempts drop dramatically.
Non-Repudiation — Cryptographically linked log entries ensure a user cannot deny performing an action. Timestamps, IPs, and JWT fingerprints make each record provable.
Audit Event Categories
📄 CRUD Operations
CREATE— Document created, with owner, department, classification levelREAD— Document accessed, including partial reads and search-result exposuresUPDATE— Field-level diff recorded (old value → new value)DELETE— Soft-delete recorded; hard-deletes require executive approval and double-log
🛡️ Role & Access Events
ROLE_ASSIGNED— User granted a role, by whom, effective dateROLE_REVOKED— Role removed, reason, revoking authorityROLE_ESCALATED— Temporary privilege elevation with TTLPERMISSION_DENIED— Failed access attempt logged with full context
🔐 Classification & NTK Events
CLASSIFICATION_CHECK— Gate 2 evaluation result (pass/fail)NTK_REQUESTED— Need-to-know approval initiatedNTK_APPROVED— Approval granted with CRUD scope and expiryNTK_DENIED— Approval rejected, reason recordedNTK_EXPIRED— Automatic revocation at expiry timestamp
🔒 Authentication Events
LOGIN_SUCCESS— SSO session created, IdP source, MFA statusLOGIN_FAILED— Failed attempt with IP, user-agent, failure reasonLOGOUT— Session terminated, duration, trigger (user/timeout/admin)TOKEN_REFRESHED— Refresh token rotation event
📤 Share & Export Events
DOCUMENT_SHARED— Who shared, with whom, permission scopeSHARE_REVOKED— Sharing permission removedDOCUMENT_EXPORTED— Download or print event with formatBULK_EXPORT— Mass data export flagged for review
⚙️ System & Admin Events
POLICY_CHANGED— RLS policy or Keycloak policy modifiedSCHEMA_ALTERED— Database schema change detectedCONFIG_UPDATED— System configuration change with diffAUDIT_LOG_ACCESSED— Meta-audit: who reviewed the audit logs
Audit Trail Flow
What Gets Audited Per Role
| Audit Event | Employee | Manager | Director | Executive |
|---|---|---|---|---|
| CREATE document | ✅ | ✅ | ✅ | ✅ |
| READ document | ✅ | ✅ | ✅ | ✅ |
| UPDATE document | ✅ | ✅ | ✅ | ✅ |
| DELETE document | ✅ | ✅ | ✅ | ✅ |
| ROLE_ASSIGNED | — | ✅ | ✅ | ✅ |
| ROLE_REVOKED | — | ✅ | ✅ | ✅ |
| NTK_APPROVED | — | — | ✅ | ✅ |
| CLASSIFICATION_CHECK | ✅ | ✅ | ✅ | ✅ |
| LOGIN / LOGOUT | ✅ | ✅ | ✅ | ✅ |
| PERMISSION_DENIED | ✅ | ✅ | ✅ | ✅ |
| DOCUMENT_SHARED | — | ✅ | ✅ | ✅ |
| DOCUMENT_EXPORTED | ✅ | ✅ | ✅ | ✅ |
| AUDIT_LOG_ACCESSED | — | — | ✅ | ✅ |
🗄️ Audit Database Schema
The audit system uses two core tables: an append-only event log and a separate session tracking table. Both are protected by PostgreSQL policies that prevent UPDATE and DELETE operations.
-- Comprehensive audit event log (append-only) CREATE TABLE audit_events ( id BIGSERIAL PRIMARY KEY, event_id UUID DEFAULT gen_random_uuid() UNIQUE, event_type VARCHAR(50) NOT NULL, category VARCHAR(30) NOT NULL, severity VARCHAR(10) DEFAULT 'INFO', timestamp TIMESTAMPTZ DEFAULT NOW(), -- Actor user_id VARCHAR(255) NOT NULL, user_email VARCHAR(255), user_role VARCHAR(50), ip_address INET, user_agent TEXT, session_id VARCHAR(255), -- Target resource resource_type VARCHAR(50), resource_id VARCHAR(255), resource_name VARCHAR(255), -- Security gates classification_level INTEGER, classification_result VARCHAR(10), ntk_project_code VARCHAR(50), ntk_result VARCHAR(10), role_gate_result VARCHAR(10), -- Payload action_detail JSONB, old_values JSONB, new_values JSONB, metadata JSONB ); -- Performance indexes CREATE INDEX idx_audit_type ON audit_events(event_type); CREATE INDEX idx_audit_user ON audit_events(user_id); CREATE INDEX idx_audit_ts ON audit_events(timestamp); CREATE INDEX idx_audit_resource ON audit_events(resource_type, resource_id); CREATE INDEX idx_audit_severity ON audit_events(severity); CREATE INDEX idx_audit_category ON audit_events(category); CREATE INDEX idx_audit_session ON audit_events(session_id); CREATE INDEX idx_audit_user_ts ON audit_events(user_id, timestamp DESC); -- Prevent tampering: revoke UPDATE/DELETE on audit table REVOKE UPDATE, DELETE ON audit_events FROM app_user; REVOKE UPDATE, DELETE ON audit_events FROM app_role; -- Session tracking table CREATE TABLE audit_sessions ( session_id VARCHAR(255) PRIMARY KEY, user_id VARCHAR(255) NOT NULL, user_email VARCHAR(255), idp_source VARCHAR(50), login_at TIMESTAMPTZ DEFAULT NOW(), logout_at TIMESTAMPTZ, duration_sec INTEGER, mfa_used BOOLEAN DEFAULT false, ip_address INET, user_agent TEXT );
🛠️ Express.js Audit Middleware
A centralized audit middleware captures every request automatically. It enriches log entries with JWT claims, IP context, and gate results before writing to the append-only audit table.
// audit-logger.js — Centralized audit service const { Pool } = require('pg'); const pool = new Pool(); const SEVERITY = { CREATE: 'INFO', READ: 'INFO', UPDATE: 'WARN', DELETE: 'WARN', PERMISSION_DENIED: 'ALERT', ROLE_ASSIGNED: 'WARN', ROLE_REVOKED: 'WARN', LOGIN_FAILED: 'ALERT', NTK_DENIED: 'ALERT', BULK_EXPORT: 'ALERT', AUDIT_LOG_ACCESSED: 'WARN' }; async function logAuditEvent({ eventType, category, userId, userEmail, userRole, ipAddress, userAgent, sessionId, resourceType, resourceId, resourceName, classificationLevel, classificationResult, ntkProjectCode, ntkResult, roleGateResult, actionDetail, oldValues, newValues, metadata }) { const severity = SEVERITY[eventType] || 'INFO'; await pool.query(` INSERT INTO audit_events ( event_type, category, severity, user_id, user_email, user_role, ip_address, user_agent, session_id, resource_type, resource_id, resource_name, classification_level, classification_result, ntk_project_code, ntk_result, role_gate_result, action_detail, old_values, new_values, metadata ) VALUES ( $1,$2,$3,$4,$5,$6,$7,$8,$9,$10, $11,$12,$13,$14,$15,$16,$17,$18,$19,$20,$21 )`, [ eventType, category, severity, userId, userEmail, userRole, ipAddress, userAgent, sessionId, resourceType, resourceId, resourceName, classificationLevel, classificationResult, ntkProjectCode, ntkResult, roleGateResult, JSON.stringify(actionDetail), JSON.stringify(oldValues), JSON.stringify(newValues), JSON.stringify(metadata) ]); // Alert on high-severity events if (severity === 'ALERT') { await triggerSecurityAlert({ eventType, userId, resourceId }); } } module.exports = { logAuditEvent };
📡 Request-Level Audit Middleware
Wrap every route with an audit middleware that automatically captures the actor, action, and outcome for every HTTP request.
// audit-middleware.js const { logAuditEvent } = require('./audit-logger'); const HTTP_TO_CRUD = { GET: 'READ', POST: 'CREATE', PUT: 'UPDATE', PATCH: 'UPDATE', DELETE: 'DELETE' }; function auditMiddleware(resourceType) { return async (req, res, next) => { const startTime = Date.now(); const originalJson = res.json.bind(res); res.json = async (body) => { const duration = Date.now() - startTime; const eventType = HTTP_TO_CRUD[req.method] || req.method; await logAuditEvent({ eventType, category: 'DATA_ACCESS', userId: req.user?.sub, userEmail: req.user?.email, userRole: req.user?.realm_role, ipAddress: req.ip, userAgent: req.get('User-Agent'), sessionId: req.user?.session_state, resourceType, resourceId: req.params.id || body?.id, resourceName: body?.title, classificationLevel: req.classificationLevel, classificationResult: req.classificationPassed ? 'PASS' : 'FAIL', ntkProjectCode: req.ntkProjectCode, ntkResult: req.ntkPassed ? 'PASS' : 'FAIL', roleGateResult: req.rolePassed ? 'PASS' : 'FAIL', actionDetail: { method: req.method, path: req.originalUrl, statusCode: res.statusCode, durationMs: duration }, oldValues: req.previousValues || null, newValues: req.method !== 'GET' ? req.body : null, metadata: { correlationId: req.headers['x-request-id'] } }); return originalJson(body); }; next(); }; } // Usage in routes app.get('/api/documents/:id', authenticate, requireRole('employee'), requireClassifiedAccess('read'), auditMiddleware('document'), getDocumentHandler ); app.put('/api/documents/:id', authenticate, requireRole('manager'), requireClassifiedAccess('update'), captureOldValues('documents'), auditMiddleware('document'), updateDocumentHandler );
🎯 PostgreSQL Trigger-Based Auditing
Defense-in-depth: even if application-level audit middleware is bypassed, database triggers guarantee that every INSERT, UPDATE, and DELETE on the documents table is captured.
-- Database trigger for automatic CRUD auditing CREATE OR REPLACE FUNCTION fn_audit_documents() RETURNS TRIGGER AS $$ BEGIN INSERT INTO audit_events ( event_type, category, severity, user_id, resource_type, resource_id, old_values, new_values, metadata ) VALUES ( TG_OP, 'DB_TRIGGER', CASE WHEN TG_OP = 'DELETE' THEN 'WARN' ELSE 'INFO' END, current_setting('app.current_user_id', true), 'document', COALESCE(NEW.id, OLD.id)::TEXT, CASE WHEN TG_OP IN ('UPDATE','DELETE') THEN row_to_json(OLD) END, CASE WHEN TG_OP IN ('INSERT','UPDATE') THEN row_to_json(NEW) END, jsonb_build_object( 'trigger', TG_NAME, 'table', TG_TABLE_NAME, 'session_user', session_user, 'timestamp', NOW() ) ); RETURN COALESCE(NEW, OLD); END; $$ LANGUAGE plpgsql SECURITY DEFINER; CREATE TRIGGER trg_audit_documents_insert AFTER INSERT ON documents FOR EACH ROW EXECUTE FUNCTION fn_audit_documents(); CREATE TRIGGER trg_audit_documents_update AFTER UPDATE ON documents FOR EACH ROW EXECUTE FUNCTION fn_audit_documents(); CREATE TRIGGER trg_audit_documents_delete AFTER DELETE ON documents FOR EACH ROW EXECUTE FUNCTION fn_audit_documents(); -- Audit trigger for role changes CREATE OR REPLACE FUNCTION fn_audit_role_changes() RETURNS TRIGGER AS $$ BEGIN INSERT INTO audit_events ( event_type, category, severity, user_id, resource_type, resource_id, old_values, new_values ) VALUES ( CASE WHEN TG_OP = 'INSERT' THEN 'ROLE_ASSIGNED' WHEN TG_OP = 'DELETE' THEN 'ROLE_REVOKED' ELSE 'ROLE_UPDATED' END, 'ROLE_MANAGEMENT', 'WARN', current_setting('app.current_user_id', true), 'user_role', COALESCE(NEW.user_id, OLD.user_id), CASE WHEN TG_OP != 'INSERT' THEN row_to_json(OLD) END, CASE WHEN TG_OP != 'DELETE' THEN row_to_json(NEW) END ); RETURN COALESCE(NEW, OLD); END; $$ LANGUAGE plpgsql SECURITY DEFINER; -- Audit trigger for NTK approval changes CREATE OR REPLACE FUNCTION fn_audit_ntk_changes() RETURNS TRIGGER AS $$ BEGIN INSERT INTO audit_events ( event_type, category, severity, user_id, resource_type, resource_id, ntk_project_code, ntk_result, old_values, new_values ) VALUES ( CASE WHEN TG_OP = 'INSERT' THEN 'NTK_REQUESTED' WHEN NEW.status = 'approved' THEN 'NTK_APPROVED' WHEN NEW.status = 'denied' THEN 'NTK_DENIED' ELSE 'NTK_UPDATED' END, 'NTK_MANAGEMENT', CASE WHEN NEW.status = 'denied' THEN 'ALERT' ELSE 'WARN' END, current_setting('app.current_user_id', true), 'ntk_approval', COALESCE(NEW.user_id, OLD.user_id), COALESCE(NEW.project_code, OLD.project_code), NEW.status, CASE WHEN TG_OP != 'INSERT' THEN row_to_json(OLD) END, row_to_json(NEW) ); RETURN NEW; END; $$ LANGUAGE plpgsql SECURITY DEFINER;
📊 Audit Querying & Reporting
Pre-built queries for compliance reporting, anomaly detection, and security reviews that auditors and security teams use during assessments.
-- 1. All CRUD operations by a specific user (last 30 days) SELECT event_type, resource_type, resource_id, timestamp, action_detail, classification_result FROM audit_events WHERE user_id = $1 AND category = 'DATA_ACCESS' AND timestamp > NOW() - INTERVAL '30 days' ORDER BY timestamp DESC; -- 2. All denied access attempts (security review) SELECT user_id, user_email, user_role, event_type, resource_id, ip_address, classification_result, ntk_result, role_gate_result, timestamp FROM audit_events WHERE severity = 'ALERT' AND (classification_result = 'FAIL' OR ntk_result = 'FAIL' OR role_gate_result = 'FAIL') ORDER BY timestamp DESC LIMIT 100; -- 3. Role change history for compliance audit SELECT event_type, user_id, old_values->>'role' AS previous_role, new_values->>'role' AS new_role, metadata->>'changed_by' AS changed_by, timestamp FROM audit_events WHERE category = 'ROLE_MANAGEMENT' ORDER BY timestamp DESC; -- 4. Anomaly: users accessing data outside business hours SELECT user_id, user_email, event_type, resource_id, timestamp, ip_address FROM audit_events WHERE EXTRACT(HOUR FROM timestamp) NOT BETWEEN 7 AND 19 AND event_type IN ('READ','UPDATE','DELETE') AND timestamp > NOW() - INTERVAL '7 days' ORDER BY timestamp DESC; -- 5. Executive summary: event counts by category SELECT category, COUNT(*) AS total_events, COUNT(*) FILTER (WHERE severity = 'ALERT') AS alerts, COUNT(DISTINCT user_id) AS unique_users FROM audit_events WHERE timestamp > NOW() - INTERVAL '24 hours' GROUP BY category ORDER BY alerts DESC;
🔑 Keycloak Event Listener Integration
Keycloak emits authentication and admin events natively. Configure the event listener to forward these into your centralized audit_events table for a single pane of glass.
// Keycloak Realm Settings → Events // Enable: Login Events + Admin Events // Event Listeners: jboss-logging, custom-audit-spi Login Events: LOGIN, LOGIN_ERROR, LOGOUT, REGISTER, CODE_TO_TOKEN, CODE_TO_TOKEN_ERROR, REFRESH_TOKEN, REFRESH_TOKEN_ERROR, FEDERATED_IDENTITY_LINK Admin Events: CREATE, UPDATE, DELETE (on Users, Roles, Clients, Groups, Identity Providers) // Custom SPI forwards to audit_events table: public class AuditEventListener implements EventListenerProvider { @Override public void onEvent(Event event) { auditService.log(AuditEvent.builder() .eventType(mapEventType(event.getType())) .category("AUTHENTICATION") .userId(event.getUserId()) .ipAddress(event.getIpAddress()) .sessionId(event.getSessionId()) .metadata(Map.of( "realmId", event.getRealmId(), "clientId", event.getClientId(), "error", event.getError() )) .build()); } }
🧠 How Auditing Protects Your Organization
Incident Response Acceleration
When a breach is detected, the audit trail answers: Who accessed what? When? From where? What data was viewed or modified? Mean Time to Identify (MTTI) drops from weeks to hours when audit data is comprehensive.
Privilege Creep Detection
Over time, users accumulate roles they no longer need. Audit logs reveal patterns: a user with a Manager role who never exercises manager-level operations is a candidate for downgrade.
Zero-Trust Verification
Auditing completes the zero-trust loop: authenticate, authorize, then verify. Every action is recorded and can be replayed to confirm that the three-gate model (Role + Classification + NTK) was enforced.
Legal & Regulatory Shield
In litigation or regulatory investigation, complete audit trails demonstrate due diligence. They prove your organization enforced access controls, detected anomalies, and responded appropriately.
Behavioral Analytics Foundation
Audit data feeds UEBA systems. Machine learning models baseline normal patterns and flag deviations: an employee suddenly reading 500 classified documents at 2 AM triggers automatic investigation workflows.
Continuous Compliance
Rather than scrambling before annual audits, continuous audit logging means your compliance posture is always audit-ready. Automated reports can be generated for SOC 2, ISO 27001, and NIST 800-53 on demand.
🛡️ Audit Log Security & Retention
Immutability
Audit tables have UPDATE/DELETE revoked. Write-ahead logs are streamed to a separate, isolated storage system. Checksums verify integrity.
Retention Policy
Hot storage: 90 days (fast query). Warm: 1 year (compressed). Cold archive: 7 years (compliance). Auto-partitioned by month.
Meta-Auditing
Access to audit logs is itself audited. Only Director+ roles can query audit_events. Every query is logged as AUDIT_LOG_ACCESSED.
Testing Checklist
End-to-end verification
Database Schema
PostgreSQL tables for the document repository
-- ============================================= -- CLASSIFICATION LEVELS (reference table) -- ============================================= CREATE TABLE classification_levels ( level INTEGER PRIMARY KEY, -- 0..4 name VARCHAR(50) NOT NULL, -- Public, Internal, Confidential, Restricted, Top-Secret requires_ntk BOOLEAN DEFAULT false, description TEXT ); INSERT INTO classification_levels VALUES (0, 'Public', false, 'Publicly available information'), (1, 'Internal', false, 'General internal company data'), (2, 'Confidential', true, 'Sensitive business data requiring NTK'), (3, 'Restricted', true, 'Highly sensitive β M&A, security audits'), (4, 'Top-Secret', true, 'Board-level strategies, legal matters'); -- ============================================= -- PROJECT CODES (what data is grouped under) -- ============================================= CREATE TABLE projects ( code VARCHAR(50) PRIMARY KEY, -- e.g., PROJECT-ATLAS name VARCHAR(255) NOT NULL, owner_id VARCHAR(255) NOT NULL, -- project owner who can grant NTK department VARCHAR(100), classification_level INTEGER REFERENCES classification_levels(level), status VARCHAR(20) DEFAULT 'active', created_at TIMESTAMP DEFAULT NOW() ); -- ============================================= -- USER CLASSIFICATION & NEED-TO-KNOW -- ============================================= CREATE TABLE user_classifications ( user_id VARCHAR(255) PRIMARY KEY, classification_level INTEGER REFERENCES classification_levels(level) DEFAULT 0, granted_by VARCHAR(255) NOT NULL, -- security officer who granted clearance granted_at TIMESTAMP DEFAULT NOW(), expires_at TIMESTAMP, notes TEXT ); CREATE TABLE user_need_to_know ( id SERIAL PRIMARY KEY, user_id VARCHAR(255) NOT NULL, project_code VARCHAR(50) REFERENCES projects(code), status VARCHAR(20) DEFAULT 'pending', -- pending, approved, revoked, expired can_create BOOLEAN DEFAULT false, can_read BOOLEAN DEFAULT false, can_update BOOLEAN DEFAULT false, can_delete BOOLEAN DEFAULT false, approved_by VARCHAR(255), -- project owner or security officer approved_at TIMESTAMP, expires_at TIMESTAMP, justification TEXT, -- why the user needs access created_at TIMESTAMP DEFAULT NOW(), UNIQUE(user_id, project_code) ); CREATE INDEX idx_ntk_user ON user_need_to_know(user_id); CREATE INDEX idx_ntk_project ON user_need_to_know(project_code); CREATE INDEX idx_ntk_status ON user_need_to_know(status); -- ============================================= -- DOCUMENTS (with classification + project code) -- ============================================= CREATE TABLE documents ( id SERIAL PRIMARY KEY, title VARCHAR(255) NOT NULL, content TEXT, owner_id VARCHAR(255) NOT NULL, owner_email VARCHAR(255) NOT NULL, department VARCHAR(100), classification_level INTEGER REFERENCES classification_levels(level) DEFAULT 1, project_code VARCHAR(50) REFERENCES projects(code), -- NULL = no NTK required is_public BOOLEAN DEFAULT false, deleted BOOLEAN DEFAULT false, deleted_at TIMESTAMP, deleted_by VARCHAR(255), created_at TIMESTAMP DEFAULT NOW(), updated_at TIMESTAMP DEFAULT NOW() ); CREATE INDEX idx_documents_owner ON documents(owner_id); CREATE INDEX idx_documents_department ON documents(department); CREATE INDEX idx_documents_classification ON documents(classification_level); CREATE INDEX idx_documents_project ON documents(project_code); -- Enable Row-Level Security ALTER TABLE documents ENABLE ROW LEVEL SECURITY; -- RLS: classification gate CREATE POLICY cls_gate ON documents FOR ALL USING (classification_level <= current_setting('app.user_classification', true)::int); -- RLS: need-to-know gate (SELECT) CREATE POLICY ntk_read_gate ON documents FOR SELECT USING ( project_code IS NULL OR EXISTS ( SELECT 1 FROM user_need_to_know u WHERE u.user_id = current_setting('app.user_id', true) AND u.project_code = documents.project_code AND u.status = 'approved' AND u.can_read = true AND (u.expires_at IS NULL OR u.expires_at > NOW()) ) ); -- ============================================= -- DOCUMENT SHARES -- ============================================= CREATE TABLE document_shares ( id SERIAL PRIMARY KEY, document_id INTEGER REFERENCES documents(id), shared_by VARCHAR(255) NOT NULL, shared_with_email VARCHAR(255) NOT NULL, permissions JSONB, created_at TIMESTAMP DEFAULT NOW() ); -- ============================================= -- AUDIT LOG (tracks all access & NTK decisions) -- ============================================= CREATE TABLE document_audit_log ( id SERIAL PRIMARY KEY, document_id INTEGER REFERENCES documents(id), action VARCHAR(50) NOT NULL, user_id VARCHAR(255) NOT NULL, user_email VARCHAR(255) NOT NULL, classification_check BOOLEAN, -- did they pass classification? ntk_check BOOLEAN, -- did they pass need-to-know? access_granted BOOLEAN NOT NULL, timestamp TIMESTAMP DEFAULT NOW(), details JSONB ); CREATE INDEX idx_audit_document ON document_audit_log(document_id); CREATE INDEX idx_audit_user ON document_audit_log(user_id); CREATE INDEX idx_audit_timestamp ON document_audit_log(timestamp); CREATE INDEX idx_audit_granted ON document_audit_log(access_granted); -- ============================================= -- USER PROFILES -- ============================================= CREATE TABLE user_profiles ( user_id VARCHAR(255) PRIMARY KEY, classification_level INTEGER REFERENCES classification_levels(level) DEFAULT 0, department VARCHAR(100), preferences JSONB, bio TEXT, created_at TIMESTAMP DEFAULT NOW(), updated_at TIMESTAMP DEFAULT NOW() );