deepseek-kubernetes-sandbox/examples/dsh-agents-as-a-service/gateway/README.md

DSH agents-as-a-service gateway

A single-file, browser-usable control + data plane for DeepSeek Harness Web (DSH) sandboxes running behind the agent-sandbox CRDs.

It does four things:

1. Users — creates a SandboxClaim from a warm pool, mints a bearer token (only its SHA-256 is stored as a claim annotation), reports derived state, and deletes the claim (cascading the sandbox and PVC).
2. Browser sessions — exchanges a user token for an HttpOnly session cookie so the DSH UI works in a browser at /u/<user>/. Automation can still use Authorization: Bearer <token>.
3. Reverse proxy — proxies HTTP (including SSE) and WebSocket traffic to the sandbox's DSH Web server on :3080, including DSH's own cookie handshake.
4. Optional SSO — when AUTH_PROXY_HEADER is set, trusts the identity header an authenticating reverse proxy (for example oauth2-proxy) forwards, auto-provisions that user, and exposes POST /recreate for a clean restart.

The DSH version pin lives in versions.env (DSH_VERSION plus the optional repository RELEASE_REVISION); the example manifests and published images use the release identity built from it (currently dsh-v0.2.1-rc.5). DSH binds :3080, gates the UI and /api behind a dsh-auth-* cookie and requires a WebSocket at /api/remote.mux.

Flow

browser / curl                    gateway                         cluster
    |                               |                                |
    |  POST /users {"user":"alice"} |                                |
    |------------------------------>| create SandboxClaim ---------->|
    |                               | wait Ready                    |
    |  201 {token} + session cookie |                                |
    |<------------------------------|                                |
    |  GET /u/alice/ (cookie)       |                                |
    |------------------------------>| claim -> sandbox -> pod        |
    |                               | read pod logs (pods/log)       |
    |                               | find "dsh web: ...?token=..."  |
    |                               | GET http://<fqdn>:3080/?token= |
    |                               | capture dsh-auth-* Set-Cookie  |
    |                               | proxy with Host/Origin kept    |
    |  HTML / SSE / WebSocket       |<----------------------------->|
    |<------------------------------|                                |

The launch token is printed by DSH to its pod stdout:

dsh web: http://<podIP>:3080/?token=<base64url>

The gateway reads the tail of the pod logs, parses the last such line and performs the ?token= handshake itself with the browser's Host header. The resulting dsh-auth-* cookie is cached per (user, pod UID, Host) and injected into proxied requests. The browser never sees the launch token or the bearer token. A failed handshake returns 503 with Retry-After: 10.

API

Method	Path	Auth	Purpose
GET	/healthz	none	Liveness/readiness. Never touches the cluster.
GET	/	none	Landing page with signup + token login forms. With SSO it auto-provisions the proxy-identified user and redirects to their workspace.
POST	/users	none	Signup. JSON body {"user": "<dns-1123 label>"} or form field user. JSON returns 201 when Ready, 202 when accepted but still Waking; form returns 303 to /u/<user>/. Both set the session cookie. 409 if the user exists, 400 if invalid, 413 if the body is too large.
POST	/login	none	Form (user, token) returns 303 to /u/<user>/; JSON returns 200. Sets the session cookie. 401 on a bad token, 404 for an unknown user.
POST	/logout	none	Clears the session cookie and 303s to /.
POST	/recreate	trusted proxy header	SSO only. Deletes the authenticated user's claim (cascading sandbox + PVC, so workspace data is wiped), waits out finalizers, provisions a fresh claim and 303s to /u/<user>/. 401 without a valid identity header; cookie-authenticated calls must be same-origin.
GET	/users/{user}	bearer or cookie	Derived state: Provisioning, Waking, Ready, Suspending, Suspended.
DELETE	/users/{user}	bearer or cookie	Delete the claim (sandbox + PVC cascade). Cookie-auth mutations require a same-origin Origin/Referer.
GET	/u/{user}	—	308 redirect to /u/{user}/.
*	/u/{user}/{path}	bearer or cookie	HTTP proxy to DSH. Waits for a suspended sandbox to wake. Wakes on connect.
WS	/u/{user}/{path}	bearer or cookie	WebSocket proxy (e.g. /api/remote.mux).

Auth

• Authorization: Bearer <token> is checked with a timing-safe SHA-256 comparison against the aaas.example.com/token-sha256 claim annotation.
• The browser session cookie is dsh_gw_session=<user>:<token> (HttpOnly, SameSite=Lax, Path=/, optional Secure). A session for user A never authorizes user B.
• Cookie-authenticated mutations (POST, PUT, PATCH, DELETE) and cookie-authenticated WebSocket upgrades require the request Origin (or Referer fallback) to match the Host header. Missing or mismatched cross-origin metadata is rejected (403 HTTP, close code 4401 for WS).
• The gateway's bearer token, the browser Cookie header and any ?token= query parameter are never forwarded upstream. Only the cached dsh-auth-* cookie is injected.
• Upstream Set-Cookie headers are dropped. Location headers that are absolute paths are rewritten under /u/<user>/, and any token query parameter is stripped.

SSO (opt-in)

By default the gateway is the only authenticator: users sign up with a token or the session cookie, and the gateway ignores all identity headers. Set AUTH_PROXY_HEADER to enable SSO behind an authenticating reverse proxy such as oauth2-proxy:

kubectl -n dsh-demo set env deploy/dsh-gateway AUTH_PROXY_HEADER=X-Forwarded-User

When set, the gateway treats a request carrying that header as authenticated as that user, provided the value is a valid DNS-1123 label; the header is never trusted when the variable is unset. The landing page then auto-provisions the user (creates their claim idempotently, mints a rotated session token) and redirects to /u/<user>/, and POST /recreate becomes available for a clean restart. The base manifests leave it unset on purpose; the SSO exposure variant documents how to enable it on the Deployment (kubectl set env). That variant also ships 75-networkpolicies.yaml, which restricts the gateway to oauth2-proxy and oauth2-proxy's egress to the gateway, DNS and public HTTPS.

Trust boundary — this is the security-critical part. A trusted identity header is only safe when nothing else can reach the gateway. The gateway Service must stay ClusterIP and be reachable exclusively through the authenticating proxy; if a client can reach dsh-gateway:8080 directly (a second Ingress/Gateway/Route, a port-forward on a shared host, a broad NetworkPolicy), it can send X-Forwarded-User itself and impersonate any user. The variant's 75-networkpolicies.yaml restricts pod traffic to the proxy; a custom exposure without an equivalent policy has no network enforcement. Treat AUTH_PROXY_HEADER as "the proxy already authenticated this request", not as an authentication mechanism of its own. Disable SSO by removing the variable: the header is then ignored and bearer-token/session auth applies again.

Proxy failure semantics

Situation	Result
Unknown user / claim	404
Missing or invalid credentials	401
Cross-origin cookie mutation	403
Request body over MAX_BODY_MB	413
Client aborts mid-body, before the response starts	499 (CLIENT_CLOSED_REQUEST). Nothing is forwarded to DSH and nothing can be delivered; it is not an error, so no traceback is logged. Covers the body-reading control routes (POST /users, POST /login) as well as the proxy.
Client aborts mid-stream (SSE, long response)	The response already started, so its upstream status stands (typically 200); the stream simply ends. No traceback, and the in-flight slot is released as the relay unwinds.
Sandbox not provisioned, wake timeout, handshake failure	503 + Retry-After: 10
Upstream connection/protocol failure	502
WS auth failure (unknown user, bad token, cross-origin)	accept, then close 4401
WS upstream/handshake failure	accept, then close 1011

A close code can only be delivered after the WebSocket upgrade is accepted; Starlette would turn an early close into an HTTP 403, which is why WS auth failures are 101, then close 4401.

An aborted request is deliberately silent: uvicorn runs with --no-access-log, and the 499 cannot reach a peer that has already left (uvicorn's send() is a no-op once the socket is closed). Sandbox resolution, the wake and the DSH handshake run before the body is read, so an aborted upload can cost a wake or a handshake but never reaches DSH.

Configuration

All values are read from the environment by Settings.from_env(); every one can also be injected into create_app for tests.

Variable	Default	Meaning
NAMESPACE	dsh-demo	Namespace holding the claims, sandboxes and pods.
POOL	dsh-pool	warmPoolRef.name for new claims.
IDLE_TIMEOUT	60	Seconds of inactivity before the sweeper suspends a sandbox. Only claims the gateway created (annotation aaas.example.com/managed-by=dsh-gateway) are ever suspended.
WAKE_TIMEOUT	120	Seconds to wait for a sandbox to become Ready (signup and wake-on-connect).
DSH_PORT	3080	Upstream DSH port.
DSH_CONTAINER	dsh-web	Container whose logs carry the launch token.
DSH_LOG_TAIL_LINES	2000	How many log lines to scan for the launch token.
COOKIE_NAME	dsh_gw_session	Browser session cookie name.
COOKIE_SECURE	false	Add Secure to the session cookie when served over TLS.
COOKIE_MAX_AGE	604800	Session cookie lifetime in seconds.
MAX_BODY_MB	32	Maximum buffered request body (and 413 threshold).
HTTP_TIMEOUT	300	Upstream connect/read timeout in seconds. Proxied streams (SSE, long responses) keep the connect/write budget but have no read timeout, so an idle event stream is not cut every five minutes.
SWEEP_INTERVAL	15	Seconds between idle sweeps.
AUTH_PROXY_HEADER	unset	Opt-in SSO: name of the trusted identity header set by an authenticating reverse proxy (e.g. X-Forwarded-User). Unset disables SSO and the header is never trusted. Enable only when the Service is reachable exclusively through that proxy (see SSO).

The shipped manifests/50-gateway.yaml sets IDLE_TIMEOUT=360 and WAKE_TIMEOUT=240 for the demo; the e2e script overrides them to 60/120 so the smoke test does not wait minutes for the sweeper.

Required RBAC

The gateway's ServiceAccount needs, in its namespace:

• sandboxclaims (extensions.agents.x-k8s.io): get, list, create, delete
• sandboxes (agents.x-k8s.io): get, list, patch
• pods: get, list
• pods/log: get

Running locally

The module has no import-time cluster access and the Kubernetes adapter is lazy, so /healthz works without a kubeconfig:

python -m pip install -r requirements-dev.txt
python -m pytest -q
python -m ruff check .

# Uvicorn factory (loads in-cluster credentials only on the first claim call)
uvicorn gateway:create_app --factory --host 0.0.0.0 --port 8080

Container image, from the repository root:

make gateway    # builds dsh-aaas-gateway:dev from this directory

The equivalent direct build is docker build -t dsh-aaas-gateway:dev examples/dsh-agents-as-a-service/gateway. For kind, retag it to the image reference in examples/dsh-agents-as-a-service/manifests/50-gateway.yaml and kind load it. Published images are nested below the main image as <GHCR_IMAGE>/dsh-agents-as-a-service/gateway:<identity>, where the identity comes from versions.env — see docs/versioning.md.

The image runs as UID 10000, sets PYTHONDONTWRITEBYTECODE=1 and HOME=/tmp (read-only-rootfs compatible), listens on 8080, and disables uvicorn's access log so bearer tokens in URLs are not logged.

Known limitations

• Log scraping is fragile. The launch token is discovered from pod logs. Log rotation, a very chatty container, or DSH changing its startup message breaks the handshake. Prefer a first-class token/credential API when DSH offers one.
• Single replica, in-memory state. The handshake cookie cache, idle activity clocks and in-flight counters live in one process. Running more than one replica would need shared state (or sticky routing), otherwise each replica handshakes independently and the sweeper may race another replica.
• In-memory token cache has no TTL. It is keyed by pod UID, so pod restarts invalidate it naturally; otherwise entries survive until the pod or claim is deleted.
• No TLS termination. Run behind an ingress/load balancer; set COOKIE_SECURE=true there.
• Origin-based CSRF protection only. Cookie-auth mutations require a same-origin Origin/Referer; there is no per-form CSRF token, rate limiting or login throttling.
• HTTP request bodies are buffered up to MAX_BODY_MB; uploads are not streamed. Responses (including SSE) are streamed.
• Upstream cookies are not forwarded. DSH cannot rotate its own dsh-auth-* cookie through the gateway; on an upstream 401 the gateway re-runs the handshake once and replays the request.
• Only DSH on :3080 is proxied. There is no dashboard/API-port split and no platform API key injection.
• SSO trusts a header. With AUTH_PROXY_HEADER set, any client that can reach the gateway Service directly can spoof the identity header. The gateway cannot enforce the trust boundary itself: keep the Service ClusterIP and reachable only through the authenticating proxy. The SSO exposure variant ships a NetworkPolicy that restricts pod traffic to the proxy; without it there is no mTLS or signed identity assertion.