nicolas-georger/deepseek-kubernetes-sandbox
1
0
Fork 0
mirror of https://github.com/sredevopsorg/deepseek-kubernetes-sandbox.git synced 2026-11-03 01:13:24 +00:00
Code Projects Releases Activity

feat(repo)!: restructure around the dsh-harness image and one example per implementation

Browse Source 
- the root Dockerfile is the primary artifact; versions.env is the single source of truth
- merge gateway-api/dsh-demo into examples/dsh-agents-as-a-service (manifests/, exposure variants)
- move the single-tenant template into examples/minimal-single-tenant
- publish the gateway as a nested package under the repository image
- mirror the DSH ref for release identities (first release dsh-v0.2.1-rc.2-r2)
- CI: lint/python/manifests/gateway-image, reusable multi-arch build, tag-driven release, weekly e2e
- docs: README, docs/{main-image,examples,gitflow,versioning}, example READMEs, AGENTS.md, governance
- SSO is opt-in again; gateway pull policy and cookie defaults match the documented demo

BREAKING CHANGE: example manifest paths, the gateway image name, and the removal
of deploy/ and gateway-api/ affect existing deployments; see the migration table
in README.md.
This commit is contained in:
SREDevOpsAgents
2026-10-25 06:37:28 -03:00
parent 53e55fc851
commit e6e109d9a9
68 changed files with 3713 additions and 1643 deletions
Download Patch File Download Diff File Expand all files Collapse all files

27
examples/dsh-agents-as-a-service/manifests/00-prereqs.yaml Normal file
View File

@@ -0,0 +1,27 @@
# DSH "Agents as a Service" — prerequisites: namespace + shared provider key.
#
# The dsh-provider-keys Secret is deliberately NOT committed with a usable
# value. DEEPSEEK_API_KEY reaches every tenant sandbox (injected via envFrom
# on the SandboxTemplate, stamped before any user exists — a warm-pool
# constraint), so a real key committed here would be shared by every install
# of this example. Generate a fresh value per install (README step 1;
# scripts/e2e-kind.sh uses a throwaway):
#
# kubectl -n dsh-demo create secret generic dsh-provider-keys \
# --from-literal=DEEPSEEK_API_KEY="sk-..."
#
# All the infrastructure in this example works without a real key; only
# actual model calls need one.
apiVersion: v1
kind: Namespace
metadata:
name: dsh-demo
---
apiVersion: v1
kind: Secret
metadata:
name: dsh-provider-keys
namespace: dsh-demo
type: Opaque
# stringData:
# DEEPSEEK_API_KEY: "sk-your-key-here"

128
examples/dsh-agents-as-a-service/manifests/10-sandboxtemplate.yaml Normal file
View File

@@ -0,0 +1,128 @@
# The reusable DSH Web blueprint. Warm pools stamp pod env BEFORE any user
# exists, so per-user secrets cannot live in pod env; the shared provider key
# is injected via envFrom, and claims that would bypass the warm pool are
# rejected outright (see 40-claim-rejected.yaml).
#
# A stripped-down single-tenant variant of this template (no warm pool, no
# policy enforcement) lives in examples/minimal-single-tenant/.
apiVersion: extensions.agents.x-k8s.io/v1beta1
kind: SandboxTemplate
metadata:
name: dsh-web
namespace: dsh-demo
spec:
# Claims must not inject env or volumeClaimTemplates: with Disallowed, such
# claims are REJECTED instead of silently taking the cold (non-warm-pool)
# provisioning path. Demonstrated in step 6.
envVarsInjectionPolicy: Disallowed
volumeClaimTemplatesPolicy: Disallowed
# Per-sandbox headless Service -> stable DNS (sandbox.status.serviceFQDN).
# The gateway routes each user to their DSH Web instance by this name.
service: true
# The example gateway does not manage NetworkPolicy, so it is left
# unmanaged here. A production platform would use
# networkPolicyManagement: Managed and restrict ingress to its gateway on
# :3080 — with the shared provider key, that NetworkPolicy is the per-user
# isolation boundary.
networkPolicyManagement: Unmanaged
podTemplate:
spec:
# DSH Web is multi-arch (amd64 + arm64); no arch nodeSelector needed.
restartPolicy: "OnFailure"
securityContext:
runAsUser: 1000 # the 'node' user in the DSH image
runAsGroup: 1000
fsGroup: 1000
runAsNonRoot: true
# DSH's credentials plugin refuses to start when .credentials.yaml has
# any group/other permission bit (it creates the file 0600). The
# default kubelet fsGroup policy ("Always") recursively ORs 0660 into
# every file each time the pod starts, so a resumed sandbox comes back
# with 0660 and crash-loops before the gateway can proxy to it.
# OnRootMismatch skips that walk once the volume root already carries
# the fsGroup ownership/mode applied on the first mount.
# Keep in sync with the minimal variant:
# examples/minimal-single-tenant/manifests/sandbox-template.yaml.
fsGroupChangePolicy: OnRootMismatch
containers:
- name: dsh-web
image: ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox:dsh-v0.2.1-rc.2-r2
imagePullPolicy: IfNotPresent
ports:
- name: web
containerPort: 3080
envFrom:
- secretRef:
name: dsh-provider-keys
env:
# Empty => DSH derives links from the request Host, which the
# gateway forwards (plus X-Forwarded-*). Point this at the
# gateway base URL if you want hard-coded absolute links instead.
- name: DSH_PUBLIC_URL
value: ""
# DSH Host/Origin-fences /api. The gateway forwards the browser's
# Host, so DSH must trust the public gateway host. Empty is right
# for port-forward/localhost; for either exposure variant set this
# to the public host (e.g. dsh.example.com for
# manifests/exposure/ingress/60-ingress.yaml, or
# dsh-demo.example.com for the gateway-api-sso/ HTTPRoute).
- name: DSH_TRUSTED_HOSTS
value: ""
# Probes target /favicon.svg: DSH gates / behind its launch-token
# cookie and answers 401 to an unauthenticated GET, which would
# otherwise fail readiness and then get the pod killed by liveness.
# /favicon.svg is served without auth and is the cheapest liveness
# signal that proves the HTTP server is actually serving requests.
#
# Gate liveness/readiness behind a startup probe: a cold start on a
# small node takes tens of seconds to load plugins and start serving
# (a resumed sandbox starts cold too). Without it the kubelet can kill
# DSH mid-startup, and a SIGTERM'd DSH exits 0, which the sandbox
# controller records as finished. 24 x 5s = 120s, matching the
# gateway's WAKE_TIMEOUT.
startupProbe:
httpGet:
path: /favicon.svg
port: 3080
periodSeconds: 5
failureThreshold: 24
readinessProbe:
httpGet:
path: /favicon.svg
port: 3080
initialDelaySeconds: 5
periodSeconds: 5
failureThreshold: 30
livenessProbe:
httpGet:
path: /favicon.svg
port: 3080
initialDelaySeconds: 15
periodSeconds: 15
failureThreshold: 3
# The container only needs to serve HTTP; drop every capability and
# disallow privilege escalation. DSH writes only to DSH_HOME (PVC).
securityContext:
allowPrivilegeEscalation: false
capabilities:
drop: ["ALL"]
seccompProfile:
type: RuntimeDefault
resources:
requests: {cpu: 250m, memory: 512Mi}
limits: {cpu: "1", memory: 1Gi}
volumeMounts:
- name: dsh-home
mountPath: /home/node/.dsh # DSH_HOME: sessions, config, state
volumeClaimTemplates:
- metadata:
name: dsh-home
spec:
accessModes: ["ReadWriteOnce"]
resources:
requests:
storage: 5Gi
# storageClassName omitted => cluster default (kind & GKE both work)

12
examples/dsh-agents-as-a-service/manifests/20-sandboxwarmpool.yaml Normal file
View File

@@ -0,0 +1,12 @@
# Pre-warmed spares: image pulled, PVC provisioned, DSH Web booted, probe
# green — everything a signup would otherwise wait for is pre-paid, so
# claiming an instance takes ~2s instead of a full cold provision.
apiVersion: extensions.agents.x-k8s.io/v1beta1
kind: SandboxWarmPool
metadata:
name: dsh-pool
namespace: dsh-demo
spec:
replicas: 2
sandboxTemplateRef:
name: dsh-web

19
examples/dsh-agents-as-a-service/manifests/30-claim-alice.yaml Normal file
View File

@@ -0,0 +1,19 @@
# "User signup": one claim per user. This is the entire per-user resource —
# the platform pattern keeps it deliberately minimal:
#
# - warmPoolRef ONLY. No spec.env and no spec.volumeClaimTemplates: either
# would force a cold start past the warm pool (and this template's
# Disallowed policies reject them outright — see 40-claim-rejected.yaml).
# - additionalPodMetadata is the one warm-compatible per-claim
# customization. Label keys must use the sandbox.users.io domain.
apiVersion: extensions.agents.x-k8s.io/v1beta1
kind: SandboxClaim
metadata:
name: dsh-alice
namespace: dsh-demo
spec:
warmPoolRef:
name: dsh-pool
additionalPodMetadata:
labels:
sandbox.users.io/dsh-user: alice

18
examples/dsh-agents-as-a-service/manifests/40-claim-rejected.yaml Normal file
View File

@@ -0,0 +1,18 @@
# NEGATIVE demo: this claim carries spec.env, which the template's
# envVarsInjectionPolicy: Disallowed rejects. The claim never binds a
# sandbox — inspect it to see the rejection (step 6 of the README).
#
# Why a platform wants rejection over acceptance: env injection forces a
# cold start that bypasses the warm pool, silently turning a ~2s signup into
# a cold provision. Disallowed turns that mistake into a loud error.
apiVersion: extensions.agents.x-k8s.io/v1beta1
kind: SandboxClaim
metadata:
name: dsh-mallory
namespace: dsh-demo
spec:
warmPoolRef:
name: dsh-pool
env:
- name: SNEAKY_OVERRIDE
value: "should-not-work"

187
examples/dsh-agents-as-a-service/manifests/50-gateway.yaml Normal file
View File

@@ -0,0 +1,187 @@
# The gateway: control+data plane that turns the resources into a usable
# browser-facing service. It is a Starlette ASGI app (see gateway/) listening
# on :8080 and exposing:
# GET /healthz unauthenticated health probe
# GET / landing page (signup / token login)
# POST /users, POST /login, ... session cookie auth
# /u/<user>/... HTTP + WebSocket proxy to the user's DSH
#
# Image: the release image published for this example, pinned by tag.
# imagePullPolicy: IfNotPresent lets a local kind build loaded under the same
# tag be used without a registry round-trip (scripts/e2e-kind.sh builds
# dsh-aaas-gateway:e2e and overrides this image).
#
# SSO IS OPT-IN: this manifest runs the gateway with its own bearer/session
# auth only, so AUTH_PROXY_HEADER is intentionally NOT set (see the env
# block). To put GitHub OAuth in front of it, use
# manifests/exposure/gateway-api-sso/.
#
# This Service is ClusterIP and is the only intended entry point: expose it
# through exactly ONE of the documented variants in manifests/exposure/
# (ingress/ or gateway-api-sso/), never both and never directly.
apiVersion: v1
kind: ServiceAccount
metadata:
name: dsh-gateway
namespace: dsh-demo
---
# Least privilege:
# - claims are created/deleted per user (signup/delete);
# - sandboxes are read, and patched only to flip operatingMode (suspend/resume);
# - pods are read, and pods/log is read to capture DSH's launch token from
# the startup log line (`dsh web: ...?token=...`).
# configmaps are not needed: all state is in memory / claim annotations.
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: dsh-gateway
namespace: dsh-demo
rules:
- apiGroups: ["extensions.agents.x-k8s.io"]
resources: ["sandboxclaims"]
verbs: ["get", "list", "create", "delete"]
- apiGroups: ["agents.x-k8s.io"]
resources: ["sandboxes"]
verbs: ["get", "list", "patch"]
- apiGroups: [""]
resources: ["pods"]
verbs: ["get", "list"]
- apiGroups: [""]
resources: ["pods/log"]
verbs: ["get"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: dsh-gateway
namespace: dsh-demo
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: dsh-gateway
subjects:
- kind: ServiceAccount
name: dsh-gateway
namespace: dsh-demo
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: dsh-gateway
namespace: dsh-demo
spec:
# Single replica BY DESIGN: the idle clock, in-flight counters and the
# per-(user, pod, Host) DSH cookie cache are in-process memory. Running two
# replicas would fragment that state (double suspend/resume races, repeated
# log-token handshakes). Scale only after moving state to a shared store.
replicas: 1
selector:
matchLabels:
app: dsh-gateway
template:
metadata:
labels:
app: dsh-gateway
spec:
serviceAccountName: dsh-gateway
securityContext:
runAsNonRoot: true
runAsUser: 10000 # matches USER in gateway/Dockerfile
runAsGroup: 10000
seccompProfile:
type: RuntimeDefault
containers:
- name: gateway
image: ghcr.io/sredevopsorg/deepseek-kubernetes-sandbox/dsh-agents-as-a-service/gateway:dsh-v0.2.1-rc.2-r2
imagePullPolicy: IfNotPresent
securityContext:
allowPrivilegeEscalation: false
readOnlyRootFilesystem: true
capabilities:
drop: ["ALL"]
ports:
- name: http
containerPort: 8080
env:
- name: NAMESPACE
value: dsh-demo
- name: POOL
value: dsh-pool
# Seconds a sandbox may sit idle before the sweeper suspends it
# (pod deleted, PVC kept), and the max time a proxy request waits
# for a suspended sandbox to become Ready again. Only claims this
# gateway created are swept: a claim applied by hand from the
# resource-layer manifests carries no aaas.example.com/managed-by
# annotation and is never suspended.
#
# These are the demo values (360/240). scripts/e2e-kind.sh
# overrides both to 60/120 with `kubectl set env` so the smoke
# test does not wait six minutes for the sweeper.
- name: IDLE_TIMEOUT
value: "360"
- name: WAKE_TIMEOUT
value: "240"
# Container name inside the per-user sandbox pods whose logs carry
# the DSH launch token.
- name: DSH_CONTAINER
value: dsh-web
# Cookie dsh_gw_session is HttpOnly + SameSite=Lax. "false" keeps
# the plain-HTTP demo usable through port-forward: a Secure cookie
# is never sent over HTTP, so login would silently fail.
# HTTPS deployments (the ingress/ variant with TLS, or
# gateway-api-sso/, which terminates TLS at the Gateway) MUST set
# this to "true".
- name: COOKIE_SECURE
value: "false"
# SSO IS OPT-IN AND DISABLED HERE. With AUTH_PROXY_HEADER unset the
# gateway trusts only its own bearer/session auth and ignores every
# identity header. Enable it ONLY behind
# manifests/exposure/gateway-api-sso/, where oauth2-proxy is the
# sole route to this Service: set it to X-Forwarded-User, keep the
# Service ClusterIP and reachable only through the proxy, and
# terminate TLS in front of the gateway. Otherwise anyone who can
# reach :8080 can impersonate any user by sending that header.
# - name: AUTH_PROXY_HEADER
# value: X-Forwarded-User
# Reject proxied request bodies larger than this (413).
- name: MAX_BODY_MB
value: "32"
# /healthz is unauthenticated and returns 200 without touching the
# cluster, so it is safe for both readiness and liveness.
readinessProbe:
httpGet:
path: /healthz
port: 8080
initialDelaySeconds: 5
periodSeconds: 5
failureThreshold: 3
livenessProbe:
httpGet:
path: /healthz
port: 8080
initialDelaySeconds: 10
periodSeconds: 10
failureThreshold: 3
resources:
requests: {cpu: 50m, memory: 128Mi}
limits: {cpu: 500m, memory: 256Mi}
volumeMounts:
# readOnlyRootFilesystem leaves no writable HOME/tmp; the image sets
# HOME=/tmp, so back it with scratch space. No state lives here.
- name: tmp
mountPath: /tmp
volumes:
- name: tmp
emptyDir: {}
---
apiVersion: v1
kind: Service
metadata:
name: dsh-gateway
namespace: dsh-demo
spec:
selector:
app: dsh-gateway
ports:
- port: 8080
targetPort: 8080

135
examples/dsh-agents-as-a-service/manifests/exposure/README.md Normal file
View File

@@ -0,0 +1,135 @@
# Exposing the gateway
The base manifests deploy the gateway as a ClusterIP Service
(`dsh-gateway:8080`) and expose nothing outside the cluster. WebSocket
upgrades on `/u/<user>/**` are part of the contract, so the entry point must
forward `Upgrade`/`Connection` headers (Traefik does by default).
There are two **mutually exclusive** exposure variants. Pick exactly one:
| Variant | Entry point | Authentication | TLS |
| --- | --- | --- | --- |
| [`ingress/60-ingress.yaml`](ingress/60-ingress.yaml) | Traefik `Ingress` | the gateway's own signup/session cookie | optional, recommended |
| [`gateway-api-sso/`](gateway-api-sso/72-oauth2-proxy.yaml) | Gateway API `Gateway` + `HTTPRoute` | GitHub OAuth via oauth2-proxy, then the gateway cookie | required (the Gateway listener terminates it) |
Applying both gives the gateway two entry points, and the SSO variant is only
safe when the gateway is reachable **exclusively** through oauth2-proxy. Do not
mix them.
## Applying a variant
Run these commands from the example root
(`examples/dsh-agents-as-a-service/`), the same working directory the example
README uses.
`kubectl apply -f manifests/` is **not recursive**: it applies only the
top-level numbered files and does not descend into `exposure/`. Apply the base
resources first, then the chosen variant explicitly:
```bash
# Base resources: namespace, template, warm pool, claims, gateway.
kubectl apply -f manifests/
# Plain Ingress exposure:
kubectl apply -f manifests/exposure/ingress/
# ... or SSO exposure (create the two Secrets and replace the example.com
# placeholders first — see below):
kubectl apply -f manifests/exposure/gateway-api-sso/
```
**Never use `kubectl apply -R`** (recursive): it would apply both variants at
once and hand out a second, unauthenticated path to the same gateway. Apply the
variant directory on purpose, or the individual files by name.
`kubectl` only reads `.yaml`, `.yml` and `.json` files from a directory, so
the `*.yaml.example` secret templates are skipped automatically; create the real
Secrets imperatively (commands below) before applying the directory.
## `ingress/`: plain Ingress, no SSO
`60-ingress.yaml` routes a host through the cluster's Traefik
(`ingressClassName: traefik`; k3s ships one) to the gateway Service.
1. Replace the `dsh.example.com` host and the `dsh-example-tls` Secret name
with real values, or delete the `tls:` block to serve plain HTTP.
2. Set `DSH_TRUSTED_HOSTS` in the SandboxTemplate to the same public host:
the gateway forwards the browser's `Host`, and DSH rejects unknown hosts on
`/api` (see `../10-sandboxtemplate.yaml`).
3. Set `COOKIE_SECURE=true` on the gateway once the Ingress serves HTTPS:
`kubectl -n dsh-demo set env deploy/dsh-gateway COOKIE_SECURE=true`. The
base manifest keeps `false` so the plain-HTTP port-forward demo works; a
`Secure` cookie is never sent over HTTP, which silently breaks login.
4. Do not attach buffering or compression middlewares to this router: they
break WebSocket upgrades and streaming.
Verify: `curl -sI https://<host>/healthz` returns `200`.
## `gateway-api-sso/`: TLS + GitHub OAuth
`70-gateway.yaml` defines the TLS `Gateway`, `71-httproute.yaml` sends every
request to oauth2-proxy, and `72-oauth2-proxy.yaml` authenticates the browser
against GitHub and proxies to the gateway with `X-Forwarded-User` set.
Prerequisites:
- Gateway API CRDs and a Gateway implementation installed;
`gatewayClassName: traefik` matches the Traefik Gateway provider. The CRD
version is pinned in `versions.env` (`GATEWAY_API_VERSION`).
- The OAuth App credentials and a cookie secret, from
`73-oauth2-proxy-secret.yaml.example`.
- A TLS certificate/key for the public host, from
`74-tls-secret.yaml.example`.
- oauth2-proxy is pinned in the manifest to
`quay.io/oauth2-proxy/oauth2-proxy:v7.15.5` (`versions.env`:
`OAUTH2_PROXY_IMAGE`).
Steps:
```bash
# 1. Real Secrets (the committed files are *.yaml.example templates).
kubectl -n dsh-demo create secret generic oauth2-proxy \
--from-literal=client-id='<GitHub OAuth App client ID>' \
--from-literal=client-secret='<GitHub OAuth App client secret>' \
--from-literal=cookie-secret="$(python3 -c 'import secrets; print(secrets.token_urlsafe(32))')"
kubectl -n dsh-demo create secret tls dsh-demo-tls --cert=tls.crt --key=tls.key
# 2. Replace dsh-demo.example.com (70/71/72) and your-github-org (72) with
# the real public host and allowed GitHub organization.
# 3. Enable SSO on the gateway and trust the public host in the template.
kubectl -n dsh-demo set env deploy/dsh-gateway \
AUTH_PROXY_HEADER=X-Forwarded-User COOKIE_SECURE=true
# then set DSH_TRUSTED_HOSTS=dsh-demo.example.com in the SandboxTemplate
# (see ../10-sandboxtemplate.yaml) and re-apply it.
# 4. Apply the variant.
kubectl apply -f manifests/exposure/gateway-api-sso/
```
### SSO trust boundary
`AUTH_PROXY_HEADER` is opt-in and deliberately unset in the base gateway
manifest: the gateway ignores identity headers until it is set. Enabling SSO
takes **both** sides:
1. `AUTH_PROXY_HEADER=X-Forwarded-User` on `deploy/dsh-gateway` (step 3
above). Without it, oauth2-proxy's login has no effect on the gateway and
users are still asked for the gateway's own token.
2. The gateway Service must be reachable **only through oauth2-proxy**: keep it
ClusterIP, route all browser traffic through the `HTTPRoute` →
oauth2-proxy → `dsh-gateway` chain, and never apply `ingress/` (or any
other path to the Service) while SSO is enabled. A client that can reach
`dsh-gateway:8080` directly can set `X-Forwarded-User` itself and
impersonate any user.
Also terminate TLS in front of the gateway (the Gateway listener does) and
leave `COOKIE_SECURE=true` so the session cookie never travels in clear text.
## Switching variants
Delete the variant you had before applying the other one, e.g.
`kubectl delete -f manifests/exposure/ingress/`, and revert
`AUTH_PROXY_HEADER`/`COOKIE_SECURE` on the gateway. Verify with
`kubectl -n dsh-demo get ingress,gateway,httproute` that exactly one path to
the gateway remains.

25
examples/dsh-agents-as-a-service/manifests/exposure/gateway-api-sso/70-gateway.yaml Normal file
View File

@@ -0,0 +1,25 @@
# Browser entry point (TLS) for the gateway-api-sso variant: terminates TLS
# for dsh-demo.example.com and routes through 71-httproute.yaml to
# oauth2-proxy, which authenticates the browser before the gateway.
# Mutually exclusive with ../../ingress/ (see ../../README.md). The
# "dsh-demo-tls" Secret comes from 74-tls-secret.yaml.example.
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
name: dsh-gateway
namespace: dsh-demo
labels:
app.kubernetes.io/part-of: dsh-demo
spec:
gatewayClassName: traefik
listeners:
- name: https
port: 8443
protocol: HTTPS
tls:
mode: Terminate
certificateRefs:
- name: dsh-demo-tls
allowedRoutes:
namespaces:
from: Same

23
examples/dsh-agents-as-a-service/manifests/exposure/gateway-api-sso/71-httproute.yaml Normal file
View File

@@ -0,0 +1,23 @@
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: dsh-gateway
namespace: dsh-demo
labels:
app.kubernetes.io/part-of: dsh-demo
spec:
parentRefs:
- name: dsh-gateway
namespace: dsh-demo
sectionName: https
hostnames:
# TODO: replace with the actual public FQDN before applying.
- dsh-demo.example.com
rules:
- backendRefs:
# Route through oauth2-proxy (GitHub OAuth) first; it authenticates the
# browser and forwards to dsh-gateway:8080 with X-Forwarded-User set.
# The gateway only trusts that header when AUTH_PROXY_HEADER is set on
# its Deployment — see 72-oauth2-proxy.yaml and ../README.md.
- name: oauth2-proxy
port: 4180

134
examples/dsh-agents-as-a-service/manifests/exposure/gateway-api-sso/72-oauth2-proxy.yaml Normal file
View File

@@ -0,0 +1,134 @@
# oauth2-proxy — GitHub OAuth login in front of the dsh-gateway.
#
# Runs as a reverse proxy: it authenticates the browser against GitHub
# (restricted to a configured organization), then forwards the request to
# the dsh-gateway Service with the authenticated identity in X-Forwarded-User
# (plus X-Auth-Request-* via --set-xauthrequest). The gateway trusts that
# header only when AUTH_PROXY_HEADER is set on its Deployment.
#
# ENABLING SSO TAKES BOTH SIDES (see ../../../exposure/README.md):
# 1. Set AUTH_PROXY_HEADER=X-Forwarded-User on deploy/dsh-gateway, otherwise
# the gateway ignores the header and the login is cosmetic:
# kubectl -n dsh-demo set env deploy/dsh-gateway \
# AUTH_PROXY_HEADER=X-Forwarded-User COOKIE_SECURE=true
# 2. Make dsh-gateway reachable ONLY through this proxy: keep the Service
# ClusterIP, route browser traffic through 71-httproute.yaml, and do not
# apply ../ingress/ (or anything else that reaches the gateway). A
# client that can reach dsh-gateway:8080 can spoof X-Forwarded-User and
# impersonate any user.
# TLS terminates at the Gateway listener (70-gateway.yaml); keep
# --cookie-secure=true and COOKIE_SECURE=true on the gateway.
#
# The image is pinned to quay.io/oauth2-proxy/oauth2-proxy:v7.15.5
# (versions.env: OAUTH2_PROXY_IMAGE). Credentials come from the
# "oauth2-proxy" Secret (see 73-oauth2-proxy-secret.yaml.example); this
# manifest never carries secrets.
apiVersion: apps/v1
kind: Deployment
metadata:
name: oauth2-proxy
namespace: dsh-demo
labels:
app: oauth2-proxy
app.kubernetes.io/part-of: dsh-demo
spec:
replicas: 1
selector:
matchLabels:
app: oauth2-proxy
template:
metadata:
labels:
app: oauth2-proxy
app.kubernetes.io/part-of: dsh-demo
spec:
securityContext:
runAsNonRoot: true
runAsUser: 2000
runAsGroup: 2000
seccompProfile:
type: RuntimeDefault
containers:
- name: oauth2-proxy
image: quay.io/oauth2-proxy/oauth2-proxy:v7.15.5
imagePullPolicy: IfNotPresent
securityContext:
allowPrivilegeEscalation: false
readOnlyRootFilesystem: true
capabilities:
drop: ["ALL"]
args:
- --http-address=0.0.0.0:4180
- --provider=github
# TODO: replace with the GitHub organization allowed to sign in.
- --github-org=your-github-org
- --email-domain=*
# Reverse-proxy target: the gateway backend Service.
- --upstream=http://dsh-gateway:8080
# Public callback (TLS terminates at the Gateway listener).
# TODO: replace with the actual public FQDN.
- --redirect-url=https://dsh-demo.example.com/oauth2/callback
- --cookie-secure=true
- --cookie-name=_oauth2_proxy
# Also expose X-Auth-Request-User/Email/Groups upstream.
- --set-xauthrequest
# Single provider: skip the provider-selection page.
- --skip-provider-button
env:
- name: OAUTH2_PROXY_CLIENT_ID
valueFrom:
secretKeyRef:
name: oauth2-proxy
key: client-id
- name: OAUTH2_PROXY_CLIENT_SECRET
valueFrom:
secretKeyRef:
name: oauth2-proxy
key: client-secret
- name: OAUTH2_PROXY_COOKIE_SECRET
valueFrom:
secretKeyRef:
name: oauth2-proxy
key: cookie-secret
ports:
- name: http
containerPort: 4180
livenessProbe:
httpGet:
path: /ping
port: http
initialDelaySeconds: 5
periodSeconds: 10
timeoutSeconds: 1
failureThreshold: 3
readinessProbe:
httpGet:
path: /ready
port: http
initialDelaySeconds: 5
periodSeconds: 10
timeoutSeconds: 5
failureThreshold: 3
resources:
requests:
cpu: 10m
memory: 32Mi
limits:
cpu: 200m
memory: 128Mi
---
apiVersion: v1
kind: Service
metadata:
name: oauth2-proxy
namespace: dsh-demo
labels:
app: oauth2-proxy
app.kubernetes.io/part-of: dsh-demo
spec:
selector:
app: oauth2-proxy
ports:
- name: http
port: 4180
targetPort: http

39
examples/dsh-agents-as-a-service/manifests/exposure/gateway-api-sso/73-oauth2-proxy-secret.yaml.example Normal file
View File

@@ -0,0 +1,39 @@
# EXAMPLE ONLY — do not apply as-is.
#
# oauth2-proxy reads its GitHub OAuth credentials and cookie-signing secret from
# this Secret. Populate real values and apply:
#
# kubectl -n dsh-demo create secret generic oauth2-proxy \
# --from-literal=client-id='<GitHub OAuth App client ID>' \
# --from-literal=client-secret='<GitHub OAuth App client secret>' \
# --from-literal=cookie-secret="$(python3 -c 'import secrets; print(secrets.token_urlsafe(32))')"
#
# The cookie secret must decode (base64) to exactly 16, 24, or 32 bytes.
# oauth2-proxy base64-decodes it, so it must be URL-safe, unpadded base64.
# Generate it with either of:
#
# python3 -c 'import secrets; print(secrets.token_urlsafe(32))'
# openssl rand -base64 32 | tr -- '+/' '-_' | tr -d '='
#
# Do NOT use plain `openssl rand -base64 32`: that yields 44 characters of
# standard base64 (with "=" and possibly "+"/"/"), which oauth2-proxy cannot
# decode and therefore treats as 44 raw bytes, failing with "cookie_secret must
# be 16, 24, or 32 bytes ... but is 44 bytes".
#
# GitHub OAuth App (https://github.com/settings/developers):
# Homepage URL https://dsh-demo.example.com
# Authorization callback https://dsh-demo.example.com/oauth2/callback
# (replace dsh-demo.example.com with the actual public FQDN)
#
# NOTE: pass the base64 value as-is via --from-literal (kubectl stores the
# literal; it does not base64-encode it a second time).
apiVersion: v1
kind: Secret
metadata:
name: oauth2-proxy
namespace: dsh-demo
type: Opaque
stringData:
client-id: "<github-oauth-app-client-id>"
client-secret: "<github-oauth-app-client-secret>"
cookie-secret: "<urlsafe-unpadded-base64-of-32-random-bytes>"

17
examples/dsh-agents-as-a-service/manifests/exposure/gateway-api-sso/74-tls-secret.yaml.example Normal file
View File

@@ -0,0 +1,17 @@
# EXAMPLE ONLY — do not apply as-is.
#
# Copy to tls-secret.yaml and populate with real base64-encoded PEM material,
# then apply. The secret must live in the dsh-demo namespace and be named
# "dsh-demo-tls" (referenced by 70-gateway.yaml).
#
# kubectl -n dsh-demo create secret tls dsh-demo-tls \
# --cert=tls.crt --key=tls.key --dry-run=client -o yaml > tls-secret.yaml
apiVersion: v1
kind: Secret
metadata:
name: dsh-demo-tls
namespace: dsh-demo
type: kubernetes.io/tls
data:
tls.crt: <base64-encoded-certificate>
tls.key: <base64-encoded-private-key>

52
examples/dsh-agents-as-a-service/manifests/exposure/ingress/60-ingress.yaml Normal file
View File

@@ -0,0 +1,52 @@
# OPTIONAL browser entry point: expose the gateway through Traefik.
#
# This is the "no SSO" exposure variant and is MUTUALLY EXCLUSIVE with
# ../gateway-api-sso/ — apply exactly one of them (see ../README.md).
#
# Requires the base manifests (Service dsh-gateway:8080 from
# manifests/50-gateway.yaml) and a Traefik install with an IngressClass named
# "traefik" (k3s ships one). Replace the placeholder host below with a name
# that resolves to your ingress address, and either supply a real TLS Secret or
# delete the `tls:` block for plain HTTP.
#
# Two things must match this host:
# 1. SandboxTemplate (manifests/10-sandboxtemplate.yaml): set
# DSH_TRUSTED_HOSTS to this host so DSH accepts the forwarded Host/Origin
# on /api.
# 2. Gateway Deployment (manifests/50-gateway.yaml): set COOKIE_SECURE to
# "true" once this Ingress serves HTTPS, so the session cookie is Secure.
# The base manifest sets "false" for the plain-HTTP port-forward demo.
#
# WebSocket support is REQUIRED: the browser talks to /u/<user>/... and DSH
# uses WebSockets for live session traffic. Traefik upgrades WS/SSE
# transparently for ordinary HTTP routers, but do NOT attach buffering
# middlewares (e.g. compress, buffering) to this router — they break streaming
# and upgrades. `path: /` Prefix covers the landing page and every per-user
# proxy path, so no per-sandbox Ingress is needed.
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: dsh-gateway
namespace: dsh-demo
# Typical Traefik + cert-manager wiring (uncomment and adapt):
# annotations:
# traefik.ingress.kubernetes.io/router.entrypoints: websecure
# cert-manager.io/cluster-issuer: letsencrypt-prod
annotations: {}
spec:
ingressClassName: traefik
tls:
- hosts:
- dsh.example.com # placeholder
secretName: dsh-example-tls # placeholder; create/PKI-provision this Secret
rules:
- host: dsh.example.com # placeholder
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: dsh-gateway
port:
number: 8080