1
0
mirror of https://github.com/sredevopsorg/deepseek-kubernetes-sandbox.git synced 2026-11-03 01:13:24 +00:00

4 Commits

Author SHA1 Message Date
Nicolás Georger
f4c0b52ee9 Merge branch 'chore/changelog-dsh-v0.2.1-rc.5-r6' into fix/security-gateway-sessions
Signed-off-by: Nicolás Georger <info@sredevops.org>
2026-10-31 07:38:56 -03:00
github-actions[bot]
96837a43b6 chore(changelog): refresh for dsh-v0.2.1-rc.5-r6 2026-10-31 09:31:08 +00:00
SREDevOpsAgents
2eb5c82ace Merge remote-tracking branch 'origin/main' into fix/security-gateway-sessions
# Conflicts:
#	docs/versioning.md
#	versions.env
2026-10-31 00:55:39 -03:00
SREDevOpsAgents
ae8aba8241 fix(gateway)!: replace token cookies with revocable sessions and harden abuse controls
- Replace user:token cookies with opaque server-side sessions, revocable on logout, delete, recreate and expiry; bind every credential to the path user and add cross-user regression tests.
- Rate-limit /users and /login, add optional signup invite codes and form same-origin checks, and unify login failures to 401.
- Require explicit ALLOW_INSECURE_COOKIES for plain HTTP, use __Host-dsh_gw_session when secure, and fix secure cookie deletion.
- Strip client-supplied forwarding/SSO headers, add an optional upstream allowlist, per-user in-flight and WebSocket message caps, credential-free handshake errors, and sweep-time state pruning.
- Digest-pin base images, hash-pin gateway dependencies, verify DSH_COMMIT in the build, and run the gateway image as non-root with a pinned Python base.
- Update manifests, production checklist, architecture/security docs and add an SSO preflight script.

BREAKING CHANGE: existing dsh_gw_session=<user>:<token> cookies are invalid; users must sign in again. Unknown users on POST /login now return 401 instead of 404, and the gateway refuses COOKIE_SECURE=false unless ALLOW_INSECURE_COOKIES=1.
2026-10-31 00:54:05 -03:00
9 changed files with 16 additions and 97 deletions

5
.entire/.gitignore vendored
View File

@@ -1,5 +0,0 @@
tmp/
settings.local.json
metadata/
logs/
redactors/local/

View File

@@ -1,16 +0,0 @@
{
"version": 1,
"agent": "dsh",
"bridge": "@sredevopsorg/dsh-entire-bridge",
"hook_names": [
"session-start",
"turn-start",
"turn-end",
"compaction",
"session-end",
"subagent-start",
"subagent-end"
],
"installed_at": "2026-11-02T06:02:27.816Z",
"local_dev": false
}

View File

@@ -1,9 +0,0 @@
{
"enabled": true,
"telemetry": true,
"checkpoints": {
"primary": {
"type": "git-refs"
}
}
}

View File

@@ -339,7 +339,7 @@ The gateway runs with a namespace-scoped ServiceAccount (`manifests/50-gateway.y
| API group | Resource | Verbs | Why |
| --- | --- | --- | --- |
| `extensions.agents.x-k8s.io` | `sandboxclaims` | `get`, `list`, `create`, `patch`, `delete` | signup, state, account deletion, token rotation |
| `extensions.agents.x-k8s.io` | `sandboxclaims` | `get`, `list`, `create`, `delete` | signup, state, account deletion |
| `agents.x-k8s.io` | `sandboxes` | `get`, `list`, `patch` | resolve DNS/pod, wake/suspend |
| `""` (core) | `pods` | `get`, `list` | resolve the pod behind a sandbox |
| `""` (core) | `pods/log` | `get` | launch-token handshake |
@@ -404,7 +404,6 @@ Notes and limits:
| DSH log line changed or rotated away | Proxy returns `503` (missing launch token) | Fix the parser or restart the pod so the token line is re-emitted |
| Probe on `/` | Pod `CrashLoopBackOff`, `401` in probe logs | Use `/favicon.svg` for readiness/liveness (shipped templates do) |
| Warm pool exhausted | JSON signup returns `202` (state `Waking`); form signup is slow | Increase `replicas` in the warm pool |
| Kubernetes API error (API down or throttled, missing RBAC verb) | HTTP `503` + `Retry-After`; WebSocket close `1011` | One credential-free log line with the API `status`/`reason`, plus a fixed body; the Kubernetes `Status` payload is never proxied. Check the ServiceAccount Role and the API server — RBAC is evaluated per request, so a Role fix needs no restart |
| Gateway restart | Idle sandboxes stop suspending; handshake cache and browser sessions lost | Idle clocks and handshake cache rebuild lazily; sessions do not, so browsers must sign in again after a rollout. Use a shared session store for production |
| Gateway configured `COOKIE_SECURE=false` without `ALLOW_INSECURE_COOKIES=1` | Gateway pod exits at startup / `CrashLoopBackOff` | Intentional guard against an insecure cookie on an HTTPS deployment; set `ALLOW_INSECURE_COOKIES=1` only for the local plain-HTTP demo, or set `COOKIE_SECURE=true` |
| Auth rate limit hit | `429` + `Retry-After` on signup/login | Per-client in-memory bucket (`AUTH_RATE_LIMIT`/`AUTH_RATE_WINDOW`); wait for the window or tune it. Behind a proxy, keying uses the rightmost `X-Forwarded-For` entry |

View File

@@ -175,11 +175,9 @@ oauth2-proxy's egress to the gateway, DNS and public HTTPS.
| Sandbox not provisioned, wake timeout, handshake failure | `503` + `Retry-After: 10`. The handshake error body is a fixed message: it never contains the launch token, the handshake URL or pod/credential details. |
| User over `MAX_IN_FLIGHT_PER_USER` | `503` + `Retry-After` (HTTP); the existing in-flight requests are untouched |
| Upstream connection/protocol failure | `502` |
| Kubernetes API error (API unreachable/throttled, RBAC `403`, `5xx`) | `503` + `Retry-After: 10`. The body is a fixed message; the Kubernetes `Status` payload (which can name cluster objects) is never proxied, and the API `status`/`reason` is logged once |
| WS auth failure (unknown user, bad token, cross-origin) | accept, then close `4401` |
| WS per-user concurrency cap | accept, then close `1011` |
| WS upstream/handshake failure | accept, then close `1011` |
| WS Kubernetes API error | accept, then close `1011` |
| WS message from DSH over `WS_MAX_MESSAGE_BYTES` | the upstream/relay closes the connection (websockets' message-too-big path) |
A close code can only be delivered after the WebSocket upgrade is accepted;
@@ -232,8 +230,7 @@ would never be sent). HTTPS deployments must set `COOKIE_SECURE=true`.
The gateway's ServiceAccount needs, in its namespace:
* `sandboxclaims` (`extensions.agents.x-k8s.io`): get, list, create, patch, delete
(patch rotates the token-hash annotation when a session is minted)
* `sandboxclaims` (`extensions.agents.x-k8s.io`): get, list, create, delete
* `sandboxes` (`agents.x-k8s.io`): get, list, patch
* `pods`: get, list
* `pods/log`: get

View File

@@ -1510,13 +1510,7 @@ against the JSON API. The browser session is an HttpOnly cookie.</p>
user = websocket.path_params["user"]
if not valid_user(user):
return await self._ws_reject(websocket, 4401)
try:
claim = await self.k8s.get_claim(user)
except ApiException:
# Starlette has no exception handler for WebSocket scopes, so an
# unhandled API error would escape as a traceback and an unclean
# close. 1011 keeps the HTTP 503 meaning on the WS channel.
return await self._ws_reject(websocket, 1011)
if claim is None:
return await self._ws_reject(websocket, 4401)
token, cookie_user = self.credentials(websocket.headers, websocket.cookies)
@@ -1752,28 +1746,6 @@ async def client_gone(request: Request, exc: ClientDisconnect) -> Response:
return Response(status_code=CLIENT_CLOSED_REQUEST)
async def k8s_api_error(request: Request, exc: ApiException) -> Response:
"""Answer a Kubernetes API failure with 503 instead of a bare 500.
The gateway is the only path between the browser and the cluster, so an
API outage, a throttle or a missing RBAC verb is a server-side
availability problem, not a client error. A 503 keeps ingress and
browser retry semantics honest, and the body is a fixed message: the
Kubernetes ``Status`` payload can name cluster objects and is never
proxied. The API ``status``/``reason`` is logged once, credential-free.
"""
print(
"gateway: kubernetes API error: "
f"status={getattr(exc, 'status', None)} reason={getattr(exc, 'reason', None)}",
flush=True,
)
return JSONResponse(
{"error": "cluster API unavailable, retry shortly"},
status_code=503,
headers={"Retry-After": "10"},
)
def create_app(
settings: Settings | None = None,
k8s=None,
@@ -1806,7 +1778,7 @@ def create_app(
]
app = Starlette(
routes=routes,
exception_handlers={ClientDisconnect: client_gone, ApiException: k8s_api_error},
exception_handlers={ClientDisconnect: client_gone},
lifespan=_lifespan(gateway, http_client if owns_http_client else None),
)
app.state.gateway = gateway

View File

@@ -1176,14 +1176,12 @@ async def test_delete_missing_claim_is_404(env):
@pytest.mark.asyncio
async def test_unexpected_api_error_returns_503(env):
async def test_unexpected_api_error_propagates(env):
add_user(env.k8s)
env.k8s.get_claim_error = ApiException(status=500)
async with asgi_client(env) as client:
response = await client.get("/users/alice")
assert response.status_code == 503
assert response.headers["retry-after"] == "10"
assert response.json() == {"error": "cluster API unavailable, retry shortly"}
with pytest.raises(ApiException):
await client.get("/users/alice")
@pytest.mark.asyncio
@@ -1833,16 +1831,6 @@ def test_ws_unauthenticated_closes_4401(env):
assert exc.value.code == 4401
def test_ws_kubernetes_api_error_closes_1011(env):
add_user(env.k8s)
env.k8s.get_claim_error = ApiException(status=503)
with TestClient(env.app) as client:
with client.websocket_connect("/u/alice/api/remote.mux") as ws:
with pytest.raises(WebSocketDisconnect) as exc:
ws.receive_text()
assert exc.value.code == 1011
def test_ws_cookie_cross_origin_closes_4401(env):
add_user(env.k8s)
with TestClient(env.app) as client:
@@ -2161,19 +2149,6 @@ def test_remote_user_returns_valid_and_rejects_invalid(env):
assert g.remote_user({}) is None
@pytest.mark.asyncio
async def test_sso_landing_returns_503_when_claim_patch_is_forbidden(env):
"""A missing RBAC verb on ``patch sandboxclaims`` used to escape as a 500."""
enable_sso(env)
env.k8s.claims["alice"] = {"metadata": {"name": claim_name("alice")}}
env.k8s.patch_error = ApiException(status=403)
async with asgi_client(env) as client:
response = await client.get("/", headers={"X-Forwarded-User": "alice"})
assert response.status_code == 503
assert response.headers["retry-after"] == "10"
assert response.json() == {"error": "cluster API unavailable, retry shortly"}
@pytest.mark.asyncio
async def test_sso_landing_new_user_shows_provisioning(env):
enable_sso(env)

View File

@@ -26,8 +26,7 @@ metadata:
namespace: dsh-demo
---
# Least privilege:
# - claims are created/deleted per user (signup/delete) and patched only to
# rotate the token-hash annotation when a session is minted;
# - claims are created/deleted per user (signup/delete);
# - sandboxes are read, and patched only to flip operatingMode (suspend/resume);
# - pods are read, and pods/log is read to capture DSH's launch token from
# the startup log line (`dsh web: ...?token=...`).
@@ -40,7 +39,7 @@ metadata:
rules:
- apiGroups: ["extensions.agents.x-k8s.io"]
resources: ["sandboxclaims"]
verbs: ["get", "list", "create", "patch", "delete"]
verbs: ["get", "list", "create", "delete"]
- apiGroups: ["agents.x-k8s.io"]
resources: ["sandboxes"]
verbs: ["get", "list", "patch"]

View File

@@ -29,6 +29,13 @@ NODE_RUNTIME_IMAGE=docker.io/node:24-trixie-slim@sha256:54f84b4cf8432b21aaffd5d3
# Base image for the example gateway (examples/dsh-agents-as-a-service/gateway).
GATEWAY_PYTHON_IMAGE=docker.io/python:3.12-slim@sha256:aa943ce411423c7582288f18707ff66eddcbaeb510502d747a72aeb34f30d159
# Base images for the main DSH Web image (root Dockerfile), pinned by digest.
NODE_BUILD_IMAGE=docker.io/node:24-trixie@sha256:8c761eca2bf7eec965f946cf47b8090143cb41f45f2f8699af0cc0298d2d2d19
NODE_RUNTIME_IMAGE=docker.io/node:24-trixie-slim@sha256:54f84b4cf8432b21aaffd5d3ce8d41e12cea0d10d33c09d2f3c9879d49d6b446
# Base image for the example gateway (examples/dsh-agents-as-a-service/gateway).
GATEWAY_PYTHON_IMAGE=docker.io/python:3.12-slim@sha256:aa943ce411423c7582288f18707ff66eddcbaeb510502d747a72aeb34f30d159
# Control-plane versions used by the examples and their manifests.
# The matching SHA-256s pin the release manifests that CI downloads and
# applies; update both the version and the digest when bumping either pin.