nicolas-georger/deepseek-kubernetes-sandbox
1
0
Fork 0
mirror of https://github.com/sredevopsorg/deepseek-kubernetes-sandbox.git synced 2026-11-03 01:13:24 +00:00
Code Projects Releases Activity

Dial the sandbox pod IP for WebSocket upgrades (#10)

Browse Source 
DSH's Host fence (isTrustedApiRequest) only trusts loopback, an allow-listed
host, or its own bind address, and requires any Origin to match the Host
authority. The gateway forwarded the browser Host as an additional header,
but websockets derives Host from the URI and cannot be overridden, so the
request carried a duplicate/mismatched Host plus the browser Origin and DSH
answered 403. The gateway then closed with 1011, which the DSH client retries
forever:

  [connection] connection lost, retry #1 ... #55

Dial the pod IP instead - it is the bind address, so the derived Host passes
the fence - and forward no Origin; CSWSH is already rejected by same_origin()
before the upstream connection. Fall back to the service FQDN when
status.podIPs is empty and bracket IPv6 literals.

Reproduced against dsh-v0.2.1-rc.2: pod-IP upgrade with no Origin opens;
the same upgrade carrying the browser Origin gets HTTP 403.

Co-authored-by: SREDevOpsAgents <agents@sredevops.org>
This commit is contained in:
SREDevOps.org Agent
2026-10-21 15:09:51 -03:00
committed by GitHub
parent 163abaeac0
commit abe5b4d5bb
3 changed files with 34 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files

7
examples/dsh-agents-as-a-service/README.md
View File

@@ -313,6 +313,11 @@ the DSH web GUI is fully interactive. Requirements:
`60-ingress.yaml` is a starting point (Traefik handles upgrades by default),
- idle suspension is disabled for users with in-flight proxied requests or open WebSockets, so
a long-running chat stream is not cut off mid-turn,
- the upstream upgrade dials the sandbox **pod IP** and forwards no `Host`/`Origin`: DSH's Host
fence only trusts its own bind address or a loopback/allow-listed host, and `websockets`
derives `Host` from the URI (it cannot be overridden). Forwarding the browser's origin here
makes DSH answer `403`, which shows up in the browser as an endless
`[connection] connection lost, retry #N` loop,
- missing/invalid gateway auth is rejected right after the upgrade with close code `4401`
(WebSocket auth has no HTTP `401` path); cross-origin or Origin-less upgrades with the cookie
are rejected the same way (CSWSH protection), so clients must send a matching `Origin`;
@@ -336,6 +341,8 @@ redirects `/u/<user>` to the trailing-slash form with `308` so relative URLs res
| `409` on signup | User already has a claim; use `POST /login` with the existing token or delete the user first |
| Cookie set but browser keeps getting `401` | `COOKIE_SECURE=true` while serving over plain HTTP (secure cookies are not sent); use HTTPS or set `false` for local demos |
| WebSocket closes immediately (`4401`) | Missing/expired gateway cookie, or the upgrade's `Origin` does not match `Host` (CSWSH check); DSH rejecting `Host` instead surfaces as close `1011` — add the host to `DSH_TRUSTED_HOSTS` |
| Console spams `[connection] connection lost, retry #N` | The upstream WS upgrade is being rejected (DSH answers `403` to the forward, surfaced as close `1011`). The gateway dials the sandbox **pod IP** and forwards no `Host`/`Origin` for upgrades; a gateway image from before that fix, or a sandbox with an empty `status.podIPs`, hits the loop |
| `manifest.webmanifest` returns `401` in the console | Browsers fetch the Web App Manifest without credentials, so the gateway's session cookie is absent and the request fails closed. The UI is unaffected; only the PWA install prompt is lost |
| Signup is slow / `202` then `Waking` | Warm pool exhausted (`kubectl get sandboxes -l agents.x-k8s.io/warm-pool-sandbox`); scale the pool |
| `403` from the Kubernetes API in gateway logs | The gateway ServiceAccount is missing claim/sandbox/pod-log verbs (see `50-gateway.yaml`) |
| Step 3: `pod "$SB" not found` after deploying the gateway | The claim was suspended by the idle sweeper. Claims created by the gateway carry `aaas.example.com/managed-by=dsh-gateway`; hand-applied claims (steps 1–7) do not and are never swept. Upgrade the gateway to a build that honors the marker, or resume the sandbox: `kubectl -n dsh-demo patch sandbox "$SB" --type merge -p '{"spec":{"operatingMode":"Running"}}'` |

21
examples/dsh-agents-as-a-service/gateway/gateway.py
View File

@@ -669,6 +669,15 @@ class Gateway:
or sandbox["metadata"]["name"]
)
@staticmethod
def _pod_ip(sandbox: dict) -> str | None:
"""The pod IP is the authority DSH accepts in its Host fence."""
for entry in (sandbox.get("status") or {}).get("podIPs") or []:
ip = entry.get("ip") if isinstance(entry, dict) else entry
if ip:
return ip
return None
def _clear_handshake(self, user: str) -> None:
for key in [k for k in self.state.handshake if k[0] == user]:
self.state.handshake.pop(key, None)
@@ -1141,7 +1150,15 @@ against the JSON API. The browser session is an HttpOnly cookie.</p>
host = websocket.headers.get("host", "")
cookie, _, _ = await self.dsh_cookie(user, sandbox, fqdn, host)
path = websocket.path_params.get("path", "")
url = f"ws://{fqdn}:{self.settings.dsh_port}/{path}"
# DSH's Host fence accepts its own bind address and requires
# any Origin to have the same authority. websockets derives the
# Host header from the URI and cannot be overridden, so dial
# the pod IP instead of the service FQDN and forward no Origin;
# CSWSH is already rejected by same_origin() above.
upstream_host = self._pod_ip(sandbox) or fqdn
if ":" in upstream_host and not upstream_host.startswith("["):
upstream_host = f"[{upstream_host}]" # IPv6 literal
url = f"ws://{upstream_host}:{self.settings.dsh_port}/{path}"
query = filtered_query(websocket.url.query)
if query:
url = f"{url}?{query}"
@@ -1149,9 +1166,9 @@ against the JSON API. The browser session is an HttpOnly cookie.</p>
key: value
for key, value in websocket.headers.items()
if key.lower() not in OWNED_REQUEST_HEADERS
and key.lower() != "origin"
and not key.lower().startswith("sec-websocket-")
}
headers["host"] = host
headers["cookie"] = cookie
headers["x-forwarded-host"] = host
headers["x-forwarded-proto"] = websocket.url.scheme

11
examples/dsh-agents-as-a-service/gateway/test_gateway.py
View File

@@ -38,6 +38,7 @@ from gateway import (
TOKEN = "s3cret-token"
POD_UID = "pod-uid-1"
POD_IP = "10.0.0.7"
FQDN = "dsh-alice-sbx.test-ns.svc.cluster.local"
@@ -206,6 +207,7 @@ def add_user(k8s, user="alice", token=TOKEN, ready=True, pod_name=None, suffix="
"status": {
"conditions": [{"type": "Ready", "status": "True"}] if ready else [],
"serviceFQDN": FQDN,
"podIPs": [POD_IP],
},
}
k8s.pods[pod_name] = {"metadata": {"name": pod_name, "uid": POD_UID}}
@@ -1202,13 +1204,16 @@ def test_ws_bridges_frames_and_injects_cookie(env):
assert ws.receive_text() == "pong"
url, kwargs = env.ws_connect.calls[0]
assert url.startswith(f"ws://{FQDN}:3080/api/remote.mux")
# DSH's Host fence: dial the pod IP (its bind address) and never forward
# the browser's Host/Origin, which would make the upgrade 403.
assert url.startswith(f"ws://{POD_IP}:3080/api/remote.mux")
assert "token=secret" not in url
assert "x=1" in url
headers = kwargs["additional_headers"]
assert "authorization" not in headers
assert headers["cookie"] == "dsh-auth-a=one"
assert headers["host"] == "testserver"
assert "host" not in headers
assert "origin" not in headers
assert env.ws_connect.instances[0].sent == ["hello"]
@@ -1228,7 +1233,7 @@ def test_ws_cookie_same_origin_bridges_frames(env):
headers = env.ws_connect.calls[0][1]["additional_headers"]
assert headers["cookie"] == "dsh-auth-a=one"
assert headers["origin"] == "http://testserver"
assert "origin" not in headers
def test_ws_handshake_failure_closes_1011(env):