From bf243064218788b955b1889b1281a86b47ec5f4c Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Nicol=C3=A1s=20Georger?= Date: Mon, 26 Oct 2026 08:10:59 -0300 Subject: [PATCH] feat(skills): add agent-sandbox platform skill (#20) Add the project-local dsh-agent-sandbox-platform skill covering platform operations, the optional agent-sandbox MCP server, feature-extension recipes, and a read-only preflight. Document project skills in README, docs and AGENTS.md. Co-authored-by: SREDevOpsAgents --- .../dsh-agent-sandbox-platform/SKILL.md | 151 +++++++++ .../references/extending-the-repo.md | 131 ++++++++ .../references/mcp-deployment.md | 94 ++++++ .../references/mcp-server.md | 103 ++++++ .../references/platform-runbook.md | 176 +++++++++++ .../references/troubleshooting.md | 71 +++++ .../scripts/platform-preflight.sh | 298 ++++++++++++++++++ AGENTS.md | 3 + README.md | 10 + docs/README.md | 9 + 10 files changed, 1046 insertions(+) create mode 100644 .dsh/skills/dsh-agent-sandbox-platform/SKILL.md create mode 100644 .dsh/skills/dsh-agent-sandbox-platform/references/extending-the-repo.md create mode 100644 .dsh/skills/dsh-agent-sandbox-platform/references/mcp-deployment.md create mode 100644 .dsh/skills/dsh-agent-sandbox-platform/references/mcp-server.md create mode 100644 .dsh/skills/dsh-agent-sandbox-platform/references/platform-runbook.md create mode 100644 .dsh/skills/dsh-agent-sandbox-platform/references/troubleshooting.md create mode 100755 .dsh/skills/dsh-agent-sandbox-platform/scripts/platform-preflight.sh diff --git a/.dsh/skills/dsh-agent-sandbox-platform/SKILL.md b/.dsh/skills/dsh-agent-sandbox-platform/SKILL.md new file mode 100644 index 0000000..a573f96 --- /dev/null +++ b/.dsh/skills/dsh-agent-sandbox-platform/SKILL.md @@ -0,0 +1,151 @@ +--- +name: dsh-agent-sandbox-platform +description: >- + Operate and extend this repository's DeepSeek Harness-on-agent-sandbox platform: + warm pools, claims, sandbox suspend/resume, the gateway, and the optional + agent-sandbox MCP server, with version-aware safety and a read-only preflight. +whenToUse: >- + Use when working in deepseek-kubernetes-sandbox or with Kubernetes agent-sandbox: + inspect or provision sandboxes, debug warm pools or claims, use + mcp__*__create_sandbox/execute_command, connect the agent-sandbox MCP server, or + add a manifest, gateway route, exposure variant, example, or version bump here. +metadata: + author: sredevopsorg + version: "1.0.0" +--- + +# DSH agent-sandbox platform + +This repository runs DeepSeek Harness Web (DSH) as one sandbox per user on +[agent-sandbox](https://github.com/kubernetes-sigs/agent-sandbox), with an +optional official MCP server for sandbox execution. For generic cluster work +load `kubernetes-kubectl-operations`; for profile MCP rows load +`cordis-plugin-development`. + +## 0. Facts before action + +1. Read `AGENTS.md`, `versions.env`, the example README/architecture. The + control-plane pin is `AGENT_SANDBOX_VERSION`; release identity is + `DSH_VERSION` + `RELEASE_REVISION`. +2. Establish context, user and namespace; never assume `dsh-demo`. +3. Run `scripts/platform-preflight.sh --namespace ` (read-only). +4. If `mcp__*__create_sandbox`-style tools are absent, use `kubectl`; do not + install or enable a server on your own. + +## 1. Non-negotiables + +- **Read-only by default.** Change a cluster only when the user asked for that + exact change; show the command and rollback first. +- **Scope every command** with `--context` and `-n`. `--all`/`-A` mutations and + deleting namespaces, PVCs, PVs, CRDs, or cluster-scoped objects need an + explicit, named confirmation. +- **Secrets and logs stay private.** Never print Secret data, kubeconfigs, or + node logs. Pod logs can contain the DSH launch token. +- **The MCP server has no authentication.** It is a full-access management + endpoint; never expose it. A DSH profile row affects every session. +- **Clean up what you create.** Delete MCP-created sandboxes in the same + session; never leave orphaned claims running. +- **Bound output.** Use `--tail`, `--since`, selectors, JSONPath, and + head/tail excerpts instead of whole manifests or huge logs. + +## 2. Choose the path + +| Need | Path | +| --- | --- | +| Inspect or change claims, pools, sandboxes, PVCs, gateway | `kubectl` + [platform runbook](references/platform-runbook.md) | +| Run code or move files in a sandbox | agent-sandbox MCP, only for a runtime-capable template ([MCP reference](references/mcp-server.md)) | +| Add or change a repository feature | [repo extension guide](references/extending-the-repo.md) | + +The MCP server cannot manage this example's DSH claims: its tools are scoped to +claims created by the same MCP session. Use `kubectl` for the platform. + +## 3. Operate the platform + +Resolve user -> claim -> sandbox -> pod: + +```bash +SB=$(kubectl -n "$NS" get sandboxclaim "$CLAIM" -o jsonpath='{.status.sandbox.name}') +kubectl -n "$NS" get sandbox "$SB" -o jsonpath='{.status.conditions[?(@.type=="Ready")].status}{" "}{.status.conditions[?(@.type=="Ready")].reason}{"\n"}{.status.serviceFQDN}{"\n"}' +POD=$(kubectl -n "$NS" get sandbox "$SB" -o jsonpath='{.metadata.annotations.agents\.x-k8s\.io/pod-name}'); POD=${POD:-$SB} +``` + +- Read runtime state from the Sandbox, not the claim: `spec.operatingMode`, + `Suspended`, `Ready`. The claim tracks binding, not liveness. +- Suspend = patch `operatingMode: Suspended` (pod deleted, PVC + Service kept); + resume = `Running` + wait for `Ready`. The gateway wakes on connect. +- Warm-pool spares carry `agents.x-k8s.io/warm-pool-sandbox`; adopted sandboxes + lose it. A pool with no spares makes signup cold and slow. +- The DSH launch token appears once in pod logs; hand the user the `?token=` URL, + never the token in reports. Probe `/favicon.svg`. +- The idle sweeper only suspends claims the gateway created. + +Full commands, security boundaries and failure matrix: +[platform runbook](references/platform-runbook.md). + +## 4. Use the agent-sandbox MCP server within its limits + +Verified from the upstream server: + +- Tools: `create_sandbox`, `delete_sandbox`, `execute_command`, `upload_file`, + `download_file`, `list_files`, `file_exists`, `get_sandbox_status`; resource + `sandboxes://{namespace}`. +- **Session-scoped:** every tool and the resource filter claims by + `K8S_SANDBOX_SESSION_ID_LABEL_KEY=`. It cannot read, exec in, + or delete DSH platform claims or claims from a prior/reconnected session. +- **Streamable HTTP only:** stdio has no MCP session id and fails. A DSH row must + use `transport: streamable-http`; health endpoints are `/healthz` and `/readyz` + at the root, the MCP endpoint is `/mcp`. +- **Runtime required:** connection types target a sandbox runtime on 8888, or + sandboxd on 8080/9090. The DSH template listens only on 3080, so + `execute_command` cannot work against it. A sandboxd sidecar executes in its + own container, not DSH's. +- **Timeouts:** `create_sandbox` waits up to 180 s, but the DSH MCP client's + default call timeout is 60 s; raise `toolCallTimeoutMs`. + `shutdown_after_seconds` defaults to 300 s (`null` disables it). +- **Output:** servers before v1.0.8 return unbounded `execute_command` output. + v1.0.8 adds `max_output_chars` (128-300,000, default 30,000) and truncation + metadata. Never pass the new argument to an older server; redirect verbose + output and read a file instead. + +Deployment, connection matrix and DSH row: +[MCP deployment](references/mcp-deployment.md). Limits and cleanup: +[MCP reference](references/mcp-server.md). + +## 5. Extend the project + +Read `docs/examples.md` and the testing table in `AGENTS.md` before editing. + +- **Manifest step:** numbered `manifests/NN-*.yaml`, update the README step, run + `make check`; CI dry-runs it. +- **Gateway:** edit `gateway/gateway.py` and `gateway/test_gateway.py`, update + the gateway README API table and architecture doc, run `make test` and + `make lint`. Keep the HTTP/WebSocket contract stable. +- **Exposure variant:** one variant under `manifests/exposure//`, + update the exposure README and trusted-host/cookie settings. Variants are + alternatives, never a stack. +- **New example:** follow `docs/examples.md`, add the index entry, keep scripts + executable/idempotent/self-cleaning, run `make check`. +- **Version bump:** edit `versions.env` only, update prerequisites, run + `make check` and the e2e. Never mix a bump with an unrelated feature. +- **MCP integration:** pin the build to the cluster's tag, least-privilege SA, + ClusterIP + auth proxy, raised DSH timeout, no secrets in the repo. +- **Docs:** update the affected doc in the same change. + +Details and review checklist: +[repo extension guide](references/extending-the-repo.md). + +## 6. Report and clean up + +State context/namespace, bounded evidence, conclusion, exact change, rollback, +and resources deleted. Disable any MCP row enabled only for the task. Do not +leave sandboxes, port-forwards, or temporary files behind. + +## References + +- [Platform runbook](references/platform-runbook.md) +- [MCP deployment](references/mcp-deployment.md) +- [MCP server reference](references/mcp-server.md) +- [Repo extension guide](references/extending-the-repo.md) +- [Troubleshooting](references/troubleshooting.md) +- Upstream MCP server README: + https://github.com/kubernetes-sigs/agent-sandbox/blob/main/clients/integrations/mcp-server/README.md diff --git a/.dsh/skills/dsh-agent-sandbox-platform/references/extending-the-repo.md b/.dsh/skills/dsh-agent-sandbox-platform/references/extending-the-repo.md new file mode 100644 index 0000000..692903f --- /dev/null +++ b/.dsh/skills/dsh-agent-sandbox-platform/references/extending-the-repo.md @@ -0,0 +1,131 @@ +# Extending the repository + +Read `AGENTS.md`, `docs/examples.md`, and the example README before editing. +This repository is image-first: the Dockerfile is the primary artifact, every +implementation is a self-contained example under `examples//`, and +`versions.env` is the only source of pins. Documentation is part of the change. + +## Invariants + +- Never hard-code a version, image tag, or registry path; edit `versions.env`. + For the DSH release identity use `make set-version`, never ad-hoc `sed`. +- Never commit a secret. Secret-shaped manifests are `*.yaml.example` with + placeholders; keys are documented only in `.env.sample`. +- Manifests are numbered `NN-*.yaml` and apply in order with plain `kubectl`. + Exposure variants live under `manifests/exposure//`, are documented + by one README, and are mutually exclusive. +- Keep the gateway HTTP/WebSocket contract stable unless the change is explicitly + about it. +- Do not edit `CHANGELOG.md` (generated by git-cliff) or rewrite git history. +- Keep diffs scoped; do not mix a version bump with an unrelated feature. + +## Change matrix + +| Change | Edit | Minimum verification | +| --- | --- | --- | +| Documentation only | `*.md`, `docs/` | `make check`; relative links resolve | +| Example manifest | `examples//manifests/NN-*.yaml`, README step | `make check`; server-side dry-run in CI | +| Gateway behavior/route | `gateway/gateway.py`, `gateway/test_gateway.py`, gateway README, architecture | `make test`, `make lint`; update tests and API docs | +| Exposure variant | `manifests/exposure//`, exposure README, example README | `make check`; apply exactly one variant | +| New example | full `examples//` layout, `examples/README.md` | `make check`; CI dry-run; optional e2e | +| Version bump | `versions.env`, affected docs/manifests | `make check`; CI installs the pin; e2e on a cluster | +| MCP integration | deployment/RBAC, DSH row, docs | read-only `/readyz`; no secret in repo | +| Workflow | `.github/workflows/` | actionlint; SHA-pinned actions; no hard-coded versions | + +## Recipes + +### Add or change a resource step + +1. Keep numbering stable; add `NN-*.yaml` at the end unless order matters. +2. Update the matching numbered step in the example README. +3. Keep cluster-specific state (node affinity, tuned replica counts) out of the + shipped manifest; document it as a local patch. +4. Run `make check`. CI additionally parses every manifest and runs a + server-side dry-run against the agent-sandbox CRDs. + +### Change the gateway + +The gateway is a single-file Starlette app. A change usually touches: + +- `gateway/gateway.py` (route, auth, proxy, sweeper); +- `gateway/test_gateway.py` (unit tests; no cluster required); +- `gateway/README.md` (API table, configuration, auth semantics); +- `examples/dsh-agents-as-a-service/docs/architecture.md` when flows change; +- `examples/dsh-agents-as-a-service/README.md` when the walkthrough changes. + +Run `make test` and `make lint`. Preserve session-cookie auth, same-origin rules +for cookie mutations and WebSocket upgrades, and the "waking"/`502`/`503` +semantics unless the change is explicitly about them. Do not expose a trusted +identity header unless the Service is reachable only through the proxy. + +### Add an exposure variant + +Create `manifests/exposure//NN-*.yaml`, update +`manifests/exposure/README.md` and the example README, and state the trust +boundary. Variants are alternatives: never apply two paths to the same gateway. +Set `DSH_TRUSTED_HOSTS` in the template for the public host and +`COOKIE_SECURE=true` for HTTPS. + +### Add an example + +Follow `docs/examples.md`: `README.md`, `manifests/NN-*.yaml`, optional +`manifests/exposure/` with a README, optional components with Dockerfile + +README + tests, and executable, idempotent, self-cleaning scripts. Add the +literal `examples//` entry to `examples/README.md`; `make check` enforces +the layout. Publish component images under +`////`. + +### Bump a version + +- **DSH release identity:** `make set-version DSH_VERSION= + [RELEASE_REVISION=rN]`, then `make check`, then build and smoke-test the image. + Tags are immutable; fix a bad release with `-r(N+1)`. +- **agent-sandbox:** edit `AGENT_SANDBOX_VERSION` in `versions.env`, update the + prerequisites in the example README, and note behavior changes (connection + types, MCP output limits, CRD fields). Install the new release YAML on a kind + cluster, then run `make check` and `make e2e`. Do not bump it inside an + unrelated PR. + +### Integrate the agent-sandbox MCP server + +Build from the cluster's pinned tag, give it a dedicated namespace and +least-privilege ServiceAccount, keep it ClusterIP behind an authenticating +proxy, and document how to reach `/readyz`. Add a DSH profile row only with user +approval; a row affects every session. Never commit its credentials or a +`cluster-admin` binding. + +### Documentation + +Update the doc whose behavior changed in the same commit: root README for the +front door, `docs/` for repository-wide contracts, the example README for that +walkthrough, the gateway README for API/config/auth. Skill docs live in +`.dsh/skills/`; keep `SKILL.md` under ~8 KB and put depth in `references/`. +`scripts/check-versions.sh` scans every `*.md` for stale `dsh-v…` tokens, so +refer to "the release identity in `versions.env`" rather than copying a tag. + +## Verification and review + +Run at least `make check`; add `make test` and `make lint` for gateway/script +changes. For manifests, use `kubectl apply --dry-run=server` against a cluster +with the CRDs. For behavior changes, run the example or `make e2e`. + +Before reporting, review: + +- [ ] Diff is scoped; no generated files or unrelated changes. +- [ ] Docs and examples match the new behavior. +- [ ] Pins come only from `versions.env`; no stale version tokens. +- [ ] No secrets or usable credentials; secret-shaped files are `.example`. +- [ ] Manifests parse; exposure variants remain mutually exclusive. +- [ ] Gateway tests and public contract unchanged or intentionally updated. +- [ ] Skills: frontmatter valid, body within budget, scripts shellcheck-clean. +- [ ] `CHANGELOG.md` untouched; branch/commit follow `docs/gitflow.md`. + +## Safe follow-ups enabled by this skill + +- A read-only preflight job or scheduled report using + `scripts/platform-preflight.sh`. +- An MCP readiness check in the example e2e (health endpoints only). +- A separate runtime warm pool for untrusted execution, leaving the DSH template + unchanged. +- A combined DSH + sandboxd image only with an explicit maintenance/exec design + and a documented trust boundary. diff --git a/.dsh/skills/dsh-agent-sandbox-platform/references/mcp-deployment.md b/.dsh/skills/dsh-agent-sandbox-platform/references/mcp-deployment.md new file mode 100644 index 0000000..e22c49c --- /dev/null +++ b/.dsh/skills/dsh-agent-sandbox-platform/references/mcp-deployment.md @@ -0,0 +1,94 @@ +# Deploying the agent-sandbox MCP server + +Read [the MCP reference](mcp-server.md) first. This file covers the connection +matrix, a safe deployment shape, and the DSH profile row. The server is +unauthenticated; its service account is the security boundary. + +## Connection matrix + +| `K8S_SANDBOX_CONNECTION__TYPE` | Required env | Use when | +| --- | --- | --- | +| `gateway` | `GATEWAY_NAME`, `GATEWAY_NAMESPACE=default`, `GATEWAY_READY_TIMEOUT=180`, `SERVER_PORT=8888` | Gateway API + implementation exist | +| `direct` | `API_URL` (router URL), `SERVER_PORT=8888` | sandbox-router is deployed | +| `in-cluster` | `USE_POD_IP=false` (DNS) or `true` (`status.podIPs`), `SERVER_PORT=8888` | Bypass the router from inside the cluster | +| `sandboxd-in-cluster` (v1.0.8+) | `MODE=in-cluster-service` or `in-cluster-pod-ip`, `REST_PORT=8080`, `GRPC_PORT=9090` | Template runs sandboxd | + +Also set `K8S_SANDBOX_PROBE_NAMESPACE` to a namespace the server's service +account can list claims in, and optionally +`K8S_SANDBOX_SESSION_ID_LABEL_KEY` (default +`mcp.k8s-agent-sandbox/session-id`). + +For `sandboxd-in-cluster` service mode the template needs `spec.service: true`. +The default managed NetworkPolicy admits only sandbox-router, so add an ingress +rule for the MCP server on 8080/9090. Supplying `spec.networkPolicy` replaces the +secure defaults: also add the egress rules the sandbox needs, because an omitted +egress list denies all outbound traffic. RBAC for that mode is +`sandboxclaims` create/get/list/watch/delete and `sandboxes` get/list/watch; no +`pods/portforward`. + +## Build and deploy (sketch) + +No official image reference is documented in the upstream README; pin what you +build. + +```bash +# AGENT_SANDBOX_VERSION comes from versions.env; never hard-code it. +tmp="$(mktemp -d)"; trap 'rm -rf "$tmp"' EXIT +git clone --depth 1 --branch "$AGENT_SANDBOX_VERSION" \ + https://github.com/kubernetes-sigs/agent-sandbox.git "$tmp" +docker build -t "$MCP_IMAGE" "$tmp/clients/integrations/mcp-server" +``` + +Kubernetes shape: + +- Dedicated namespace and ServiceAccount; a namespace-scoped Role granting only + `sandboxclaims` create/get/list/watch/delete and `sandboxes` get/list/watch. + No Secrets, no `pods/exec`, no cluster-admin. +- Deployment: one replica, non-root, `readOnlyRootFilesystem`, dropped + capabilities, `K8S_SANDBOX_CONNECTION__TYPE=in-cluster`, + `K8S_SANDBOX_PROBE_NAMESPACE`, resource limits. +- Probes: `GET /healthz` (liveness; never calls the API) and `GET /readyz` + (readiness; lists claims; use `periodSeconds: 15`, not aggressive polling). +- ClusterIP Service on `8000`. Add a NetworkPolicy that allows only the intended + client namespace/pod. Never expose it through Ingress/LoadBalancer without an + authenticating proxy. The service account is the real boundary. + +## Connect it to DSH + +Add a profile MCP row (through the Plugins page/`plugin_manager`; see +`cordis-plugin-development`). A row affects every session and needs approval. + +```yaml +- insert: + - id: mcp-agent-sandbox + name: '@deepseek-ai/dsh-mcp-client' + config: + serverName: agent-sandbox + transport: streamable-http + url: http://agent-sandbox-mcp.example.svc.cluster.local:8000/mcp + toolCallTimeoutMs: 240000 + failOnStartupError: false +``` + +- `serverName` must match `[A-Za-z0-9_-]{1,32}`; tools appear as + `mcp____`. Check the actual names in the tool list; do not + assume hyphens/underscores. +- Raise `toolCallTimeoutMs` above `sandbox_ready_timeout` with headroom; the DSH + default is 60 s and would cancel a 180 s creation. +- If the server is behind an authenticating proxy, pass its token via the row's + `headers` and an environment expression; never commit it. +- To reach a locally port-forwarded server, use a loopback URL; the MCP session + still needs streamable HTTP. +- Remove the row when it was enabled only for one task. + +## Deployment safety checklist + +- [ ] Build tag equals the cluster's `AGENT_SANDBOX_VERSION`. +- [ ] Dedicated namespace/SA; RBAC limited to claims and sandboxes. +- [ ] Service is ClusterIP; no public route without authentication. +- [ ] NetworkPolicy admits only the DSH client and needed sandbox traffic. +- [ ] `/readyz` succeeds with `K8S_SANDBOX_PROBE_NAMESPACE` set to a listable + namespace. +- [ ] No Secret data in manifests, logs, or the repository. +- [ ] Cleanup path documented: delete session-owned claims or wait for the + shutdown timeout. diff --git a/.dsh/skills/dsh-agent-sandbox-platform/references/mcp-server.md b/.dsh/skills/dsh-agent-sandbox-platform/references/mcp-server.md new file mode 100644 index 0000000..2785ef8 --- /dev/null +++ b/.dsh/skills/dsh-agent-sandbox-platform/references/mcp-server.md @@ -0,0 +1,103 @@ +# agent-sandbox MCP server reference + +Upstream: [clients/integrations/mcp-server](https://github.com/kubernetes-sigs/agent-sandbox/tree/main/clients/integrations/mcp-server). +It is a Python/FastAPI + FastMCP service (streamable HTTP on `:8000`, MCP +endpoint `/mcp`) that provisions sandboxes from a warm pool and runs commands or +file operations through a sandbox runtime. It ships with **no authentication**. + +Build the server from the same tag as the cluster: read `AGENT_SANDBOX_VERSION` +in `versions.env`, and check the installed controller image. Do not build from +`main` against an older cluster; tool schemas and connection types differ. + +## Tool surface + +| Tool | Arguments (defaults) | Returns / notes | +| --- | --- | --- | +| `create_sandbox` | `warmpool`, `namespace`, `sandbox_ready_timeout=180`, `labels`, `shutdown_after_seconds=300` (`null` disables), `pod_labels`, `pod_annotations` | `sandbox_claim_name`; adds the MCP session label | +| `delete_sandbox` | `sandbox_claim_name`, `namespace` | Deletes the session-owned claim; PVC cascades | +| `execute_command` | `sandbox_claim_name`, `namespace`, `command`, `timeout=60` (max 600), `max_output_chars` only on v1.0.8+ (128-300,000, default 30,000) | `exit_code`, `stdout`, `stderr`; v1.0.8 adds `*_total_chars` and `*_truncated` | +| `upload_file` | `sandbox_claim_name`, `namespace`, `path`, `content`, `binary=false`, `timeout=60` | Bytes written; base64 when `binary=true` | +| `download_file` | `sandbox_claim_name`, `namespace`, `path`, `binary=false`, `timeout=60` | Content + bytes read | +| `list_files` | `sandbox_claim_name`, `namespace`, `path`, `timeout=60`, `max_entries=1000` (max 10,000) | Entries (`name`, `size`, `type`, `mod_time`), `total_entries`, `truncated` | +| `file_exists` | `sandbox_claim_name`, `namespace`, `path`, `timeout=60` | `exists`; a missing path is not an error | +| `get_sandbox_status` | `sandbox_claim_name`, `namespace` | `SandboxReady` / `SandboxNotReady` / `SandboxNotFound`, `ready`, `message` | + +Resource: `sandboxes://{namespace}` lists sandboxes for the current MCP session. + +## Session scoping is the most important behavior + +Every tool, including `get_sandbox_status`, calls `get_sandbox()` -> +`ensure_session_owns()`, which lists sandboxes with the label selector +`=`. The resource uses the same +selector. Consequences: + +- The server can only see/exec/delete claims **it created in the same MCP + session**. It cannot inspect or manage DSH platform claims created by the + gateway or `kubectl`, or claims from another session. +- On a DSH MCP reconnect, the MCP session changes; earlier claims become + invisible to the tools. They keep running until `shutdown_after_seconds` + expires, or until you remove them by label with a confirmed `kubectl delete`. +- A "Sandbox claim '' is not found in namespace ''" error usually + means wrong namespace **or** the claim belongs to a different session. +- Use `kubectl` for global inventory; use the MCP only for sandboxes it created + in this task. + +Session identity requires a session-providing transport. With stdio, +`ctx.session_id` is `None` and creation fails. Connect DSH over +`streamable-http`. + +## Runtime prerequisite + +The server talks to a sandbox runtime, not to arbitrary pods: + +- `gateway`, `direct`, `in-cluster`: runtime HTTP on `SERVER_PORT=8888` + (sandbox-router API or direct pod runtime). +- `sandboxd-in-cluster` (v1.0.8+): sandboxd REST `8080`, gRPC `9090`. + +The shipped DSH template runs only DSH Web on `3080`, so no connection type can +exec into it. A sandboxd **sidecar** executes commands in the sidecar's own +container, not in the DSH container; shared volumes share files, not binaries or +processes. To exec inside DSH, build a combined image (sandboxd injected into the +DSH image) or use `kubectl exec` with explicit RBAC and confirmation. For +untrusted code, deploy a separate runtime warm pool rather than modifying the +DSH template. + +## Version differences + +| Area | Before v1.0.8 | v1.0.8+ | +| --- | --- | --- | +| Connection types | `gateway`, `direct`, `in-cluster` | Adds `sandboxd-in-cluster` | +| `execute_command` output | Unbounded | Bounded by `max_output_chars`; four metadata fields | +| Failed pod recovery | `Retain` only | `spec.failedPodPolicy: Recreate` | +| Router install | `kubectl apply -f sandbox-router/deploy/` | `kubectl apply -k sandbox-router/deploy/`; projected token | + +Never pass `max_output_chars` to a pre-v1.0.8 server. On older servers, redirect +verbose output to a file and `download_file` it, or pipe through `tail`/`head`. + +## Safety and cleanup + +- No auth: anyone reaching `:8000` gets every tool and the service account's + access. Keep it ClusterIP, authenticated, and namespace-scoped. +- Treat commands as untrusted code: they can read files and environment + variables inside the sandbox, including provider keys in a combined image. +- `delete_sandbox` cascades the claim, sandbox, and PVC: confirm the target and + namespace. Never delete a namespace or PVC as cleanup. +- Clean up in the same session that created the sandbox. If the session is gone, + list claims by the session label and delete them explicitly (requires write + RBAC); otherwise rely on the shutdown timeout. + +## Troubleshooting + +| Symptom | Cause / fix | +| --- | --- | +| Tools missing | Row not enabled, wrong URL, connection failed; check logs and `/readyz` | +| 404 on connect | URL is missing `/mcp` or points at the wrong port | +| `ctx.session_id is None` | Connected over stdio; use streamable HTTP | +| `Sandbox claim ... is not found` | Wrong namespace, or claim created by another MCP session | +| Creation times out at ~60 s | MCP session alive but DSH cancelled the call; raise `toolCallTimeoutMs` | +| `execute_command` connection error | No runtime on 8888/8080/9090, or wrong connection type; the DSH template is unsupported | +| Output flood | Pre-v1.0.8 server; bound the command output yourself | +| Empty sandbox list | The resource is session-scoped; use `kubectl` for global inventory | + +Deployment, connection matrix, RBAC, and the DSH row: +[MCP deployment](mcp-deployment.md). diff --git a/.dsh/skills/dsh-agent-sandbox-platform/references/platform-runbook.md b/.dsh/skills/dsh-agent-sandbox-platform/references/platform-runbook.md new file mode 100644 index 0000000..c0de958 --- /dev/null +++ b/.dsh/skills/dsh-agent-sandbox-platform/references/platform-runbook.md @@ -0,0 +1,176 @@ +# Platform runbook + +Scope: the resource layer and gateway in +`examples/dsh-agents-as-a-service/`, one namespace, one DSH Web sandbox per +user. Read the example README and `AGENTS.md` first. All commands are read-only +unless a section is explicitly a change; confirm before changing anything. + +## 1. Context, version, CRDs + +```bash +kubectl config current-context +kubectl auth whoami +kubectl config view --minify -o jsonpath='{..namespace}{"\n"}' +kubectl get crd | grep -E 'agents\.x-k8s\.io' +kubectl -n agent-sandbox-system get deploy agent-sandbox-controller \ + -o jsonpath='{.spec.template.spec.containers[0].image}{"\n"}' +``` + +Compare the controller image tag with `AGENT_SANDBOX_VERSION` in +`versions.env`. The example needs the extension CRDs plus core `sandboxes`; +a missing CRD is a prerequisite failure, not an application bug. + +## 2. Resource model + +| Object | Role | Key fields | +| --- | --- | --- | +| `SandboxTemplate` | Blueprint; pinned image, probes, security context, PVC template | `envVarsInjectionPolicy`, `volumeClaimTemplatesPolicy`, `service`, `podTemplate` | +| `SandboxWarmPool` | Pre-warmed spares from one template | `spec.replicas`, `sandboxTemplateRef` | +| `SandboxClaim` | One tenant/user request; adopts a spare | `spec.warmPoolRef`, `additionalPodMetadata`, `status.sandbox.name` | +| `Sandbox` | Adopted or directly created pod + Service + PVC | `spec.operatingMode`, `status.conditions`, `status.serviceFQDN`, `status.podIPs` | +| `Pod` | The DSH Web process | `:3080`, PVC at `/home/node/.dsh` | +| `PersistentVolumeClaim` | Per-sandbox state | `dsh-home-` | +| `Service` | Stable per-sandbox DNS | ``, `status.serviceFQDN` | + +Delete a claim to cascade-delete its sandbox and PVC (account deletion). Never +delete a PVC or namespace without an explicit, named confirmation. + +## 3. Read-only inspection + +```bash +kubectl -n "$NS" get sandboxtemplates,sandboxwarmpools,sandboxclaims,sandboxes -o wide +kubectl -n "$NS" get pods,pvc,svc -o wide +kubectl -n "$NS" get events --sort-by=.lastTimestamp | tail -30 +kubectl -n "$NS" get sandboxclaim "$CLAIM" -o yaml +kubectl -n "$NS" get sandbox "$SB" -o yaml +``` + +Bounded logs: `kubectl -n "$NS" logs "$POD" --tail=200` (add `--previous` +only after a restart). + +## 4. Claim -> sandbox -> pod + +```bash +SB=$(kubectl -n "$NS" get sandboxclaim "$CLAIM" -o jsonpath='{.status.sandbox.name}') +POD=$(kubectl -n "$NS" get sandbox "$SB" -o jsonpath='{.metadata.annotations.agents\.x-k8s\.io/pod-name}') +POD=${POD:-$SB} +kubectl -n "$NS" get sandbox "$SB" -o jsonpath='{range .status.conditions[*]}{.type}={.status} {.reason}{"\n"}{end}' +kubectl -n "$NS" get sandbox "$SB" -o jsonpath='{.spec.operatingMode}{" "}{.status.serviceFQDN}{" "}{.status.podIPs}{"\n"}' +``` + +Rules: + +- Resolve user -> sandbox only through `claim.status.sandbox.name`; the sandbox + keeps its pool-generated name, not the claim name. +- A claim tracks binding; for runtime state read `Sandbox.spec.operatingMode` + and the `Suspended`/`Ready` conditions. A suspended sandbox is not serving + even when the claim still exists. +- A missing `claim.status.sandbox.name` means the claim did not bind: inspect + the Ready condition `reason` and events. The pod is normally named after the + sandbox; use the annotation when present. + +## 5. Suspend, resume, wake + +Suspend keeps the PVC and Service, deletes the pod: + +```bash +kubectl -n "$NS" patch sandbox "$SB" --type merge \ + -p '{"spec":{"operatingMode":"Suspended"}}' +kubectl -n "$NS" wait --for=delete "pod/$POD" --timeout=120s +kubectl -n "$NS" get pvc "dsh-home-$SB" +kubectl -n "$NS" get svc "$SB" +``` + +Resume and verify state: + +```bash +kubectl -n "$NS" patch sandbox "$SB" --type merge \ + -p '{"spec":{"operatingMode":"Running"}}' +kubectl -n "$NS" wait sandbox "$SB" --for=condition=Ready --timeout=180s +kubectl -n "$NS" exec "$SB" -- cat /home/node/.dsh/marker.txt +``` + +The gateway wakes a suspended sandbox on the first request. The idle sweeper +only suspends claims it created (`aaas.example.com/managed-by=dsh-gateway`); +hand-applied claims are never swept. + +## 6. Warm pool + +```bash +kubectl -n "$NS" get sandboxwarmpool "$POOL" -o yaml +kubectl -n "$NS" get sandboxes -l agents.x-k8s.io/warm-pool-sandbox -o wide +kubectl -n "$NS" get sandboxwarmpool "$POOL" -o jsonpath='{.spec.replicas}{"\n"}' +``` + +Adopted sandboxes lose the warm-pool label; count current spares against +`spec.replicas`. Zero spares or `spec.replicas: 0` means cold, slow signup. +Scale only when asked: + +```bash +kubectl -n "$NS" patch sandboxwarmpool "$POOL" --type merge -p '{"spec":{"replicas":2}}' +``` + +## 7. DSH runtime specifics + +- DSH Web listens on `3080` in-cluster. Port-forward to a free local port such + as `18081`, never `3080` if a local DSH already uses it. +- The launch token is printed once as `dsh web: http://:3080/?token=...`. + Treat it as a credential: do not paste it into reports; hand the user the URL. +- `/` returns `401` without the token. Probes must target `/favicon.svg`. +- `DSH_TRUSTED_HOSTS` must include any public host that a proxy forwards; DSH + fences on `Host`/`Origin`. +- State lives at `/home/node/.dsh`. DSH requires `.credentials.yaml` to be + `0600`; the template uses `fsGroupChangePolicy: OnRootMismatch`. If a resumed + sandbox crash-loops with a mode-660 error, suspend it, fix the file with a + throwaway pod, patch the policy, then resume. + +## 8. Gateway layer + +```bash +kubectl -n "$NS" get deploy,svc,pods -l app=dsh-gateway -o wide +kubectl -n "$NS" get endpoints dsh-gateway +kubectl -n "$NS" logs deploy/dsh-gateway --tail=100 +``` + +The gateway is a ClusterIP Service on `8080`; it proxies under `/u/{user}/` and +derives state (`Provisioning`, `Waking`, `Ready`, `Suspending`, `Suspended`). +It performs the DSH launch-token handshake server-side. With SSO enabled +(`AUTH_PROXY_HEADER`), the Service must be reachable only through the +authenticating proxy; never add a second path or port-forward it on a shared +host. A resource-only install has no gateway: that is a warning, not a failure. + +## 9. Security boundaries + +- One `DEEPSEEK_API_KEY` is shared by all sandboxes; the example is not + tenant-isolated. +- Secret data, kubeconfigs, and pod logs are sensitive; RBAC may allow Secret + access, policy does not. Keep gateway RBAC namespace-scoped; it needs pod logs + to find the launch token. +- TLS terminates outside the gateway; set `COOKIE_SECURE=true` for HTTPS. + +## 10. Failure matrix + +| Symptom | First evidence | Likely cause / fix | +| --- | --- | --- | +| Claim not Ready, no sandbox | Ready condition reason, events | Empty pool, image pull, quota, unschedulable | +| `EnvVarsInjectionRejected` | claim Ready reason | Claim carries env/volume templates; template policy rejects it | +| Pod Pending | `kubectl describe pod` | Requests/taints/affinity, unbound PVC | +| CrashLoopBackOff | logs, `--previous` | Startup failure, bad config, probe 401, fsGroup mode 660 | +| ImagePullBackOff | describe events | Wrong tag, private registry, missing pull secret | +| DSH 401 in browser | gateway logs | Launch-token handshake failed; check gateway RBAC `pods/log` | +| 502/503 through gateway | sandbox Ready, gateway logs | Wake timeout, DNS stale, token not in logs yet | +| Warm pool empty | sandboxes label count | Scale the pool or wait for replenishment | +| Suspended but expected running | `operatingMode` | Resume it; the gateway only wakes on a request | +| WS closes `1011` | gateway logs | DSH rejected the upstream Host/authority; check trusted hosts | + +## 11. Cleanup and reporting + +Delete only claims/sandboxes you created, by name: + +```bash +kubectl -n "$NS" delete sandboxclaim "$CLAIM" --dry-run=server +kubectl -n "$NS" delete sandboxclaim "$CLAIM" +``` + +Report context/namespace, evidence, conclusion, change, rollback, and what was +deleted. Leave the namespace, pools, templates, and unrelated claims alone. diff --git a/.dsh/skills/dsh-agent-sandbox-platform/references/troubleshooting.md b/.dsh/skills/dsh-agent-sandbox-platform/references/troubleshooting.md new file mode 100644 index 0000000..894918f --- /dev/null +++ b/.dsh/skills/dsh-agent-sandbox-platform/references/troubleshooting.md @@ -0,0 +1,71 @@ +# Troubleshooting + +Start with evidence, not fixes: resolve context/namespace, read conditions and +events, then compare desired vs observed state. Keep output bounded. Never +"fix" an access error by broadening credentials. + +## Platform + +| Symptom | First evidence | Likely cause / fix | +| --- | --- | --- | +| `Forbidden` on every call | `kubectl auth whoami`, `kubectl auth can-i --list -n ` | Wrong context or RBAC. Do not retry with broader credentials; ask for the right kubeconfig. | +| CRDs not found | `kubectl get crd | grep agents.x-k8s.io` | agent-sandbox extensions not installed. Install the pinned release YAML; that is a prerequisite, not a code change. | +| Claim created, no sandbox | `claim.status.conditions[?(@.type=="Ready")]`, events | Empty pool, image pull, quota, unschedulable; or policy rejection. | +| Claim rejected | Ready reason `EnvVarsInjectionRejected` | Claim carries `env`/`volumeClaimTemplates` while the template disallows injection. Remove them; do not bypass the pool. | +| User is suspended but the claim exists | `Sandbox.spec.operatingMode`, `Suspended` condition | Resume with `operatingMode: Running`; a gateway request would wake it. | +| Warm pool exhausted | `kubectl get sandboxes -l agents.x-k8s.io/warm-pool-sandbox` | Scale `spec.replicas` (with confirmation) or accept slow cold provisioning. | +| Pod `Pending` | `kubectl describe pod` | Requests/limits, taints, affinity, unbound PVC, storage class. | +| `ImagePullBackOff` | describe events | Wrong image/tag, private GHCR package, missing pull secret; on kind, load the image under the exact manifest ref. | +| `CrashLoopBackOff`, probe 401 | logs, `describe` | Probe path is `/`; use `/favicon.svg`. | +| Restart with mode-660 error | logs mention `.credentials.yaml` | kubelet fsGroup widened the file; suspend, `chmod 600` from a throwaway pod, patch `fsGroupChangePolicy: OnRootMismatch`, resume. | +| PVC `Pending` | `kubectl describe pvc`, StorageClass | No default provisioner, access mode, zone binding. | +| Launch token not found | `kubectl logs --tail=2000 | grep 'dsh web:'` | Pod not ready yet, or logs rotated. Do not guess tokens; wait for readiness. | +| Hand-applied sandbox disappeared | claim annotations, gateway logs | Idle sweeper only manages claims with `aaas.example.com/managed-by=dsh-gateway`; if it was managed, recreate/resume. | +| Gateway 401 | gateway logs, cookie flags | Missing/expired cookie, DSH handshake failed, or `COOKIE_SECURE=true` over HTTP. | +| Gateway 502/503 | sandbox Ready, gateway logs | Wake timeout, stale DNS, token not yet in logs; check `WAKE_TIMEOUT` and RBAC `pods/log`. | +| WebSocket closes 1011 | gateway logs, `Host`/Origin | DSH rejected the upstream authority; check `DSH_TRUSTED_HOSTS` and the gateway's pod-IP dial path. | +| WebSocket closes 4401 | cookie, Origin | Missing gateway auth or cross-origin upgrade; send a matching `Origin`. | +| SSO header ignored | gateway env | `AUTH_PROXY_HEADER` unset; setting it requires the proxy to be the only path to the ClusterIP Service. | +| Someone is treated as another user | service paths | Trust-boundary violation: the gateway is directly reachable. Remove the extra path and disable SSO until fixed. | + +## Agent-sandbox MCP + +| Symptom | First evidence | Likely cause / fix | +| --- | --- | --- | +| No `mcp__*__create_sandbox` tools | tool list, DSH logs | Server not deployed or row not enabled. Use `kubectl`; connect only with approval. | +| 404 on connect | curl `/mcp`, `/healthz` | Wrong path/port; MCP endpoint is `/mcp`, health is at the root. | +| `/readyz` 503 | response body, server logs | API unreachable or namespace `K8S_SANDBOX_PROBE_NAMESPACE` not listable; fix RBAC/config. | +| `ctx.session_id is None` | server error | Connected over stdio; use streamable HTTP. | +| `Sandbox claim ... is not found` | server error, `kubectl get claim` | Wrong namespace, or the claim belongs to another MCP session. List with `kubectl`; use the MCP only for its own claims. | +| Creation cancelled at ~60 s | MCP call log | DSH `toolCallTimeoutMs` is 60 s while creation waits up to 180 s; raise the timeout. | +| `execute_command` connection error | server logs, template ports | No runtime on 8888/8080/9090. The DSH template only listens on 3080; use a runtime template or a combined image. | +| Output truncated / flooded | tool result | Pre-v1.0.8 is unbounded; v1.0.8 bounds output. Redirect verbose output to a file and read/download it; never pass `max_output_chars` to an old server. | +| Orphaned claims after reconnect | `kubectl get claims -l mcp.k8s-agent-sandbox/session-id` | MCP sessions are transient; wait for `shutdown_after_seconds` or delete by label with explicit confirmation. | +| `Forbidden` from the MCP server | server logs | Its ServiceAccount lacks claim/sandbox verbs, or RBAC is in another namespace; fix the Role, not the kubeconfig. | +| Sandbox expired | `get_sandbox_status` -> `SandboxNotFound` | `shutdown_after_seconds` elapsed; create a new sandbox and keep the task inside the budget. | + +## Repository checks and CI + +| Symptom | Cause / fix | +| --- | --- | +| `check-versions: stale version reference in docs: dsh-v...` | A copied release tag in a Markdown file. Refer to the identity in `versions.env`; run `make set-version` only for a real bump. | +| `check-versions: hard-coded version outside versions.env` | A tag in `.github/workflows` or the Makefile; move it to `versions.env`. | +| `check-examples` layout error | Missing `README.md`/`manifests/`, stray YAML at the example root, missing exposure README, non-executable script, or missing `examples/README.md` entry. | +| Manifest dry-run fails in CI | Invalid YAML or CRD schema; parse locally and run `kubectl apply --dry-run=server` against the pinned CRDs. | +| `shellcheck` failure | Quote expansions, use arrays for kubectl args, avoid unused variables; run shellcheck locally. | +| Lychee link failure | A relative/absolute link does not resolve; the link checker runs offline on the listed docs. | +| PR-title check failure | Use Conventional Commits: `feat(scope): ...`, `fix(scope): ...`, `docs(scope): ...`. | +| Changelog conflict | Never edit `CHANGELOG.md`; it is generated from commits. | + +## Do not + +- Do not use `--force --grace-period=0` to clear a stuck resource before + understanding why it is stuck. +- Do not delete namespaces, PVCs, PVs, CRDs, or cluster-scoped objects as a + cleanup shortcut. +- Do not read Secret data, kubeconfigs, or node logs to "debug" a permission + problem, and do not paste pod logs containing the DSH launch token into a + report. +- Do not expose the unauthenticated MCP server or enable `AUTH_PROXY_HEADER` + while another path to the gateway exists. +- Do not retry a `Forbidden` with a stronger kubeconfig; fix the RBAC scope. diff --git a/.dsh/skills/dsh-agent-sandbox-platform/scripts/platform-preflight.sh b/.dsh/skills/dsh-agent-sandbox-platform/scripts/platform-preflight.sh new file mode 100755 index 0000000..a38bcab --- /dev/null +++ b/.dsh/skills/dsh-agent-sandbox-platform/scripts/platform-preflight.sh @@ -0,0 +1,298 @@ +#!/usr/bin/env bash +# Read-only preflight for the DSH agents-as-a-service platform. +# +# Checks the kubectl target, agent-sandbox CRDs/version, warm pools, claims, +# sandboxes, gateway and (optionally) an agent-sandbox MCP server. It never +# mutates the cluster, never reads Secret data, and never port-forwards. +# +# Usage: +# platform-preflight.sh [--context NAME] [--namespace NS] [--pool NAME] +# [--template NAME] [--mcp-url URL] [--help] +# +# Exit codes: 0 = no failures, 1 = at least one failure, 2 = usage error. +set -euo pipefail + +SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)" +REPO_ROOT="$(git -C "$SCRIPT_DIR" rev-parse --show-toplevel 2>/dev/null || true)" + +CTX="" +NS="${NAMESPACE:-dsh-demo}" +POOL="" +TEMPLATE="" +MCP_URL="" + +PASS=0 +WARN=0 +FAIL=0 + +usage() { + cat <<'EOF' +Read-only preflight for the DSH agents-as-a-service platform. + +Options: + --context NAME kubectl context (default: current context) + --namespace NS namespace to inspect (default: $NAMESPACE or dsh-demo) + --pool NAME check only this warm pool + --template NAME check only this SandboxTemplate + --mcp-url URL also check an MCP server's /healthz and /readyz + --help show this help + +The script makes no changes and never reads Secret data. +EOF +} + +while [ "$#" -gt 0 ]; do + case "$1" in + --context) CTX="${2:-}"; shift 2 ;; + --namespace) NS="${2:-}"; shift 2 ;; + --pool) POOL="${2:-}"; shift 2 ;; + --template) TEMPLATE="${2:-}"; shift 2 ;; + --mcp-url) MCP_URL="${2:-}"; shift 2 ;; + --help|-h) usage; exit 0 ;; + *) echo "unknown argument: $1" >&2; usage >&2; exit 2 ;; + esac +done + +if [ -z "$NS" ]; then + echo "namespace must not be empty" >&2 + exit 2 +fi + +section() { printf '\n== %s ==\n' "$1"; } +pass() { PASS=$((PASS + 1)); printf 'PASS %s\n' "$1"; } +warn() { WARN=$((WARN + 1)); printf 'WARN %s\n' "$1"; } +fail() { FAIL=$((FAIL + 1)); printf 'FAIL %s\n' "$1"; } +info() { printf 'INFO %s\n' "$1"; } + +command -v kubectl >/dev/null 2>&1 || { + echo "FAIL kubectl is not installed" >&2 + exit 1 +} + +if [ -n "$CTX" ]; then + if ! kubectl config get-contexts -o name 2>/dev/null | grep -Fxq "$CTX"; then + echo "FAIL context '$CTX' is not in the kubeconfig" >&2 + exit 1 + fi + KUBECTL=(kubectl --context "$CTX" --request-timeout=15s) +else + KUBECTL=(kubectl --request-timeout=15s) +fi + +k() { "${KUBECTL[@]}" "$@"; } + +section "Target" +if current="$(k config current-context 2>/dev/null)"; then + info "context: $current" + pass "kubeconfig is readable" +else + fail "cannot read the current kubectl context" +fi +if user="$(k auth whoami -o jsonpath='{.status.userInfo.username}' 2>/dev/null)" && [ -n "$user" ]; then + info "identity: $user" +else + fallback_user="$(k config view --minify -o jsonpath='{.users[0].name}' 2>/dev/null || true)" + info "identity: ${fallback_user:-unknown}" +fi +if k get namespace "$NS" -o name >/dev/null 2>&1; then + pass "namespace $NS exists" +else + fail "namespace $NS is missing or not readable" +fi + +if [ -n "$REPO_ROOT" ] && [ -f "$REPO_ROOT/versions.env" ]; then + # shellcheck source=/dev/null + . "$REPO_ROOT/versions.env" + info "expected agent-sandbox pin: ${AGENT_SANDBOX_VERSION:-unknown}" +else + warn "versions.env not found; skipping the pin comparison" +fi + +section "Capabilities (informational; never widen them)" +for check in "create sandboxclaims" "delete sandboxclaims" "create pods/exec"; do + verb="${check%% *}" + resource="${check#* }" + if [ "$(k auth can-i "$verb" "$resource" -n "$NS" 2>/dev/null || echo unknown)" = "yes" ]; then + info "can $verb $resource in $NS: yes" + else + info "can $verb $resource in $NS: no" + fi +done +if [ "$(k auth can-i get secrets -n "$NS" 2>/dev/null || echo unknown)" = "yes" ]; then + warn "RBAC allows reading Secrets; policy still forbids printing Secret data" +else + info "can get secrets in $NS: no" +fi + +section "Agent Sandbox CRDs" +for crd in \ + sandboxtemplates.extensions.agents.x-k8s.io \ + sandboxwarmpools.extensions.agents.x-k8s.io \ + sandboxclaims.extensions.agents.x-k8s.io \ + sandboxes.agents.x-k8s.io; do + if k get crd "$crd" -o name >/dev/null 2>&1; then + pass "CRD $crd" + else + fail "CRD $crd is missing or not readable" + fi +done + +section "Controller" +controller_ns="agent-sandbox-system" +controller_image="$(k -n "$controller_ns" get deployment agent-sandbox-controller \ + -o jsonpath='{.spec.template.spec.containers[0].image}' 2>/dev/null || true)" +if [ -n "$controller_image" ]; then + info "controller image: $controller_image" + controller_tag="${controller_image##*:}" + if [ -n "${AGENT_SANDBOX_VERSION:-}" ] && [ "$controller_tag" != "$AGENT_SANDBOX_VERSION" ]; then + warn "controller tag '$controller_tag' differs from AGENT_SANDBOX_VERSION '$AGENT_SANDBOX_VERSION'" + else + pass "controller version matches versions.env" + fi +else + warn "controller deployment not found in $controller_ns; cannot verify the installed version" +fi + +section "Templates" +templates="$(k -n "$NS" get sandboxtemplates \ + -o jsonpath='{range .items[*]}{.metadata.name}{"\n"}{end}' 2>/dev/null || true)" +if [ -z "$templates" ]; then + warn "no SandboxTemplates in $NS" +fi +if [ -n "$TEMPLATE" ] && ! printf '%s\n' "$templates" | grep -Fxq "$TEMPLATE"; then + fail "SandboxTemplate $TEMPLATE not found in $NS" +fi +for tpl in $templates; do + [ -n "$TEMPLATE" ] && [ "$tpl" != "$TEMPLATE" ] && continue + desc="$(k -n "$NS" get sandboxtemplate "$tpl" \ + -o jsonpath='{range .spec.podTemplate.spec.containers[*]}{.name}{":"}{range .ports[*]}{.containerPort}{","}{end}{";"}{end}' \ + 2>/dev/null || true)" + info "template $tpl containers: ${desc:-}" + case "$desc" in + *sandboxd*|*:8080,*|*:9090,*) + pass "template $tpl exposes a sandboxd runtime (MCP sandboxd-in-cluster)" ;; + *:8888,*) + pass "template $tpl exposes a runtime HTTP port (MCP gateway/direct/in-cluster)" ;; + *:3080,*) + warn "template $tpl is DSH Web only (:3080); MCP execute_command cannot run against it" ;; + *) + warn "template $tpl exposes no known sandbox runtime port" ;; + esac +done + +section "Warm pools" +pools="$(k -n "$NS" get sandboxwarmpools \ + -o jsonpath='{range .items[*]}{.metadata.name}{"\n"}{end}' 2>/dev/null || true)" +if [ -z "$pools" ]; then + warn "no SandboxWarmPools in $NS; claims will provision cold" +fi +if [ -n "$POOL" ] && ! printf '%s\n' "$pools" | grep -Fxq "$POOL"; then + fail "SandboxWarmPool $POOL not found in $NS" +fi +for pool in $pools; do + [ -n "$POOL" ] && [ "$pool" != "$POOL" ] && continue + replicas="$(k -n "$NS" get sandboxwarmpool "$pool" -o jsonpath='{.spec.replicas}' 2>/dev/null || true)" + selector="$(k -n "$NS" get sandboxwarmpool "$pool" -o jsonpath='{.status.selector}' 2>/dev/null || true)" + if [ -n "$selector" ]; then + ready="$(k -n "$NS" get sandboxes -l "$selector" \ + -o jsonpath='{range .items[*]}{.status.conditions[?(@.type=="Ready")].status}{"\n"}{end}' \ + 2>/dev/null | grep -c '^True$' || true)" + else + ready="$(k -n "$NS" get sandboxes -l agents.x-k8s.io/warm-pool-sandbox \ + -o jsonpath='{range .items[*]}{.status.conditions[?(@.type=="Ready")].status}{"\n"}{end}' \ + 2>/dev/null | grep -c '^True$' || true)" + fi + replicas="${replicas:-0}" + info "pool $pool: $ready/$replicas warm spares Ready" + if [ "$replicas" -eq 0 ]; then + warn "pool $pool has spec.replicas=0; signup will cold-provision" + elif [ "$ready" -lt "$replicas" ]; then + warn "pool $pool is below its desired spare count" + else + pass "pool $pool is at capacity" + fi +done + +section "Claims" +claims="$(k -n "$NS" get sandboxclaims \ + -o jsonpath='{range .items[*]}{.metadata.name}{"\t"}{.status.conditions[?(@.type=="Ready")].status}{"\t"}{.status.conditions[?(@.type=="Ready")].reason}{"\t"}{.status.sandbox.name}{"\n"}{end}' \ + 2>/dev/null || true)" +if [ -z "$claims" ]; then + warn "no SandboxClaims in $NS" +else + while IFS=$'\t' read -r name ready reason sandbox; do + [ -n "$name" ] || continue + if [ "$ready" != "True" ]; then + warn "claim $name is not Ready (status='${ready:-unknown}', reason='${reason:-unknown}')" + elif [ -z "$sandbox" ]; then + warn "claim $name is Ready but has no bound sandbox" + else + pass "claim $name -> sandbox $sandbox" + fi + done <<< "$claims" +fi + +section "Sandboxes" +sandboxes="$(k -n "$NS" get sandboxes \ + -o jsonpath='{range .items[*]}{.metadata.name}{"\t"}{.spec.operatingMode}{"\t"}{.status.conditions[?(@.type=="Ready")].status}{"\n"}{end}' \ + 2>/dev/null || true)" +if [ -z "$sandboxes" ]; then + warn "no Sandboxes in $NS" +else + while IFS=$'\t' read -r name mode ready; do + [ -n "$name" ] || continue + info "sandbox $name mode=${mode:-Running} ready=${ready:-unknown}" + done <<< "$sandboxes" +fi + +section "Storage and Services" +pvc_count="$(k -n "$NS" get pvc -o name 2>/dev/null | wc -l | tr -d ' ')" +svc_count="$(k -n "$NS" get service -o name 2>/dev/null | wc -l | tr -d ' ')" +info "persistent volume claims: ${pvc_count:-0}" +info "services: ${svc_count:-0}" +[ "${pvc_count:-0}" -gt 0 ] || warn "no PVCs in $NS" +[ "${svc_count:-0}" -gt 0 ] || warn "no Services in $NS" + +section "Gateway" +if k -n "$NS" get deployment dsh-gateway -o name >/dev/null 2>&1; then + pass "gateway deployment dsh-gateway exists" + if k -n "$NS" get service dsh-gateway -o name >/dev/null 2>&1; then + endpoints="$(k -n "$NS" get endpoints dsh-gateway \ + -o jsonpath='{range .subsets[*].addresses[*]}{.ip}{" "}{end}' 2>/dev/null || true)" + if [ -n "$endpoints" ]; then + pass "gateway endpoints: $endpoints" + else + warn "gateway Service has no ready endpoints" + fi + else + warn "gateway Service dsh-gateway is missing" + fi +else + warn "no dsh-gateway deployment; this is a resource-layer-only install" +fi + +section "MCP server" +if [ -n "$MCP_URL" ]; then + if ! command -v curl >/dev/null 2>&1; then + fail "curl is required for --mcp-url" + else + base="${MCP_URL%/}" + base="${base%/mcp}" + for endpoint in healthz readyz; do + body="$(curl -fsS --max-time 5 "$base/$endpoint" 2>/dev/null | head -c 300 || true)" + if [ -n "$body" ]; then + pass "MCP /$endpoint: $body" + else + fail "MCP /$endpoint is unreachable at $base/$endpoint" + fi + done + fi +else + info "no --mcp-url supplied; skipping server health checks" +fi + +printf '\n== Summary ==\nPASS=%d WARN=%d FAIL=%d\n' "$PASS" "$WARN" "$FAIL" +if [ "$FAIL" -gt 0 ]; then + exit 1 +fi +exit 0 diff --git a/AGENTS.md b/AGENTS.md index 71958fc..3eea7f3 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -48,6 +48,7 @@ before changing anything; it encodes the conventions the CI enforces. | examples/dsh-agents-as-a-service/ | The multi-tenant platform example (manifests, gateway, docs, scripts) | | examples/minimal-single-tenant/ | The just-one-DSH-Web quickstart | | scripts/ | check-versions.sh, check-examples.sh, check-manifests.py, set-version.sh | +| .dsh/skills/ | Project-local DSH skills (auto-discovered); dsh-agent-sandbox-platform covers platform operation, MCP limits and extension recipes | | .github/workflows/ | ci.yaml, reusable _build-image.yaml, build-*, release.yaml, e2e-kind.yaml, pr-title.yaml, codeql.yml | ## Commands @@ -75,6 +76,7 @@ needs kind plus the agent-sandbox CRDs (see the example README). | Dockerfile / versions.env | make check; make image (amd64) and a container smoke test | | Workflow | actionlint in CI; keep actions SHA-pinned; no hard-coded versions | | New example | make check; all files described in docs/examples.md | +| Project skill (.dsh/skills/) | make check; frontmatter and size check; shellcheck skill scripts; load it with the skill tool | ## Versioning and releases @@ -136,5 +138,6 @@ needs kind plus the agent-sandbox CRDs (see the example README). - [docs/examples.md](docs/examples.md) - example authoring rules - [docs/gitflow.md](docs/gitflow.md) - branches, PRs, required checks - [docs/versioning.md](docs/versioning.md) - versioning, changelog, releases +- [.dsh/skills/dsh-agent-sandbox-platform/SKILL.md](.dsh/skills/dsh-agent-sandbox-platform/SKILL.md) - platform operations, MCP limits, extension recipes - [CONTRIBUTING.md](CONTRIBUTING.md) - development setup - [SECURITY.md](SECURITY.md) - vulnerability reporting and threat notes diff --git a/README.md b/README.md index bb34e1a..75e3f34 100644 --- a/README.md +++ b/README.md @@ -132,6 +132,15 @@ The gateway is a component of the multi-tenant example and has its own [README](examples/dsh-agents-as-a-service/gateway/README.md) (API, configuration, auth/SSO, limitations). +## Agent skills + +DSH discovers project skills from `.dsh/skills//SKILL.md`. The repository +ships [`dsh-agent-sandbox-platform`](.dsh/skills/dsh-agent-sandbox-platform/SKILL.md), +which covers operating and extending the platform, the optional +[agent-sandbox MCP server](https://github.com/kubernetes-sigs/agent-sandbox/blob/main/clients/integrations/mcp-server/README.md), +and a read-only cluster preflight. Runbooks, the MCP reference, the extension +guide and troubleshooting notes live beside `SKILL.md`. + ## Repository layout | Path | Purpose | @@ -144,6 +153,7 @@ auth/SSO, limitations). | [`examples/dsh-agents-as-a-service/`](examples/dsh-agents-as-a-service/) | Multi-tenant platform example: `manifests/`, `gateway/`, `docs/`, `scripts/`. | | [`examples/minimal-single-tenant/`](examples/minimal-single-tenant/) | Single-sandbox quickstart. | | [`scripts/`](scripts/) | Repository checks: versions, example layout, manifest YAML, version bump. | +| [`.dsh/skills/`](.dsh/skills/) | Project-local DSH skills, auto-discovered from the git root. | | [`.github/`](.github/) | CI, release automation, dependency updates, PR/issue templates. | | [`.env.sample`](.env.sample) | Documented local configuration (copy to `.env`; never commit it). | diff --git a/docs/README.md b/docs/README.md index 75e94da..3fe8d88 100644 --- a/docs/README.md +++ b/docs/README.md @@ -17,6 +17,15 @@ gets a DSH Web container running before anything else. | [gitflow.md](gitflow.md) | Branches, protected `main`, squash merges, Conventional PR titles, required CI checks, release branches, hotfixes and the one-time GitHub settings checklist. | | [versioning.md](versioning.md) | Release identity (`DSH ref` + optional `-rN`), why the current identity is `dsh-v0.2.1-rc.3-r3`, channels, compatibility matrix, bumping pins, the release checklist, rollback, git-cliff and digest pinning. | +## Project skills + +DSH discovers project skills from `/.dsh/skills//SKILL.md` +(the nearest `.git` ancestor). This repository ships one: + +| Skill | Read it for | +| --- | --- | +| [dsh-agent-sandbox-platform](../.dsh/skills/dsh-agent-sandbox-platform/SKILL.md) | Operating and extending the agents-as-a-service platform, agent-sandbox MCP limits and deployment, and a read-only preflight. | + ## Example documentation | Document | Read it for |