mirror of
https://github.com/sredevopsorg/deepseek-kubernetes-sandbox.git
synced 2026-11-03 01:13:24 +00:00
a09d4ac340
* fix(example): keep agents usable when DSH's inner sandbox has no backend DSH confines bash commands itself (bubblewrap, else Landlock) and fails closed with SANDBOX_UNAVAILABLE when no runner is usable. In this hardened pod - non-root, capabilities dropped, seccomp RuntimeDefault - bubblewrap cannot start (it needs CAP_SYS_ADMIN) and the pod exposes no Landlock runner, so the shipped workspace-write default made every command fail: sandbox mode "workspace-write" is requested but no sandbox backend is usable The DSH base bundle at the pinned ref documents DSH_PERMISSION_MODE as the explicit deployment override, so set it to danger-full-access in both sandbox templates. The per-user pod is the isolation boundary in this example (its own pod, PVC and Service; non-root; all capabilities dropped), so a working agent is worth more than an inner file policy that cannot be enforced. Note that danger-full-access also turns approval prompts off. Trade-off documented rather than hidden: an agent command can now read its own DSH_HOME (including .credentials.yaml) and write anywhere in the container. Hosts where a runner works should keep workspace-write or read-only. - main-image docs: DSH_PERMISSION_MODE row plus a note on the runners and the fail-closed behavior - example README: limits bullet and a troubleshooting row with the in-pod backend check and how to restore a confined mode - architecture: tenant-isolation bullet extended - skill runbook/troubleshooting: operation guidance; never grant CAP_SYS_ADMIN or seccompProfile: Unconfined to "fix" this without an explicit decision Verified: make check (14 manifests), make lint, YAML assertions on both templates, and the override expression at the pinned dsh-v0.2.1-rc.5 base bundle (mode: !!js process.env.DSH_PERMISSION_MODE ?? 'workspace-write'). * fix(image): install bubblewrap for DSH's command sandbox DSH's Linux sandbox runner chain is bubblewrap, then Landlock, and it fails closed with SANDBOX_UNAVAILABLE when neither works. The image shipped neither, so a sandboxed agent had no way to execute confined commands. Add bubblewrap to the runtime stage. Debian ships it unsetuid (0.12.0), so it still needs unprivileged user namespaces plus the namespace/mount syscalls: in a hardened pod (all capabilities dropped, RuntimeDefault seccomp) it usually cannot create one, and DSH then falls back to Landlock or, failing that, to the DSH_PERMISSION_MODE=danger-full-access override the templates already set. Where the host allows an unprivileged user namespace, the first-choice runner now exists, so the confined modes can be restored. Docs corrected to match: bubblewrap is no longer "absent"; the blocker is the unsetuid binary's namespace requirement, not a missing CAP_SYS_ADMIN grant. Verified: make check, make lint; the package exists in the pinned base (node:24-trixie-slim, bubblewrap 0.12.0-1~deb13u1); a throwaway container with the pod's flags (--user 1000 --cap-drop ALL --security-opt no-new-privileges) still cannot create a namespace, which is why the override stays. No local image build was run. * ci(image): build and smoke-test the main image on PRs that touch it The Dockerfile and the pins in versions.env could only be validated by a merge to main or by running `make image` by hand; neither gives a PR-time signal. Add ci-image.yaml: on pull requests that change Dockerfile, .dockerignore or versions.env (and on manual dispatch) it builds linux/amd64 natively with the pins sourced from versions.env - no registry login, no push - loads the image locally and smoke-tests it: - the packaged CLI runs and reports the pinned version - the container is the non-root node user (1000) and owns /workspace + DSH_HOME - the bubblewrap command-sandbox runner is present - the OCI version label matches versions.env - a container serves /favicon.svg (the same unauthenticated probe the shipped manifests use), with logs filtered so the launch token is never printed Path-filtered like the other on-demand checks, so it is not a required status check; the published multi-arch build stays in build-main.yaml and release.yaml. Docs updated in the same change: AGENTS.md repository map and testing table, docs/gitflow.md checks section, docs/main-image.md build section. Verified: actionlint 1.7.12 and yamllint 1.38.0 (CI versions) clean over the whole repository, plus make check and make lint. --------- Co-authored-by: SREDevOpsAgents <agents@sredevops.org>
7.8 KiB
7.8 KiB
AGENTS.md
Guidance for AI coding agents (and humans) working in this repository. Read it before changing anything; it encodes the conventions the CI enforces.
What this repository is
- The root is the primary artifact. The Dockerfile packages DeepSeek Harness Web (dsh web) as a slim, non-root, multi-arch image. Everything else exists to support it or to demonstrate an implementation on top of it.
- Every implementation is a self-contained example under examples//. There is exactly one example for the agents-as-a-service platform; do not split it into parallel directories.
- versions.env is the single source of truth for the DSH ref, pnpm, the control-plane versions and the image names.
- Docs describe the shipped behavior, not intentions. If you change a manifest, port, env var or workflow, update the docs in the same change.
- CHANGELOG.md is generated by git-cliff from Conventional Commits. Never edit it by hand.
Golden rules
- Keep the Dockerfile working for both linux/amd64 and linux/arm64; it must stay the first-class artifact.
- Never hard-code a version, image tag or registry path. Edit versions.env and run make set-version when bumping.
- Never commit a real secret or usable credential. Secret-shaped YAML lives in *.yaml.example files; keys are documented in .env.sample only.
- Every manifest under examples//manifests/ must parse and pass a server-side dry-run in CI. Keep manifests numbered (NN-.yaml), and keep exposure variants under manifests/exposure// documented by the single manifests/exposure/README.md (variants are mutually exclusive).
- Do not edit CHANGELOG.md, and do not move or delete published tags. Releasing is documented in docs/versioning.md.
- Keep the gateway HTTP/WebSocket contract stable unless the change is explicitly about it; clients and examples depend on it.
Repository map
| Path | What lives there |
|---|---|
| Dockerfile | Main DSH Web image (build + runtime stages, entrypoint) |
| versions.env | Every pinned version and image name |
| Makefile | Canonical local entrypoint |
| build.sh | Deprecated shim that forwards to make |
| docs/ | main-image.md, examples.md, gitflow.md, versioning.md |
| examples/ | One implementation per directory (contract in docs/examples.md) |
| examples/dsh-agents-as-a-service/ | The multi-tenant platform example (manifests, gateway, docs, scripts) |
| examples/minimal-single-tenant/ | The just-one-DSH-Web quickstart |
| scripts/ | check-versions.sh, check-examples.sh, check-manifests.py, set-version.sh |
| .dsh/skills/ | Project-local DSH skills (auto-discovered); dsh-agent-sandbox-platform covers platform operation, MCP limits and extension recipes |
| .github/workflows/ | ci.yaml (lint, dependency-review), ci-python.yaml, ci-manifests.yaml, ci-image.yaml (amd64 build + smoke test on Dockerfile/versions.env changes), reusable _build-image.yaml, build-*, release.yaml, e2e-kind.yaml, pr-title.yaml, codeql.yml |
Commands
make check # versions, example layout and manifest YAML (run before every commit)
make test # gateway unit tests (109+ tests)
make lint # ruff + shellcheck (+ yamllint/actionlint when installed)
make image # build the main DSH Web image (slow: clones and builds DSH)
make gateway # build the example gateway image (fast)
make e2e # full kind walkthrough; needs a cluster with agent-sandbox
make changelog # regenerate CHANGELOG.md with git-cliff (release only)
make release-dry-run
make set-version DSH_VERSION=dsh-vX.Y.Z RELEASE_REVISION=rN
No cluster is needed for check, test, lint, gateway or image. Only make e2e needs kind plus the agent-sandbox CRDs (see the example README).
Testing expectations per change
| Change | Minimum verification |
|---|---|
| Documentation only | make check; verify relative links |
| Example manifest | make check; kubectl apply --dry-run=server in CI |
| Optional hardening (gVisor, nodeSelector, NetworkPolicy, provider keys) | make check; server dry-run in CI; manual verification on a capable cluster |
| Gateway code | make test; make lint; add or update tests |
| Dockerfile / versions.env | make check; the CI build-amd64 job (ci-image.yaml) builds and smoke-tests the image on the PR; run make image locally only when you need the image |
| Workflow | actionlint in CI; keep actions SHA-pinned; no hard-coded versions |
| New example | make check; all files described in docs/examples.md |
| Project skill (.dsh/skills/) | make check; frontmatter and size check; shellcheck skill scripts; load it with the skill tool |
Versioning and releases
- Release identity = DSH_VERSION, optionally with a repository revision suffix for re-releases at the same DSH ref (current identity: dsh-v0.2.1-rc.5).
- The current identity is dsh-v0.2.1-rc.5 (stored in versions.env). It is the first release at this DSH ref, so it carries no revision suffix; a repository-only change at the same ref publishes -r2.
- Channels: release tags are immutable; latest follows the newest release; main is the development channel for both image packages.
- A broken release is fixed by publishing -r(N+1), never by re-tagging.
- The full procedure, compatibility table and rollback rules are in docs/versioning.md.
Branch and PR conventions
- Trunk-based with release branches, documented in docs/gitflow.md.
- Branch names: feat/, fix/, docs/, chore/, refactor/, test/.
- Conventional Commit subjects and PR titles (feat(scope): ..., fix(scope): ...).
- PRs are squash-merged; main stays linear and protected.
- Required checks: lint, python, manifests, pr-title, CodeQL, dependency-review.
- Keep diffs scoped; update docs with the code; never mix a version bump with unrelated changes.
Adding an example
- Read docs/examples.md and examples/README.md.
- Create examples// with README.md, manifests/NN-*.yaml, optional manifests/exposure// (plus README), optional components with their own Dockerfile/README/tests, and scripts/ (executable, idempotent, self-cleaning).
- Publish component images under ////.
- Add the example to examples/README.md; scripts/check-examples.sh enforces the layout.
Security notes
- AUTH_PROXY_HEADER is opt-in. Only enable it when the gateway Service is reachable exclusively through the authenticating proxy; otherwise any client can spoof X-Forwarded-User.
- The example gateway reads the DSH launch token from pod logs. Treat pod logs as sensitive and keep its RBAC namespace-scoped.
- The example shares provider keys (dsh-provider-keys plus the optional dsh-provider-keys-extra) across sandboxes; do not describe it as tenant-isolated.
- gVisor, nodeSelector and the NetworkPolicies are opt-in and are not active in the base manifests; do not describe the base example as hardened.
- TLS terminates outside the gateway; set COOKIE_SECURE=true for HTTPS and keep the base manifest safe for local HTTP.
Ask before
- Changing release workflows, tag formats or published image names.
- Editing the gateway's public routes or auth semantics.
- Adding a new top-level tool, service or dependency to the main image.
- Rewriting generated files (CHANGELOG.md) or git history.
Where to look first
- docs/main-image.md - container contract and tags
- docs/examples.md - example authoring rules
- docs/gitflow.md - branches, PRs, required checks
- docs/versioning.md - versioning, changelog, releases
- .dsh/skills/dsh-agent-sandbox-platform/SKILL.md - platform operations, MCP limits, extension recipes
- CONTRIBUTING.md - development setup
- SECURITY.md - vulnerability reporting and threat notes