mirror of
https://github.com/sredevopsorg/deepseek-kubernetes-sandbox.git
synced 2026-11-03 09:23:25 +00:00
feat(example): start DSH's folder picker on the persisted workspace (#36)
The directory picker has no start-path setting: it lists $HOME whenever it opens. With the image default HOME=/home/node the folder explorer started on a throwaway directory, so point HOME at the PVC-backed /workspace in both sandbox templates. `~` and $HOME-relative tool caches/config follow; DSH_HOME stays /home/node/.dsh, so sessions, config and credentials are unaffected. Document the trade-off in the image contract and both examples, add troubleshooting for sandboxes whose pod env was stamped before the change, and assert HOME/~ in the e2e walkthrough. Co-authored-by: SREDevOpsAgents <agents@sredevops.org>
This commit is contained in:
committed by
GitHub
parent
a09d4ac340
commit
82ee0d0cdb
@@ -125,6 +125,15 @@ kubectl -n "$NS" patch sandboxwarmpool "$POOL" --type merge -p '{"spec":{"replic
|
||||
throwaway pod, patch the policy, then resume.
|
||||
- Agent files live at `/workspace`, the same PVC mounted with
|
||||
`subPath: workspace`; it survives suspend/resume and dies with the PVC.
|
||||
- The templates set `HOME=/workspace` (the image default is `/home/node`). DSH's
|
||||
directory picker lists `$HOME` when it opens and has no start-path setting, so
|
||||
this is what makes the folder explorer start on the persisted workspace; `~`
|
||||
and `$HOME`-relative tool caches/config also resolve there. `DSH_HOME` remains
|
||||
`/home/node/.dsh`, so state and credentials stay out of the workspace. Pod env
|
||||
is stamped at sandbox creation: a sandbox created before the override keeps
|
||||
the old `HOME` until it is recreated (or its `podTemplate` is patched and the
|
||||
sandbox resumed). Verify with
|
||||
`kubectl -n "$NS" exec "$POD" -- printenv HOME`.
|
||||
- DSH confines bash with its own file sandbox (`bubblewrap`, else Landlock) and
|
||||
fails closed with `SANDBOX_UNAVAILABLE` when neither works. The image ships
|
||||
`bubblewrap`, but it is unsetuid and needs an unprivileged user namespace that
|
||||
|
||||
@@ -19,6 +19,7 @@ events, then compare desired vs observed state. Keep output bounded. Never
|
||||
| `CrashLoopBackOff`, probe 401 | logs, `describe` | Probe path is `/`; use `/favicon.svg`. |
|
||||
| Restart with mode-660 error | logs mention `.credentials.yaml` | kubelet fsGroup widened the file; suspend, `chmod 600` from a throwaway pod, patch `fsGroupChangePolicy: OnRootMismatch`, resume. |
|
||||
| Every bash command fails with `SANDBOX_UNAVAILABLE` | DSH tool result names the requested mode and backends | DSH's own command sandbox has no usable runner: `bubblewrap` ships in the image but is unsetuid and needs an unprivileged user namespace this hardened pod does not grant, and no Landlock runner is available. The example template sets `DSH_PERMISSION_MODE=danger-full-access`; do not grant capabilities to restore it unless the operator explicitly accepts that downgrade. |
|
||||
| Folder picker opens on `/home/node`, not the workspace | `kubectl exec <pod> -- printenv HOME` | The sandbox predates the template's `HOME=/workspace` and pod env is stamped at creation. Recreate the sandbox (PVC survives) or patch its `podTemplate` with `HOME`, then resume. Workspace dotfiles (`.cache`, `.config`, `.npm`, `.agents`) are the expected trade-off of the override, not a fault. |
|
||||
| PVC `Pending` | `kubectl describe pvc`, StorageClass | No default provisioner, access mode, zone binding. |
|
||||
| Launch token not found | `kubectl logs <pod> --tail=2000 | grep 'dsh web:'` | Pod not ready yet, or logs rotated. Do not guess tokens; wait for readiness. |
|
||||
| Sandbox reports `Finished=True` | pod phase, `kubectl get events` | The backing Pod reached `Failed` (eviction, spot reclaim) and `failedPodPolicy` defaults to `Retain`. Delete the Pod to force a replacement, or set `Recreate` (v1.0.8+) on the Sandbox/template; the claim and PVCs survive either way. |
|
||||
|
||||
@@ -80,11 +80,26 @@ The entrypoint is `tini -- /usr/local/bin/dsh-entrypoint`:
|
||||
| `DSH_TRUSTED_HOSTS` | empty | Extra `Host`/`Origin` allow-list entries, comma- or space-separated. `localhost`/`127.0.0.1` are always trusted. Required when a proxy forwards the browser's `Host`. |
|
||||
| `DEEPSEEK_API_KEY` | empty | Provider credential read by DSH. Inject it with `-e`/`envFrom`; never bake it into the image. Only model calls fail when it is missing. |
|
||||
| `DSH_HOME` | `/home/node/.dsh` | DSH state directory (conversations, sessions, config, credentials). Mount a volume here to persist it. |
|
||||
| `HOME` | `/home/node` | OS home: where `~` resolves and where tools keep caches and config. DSH's directory picker has no start-path setting, so it lists `$HOME` when opened. The shipped sandbox templates override it to `/workspace` so the picker opens on the persisted workspace; the trade-offs are below. |
|
||||
| `DSH_PERMISSION_MODE` | `workspace-write` | Deployment default for DSH's file policy: `read-only`, `workspace-write`, or `danger-full-access` (which also turns approval prompts off). The confined modes need `bubblewrap` or Landlock inside the container; where neither is usable every bash command fails closed with `SANDBOX_UNAVAILABLE`. See the note below. |
|
||||
|
||||
The image also sets `NODE_ENV=production`, `HOME=/home/node`, and a `PATH` that
|
||||
includes `/app/node_modules/.bin`.
|
||||
|
||||
`HOME` decides where DSH's folder picker starts. DSH has no start-path setting:
|
||||
the picker's browse backend lists `$HOME` when the dialog opens and labels that
|
||||
listing *Home*. With the image default (`/home/node`) that listing is outside a
|
||||
mounted volume; the example sandbox templates
|
||||
([minimal](../examples/minimal-single-tenant/manifests/sandbox-template.yaml),
|
||||
[agents as a service](../examples/dsh-agents-as-a-service/manifests/10-sandboxtemplate.yaml))
|
||||
set `HOME=/workspace` so the picker opens on the persisted agent workspace. `~`
|
||||
and every tool that keeps caches or config under `$HOME` (`npm`/`pnpm`, `uv`,
|
||||
`git`, `gh`, DSH's global `~/.agents` instructions and skills) follow, so those
|
||||
dotfiles then live on the workspace volume too. `DSH_HOME` is separate and stays
|
||||
`/home/node/.dsh`, so sessions, config and credentials stay out of the
|
||||
workspace. An existing sandbox stamps this environment at pod creation, so the
|
||||
override only applies to sandboxes created after it was added.
|
||||
|
||||
DSH confines bash commands itself (a second layer inside whatever container
|
||||
sandbox you already run). Its Linux runners are `bubblewrap` and then Landlock,
|
||||
and it fails closed rather than running unconfined. The image ships
|
||||
@@ -126,7 +141,7 @@ procedure.
|
||||
| --- | --- | --- |
|
||||
| Exposed port | `3080/tcp` | `EXPOSE 3080`; one port serves the UI, assets, API, SSE and WebSocket traffic. |
|
||||
| State volume | `/home/node/.dsh` | Declared with `VOLUME`; use a named volume (Docker) or PVC (Kubernetes). |
|
||||
| Working directory | `/workspace` | The agent workspace. Mount a volume here if agents should keep files; the agents-as-a-service example mounts the same PVC's `workspace` subPath here. |
|
||||
| Working directory | `/workspace` | The agent workspace. Mount a volume here if agents should keep files; the agents-as-a-service example mounts the same PVC's `workspace` subPath here and points `HOME` at it, so DSH's folder picker opens on it. |
|
||||
| Stop signal | `SIGTERM` | `tini` forwards it to `dsh web`; allow a graceful shutdown period. |
|
||||
| User | `node`, UID/GID `1000` | The process is non-root. A PVC mounted at `DSH_HOME` must be writable by UID 1000. |
|
||||
|
||||
|
||||
@@ -201,6 +201,7 @@ kubectl -n dsh-demo port-forward "pod/$SB" 18081:3080
|
||||
## 4. Write state on the PVC
|
||||
|
||||
```bash
|
||||
kubectl -n dsh-demo exec "$SB" -- printenv HOME # /workspace
|
||||
kubectl -n dsh-demo exec "$SB" -- sh -c 'echo "remember me" > /home/node/.dsh/marker.txt'
|
||||
kubectl -n dsh-demo exec "$SB" -- sh -c 'echo "remember me too" > /workspace/marker.txt'
|
||||
```
|
||||
@@ -210,6 +211,20 @@ kubectl -n dsh-demo exec "$SB" -- sh -c 'echo "remember me too" > /workspace/mar
|
||||
separate, so `.credentials.yaml` and the session store are never part of the
|
||||
agent-visible workspace.
|
||||
|
||||
The template also sets `HOME=/workspace`. DSH's directory picker (the folder
|
||||
explorer used to choose a workspace) has no start-path setting: when it opens,
|
||||
its browse backend lists `$HOME`, so with the image default (`/home/node`, which
|
||||
is not on a volume) it would start outside the persisted tree. Pointing `HOME`
|
||||
at the workspace makes the picker open on it (labelled *Home* in its breadcrumbs)
|
||||
and makes `~` in agent commands mean `/workspace`. The trade-off is that tools
|
||||
keeping caches or config under `$HOME` (`npm`/`pnpm`, `uv`, `git`, `gh`, and
|
||||
DSH's global `~/.agents` instructions and skills) write dotfiles onto the PVC
|
||||
inside the agent workspace, where the agent and the file explorer can see them.
|
||||
`DSH_HOME` is explicit (`/home/node/.dsh`), so sessions, config and credentials
|
||||
are unaffected. Pod env is stamped when a sandbox is created: a sandbox from
|
||||
before this setting keeps `HOME=/home/node` until it is recreated (see
|
||||
[Troubleshooting](#troubleshooting)).
|
||||
|
||||
## 5. Suspend: delete the pod, keep the PVC + Service
|
||||
|
||||
```bash
|
||||
@@ -599,6 +614,8 @@ is non-recursive, so the optional manifest is not applied by the base step.
|
||||
| Sandbox cannot reach a model API after applying the optional NetworkPolicy | The policy allows DNS and public TCP `443` only; a proxy, custom base URL, port `80` or non-443 endpoint needs an explicit egress rule |
|
||||
| Gateway cannot reach a sandbox after applying an exposure NetworkPolicy | Check that the CNI enforces NetworkPolicy and that the Traefik/oauth2-proxy pod labels match; kubelet probes may also need the node CIDR |
|
||||
| New provider key not visible in a sandbox | Pod env is stamped at creation: delete the warm-pool spares and recreate claimed pods. A key saved in DSH's credential store also wins over env |
|
||||
| Folder picker opens on `/home/node` instead of the workspace | The sandbox was created before the template set `HOME=/workspace`, and pod env is stamped at creation. Verify with `kubectl -n dsh-demo exec "$SB" -- printenv HOME`; patch the sandbox's `podTemplate` with the env (or recreate the sandbox — the PVC keeps DSH state and workspace) and resume |
|
||||
| File explorer and `ls /workspace` show `.cache`, `.config`, `.local`, `.npm`, `.agents` | Expected with `HOME=/workspace`: `~`-relative tool caches and DSH's global agent instructions/skills live on the PVC inside the workspace. Remove the `HOME` env from the template (and recreate the sandbox) if you would rather keep the picker on the ephemeral image home; point caches elsewhere per tool if only they are unwanted |
|
||||
| Sandbox reports `Finished=True` and every gateway request fails | The sandbox Pod reached phase `Failed` (node eviction, spot reclaim) and the default `Retain` keeps it. Delete the Pod so the controller recreates it, or set `failedPodPolicy: Recreate` (agent-sandbox v1.0.8+; see optional hardening) |
|
||||
|
||||
## Limitations
|
||||
|
||||
@@ -166,6 +166,15 @@ volume with `subPath: workspace`, which keeps `.credentials.yaml` out of the age
|
||||
The idle sweeper never touches the volume; deleting the claim still garbage-collects it, so
|
||||
`POST /recreate` wipes state and workspace together.
|
||||
|
||||
**Workspace as home.** The template sets `HOME=/workspace`. DSH has no start-path setting for its
|
||||
directory picker: the browse backend lists `$HOME` when the dialog opens, so with the image default
|
||||
(`/home/node`, off the PVC) the picker would start on a throwaway directory. The override makes the
|
||||
picker open on the persisted workspace and makes `~` resolve there; the cost is that tool caches
|
||||
and config kept under `$HOME` (`npm`/`pnpm`, `uv`, `git`, `gh`, and DSH's global `~/.agents`
|
||||
instructions and skills) become dotfiles in the workspace. `DSH_HOME` stays `/home/node/.dsh`, so
|
||||
sessions, config and credentials remain outside the agent-visible tree. Pod env is stamped when a
|
||||
sandbox is created, so an existing sandbox keeps the old `HOME` until it is recreated.
|
||||
|
||||
**Ownership boundary.** The gateway shares its namespace with claims created outside it — the
|
||||
example's resource-layer walkthrough applies `dsh-alice` by hand, and every claim is named
|
||||
`dsh-<user>`. Claims the gateway creates carry `aaas.example.com/managed-by=dsh-gateway`; the
|
||||
|
||||
@@ -89,6 +89,23 @@ spec:
|
||||
name: dsh-provider-keys-extra
|
||||
optional: true
|
||||
env:
|
||||
# Point the OS home at the persisted agent workspace. DSH's
|
||||
# directory picker (the folder explorer used to choose a workspace)
|
||||
# has no start-path setting: when it opens it lists $HOME, and
|
||||
# os.homedir() follows this variable. The image default is
|
||||
# /home/node, which is not on a volume, so the picker would open on
|
||||
# a throwaway directory; /workspace is the PVC-backed WORKDIR. `~`
|
||||
# in agent commands resolves here too. Trade-off: tools that keep
|
||||
# caches/config under $HOME (npm/pnpm, uv, git, gh, and DSH's global
|
||||
# ~/.agents instructions and skills) now write dotfiles onto the PVC
|
||||
# inside the agent workspace, so they are visible to the agent and
|
||||
# in the file explorer. DSH_HOME is explicit (/home/node/.dsh), so
|
||||
# sessions, config and credentials are unaffected. Existing
|
||||
# sandboxes stamp pod env at creation and keep the old HOME until
|
||||
# recreated. Keep in sync with the minimal variant:
|
||||
# examples/minimal-single-tenant/manifests/sandbox-template.yaml.
|
||||
- name: HOME
|
||||
value: /workspace
|
||||
# Empty => DSH derives links from the request Host, which the
|
||||
# gateway forwards (plus X-Forwarded-*). Point this at the
|
||||
# gateway base URL if you want hard-coded absolute links instead.
|
||||
|
||||
@@ -72,6 +72,15 @@ pass "claim Ready in ${ELAPSED}s, adopted $SB"
|
||||
|
||||
echo "=== 3. write state on the PVC"
|
||||
POD="$SB"
|
||||
# HOME must point at the PVC-backed workspace: DSH's directory picker lists
|
||||
# $HOME when it opens and has no start-path setting, so this is what makes the
|
||||
# folder explorer start on the persisted tree instead of the ephemeral image
|
||||
# home. It also makes ~ resolve to /workspace inside agent commands.
|
||||
HOME_VALUE=$(kubectl -n "$NS" exec "$POD" -- printenv HOME)
|
||||
[ "$HOME_VALUE" = "/workspace" ] || fail "HOME is '$HOME_VALUE', expected /workspace"
|
||||
TILDE=$(kubectl -n "$NS" exec "$POD" -- sh -c 'cd ~ && pwd')
|
||||
[ "$TILDE" = "/workspace" ] || fail "~ resolves to '$TILDE', expected /workspace"
|
||||
pass "HOME=$HOME_VALUE and ~ resolve to the workspace"
|
||||
kubectl -n "$NS" exec "$POD" -- sh -c 'echo "remember me" > /home/node/.dsh/marker.txt'
|
||||
pass "state marker written in $POD"
|
||||
# The agent workspace must be the dsh-home PVC's workspace/ subPath, writable
|
||||
|
||||
@@ -101,6 +101,25 @@ kubectl -n dsh-demo port-forward "pod/$SB" 18081:3080
|
||||
> example or a reachable Service instead; see
|
||||
> [optional hardening](../dsh-agents-as-a-service/docs/optional-hardening.md).
|
||||
|
||||
## Folder picker and the workspace
|
||||
|
||||
The template sets `HOME=/workspace`. DSH's directory picker (the folder explorer
|
||||
used to choose a workspace) has no start-path setting: it lists `$HOME` when it
|
||||
opens, so with the image default (`/home/node`) it would start on a directory that
|
||||
is not persisted. Pointing `HOME` at the PVC-backed workspace makes the picker
|
||||
open there (labelled *Home* in its breadcrumbs) and makes `~` in agent commands
|
||||
resolve to `/workspace`. The trade-off is that `$HOME`-relative tool caches and
|
||||
config (`.cache`, `.config`, `.local`, `.npm`, `.agents`) are written into the
|
||||
workspace on the PVC. DSH state is unaffected: `DSH_HOME` stays
|
||||
`/home/node/.dsh`. The rationale and the list of affected paths are in
|
||||
[`10-sandboxtemplate.yaml`](../dsh-agents-as-a-service/manifests/10-sandboxtemplate.yaml).
|
||||
|
||||
Verify inside a running sandbox:
|
||||
|
||||
```bash
|
||||
kubectl -n dsh-demo exec "$SB" -- printenv HOME # /workspace
|
||||
```
|
||||
|
||||
## Network policy and optional hardening
|
||||
|
||||
The template omits `networkPolicyManagement`, so the agent-sandbox controller
|
||||
|
||||
@@ -69,6 +69,21 @@ spec:
|
||||
name: dsh-provider-keys-extra
|
||||
optional: true
|
||||
env:
|
||||
# Point the OS home at the persisted agent workspace. DSH's
|
||||
# directory picker (the folder explorer used to choose a workspace)
|
||||
# lists $HOME when it opens and has no start-path setting, so with
|
||||
# the image default (/home/node, not on a volume) it would open on a
|
||||
# throwaway directory. /workspace is the PVC-backed WORKDIR, so the
|
||||
# picker opens on the tree that survives suspend/resume, and `~` in
|
||||
# agent commands resolves there. Trade-off: tools that keep caches
|
||||
# or config under $HOME (npm/pnpm, uv, git, gh, DSH's global
|
||||
# ~/.agents instructions and skills) write dotfiles onto the PVC
|
||||
# inside the agent workspace. DSH_HOME is explicit
|
||||
# (/home/node/.dsh), so sessions, config and credentials are
|
||||
# unaffected. Rationale in full at
|
||||
# examples/dsh-agents-as-a-service/manifests/10-sandboxtemplate.yaml.
|
||||
- name: HOME
|
||||
value: /workspace
|
||||
# Empty => DSH derives links from the request Host. Keep it empty
|
||||
# when fronting the sandbox with a proxy/gateway.
|
||||
- name: DSH_PUBLIC_URL
|
||||
|
||||
Reference in New Issue
Block a user