ci(workflows): streamline CI and remove redundant builds (#31)

Streamline the CI and release workflows:

- ci.yaml is now PR-only (no push-to-main re-run) and keeps only lint and
  dependency-review; the gateway-image local build/smoke job is removed.
- Split the path-sensitive jobs into ci-python.yaml and ci-manifests.yaml,
  each using native on.pull_request.paths filtering so unrelated PRs skip them.
- Drop the duplicate check-manifests.py run from the lint job.
- release.yaml: skip the tests job steps for tags already on main (where PR CI
  already ran), keeping them only for release-branch tags.

Co-authored-by: SREDevOpsAgents <agents@sredevops.org>

.github/workflows/ci-manifests.yaml

@@ -0,0 +1,80 @@
+# Manifest shape validation and server-side dry-run. Split from ci.yaml so this
+# job can use native on.pull_request.paths filtering and skip PRs that do not
+# touch manifests, scripts or versions.env. The job name is the required
+# branch-protection check `manifests`.
+name: CI (manifests)
+
+on:
+  pull_request:
+    paths:
+      - examples/**/manifests/**
+      - scripts/check-manifests.py
+      - versions.env
+      - .github/workflows/ci-manifests.yaml
+  workflow_dispatch:
+
+permissions:
+  contents: read
+
+concurrency:
+  group: ci-manifests-${{ github.workflow }}-${{ github.ref }}
+  cancel-in-progress: true
+
+jobs:
+  manifests:
+    name: manifests
+    runs-on: ubuntu-latest
+    timeout-minutes: 25
+    steps:
+      - name: Checkout
+        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+        with:
+          persist-credentials: false
+      - name: Create kind cluster
+        uses: helm/kind-action@1544676c07bb3570fcb70166beaa07c4528fd4aa # v1.15.1
+        with:
+          version: v0.33.0
+          cluster_name: dsh-ci
+      - name: Read versions.env
+        run: |
+          set -euo pipefail
+          while IFS= read -r line; do
+            case "$line" in ''|'#'*) continue ;; esac
+            echo "$line" >> "$GITHUB_ENV"
+          done < versions.env
+      - name: Install agent-sandbox and Gateway API CRDs
+        run: |
+          set -euo pipefail
+          kubectl apply -f "https://github.com/kubernetes-sigs/agent-sandbox/releases/download/$AGENT_SANDBOX_VERSION/sandbox-with-extensions.yaml"
+          kubectl wait --for=condition=Established crd/sandboxtemplates.extensions.agents.x-k8s.io --timeout=180s
+          kubectl wait --for=condition=Established crd/sandboxwarmpools.extensions.agents.x-k8s.io --timeout=180s
+          kubectl wait --for=condition=Established crd/sandboxclaims.extensions.agents.x-k8s.io --timeout=180s
+          kubectl wait --for=condition=Established crd/sandboxes.agents.x-k8s.io --timeout=180s
+          kubectl apply -f "https://github.com/kubernetes-sigs/gateway-api/releases/download/$GATEWAY_API_VERSION/standard-install.yaml"
+          kubectl wait --for=condition=Established crd/gateways.gateway.networking.k8s.io --timeout=180s
+          kubectl wait --for=condition=Established crd/httproutes.gateway.networking.k8s.io --timeout=180s
+      - name: Set up Python
+        uses: actions/setup-python@6d4fd22b20465ccc0f7354476588e8df7c9ca0e0 # v7.1.0
+        with:
+          python-version: "3.12"
+      - name: Install check dependencies
+        run: pip install "pyyaml==6.0.3"
+      - name: Validate manifest shape
+        run: python scripts/check-manifests.py
+      - name: Validate manifests (server dry-run)
+        run: |
+          set -euo pipefail
+          kubectl create namespace dsh-demo
+          shopt -s globstar nullglob
+          count=0
+          for file in examples/*/manifests/**/*.yaml; do
+            case "$file" in *.yaml.example) continue ;; esac
+            echo "== $file =="
+            kubectl apply --dry-run=server -f "$file"
+            count=$((count + 1))
+          done
+          if [ "$count" -eq 0 ]; then
+            echo "no manifests were validated" >&2
+            exit 1
+          fi
+          echo "validated $count manifests"

.github/workflows/ci-python.yaml

@@ -0,0 +1,72 @@
+# Gateway unit tests and lint. Split from ci.yaml so this job can use native
+# on.pull_request.paths filtering and skip PRs that do not touch the gateway.
+# The job name is the required branch-protection check `python`.
+name: CI (python)
+
+on:
+  pull_request:
+    paths:
+      - examples/dsh-agents-as-a-service/gateway/**
+      - .github/workflows/ci-python.yaml
+  workflow_dispatch:
+
+permissions:
+  contents: read
+
+concurrency:
+  group: ci-python-${{ github.workflow }}-${{ github.ref }}
+  cancel-in-progress: true
+
+jobs:
+  python:
+    name: python
+    runs-on: ubuntu-latest
+    timeout-minutes: 20
+    defaults:
+      run:
+        working-directory: examples/dsh-agents-as-a-service/gateway
+    steps:
+      - name: Checkout
+        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
+        with:
+          persist-credentials: false
+      - name: Set up Python
+        uses: actions/setup-python@6d4fd22b20465ccc0f7354476588e8df7c9ca0e0 # v7.1.0
+        with:
+          python-version: "3.12"
+          cache: pip
+          cache-dependency-path: |
+            examples/dsh-agents-as-a-service/gateway/requirements.txt
+            examples/dsh-agents-as-a-service/gateway/requirements-dev.txt
+      - name: Install dependencies
+        run: |
+          set -euo pipefail
+          python -m pip install --upgrade pip
+          pip install -r requirements.txt -r requirements-dev.txt
+      - name: Lint
+        run: |
+          set -euo pipefail
+          python -m ruff check .
+          python -m ruff format --check .
+      - name: Unit tests with coverage
+        run: >-
+          python -m pytest -q
+          --cov=. --cov-report=xml:coverage.xml --cov-report=term
+          --junitxml=junit.xml
+      - name: Coverage summary
+        run: |
+          set -euo pipefail
+          {
+            echo "### Gateway test coverage"
+            python -m coverage report
+          } >> "$GITHUB_STEP_SUMMARY"
+      - name: Upload test artefacts
+        if: always()
+        uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
+        with:
+          name: gateway-test-results
+          path: |
+            examples/dsh-agents-as-a-service/gateway/junit.xml
+            examples/dsh-agents-as-a-service/gateway/coverage.xml
+          if-no-files-found: warn
+          retention-days: 7

.github/workflows/ci.yaml

@@ -1,12 +1,11 @@
-# Continuous integration: lint, unit tests, manifest dry-run, gateway image
-# smoke test and dependency review. Job names are the required branch-protection
-# checks documented in docs/gitflow.md.
+# Continuous integration: lint and dependency review. Job names are the required
+# branch-protection checks documented in docs/gitflow.md. The gateway unit tests
+# and the manifest dry-run live in ci-python.yaml / ci-manifests.yaml so each can
+# use native on.pull_request.paths filtering and skip unrelated changes.
 name: CI
 
 on:
   pull_request:
-  push:
-    branches: [main]
   workflow_dispatch:
 
 permissions:
@@ -58,7 +57,6 @@ jobs:
           set -euo pipefail
           ./scripts/check-versions.sh
           ./scripts/check-examples.sh
-          python scripts/check-manifests.py
       - name: Shellcheck
         run: shellcheck -x build.sh scripts/*.sh examples/*/scripts/*.sh
       - name: Actionlint
@@ -77,169 +75,6 @@ jobs:
             --offline --no-progress --verbose
             ./README.md ./CONTRIBUTING.md ./SECURITY.md ./AGENTS.md ./docs ./examples
 
-  python:
-    name: python
-    runs-on: ubuntu-latest
-    timeout-minutes: 20
-    defaults:
-      run:
-        working-directory: examples/dsh-agents-as-a-service/gateway
-    steps:
-      - name: Checkout
-        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
-        with:
-          persist-credentials: false
-      - name: Set up Python
-        uses: actions/setup-python@6d4fd22b20465ccc0f7354476588e8df7c9ca0e0 # v7.1.0
-        with:
-          python-version: "3.12"
-          cache: pip
-          cache-dependency-path: |
-            examples/dsh-agents-as-a-service/gateway/requirements.txt
-            examples/dsh-agents-as-a-service/gateway/requirements-dev.txt
-      - name: Install dependencies
-        run: |
-          set -euo pipefail
-          python -m pip install --upgrade pip
-          pip install -r requirements.txt -r requirements-dev.txt
-      - name: Lint
-        run: |
-          set -euo pipefail
-          python -m ruff check .
-          python -m ruff format --check .
-      - name: Unit tests with coverage
-        run: >-
-          python -m pytest -q
-          --cov=. --cov-report=xml:coverage.xml --cov-report=term
-          --junitxml=junit.xml
-      - name: Coverage summary
-        run: |
-          set -euo pipefail
-          {
-            echo "### Gateway test coverage"
-            python -m coverage report
-          } >> "$GITHUB_STEP_SUMMARY"
-      - name: Upload test artefacts
-        if: always()
-        uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
-        with:
-          name: gateway-test-results
-          path: |
-            examples/dsh-agents-as-a-service/gateway/junit.xml
-            examples/dsh-agents-as-a-service/gateway/coverage.xml
-          if-no-files-found: warn
-          retention-days: 7
-
-  manifests:
-    name: manifests
-    runs-on: ubuntu-latest
-    timeout-minutes: 25
-    steps:
-      - name: Checkout
-        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
-        with:
-          persist-credentials: false
-      - name: Create kind cluster
-        uses: helm/kind-action@1544676c07bb3570fcb70166beaa07c4528fd4aa # v1.15.1
-        with:
-          version: v0.33.0
-          cluster_name: dsh-ci
-      - name: Read versions.env
-        run: |
-          set -euo pipefail
-          while IFS= read -r line; do
-            case "$line" in ''|'#'*) continue ;; esac
-            echo "$line" >> "$GITHUB_ENV"
-          done < versions.env
-      - name: Install agent-sandbox and Gateway API CRDs
-        run: |
-          set -euo pipefail
-          kubectl apply -f "https://github.com/kubernetes-sigs/agent-sandbox/releases/download/$AGENT_SANDBOX_VERSION/sandbox-with-extensions.yaml"
-          kubectl wait --for=condition=Established crd/sandboxtemplates.extensions.agents.x-k8s.io --timeout=180s
-          kubectl wait --for=condition=Established crd/sandboxwarmpools.extensions.agents.x-k8s.io --timeout=180s
-          kubectl wait --for=condition=Established crd/sandboxclaims.extensions.agents.x-k8s.io --timeout=180s
-          kubectl wait --for=condition=Established crd/sandboxes.agents.x-k8s.io --timeout=180s
-          kubectl apply -f "https://github.com/kubernetes-sigs/gateway-api/releases/download/$GATEWAY_API_VERSION/standard-install.yaml"
-          kubectl wait --for=condition=Established crd/gateways.gateway.networking.k8s.io --timeout=180s
-          kubectl wait --for=condition=Established crd/httproutes.gateway.networking.k8s.io --timeout=180s
-      - name: Set up Python
-        uses: actions/setup-python@6d4fd22b20465ccc0f7354476588e8df7c9ca0e0 # v7.1.0
-        with:
-          python-version: "3.12"
-      - name: Install check dependencies
-        run: pip install "pyyaml==6.0.3"
-      - name: Validate manifest shape
-        run: python scripts/check-manifests.py
-      - name: Validate manifests (server dry-run)
-        run: |
-          set -euo pipefail
-          kubectl create namespace dsh-demo
-          shopt -s globstar nullglob
-          count=0
-          for file in examples/*/manifests/**/*.yaml; do
-            case "$file" in *.yaml.example) continue ;; esac
-            echo "== $file =="
-            kubectl apply --dry-run=server -f "$file"
-            count=$((count + 1))
-          done
-          if [ "$count" -eq 0 ]; then
-            echo "no manifests were validated" >&2
-            exit 1
-          fi
-          echo "validated $count manifests"
-
-  gateway-image:
-    name: gateway-image
-    runs-on: ubuntu-latest
-    timeout-minutes: 20
-    steps:
-      - name: Checkout
-        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
-        with:
-          persist-credentials: false
-      - name: Build gateway image
-        run: docker build -t dsh-aaas-gateway:ci examples/dsh-agents-as-a-service/gateway/
-      - name: Smoke-test /healthz without a cluster
-        run: |
-          set -euo pipefail
-          cat > /tmp/kubeconfig <<'EOF'
-          apiVersion: v1
-          kind: Config
-          clusters:
-            - name: dummy
-              cluster:
-                server: https://127.0.0.1:6443
-                insecure-skip-tls-verify: true
-          contexts:
-            - name: dummy
-              context:
-                cluster: dummy
-                user: dummy
-          users:
-            - name: dummy
-              user: {}
-          current-context: dummy
-          EOF
-          docker run -d --name gateway-smoke \
-            -p 18080:8080 \
-            -e NAMESPACE=dsh-demo \
-            -e POOL=dsh-pool \
-            -e KUBECONFIG=/tmp/kubeconfig \
-            -v /tmp/kubeconfig:/tmp/kubeconfig:ro \
-            dsh-aaas-gateway:ci
-          trap 'docker logs gateway-smoke || true; docker rm -f gateway-smoke || true' EXIT
-          code=""
-          for _ in $(seq 1 30); do
-            code=$(curl -s -o /dev/null -w '%{http_code}' http://localhost:18080/healthz || true)
-            [ "$code" = "200" ] && break
-            sleep 1
-          done
-          if [ "$code" != "200" ]; then
-            echo "gateway /healthz returned '$code' instead of 200" >&2
-            exit 1
-          fi
-          echo "gateway image healthy: GET /healthz -> 200"
-
   dependency-review:
     name: dependency-review
     if: github.event_name == 'pull_request'

.github/workflows/release.yaml

@@ -45,6 +45,7 @@ jobs:
       ghcr_image: ${{ steps.meta.outputs.ghcr_image }}
       gateway_image: ${{ steps.meta.outputs.gateway_image }}
       tag: ${{ steps.meta.outputs.tag }}
+      run_tests: ${{ steps.meta.outputs.run_tests }}
     steps:
       - name: Checkout
         uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
@@ -82,6 +83,12 @@ jobs:
               echo "tag commit $commit is not on main or a release branch" >&2
               exit 1
             fi
+            # Tests already ran in PR CI when the tag is on main; only re-run
+            # them for release-branch tags (where ci.yaml does not trigger).
+            run_tests=true
+            if git rev-parse -q --verify origin/main >/dev/null && git merge-base --is-ancestor "$commit" origin/main; then
+              run_tests=false
+            fi
           else
             echo "tag '$tag' does not exist in this repository" >&2
             exit 1
@@ -98,6 +105,7 @@ jobs:
             echo "ghcr_image=$GHCR_IMAGE"
             echo "gateway_image=$GATEWAY_IMAGE"
             echo "tag=$tag"
+            echo "run_tests=$run_tests"
           } >> "$GITHUB_OUTPUT"
 
       - name: Verify repository pins
@@ -106,6 +114,9 @@ jobs:
   tests:
     name: Unit tests and lint
     needs: meta
+    # The test steps are guarded individually rather than the whole job so a
+    # main-branch tag (run_tests=false) still leaves this job "successful" and
+    # does not cascade-skip build-main/build-gateway.
     runs-on: ubuntu-latest
     timeout-minutes: 15
     defaults:
@@ -117,6 +128,7 @@ jobs:
         with:
           persist-credentials: false
       - name: Set up Python
+        if: needs.meta.outputs.run_tests == 'true'
         uses: actions/setup-python@6d4fd22b20465ccc0f7354476588e8df7c9ca0e0 # v7.1.0
         with:
           python-version: "3.12"
@@ -125,16 +137,19 @@ jobs:
             examples/dsh-agents-as-a-service/gateway/requirements.txt
             examples/dsh-agents-as-a-service/gateway/requirements-dev.txt
       - name: Install dependencies
+        if: needs.meta.outputs.run_tests == 'true'
         run: |
           set -euo pipefail
           python -m pip install --upgrade pip
           pip install -r requirements.txt -r requirements-dev.txt
       - name: Lint
+        if: needs.meta.outputs.run_tests == 'true'
         run: |
           set -euo pipefail
           python -m ruff check .
           python -m ruff format --check .
       - name: Unit tests
+        if: needs.meta.outputs.run_tests == 'true'
         run: python -m pytest -q
 
   build-main:

AGENTS.md

@@ -49,7 +49,7 @@ before changing anything; it encodes the conventions the CI enforces.
 | examples/minimal-single-tenant/ | The just-one-DSH-Web quickstart |
 | scripts/ | check-versions.sh, check-examples.sh, check-manifests.py, set-version.sh |
 | .dsh/skills/ | Project-local DSH skills (auto-discovered); dsh-agent-sandbox-platform covers platform operation, MCP limits and extension recipes |
-| .github/workflows/ | ci.yaml, reusable _build-image.yaml, build-*, release.yaml, e2e-kind.yaml, pr-title.yaml, codeql.yml |
+| .github/workflows/ | ci.yaml (lint, dependency-review), ci-python.yaml, ci-manifests.yaml, reusable _build-image.yaml, build-*, release.yaml, e2e-kind.yaml, pr-title.yaml, codeql.yml |
 
 ## Commands
 
@@ -99,8 +99,7 @@ needs kind plus the agent-sandbox CRDs (see the example README).
 - Branch names: feat/, fix/, docs/, chore/, refactor/, test/.
 - Conventional Commit subjects and PR titles (feat(scope): ..., fix(scope): ...).
 - PRs are squash-merged; main stays linear and protected.
-- Required checks: lint, python, manifests, gateway-image, pr-title, CodeQL,
-  dependency-review.
+- Required checks: lint, python, manifests, pr-title, CodeQL, dependency-review.
 - Keep diffs scoped; update docs with the code; never mix a version bump with
   unrelated changes.
 

CONTRIBUTING.md

@@ -43,8 +43,8 @@ Install the gateway dev dependencies once:
 - Use a Conventional Commit title, for example feat(gateway): add wake metrics
   or docs(examples): clarify SSO trust boundary. PR titles are linted.
 - Keep the diff scoped. Squash merge into main; history stays linear.
-- Required checks: lint, python, manifests, gateway-image, pr-title, CodeQL,
-  and dependency-review. Branch protection is described in docs/gitflow.md.
+- Required checks: lint, python, manifests, pr-title, CodeQL, and
+  dependency-review. Branch protection is described in docs/gitflow.md.
 - Update the docs that the change affects; docs are part of the change, not a
   follow-up.
 - Never commit a real secret. Secret-shaped YAML belongs in a *.yaml.example

docs/gitflow.md

@@ -42,14 +42,24 @@ branch; the PR is squash-merged, so individual branch commits are never part of
 
 | Check | Workflow / job | What it verifies |
 | --- | --- | --- |
-| `lint` | `ci.yaml` | `scripts/check-versions.sh` (version pins and markdown tokens), ruff on the gateway, shellcheck on shell scripts, yamllint/actionlint when available. |
-| `python` | `ci.yaml` | Gateway unit tests (`pytest`). |
-| `manifests` | `ci.yaml` | `scripts/check-examples.sh`, `scripts/check-manifests.py` and a server-side dry-run of every example manifest against the agent-sandbox CRDs. |
-| `gateway-image` | `ci.yaml` | Builds the gateway image and smoke-tests `/healthz` without a cluster. |
+| `lint` | `ci.yaml` | `scripts/check-versions.sh` (version pins and markdown tokens), `scripts/check-examples.sh` (example layout), shellcheck, actionlint, yamllint, hadolint and documentation links. |
+| `python` | `ci-python.yaml` | Gateway ruff check/format and unit tests (`pytest` with coverage). Runs only when the gateway or this workflow changes. |
+| `manifests` | `ci-manifests.yaml` | `scripts/check-manifests.py` (manifest shape) and a server-side dry-run of every example manifest against the agent-sandbox CRDs. Runs only when manifests, check scripts or `versions.env` change. |
 | `pr-title` | `pr-title.yaml` | Conventional Commit PR title. |
 | `CodeQL` | `codeql.yml` | Static analysis for Python and GitHub Actions. |
 | `dependency-review` | `ci.yaml` | Dependency changes introduced by the PR (job `dependency-review`; runs on pull requests only). |
 
+`python` and `manifests` use native `on.pull_request.paths` filtering, so PRs that
+do not touch the gateway or the manifests skip them. The gateway image build and
+`/healthz` are no longer smoke-tested in a dedicated job: `/healthz` is covered by
+the gateway unit tests, the Dockerfile is linted by `lint` (hadolint), and the
+multi-arch image build is published by `build-gateway.yaml` on merge.
+
+Because a skipped path-filtered check is not a passing check, only the always-on
+checks (`lint`, `pr-title`, `CodeQL`, `dependency-review`) should be enforced as
+required status checks; `python` and `manifests` are on-demand checks that skip
+when unrelated to the change.
+
 The image build workflows (`_build-image.yaml`, `build-main.yaml`,
 `build-gateway.yaml`) and `e2e-kind.yaml` are **not** required on every PR: the
 image builds run where they are needed, and the kind end-to-end suite is reserved
@@ -105,10 +115,10 @@ respectively.
 Configure these on the repository before relying on the workflow:
 
 - [ ] **Branches → main**: require a pull request; require the checks `lint`,
-      `python`, `manifests`, `gateway-image`, `pr-title`, `CodeQL`,
-      `dependency-review`; require branches to be up to date; require conversation
-      resolution; restrict direct pushes; allow only squash merges and enable
-      "Automatically delete head branches".
+      `python`, `manifests`, `pr-title`, `CodeQL`, `dependency-review`; require
+      branches to be up to date; require conversation resolution; restrict direct
+      pushes; allow only squash merges and enable "Automatically delete head
+      branches".
 - [ ] **Branches → release/***: same protections, plus allow the release workflow
       to push tags.
 - [ ] **Tags → dsh-***: restrict creation/update/deletion to maintainers; a tag