fix(ci): make the required security gate able to fail + repair the integration guard that could not pass #45
2
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Simon AxelssonandClaude Opus 5
|
be5e598646 |
fix(ci): replace the integration anti-skip guard that could not pass
CI (SIAX Cloud) / security (push) Successful in 15s
Security Scan (SIAX Cloud) / security (push) Successful in 18s
CI (SIAX Cloud) / integration (push) Successful in 29s
CI (SIAX Cloud) / quality (push) Successful in 47s
CI (SIAX Cloud) / security (pull_request) Successful in 15s
Security Scan (SIAX Cloud) / security (pull_request) Successful in 15s
CI (SIAX Cloud) / integration (pull_request) Successful in 31s
CI (SIAX Cloud) / quality (pull_request) Successful in 38s
The guard grepped vitest's HUMAN output:
grep -qE "skipped" && ! grep -qE "Tests [0-9]+ passed"
vitest colours that summary. The real line emitted on these runners is
ESC[2m Tests ESC[22m ESC[1mESC[32m97 passedESC[39m | ...
so ANSI escapes sit between "Tests " and the digits and the
two-spaces-then-digits pattern can never match. The first grep still
matched "skipped", so the condition was true whenever a single test
skipped -- no matter how many passed.
Measured on run 13911 job 31183 (this branch, 2026-09-08): 13 files /
97 tests PASSED, 1 file / 2 tests skipped, and the step still printed
"FATAL: integration suites skipped instead of running" and exited 1.
That message was false; the suites ran.
It survived review because it is correct locally -- redirecting vitest
to a file disables its colour, so the pattern matches on a developer
machine and misfires only under CI.
Replaced with vitest's JSON reporter: structured, colourless, and
fail-closed if the report is missing. A/B measured on real reports in
BOTH directions, so the replacement is not merely green:
all skipped (0 passed / 99 pending) -> exit 1 (protection kept)
945 passed + 184 skipped -> exit 0 (false red gone)
report file absent -> exit 1 (fail closed)
The anti-skip guarantee the original guard was written for is preserved:
a run where every suite skips still fails.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
Simon AxelssonandClaude Opus 5
|
1322d26fac |
fix(ci): make the required security gate capable of failing
Branch protection on `main` requires exactly two status contexts:
CI (SIAX Cloud) / security (push)
CI (SIAX Cloud) / quality (push)
The `security` half could not fail. All three Trivy invocations in this
repo ran with `--exit-code 0`, so the scanner reported findings to stdout
and then exited 0 regardless.
Measured on main @4514d97, 2026-09-08, same scanner (Trivy 0.73.0), same
target, only the flag differing:
A --exit-code 0, clean tree -> exit 0
B --exit-code 0, planted AWS secret access key -> exit 0 (DETECTED
as CRITICAL in the report, gate still green)
C --exit-code 1, same planted secret -> exit 1
D --exit-code 1, probe removed -> exit 0
B is the defect: the required context reported success while the scanner
had a CRITICAL secret in hand. D is why this is safe to land now -- the
tree is clean today, so flipping the flag does not open the gate red.
(PR #19 measured 19 HIGH findings on 2026-08-11; those dependencies have
since been updated on main, 142 commits later.)
The other required context, `quality`, was verified real and left alone:
a planted type error fails `pnpm run typecheck` (exit 2) and a planted
failing assertion fails `pnpm run test` (exit 1, 945 passed baseline).
nightly.yml carried the same flag. A nightly scan exists to catch CVEs
published against code that did not change; with `--exit-code 0` it could
never raise that alarm.
Not fixed here, needs an owner decision: services/scrapers/requirements.txt
uses `>=` ranges only. Trivy's pip scanner cannot resolve unpinned
specifiers, so it omits the file from the scan entirely -- it never appears
as a target. Pinning those same floors with `==` surfaces 3 HIGH findings
(aiohttp CVE-2025-69223, aiohttp CVE-2026-69244, lxml CVE-2026-41066).
Deliberately NOT suppressed with `--skip-dirs`, which would turn a
measurable blind spot into declared policy.
Job names are unchanged, so both required context strings are preserved.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|