> For the complete documentation index, see [llms.txt](https://nytshift.gitbook.io/nytshift-docs/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://nytshift.gitbook.io/nytshift-docs/engineering-guide/11-testing-and-verification.md). # Testing and verification [← Development](/nytshift-docs/engineering-guide/10-development.md) · [Documentation home](/nytshift-docs/start-here/readme.md) · Next: [Production operations →](/nytshift-docs/engineering-guide/12-production-operations.md) The test strategy is evidence-oriented: pure logic is deterministic and offline, connected audits prove current external assumptions, and public releases bind a clean exact commit to a reproducible Linux artifact. ## Verification layers ```mermaid flowchart TB U[Unit and property tests] --> C[Contract generation/parity] C --> I[Route/service integration tests] I --> S[Supervisor and store recovery tests] S --> B[Production build + smoke] B --> A[Connected audits and activation evidence] A --> R[Clean Linux release artifact] R --> V[Independent receipt verification] ``` ## Core gates | Command | Proves | | ------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `pnpm preflight` | Toolchain, repository and safe-config prerequisites | | `pnpm contracts:check` | Generated schemas/OpenAPI exactly match the registry | | `pnpm typecheck` | Workspace TypeScript consistency | | `pnpm test` | Turborepo unit/integration suites | | `pnpm test:postgres` | Clean loopback PostgreSQL migration, checksum verification and customer PAPER/live-projection persistence suites; requires matching `DATABASE_URL` and `NIGHTSHIFT_TEST_DATABASE_URL` for an explicitly named audit/test database | | `pnpm css:check` | Generated route CSS exactly matches the reviewed source inventory and initial CSS budgets | | `pnpm test:py` | Python signer domain and route tests | | `pnpm lint:py` | Python static quality gate | | `pnpm test:signer` | Arcus signer runtime integration | | `pnpm test:hl-signer` | Hyperliquid signer runtime integration | | `pnpm test:signer-store` | Sensitive store backup/restore and lock behavior | | `pnpm test:local` | Supervisors, backups, registries, activation and local runtime | | `pnpm build` | Production workspace build | | `pnpm smoke:web` | Built web surface, health and route smoke | | `pnpm handoff:check` | Release handoff/documentation/hosting invariants | | `pnpm handoff:verify` | Complete composed gate | ## Deterministic test rules * No real wallet, signer key or venue credential. * No dependence on current market prices or screenshot values. * Mock data is typed `source: "mock"` and never labelled live. * Clocks, randomness and IDs are controlled where they affect results. * Exact decimals cover boundary and rounding cases. * Every store test covers malformed/corrupt data and migration/restart behavior. * PostgreSQL integration never silently skips in its dedicated CI job: the wrapper rejects missing, mismatched, remote or production-like database targets before migration. * Every protected action covers replay and ambiguity. * Tests never weaken a risk/security assertion just to go green. * `terminal-architecture-budget.test.ts` enforces the reviewed component, hook and raw-transport ceilings. * `terminal-domain-clients.test.ts` covers same-origin transport, timeout/abort limits, fixed error redaction, unregistered endpoints and invalid canonical response contracts. ### Route performance budgets The desktop Playwright gate writes `output/performance/route-budgets.json` and independently measures landing, command deck and terminal. Reviewed transfer ceilings are 350 KB JavaScript / 700 KB total for landing, 500 KB / 900 KB for the command deck and 900 KB / 1.3 MB for the terminal shell. CSS ceilings are 100 KB, 130 KB and 160 KB respectively; aggregate observed long-task time is capped at 200 ms, 250 ms and 300 ms. Licensed Advanced Charts bytes are reported separately. Public routes do not import the Privy, viem or styled-components identity runtime. Customer and terminal layouts own the identity boundary, and configured Privy/control components load only when configuration enables that capability. This boundary is covered by `identity-boundary.test.ts`; budget changes require an explicit rationale and before/after artifact comparison. ## Domain coverage ### Market data Test malformed payloads, unknown fields, symbol/cardinality mismatch, sequence gaps, stale transitions, reconnect repair, divergent references, implausible mark jumps and consumer isolation of held values. ### Paper Test exact book walking, spread/slippage rounding, capital reservations, oldest-first GTC fills, reduce-only non-flip, cross-market isolation, protection trigger timing, partial depth and stale total-equity withholding. ### AEGIS Test every allow/stop reason independently, then combinations at the strictest boundary: freshness, allowlist, account scope, fee/slippage/notional/leverage, daily loss/drawdown, oracle divergence and missing evidence. ### Execution Test one-use confirmation, idempotency, pre-sign versus post-sign failure, request replay, unknown/reconcile-only state, bracket leg identity, dead-man ambiguity, store migration/integrity and account/network mismatch. ### Agents Test token digest/expiry/revocation, exact tool/market/account scopes, quota, protocol/method/key/byte rejection, session-not-auth behavior, proposal lifecycle, paper policy immutability, DPoP replay and mainnet rejection. ## Connected audits Connected audits verify reviewed external facts and fail on drift. They do not silently rewrite registries. ```powershell pnpm audit:rhc-registry pnpm audit:rhc-bridges pnpm audit:rhc-oracles pnpm audit:arcus-funding pnpm audit:arcus-withdrawal pnpm audit:market-references -- --origin https://nytshift.xyz pnpm activation:audit --origin https://nytshift.xyz ``` A failure means the external evidence or production surface changed and requires review. Do not update a canonical registry automatically from the provider response that caused the alarm. ## Production evidence Some capabilities require private, bounded, expiring rehearsal documents: * Privy production identity flows; * licensed Advanced Charts behavior tied to exact release/tree; * managed RHC HTTP/archive/WSS capability; * live-feed soak and restart convergence; * testnet order/emergency-stop rehearsals; * remote MCP/OAuth security and interoperability. Health exposes only fixed status/reason/expiry, not the sensitive evidence document. ## Failure triage 1. Preserve the exact command and first causal failure. 2. Separate deterministic code failure from connected evidence drift. 3. Check whether generated artifacts are stale. 4. Check whether another supervisor/render/build owns a lock or port. 5. Never delete or reset a durable store to make a migration test pass. 6. Never treat a command timeout as proof the child process stopped. 7. Record gates that could not run; do not report them passed. The concise normative strategy lives in [testing.md](/nytshift-docs/normative-design/testing.md); release-specific commands live in [release-checklist.md](/nytshift-docs/evidence-and-handoff-records/release-checklist.md). --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://nytshift.gitbook.io/nytshift-docs/engineering-guide/11-testing-and-verification.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.