Testing

Everything runs from the single repo .venv — the one interpreter that carries the full stack (FastAPI + the [ai] extras). There is no "AI tests skip here" environment; every test runs the real stack.

Layout

tests/
├── conftest.py                 # Shared fixtures (db_path, db, client) + autouse isolation
├── _cassette.py                # HTTP replay layer
├── _helpers.py                 # Test helpers
├── .git/                       # Separate local Git history; no remote
├── test_*.py                   # Unit + integration tests, grouped by manifest
├── cassettes/                  # Recorded external-HTTP responses (replayed offline)
└── fixtures/                   # JSON / SQL fixture files

The backend currently has 194 test files. They live in a separate local Git repository at tests/.git, which has no remote. The product repository ignores tests/, so publishing its main branch does not publish the tests. Commit test changes with git -C tests add ... and git -C tests commit. Back up that local repository separately; Git history on one machine is not a backup. For another ALMa worktree, run python scripts/test_local.py link /path/to/worktree from this checkout. It creates a local test-repository worktree on its own branch, so source branches can change their tests independently. The source main uses test branch main; other source branches use source-<branch-name> with / replaced by -. The runner checks that the checked-out branches match. A new test branch starts from test main. Merge its commits into test main when source work is merged; for example:

git -C tests switch main
git -C tests merge --ff-only source-feature-test-suite-scope

If the test branch is checked out in another worktree, switch that worktree away first. No test commit reaches the product repository or GitHub through this workflow.

The source-to-test dependency map is scripts/test_groups.toml; see the dependency graph for the shared edges. Its full graph is generated by python scripts/test_local.py graph --format dot (Graphviz) or --format json (machine-readable). The map includes overlapping groups when a test protects several surfaces. check rejects an ungrouped test file or a direct import of an ALMa module or exported name that no longer exists. Frontend tests live next to the code under frontend/src/ as *.test.ts(x) files (Vitest).

Fixtures

The three fixtures most backend tests use (all in conftest.py, scoped per test — fresh DB each time, no state leakage):

• db_path — a throwaway SQLite path exported via DB_PATH, with auth disabled. Use when you only need the path.
• db — a real, schema-initialised sqlite3.Connection (Row factory). Use for tests that build their own state directly.
• client — a FastAPI TestClient against a fresh schema-initialised DB. Use for route + DB integration tests.

Two autouse fixtures run for every test:

• HTTP isolation — external sources funnel through requests.Session.request or curl_cffi.requests.Session.request; both are patched to replay recorded cassettes and block live network. Opt a test into real outbound HTTP with the @pytest.mark.network marker. time.sleep is no-op'd off the network so rate-limit/retry logic runs without the wall-clock waits.
• Scheduler reset — clears the scheduler's in-process job dicts between tests so a queued job can't leak into the next test.

Running tests

# changed backend components (four isolated processes by default)
python scripts/test_local.py changed --base main

# inspect selection before running
python scripts/test_local.py changed --base main --dry-run

# an explicit component group, or several overlapping groups
python scripts/test_local.py group authors discovery

# all non-network backend tests (release gate)
python scripts/test_local.py full

# serial when debugging order or process isolation
python scripts/test_local.py group feed --workers 1

# review coverage of the group map
python scripts/test_local.py list
python scripts/test_local.py check

# one file
.venv/bin/python -m pytest tests/test_author_suggestions_scoring.py

# one test
.venv/bin/python -m pytest tests/test_author_suggestions_scoring.py::test_cited_by_high_signal_weights_lead_over_consortium_middle

# with coverage
.venv/bin/python -m pytest --cov=alma --cov-report=term-missing

addopts = --strict-markers and testpaths = tests are set in pyproject.toml, so bare invocations already target the suite.

changed selects tests for changed src/alma/ paths using the map, and adds directly edited tests from the local test repository. Shared setup, migrations, unknown backend paths and mapping changes select the full suite. This is an inner-loop shortcut; run full before a merge, release or package removal. For a frontend change, changed runs the frontend typecheck and full Vitest suite; those are already short, and the source guards do not all appear in Vitest's import graph. Each parallel process receives a different set of complete test files and a private settings/secrets profile. Use --workers 1 to reproduce a failure in one process. The graph is an explicit conservative dependency map, not a coverage-derived proof that omitted tests cannot fail. Use python scripts/test_local.py full --shuffle-seed 90 to vary test-file order within each worker when checking for order dependencies.

When driving these tests through Codex's restricted shell sandbox, run the pytest command with escalated execution. That sandbox blocks asyncio's socketpair wakeup, so even a minimal FastAPI TestClient hangs during setup; it is an execution-environment failure, not an ALMa timeout.

Real-load behaviour

For perf and behaviour-under-load that pytest doesn't capture well, exercise the path against a running instance and read Activity → Operations. The per-source timing sub-panel breaks down each external retrieval lane and is the single best signal for refresh regressions.

Use unit tests for pure-function logic and the integration suite for route + DB interactions; reach for live load only when you need real timings, multi-step interaction (e.g. refresh during read), or end-to-end shape verification.

Frontend tests

cd frontend
npm run test        # Vitest, single run
npm run test:watch  # Vitest, watch mode

Vitest drives the unit/component tests that live beside the code (src/lib/*.test.ts, src/components/**/*.test.tsx, plus guards like src/test/surface-guard.test.ts). No backend is involved — pure logic and rendering.

Frontend type check

cd frontend
npm run typecheck

Strict mode, no errors expected. Run after any change to api/client.ts or any cross-component prop change.

Frontend build

cd frontend
npm run build

Catches Vite-level errors (missing imports, broken aliases) that tsc doesn't see. Fast.

What each tier is for

Tier	Tool	When to add
Unit	pytest / Vitest	Pure-function logic. Fast, isolated.
Integration	pytest + client/db fixture	Route + DB + side effects. Most ALMa tests live here.
Live load	Activity → Operations UI	Perf budgets, interaction patterns, shape verification — driven against a running instance.
Type check	npm run typecheck	After any frontend change. Bare tsc --noEmit checks nothing — tsconfig.json is references-only.

Conventions

• No mocked DB. Tests run against a real SQLite (file-backed via the fixture). Mocking the DB hides schema-mismatch bugs that production absolutely will hit.
• No live network. External HTTP replays from cassettes; block is the default and @pytest.mark.network is the deliberate opt-out.
• Test the contract, not the implementation. Assert on the HTTP response, not on internal call paths.
• Test the failure modes too. Every test for a happy path should have a sibling that exercises the obvious failure (missing field, wrong status, conflict).

What gets pinned by tests

These invariants live in tests and will fail loudly if you break them:

• No new raw conn.commit() outside the infra allowlist — the AST guard tests/test_no_raw_commit_guard.py.
• Write gating for foreground Activity rows — tests/test_activity_write_gating.py, tests/test_db_write_guardrails.py.
• Author suggestions cited_by_high_signal filters negative-signal authors (tests/test_author_suggestions_scoring.py).
• No raw surface / semantic colour in components — the frontend guard frontend/src/test/surface-guard.test.ts.

When you change one of these contracts, change the test in the same commit.