CLM Glossary Build Kit — Incremental Test Data 2 & Automated Test Proposal

Tag: S-2026-06-08-glossary-incremental-test-data Type: own-writing Author(s): Paul Miles (AI-assisted) Date of source: 2026-06-08 Date ingested: 2026-06-08 Authority weight: high — the controlled record of the incremental test dataset and the automated-test proposal. Raw file: /_raw_sources/S-2026-06-08-glossary-incremental-test-data.md (Test Approach.md)

What it claims

Two deliverables added to the kit. (1) A new self-contained Test Data 2 folder (the original Test Data left untouched) that tests the incremental-load case from kit v1.2 (Business Concept Key + GR-14). It has Batch 1 (initial load = the five existing synthetic sources, a 27-concept baseline where each concept gets a BCK) and Batch 2 (incremental load = two new sources, Source 6 Product & Pricing Glossary xlsx Tier 3, Source 7 Servicing & Collections Addendum docx Tier 6). Batch 2’s ten instances exercise every incremental path: match-no-change (Collateral, Arrears, Security collateral → attach by BCK, no new row), match-definition-differs (Default → Change Request, approved value not overwritten), match-add-alias (Customer → Counterparty), new concepts minting BCKs (Loan-to-Income Ratio, Write-off, Bureau Score), new concept via in-batch conflict (Exposure Class, S6 vs S7 → SME workshop), homonym discrimination against the existing master (Security collateral must match the collateral concept BCK-0012, not the financial-instrument concept BCK-0013), and supersession/retirement (Bureau Score supersedes the obsolete Behavioural Score legacy → Stage 7 Retired). A four-sheet Answer Key (Batch 1 Baseline BCKs, Batch 2 Incremental, Incremental Coverage, Discrimination Traps) and a README accompany it. (2) Test Approach.md: an honest proposal that an automated end-to-end functional test is feasible — Claude acting as the AI agent executes prompts P-01…P-07 over the test data and a deterministic Python script scores the outputs against the Answer Key — with explicit caveats (it validates the kit’s LLM-executed logic, not a specific vendor tool such as M365 Copilot; AI steps are judgement so scores are reported with variance over three runs; human-decision steps are simulated from the Answer Key or gated; the data is synthetic and small). It lists what is needed (a dedicated writable test folder, read access to the kit and Test Data 2, confirmed pass thresholds, and a decision on simulating vs gating human steps) and recommends running it as a design-validation/regression check. No test has been run — awaiting Paul’s go/no-go.

Notable quotes

“A green result means the process and the v1.2 incremental design behave as specified on the designed cases — a strong de-risking signal before a real pilot, not a guarantee of production performance.” (Test Approach §6).

What’s speculative vs. asserted

Asserted: the dataset contents and the expected outcomes in the Answer Key; the test capabilities Claude actually has (file tools, Python/LibreOffice sandbox, ability to follow the prompts). Speculative/conditional: the automated test’s pass rates — not yet run; whether human steps are simulated or gated is Paul’s decision; the test does not certify a vendor tool or production-scale behaviour.

Topics this feeds

CLM Glossary Acceleration Squad — incremental test data and the test proposal.

Open questions raised

Go/no-go on running the automated functional test; whether to simulate or gate the human-decision steps; the dedicated test-folder location (proposed Glossary Test Harness).