Schema is too complex. That was the answer to the JSON schema I thought was the obvious one, and getting past it changed the data model in a way I would now choose on purpose.
The grid that would not compile
An ACORD 25 certificate of liability insurance is a grid of coverage rows, and each row carries up to fifteen named limit boxes: each_occurrence, general_aggregate, products_completed_ops_aggregate, el_disease_policy_limit, and eleven more. A real certificate fills in three or four of them. The rest are blank.
The compliance engine downstream reads limits as Partial<Record<LimitKey, number>>, so writing the extraction schema to match that map looked free. Two ways to express it, two failures, both recorded in the comment that now sits above the fragment: fifteen nullable values blew the 16-union parameter cap, and fifteen optional properties hit "Schema is too complex".
What shipped is one enum and one number:
// packages/coi-extract/src/schema.ts
const limitsItems = {
type: "object",
additionalProperties: false,
required: ["key", "amount"],
properties: {
key: { type: "string", enum: LIMIT_KEYS },
amount: { type: "number", description: "Whole US dollars." },
},
};
// ...inside each policy row:
limits: {
type: "array",
description: "One entry per limit box that has a figure. Omit blank boxes entirely.",
items: limitsItems,
},
Insurance certificate verification interface with required liability coverages and amounts — Coverage types, dollar amounts, and certificate status at a glance.
The pairs are the better model regardless
A map of fifteen nullable keys asks the reader to make a statement about every box on the form, including the empty ones. Pairs let absence be expressed by absence: a blank limit box is a pair the model never emits, and there is nothing to say about it.
That matters past the schema, because absent and zero are different findings. One of the normalization tests asserts "general_aggregate" in limits is false rather than checking for 0, since the engine reports a missing limit as "not stated" and a low one as short of the requirement. Those two produce different emails to the vendor's agent.
normalizeRecord folds the pairs back into the map the engine wants, and accepts the plain map too, so fixtures and hand written records exercise the same path without carrying the schema's compromise.
One smaller quirk in the same file: form is "occurrence" | "claims_made" | null, and an enum on a union type gets rejected with Enum value 'occurrence' does not match declared type. Two anyOf branches, one enum and one null, compile fine.
The schema constrains shape, and stops there
additionalProperties: false on every object plus required on every property gets you a parse you can type. It does not get you numeric bounds, string length, or date formats. Every value constraint that matters lives after the call returns.
| Rule | What COI_SCHEMA declares | What normalizeRecord checks |
|---|---|---|
| Object keys | required on all properties, additionalProperties: false | not re-checked |
| Date is a real calendar date | type: "string", the format lives in the description | regex plus a Date.parse round trip, so 2027-02-31 is rejected |
| Expiry falls after effective date | nothing | expiresOn < effectiveOn drops the policy and flags the path |
| Limit is a plausible figure | type: "number", no min or max available | <= 0 or > 1_000_000_000 becomes a review item |
| ADDL INSD tri-state | ["boolean", "null"] | anything that is not a boolean is coerced to null, never false |
form is one of two strings or null | anyOf branches | re-checked against the two literals |
The billion dollar ceiling is not a guess about insurance. Its comment says these are almost always a policy number or a date read into a limit box, and the test that covers it feeds 8_892_310_000_000 against a fixture whose policy number is GL-889231.
Demote, never drop
The part I would port to any extraction pipeline: nothing that fails a value check is silently discarded. Each one appends a dotted path to uncertainFields, the compliance engine turns anything left there into an EXTRACTION_UNCERTAIN finding with severity review, and any review finding pushes the whole vendor to needs_review instead of compliant.
The same idea covers an unreadable issue date. Dropping it would fail the staleness check open, and passing through the garbage would fire a wrong warning, so issuedOn falls back to today's date, which makes that check a no op, and the review item carries the real problem to a person.
That's how we built CoverCheck.
One shipped product, taken apart, once a month. What it does, what it cost to build, what the pipeline behind it looks like, and what the numbers did — read off the repository and the live site, not written from memory. Join the list.
← All writing