Typed Judgments for Knowledge Graph Extraction with TypeSafe's Jev

Here is a sentence from a synthetic 10-K filing: "Management has no plans to divest the Cascade brand." Run it through a relation extractor and you will very likely get a divests(Northwind, Cascade) edge with a healthy confidence score, because the sentence has the surface shape of an asserted fact. It isn't one. It's a denial. And a knowledge graph that records it as a fact is now confidently wrong about something the source document explicitly told it was not happening.

I've been running a series of experiments on entity and relation extraction for knowledge graph construction in a repo called extraction-knowledge-graph-experiments. Every extractor in it is one of two shapes. GLiNER2.5 is a span model: everything it knows has to be anchored in a span of text. Claude, via the claude CLI, is a generative model: it answers in prose, and the work is constraining that prose back down to JSON. Both have a hole in the same place - the small semantic judgments a pipeline needs between its stages, which are neither spans nor essays:

• Is this extracted edge actually asserted, or is it hedged, denied, or conditional?
• Are these two mentions the same company, or two companies with similar names?

This post fills that hole with a third kind of model. TypeSafe calls its flagship model Jev a System One model: you send state and typed questions, and you get typed answers with calibrated probabilities. No text to parse, no schema to enforce. I used it as a pipeline stage in five notebooks and measured what it bought at each one.

What you'll learn: how to gate extracted edges on whether the source asserts them, why a perfect entity-resolution judge can be worth nothing, how to store judgments as graph properties so a filter becomes a WHERE clause, and where relation selection beats relation generation. Who this is for: developers building knowledge graph or GraphRAG pipelines who are comfortable with Python; no Cypher or TypeSafe experience needed.

The Experiments Repo, Briefly#

The repo is a sandbox for one question: how much of knowledge graph construction can you do with small, local, schema-driven models, and where exactly do you need something more? The workhorse is GLiNER2.5, a 194M-parameter encoder that does joint entity and relation extraction against an ontology you supply at runtime, on a laptop CPU. Around it sit a pipeline and synthetic corpora with hand-written gold labels, so every experiment is scored.

The corpus that matters here is BUSINESS_NEWS: ten documents about a handful of fictional companies, with 47 gold triples, 13 gold alias groups for entity resolution, and eleven planted modality traps - sentences like the divestiture denial above that look like facts and are not. GLiNER extracts 214 mentions and 170 candidate edges from it in about 14 seconds on a CPU.

Two findings from the nine earlier notebooks set up everything below. The repo's hand-rolled entity resolver scores precision 1.000 against the gold alias groups with recall short of it, so all the headroom is in recall. And in every comparison so far, the wins came from candidate generation rather than from better scoring. That one comes back three times.

What Jev Is, In One Request#

A System One request has two parts. State is what the model looks at: a string, or a JSON object of named fields. Questions are what you want to know about it, and there are exactly three kinds:

That's the whole vocabulary. Here's a real request from notebook 10 against an acquisition press release: three questions on one state, a JSON object with named fields so a question can point at one of them by path:

from typesafe_sdk import Choice, Noul, Score

EVENT_KINDS = {
    "acquisition": "One company is buying another, or part of another.",
    "earnings":    "Results, guidance, or a financial report.",
    "litigation":  "A lawsuit, investigation, or enforcement action.",
    "partnership": "An alliance, joint venture, or supply agreement.",
    "other":       "None of the above.",
}
SETTLEDNESS = [
    "Speculative -- rumoured, or attributed to unnamed sources.",
    "Announced but conditional -- agreed, pending approval or closing.",
    "Completed -- it has happened.",
]

questions = {
    "primary_event": Choice(
        instructions="What is the main corporate event `body` reports?",
        criteria=EVENT_KINDS),
    "names_a_price": Noul(
        instructions="Does `body` state a transaction price or "
                     "per-share consideration?"),
    "certainty": Score(
        instructions="How settled is the main event `body` reports?",
        criteria=SETTLEDNESS),
}
state = {"headline": doc["title"], "body": doc["text"][:900],
         "source": doc["source"]}

response = ts.ask(state, questions)   # ts wraps the client with a disk cache

The answer, in 723 tokens in and 97 out:

That document is too easy to show what the numbers are for. Ask the same three questions about document six, a wire story about an antitrust inquiry that has opened and a price war reported second-hand, and three things fall out.

A Choice returns a distribution, not a label. Its confidence summarises how concentrated the distribution is, and the docs are explicit that it means concentration, not correctness and not permission to act.

A Noul is one number. A probability of yes already carries its own confidence, and 0.5 means equally likely yes or no, not medium.

A Score is an expected value, and sometimes the expected value is a level nobody believes. On document six the model puts 0.70 on completed and 0.29 on speculative, almost nothing in between, and the expected position lands at 1.41 - on the one level with 0.01 of the mass. The confidence of 0.12 is the number that says so. An application that rounded the score and threw away the rest would record the wrong answer with no sign it had done so.

That last point is the thesis of the whole post: read the distribution, not the label.

Gating Extracted Edges On Assertion#

Of the 170 edges GLiNER decoded, which does the corpus actually assert? I asked two questions per edge, and they are deliberately not one question:

• supported, a Noul: does the document connect these two things at all, in any modality? This catches extraction errors.
• status, a Choice over four assertion statuses: how does the document present the claim? This catches the modality traps.

One question can't tell those two errors apart. Here's the pair for one edge, as the notebook builds it:

STATUS = {
    "asserted":        "The text states it as a present or past fact.",
    "hypothetical":    "A possibility, condition or proposal, not a fact.",
    "negated":         "Denied, or said not to be planned or expected.",
    "forward_looking": "Projected or guided to hold in the future.",
}
claim = "'Northwind Logistics' -- acquires -> 'Cascade Freight Systems'"
meaning = "Here, acquires means: the head is buying or has bought the tail."

questions = {
    "supported": Noul(
        instructions=f"Does the document say anything that bears on this "
                     f"claim: {claim}? {meaning} Answer yes if it discusses "
                     "the relationship at all, in any modality. Answer no if "
                     "it does not connect these two things.",
        criteria={"true":  "The document addresses this relationship.",
                  "false": "It does not connect them, or one is absent."},
    ),
    "status": Choice(
        instructions=f"How does the document present this claim: {claim}? "
                     f"{meaning} Judge only how it is presented, not "
                     "whether it is plausible.",
        criteria=STATUS,
    ),
}

How the questions are batched is the interesting part. Every question in a request shares one state and is evaluated in parallel, so the unit of work is one request per document carrying two questions per candidate edge, and each question has to name its own edge, because question ids are never sent to the model. At 24 questions per request, the 170 edges became 340 questions in 18 requests; one question per request would have re-sent each document once per edge, about 18.8x the input tokens.

import kgx.typesafe as kts

judgments = kts.judge_edges(ts, graphs, kgx.BUSINESS_NEWS, max_questions=24)
kept = [j for j in judgments if j.keep]   # supported >= 0.5, status asserted

The gate kept 114 of 170; the 56 it dropped were 31 edges the document never connected at all and 25 it connected and then hedged or denied. A few of the lowest-scoring, with GLiNER's own confidence beside them:

An edge can be decoded at 0.90 and still be one the document denies, because span-level confidence is about the decoding, not about the claim. One picture of all 170 edges makes the point:

Scoring The Gate#

The repo scores predicted triples against the 47 gold ones with an alias-tolerant matching policy, so this is comparable with every earlier notebook. One extraction pass and one inference pass, scored ungated and gated:

Precision went from 0.279 to 0.404 and recall did not move. Not approximately - identically. Of the 56 edges the gate removed, not one was a gold triple. A gate can only ever remove triples, so recall can't rise; what this says is that the two questions removed only noise.

Whether that's the right trade depends on what the graph is for, so here is the whole frontier, and it cost zero new requests: changing a threshold re-reads stored judgments; it doesn't re-ask the model.

The corners of that sweep matter too: status alone reaches F1 0.396, supported alone 0.396, together 0.409. For the decision the two signals are largely redundant; for the diagnosis they are not, because the 31-versus-25 split is what lets each drop be explained. That is the case for keeping both as data rather than one boolean.

What the whole of notebook 10 cost:

A Question With Two Readings#

Before gating any edges, I asked about the eleven modality traps on their own, with the Choice over four statuses and a Noul that collapsed them to what the graph needs. My first wording of the Noul was:

It caught 8 of 11 traps. The Choice in the same request called all eleven non-asserted, and every one of the three the Noul let through was a denial:

Can "Management has no plans to divest the Cascade brand" be recorded as a fact that currently holds? Yes: it's a true statement about management's plans. The claim I wanted judged was the divestiture; the claim the model judged was the sentence, and my instruction didn't say which to take.

Notice that the model did not come back near 0.5. The three come back at 0.66, 0.71 and 0.74, a confident answer to the reading I did not mean, and that is the primitive working as designed. A Noul near 0.5 means equally likely yes or no. It does not mean "your question was ambiguous." Ambiguity in the question has to be removed from the question; it cannot be read off the answer. The Choice never had this problem, because naming negated as a distinct outcome forces the distinction the Noul collapsed, so the fix was to make the Noul name it too:

Same model, same sentences, same Choice alongside: 11 of 11. The three denials went from 0.66, 0.71 and 0.74 to 0.13, 0.19 and 0.15, five control sentences that are plain assertions stayed at 5 of 5, and the gap between the trap and control distributions opened from 0.07 to 0.63. Without the controls, a detector that answers "not factual" to everything would also score 11 of 11. Keep the distinction in the answer type, or state it in the question. Never fold it into a probability and hope.

A Perfect Judge On The Wrong Queue#

The repo's hand-rolled resolver (normalise, block, score, cluster, canonicalise) puts pairs that scored below the merge threshold but above a floor into a review band, documented in the code as the slot an LLM adjudicator fills. On this corpus the band holds 55 pairs, most of them bare prefix matches like Northwind inside Northwind Logistics Inc., which the resolver scores at 0.88, just under the 0.9 threshold, because Northwind and Northwind Logistics should merge while Apple and Apple Bank should not, and the string evidence is identical. Those pairs need a decision that takes world knowledge.

I filled the slot with TypeSafe's own entity-alignment cookbook, moved from beer catalogues to company mentions. The shape it recommends is not the obvious one: one Score whose levels are the outcomes, instead of a probability plus two fitted thresholds. The decision rule is round(score): 0 leaves the pair unlinked, 1 sends it to the curator queue, 2 merges. Three diagnostic Nouls ride along in the same request; they don't feed the decision, they make a disagreement with the resolver attributable rather than mysterious:

LINK_LEVELS = [
    "They refer to two different real-world entities.",          # 0: unlinked
    "Closely related entities that may or may not be the same.", # 1: curator
    "One and the same real-world entity.",                       # 2: merge
]

questions = {
    "link_state": Score(
        instructions="`mention_a` and `mention_b` are two mentions of a "
                     "company, each quoted with the sentence it appeared in. "
                     "How do they relate?",
        criteria=LINK_LEVELS,
    ),
    # diagnostics: never touch the decision, they make every verdict readable
    "same_name": Noul(
        instructions="Ignoring abbreviations, legal suffixes and "
                     "possessives, do the two texts name the same thing?"),
    "same_context": Noul(
        instructions="Do the two contexts describe the same entity doing "
                     "the same kind of thing?"),
    "abbreviation": Noul(
        instructions="Is one text an abbreviation, acronym, ticker symbol "
                     "or short form of the other?"),
}

Each mention goes in with its context window, not just its surface string, because Apple and Apple Bank are inseparable as strings and separable in context:

state = {
    "mention_a": {
        "text": "Northwind Logistics Inc.", "type": "company",
        "context": "SEATTLE -- Northwind Logistics Inc. (NASDAQ: NWL) today "
                   "announced it has entered into a definitive agreement "
                   "to acquire",
    },
    "mention_b": {
        "text": "Northwind", "type": "company",
        "context": "Priya Raman, Chief Executive Officer: Thank you. "
                   "Northwind delivered full year revenue of $2.41 billion",
    },
}

I adjudicated all 55 pairs and folded the accepted merges into the resolver's clustering with union-find:

verdicts = kts.adjudicate_pairs(ts, band, MENTION)
merged = kts.apply_verdicts(resolution, verdicts)   # mention_id -> canon_id

The verdicts were flawless: twenty-eight merges on gold-labelled pairs, every one correct, and the clustering genuinely changed, from 109 clusters to 104. Then I re-scored:

Zero movement, to four decimal places. A perfect judge, and it bought nothing: every gold-labelled merge it made was already in one cluster through some third mention, and the ten merges that did change clusters are on mentions outside the gold alias groups, which the metric can't see.

Meanwhile 41 gold-same pairs were still split, and not one of them had ever been proposed by blocking: HLCN against every spelling of Halcyon, and - stranger - Torrent Microsystems against itself. Two identical strings in different clusters is not a string-similarity failure. It's a type failure. The extractor types HLCN as security (a ticker symbol, which it literally is) and Halcyon Semiconductor as company, and types Torrent Microsystems both ways in different documents. Blocking is type-scoped, so none of these pairs is ever a candidate, whatever the strings say. So hand the adjudicator the eight distinct pairs blocking refused to propose.

Eight requests. Seven merges, one review, none contradicting the gold labels, and B-cubed F1 goes from 0.9458 to 1.0000 at precision 1.0000, where the repo's best entity-resolution configurations also land, with a diagnosis attached. Read the diagnostic columns, because they are the diagnosis: Torrent Microsystems against itself merges on same_name near 1 and abbrev near 0, so the type disagreement is the only reason it was ever in doubt, while HLCN against the full company name rides entirely on abbrev at 0.98, a ticker for the company, which no string metric can see. When a merge looks wrong six months from now, there is something to read.

Block On The Distribution, Not The Label#

Hand-feeding the pairs proves the diagnosis and leaves the pipeline as broken as it was. The cause was a type label, so the fix belongs upstream of blocking, and it's one more Choice. But the ontology defines security as "an issued instrument: shares, notes, bonds, or a ticker symbol," so GLiNER did what it was told. The question the resolver needs answered is not what kind of expression is this but what real-world thing does it stand for, so the question asks about the referent, with the ontology's own type descriptions as criteria:

Choice(
    instructions="In `document`, what kind of real-world thing does the "
                 "mention 'HLCN' stand for? Judge the thing it refers to "
                 "in this document, not the form of the expression: a "
                 "ticker used to name a company stands for the company.",
    criteria={
        "company":  "A commercial issuer or private firm, "
                    "referred to by name.",
        "security": "An issued instrument: shares, notes, bonds, "
                    "or a ticker symbol.",
        # ... the other eleven types, straight from the ontology
    },
)

Re-typing all 214 mentions took ten requests. The argmax agrees with GLiNER on 192 of 214, and where it disagrees with high confidence the changes are the ones a person would make: Torrent Microsystems and Vantage Energy Partners from security to company at 1.00, adjusted EBITDA to financial_metric, antitrust inquiry to litigation. And the tickers don't flip. HLCN stays security at 0.56 with company carrying 0.41. That is not the model failing; under an ontology that defines security to include ticker symbols, a ticker is ambiguous, and the model reports it as a distribution, which is the thing a label throws away. So use the distribution: instead of swapping in the argmax, offer each ambiguous mention to every type block that carries at least 25% of the probability mass, as a clone that blocking can see and that scoring must still accept:

retyped = kts.apply_types(mentions, type_verdicts)            # argmax only
res_argmax = resolver.resolve(retyped)

# argmax plus a clone in every type block holding >= 25% of the mass
soft, clones = kts.soft_typed(mentions, type_verdicts, min_prob=0.25)
res_soft = resolver.resolve(soft)
soft_map = kts.fold_clones(res_soft.mention_to_canon, clones)

The argmax recovers a little; the distribution recovers everything. Seventeen clones across 214 mentions took the unchanged resolver to B-cubed 1.000 at precision 1.000, with no pair hand-fed and no scoring rule touched. The clones widened candidate generation; the resolver's own scoring still had to agree to every merge, and precision says it did. This is the Score lesson from the first section, applied to a Choice: a 0.56 versus 0.41 split is not a label with a wobble, it's the model asking to be allowed both readings. Type-scoped blocking that reads only the label makes a wall of that; blocking that reads the distribution makes a candidate of it.

Refusing To Merge#

A judge that merges everything scores perfectly on a corpus with no traps, so it needs testing on pairs that must not merge. The shopping corpus has twelve, Aurora 14 versus Aurora 14 Pro and the like, and the hand-rolled resolver merges three of them. I scored them beside the corpus's must-merge pairs, once with the pair question as written above and once with the shopping ontology's own guideline in the instructions, which says outright that "'Aurora 14' and 'Aurora 14 Pro' are two products, not one."

Zero false merges on the eleven testable traps, zero false rejects on the six must-merge pairs, under both wordings. The guideline turned the adjudicator's one hedge into a confident refusal, at a cost on the side it wasn't written for: Tidewater Supply versus Tidewater, a retailer and a brand, moved from merge to review. Nothing became wrong; something became a curator's job.

Judgments As Graph Data#

Both stages produce typed values, not prose, so they can survive into the graph as properties instead of being spent at the point of decision. Build the graph ungated, on the adjudicated clustering, and put the judgment on the edge: the gate becomes a WHERE clause. That's strictly more useful, because the thing a gate throws away is also information. "The corpus says Northwind might acquire Cascade, pending approval" is a fact about the corpus, and a graph that silently dropped it can't answer a question about it.

The repo's Cypher writer already emits confidence, support, docs and evidence on every relationship; a dozen more lines add assertion and supported beside them. Across the 66 canonical edges: 48 asserted, 12 negated, 4 hypothetical, 2 forward-looking. Against the loaded graph, the distinction the gate drew is now a query, and so is what the gate would have thrown away:

// facts the corpus asserts
MATCH (a)-[r]->(b)
WHERE r.assertion = 'asserted' AND r.supported >= 0.8
RETURN a.name, type(r), b.name, r.support

// floated and never asserted - what a gate alone would have thrown away
MATCH (a)-[r]->(b)
WHERE r.assertion IN ['hypothetical', 'forward_looking']
RETURN a.name, type(r), b.name, r.assertion, r.evidence

One war story. Several sentences support one canonical edge, so their judgments have to be aggregated. My first version kept the highest-supported judgment per edge and quietly lost three edges, including the headline acquisition, which is hedged in the press release at supported 0.99 and asserted in a later document at a lower one. Ranking on support alone picks the hedge. The aggregation now ranks on assertion first, and a three-line equivalence check (build gated, versus build ungated and filter) is the only reason this was caught instead of shipped: both paths produce the identical 46 triples.

Selection, Not Generation#

Candidate generation and judgment are different jobs. Recall stayed exactly where GLiNER left it, at 0.404, because a System One model can't find an edge; it has no spans and nothing to anchor. But it can select. Given two mentions the extractor already found and the short list of relations the ontology permits between their types, one Choice can pick the relation the document states, or none. So in notebook 11, code enumerates every ontology-legal, co-occurring ordered pair of GLiNER's own mentions, and the model only chooses:

RELATIONS = {   # the six the ontology allows between two companies, plus none
    "subsidiary_of": "The head is owned or controlled by the tail.",
    "acquires":      "The head is buying or has bought the tail.",
    "has_stake_in":  "The head holds an ownership interest in the tail.",
    "supplies":      "The head provides goods or inputs to the tail.",
    "partners_with": "A stated partnership, alliance, or joint venture.",
    "competes_with": "The two companies are stated rivals in a market.",
    "none":          "None of these, or only in the opposite direction.",
}

Choice(
    instructions="In `document`, which of these relationships does the text "
                 "state or discuss from 'Northwind Logistics' to 'Cascade "
                 "Freight Systems', in that direction? Pick the one it "
                 "supports, or none.",
    criteria=RELATIONS,
)

The first thing to measure is the candidate window, because it determines the whole experiment:

Which scope is fair depends on how many of GLiNER's own edges connect mentions in different paragraphs: 130 of 170. So paragraph scope is a handicap, document scope is where the selector can see what the decoder saw, and the difference is three times the requests. At paragraph scope the selector answered none to 64% of the 295 candidates, which is the none option doing its job. Scored against gold:

Same recall, half the precision, and they are not finding the same things. Both recover 19 gold triples, but only 13 are the same 13: six are found only by the selector, three subject_to edges to regulators among them, and six only by the decoder, every one a cross-paragraph pair the paragraph-scoped selector was never asked about. Where the two disagree on a pair both saw, the selector is often correcting the decoder: partners_with(Northwind, Cascade) at 0.58 becomes acquires at 1.00, and partners_with(Vantage, Northwind) at 0.97 becomes supplies at 1.00, both gold-correct.

Threshold, Not Gate#

The selector's spurious edges are not hedged claims. They're wrong relations chosen at probability just over 0.5, a fifth of them impacts, the ontology's most permissive relation. The assertion gate removes 18 of them, costs two gold triples, and buys 0.008 of precision, because the status question has nothing to say about a plainly stated non-relation. The selector's own probability is the right filter, and because it's a threshold on stored numbers, the whole frontier comes from one pass:

Keep only the selections at probability 0.95 or above, gate them, and union them with GLiNER's gated edges:

F1 0.442 is the best any configuration in the repo has produced on this corpus, and document-scope selection on its own reaches recall 0.596 - 28 of 47 gold triples, the most any system has recovered - at precision 0.169. Document scope costs three times the requests for four thousandths of F1 and four points of recall; whether that's worth it depends on what the graph is for.

Three More Experiments#

Notebooks 12 through 14 take the same model to three other stages, and each reduces to one lesson.

What Contradicts What#

The repo's agent-memory layer supersedes "I use npm" with "I've switched to pnpm" but keeps both "I use Python" and "I'm based in Berlin", which means deciding whether two values are alternatives, and whether npm and pnpm compete for the same role is not a thing a string, an embedding, or a span model knows. One Noul - are a and b alternatives, two options filling the same role? - is the whole integration:

memory = TemporalGraph(alternative_fn=JevAlternatives(ts))

Asked over every pair of the things the gold memory facts name, it puts the two planted switches at 0.92 (npm/pnpm) and 0.63 (Python/Go) and all 26 other pairs at 0.21 or below. MiniLM cosine puts Python/Go at 0.166, lower than 20 of the 26 non-alternatives, because it measures how alike two names are, and being alternatives is not that:

In the actual ingest loop it never got to show it: extraction delivered exactly one real switch, and prefers pnpm had been extracted from the session in which the user turned pnpm down, a modality trap of the kind the assertion gate catches, one stage upstream. The judge is only as good as its queue. That's the third stage in a row to say so.

A Bigger Model Is Not A Curator#

Confidence-gated routing says: let the fast model decide everything it's sure about and send the rest to something more expensive. Notebook 13 sends the 31 of 170 edge judgments the System One model was unsure about to Claude Haiku. Haiku agreed on 28 of them; the three it flipped were two coin-flip copies of the headline acquisition and one spurious impacts edge that Haiku asserted. That last one is the whole result:

Recall never moves; precision falls as escalation rises. Both judges' answers are on disk, so the band can be swept from escalate nothing to escalate everything at no cost, and the sweep is flat, then down:

The best escalation rate on this corpus is zero, for about twelve cents of Haiku at 3.6 seconds a call: low confidence meant the text is genuinely ambiguous, and a bigger model doesn't resolve ambiguity that's in the source. The resolution side is sharper. Fourteen pairs landed on the adjudicator's middle level, the curator queue, and six of them are gold-same: Northwind against Northwind Logistics Inc., deliberately left undecided at levels 0.87 to 1.24. Haiku called two of them different ("the entities have different legal names") and four unsure. Zero of six. It reasoned its way to the wrong answer fluently, at twenty times the latency, where a calibrated abstention was the better answer. The review level exists so that a person with a reference to hand can look, and a larger language model is not that person.

The Boundary Of The No-LLM Position#

GLiNER classifies the intent of the repo's eight customer-support threads near chance and their priority at chance, because intent is written down and severity is not. Notebook 14 sends the threads to Jev instead: a Choice over six intents, a Score over four priorities written as operational situations, and a Noul for whether the thread ended resolved.

What is written down, the model reads. Whether a thread ended resolved is in the customer's last turn, and the Noul puts the two open threads at 0.05 and 0.08 and the six resolved ones between 0.64 and 0.97. Priority stays at chance, and the misses are confident: thread eight's customer says "Small one", the gold says low, and the model says urgent at 2.71, on a scale of my own writing that the gold labeller never saw. Routing on confidence sent the three hedged calls to a person and let every confident disagreement through. Confidence routes model uncertainty. It cannot route definitional disagreement. Fix the question, not the threshold.

The judgments went on a Thread node as properties, and the query that motivated all of it is one a span model could never have answered:

// open threads that need a person, most urgent first
MATCH (c:Customer)-[:OPENED]->(t:Thread)
WHERE t.resolved = false
  AND (t.priority IN ['high', 'urgent'] OR t.route = 'review')
RETURN c.name, t.thread_id, t.intent, t.priority, t.priority_score, t.route
ORDER BY t.priority_score DESC

What I Took Away#

Candidate generation and judgment are different jobs. The assertion gate worked because GLiNER had already produced the candidates; the adjudicator scored perfectly and bought nothing until something produced the candidates it needed. Extract locally and exhaustively, judge selectively, keep the judgments as data.

Read the distribution, not the label. A Score of 1.41 at confidence 0.12 is two readings, not one, and a type Choice at 0.56 versus 0.41 is a request to be allowed both. Seventeen clones took an unchanged resolver to perfect recall.

Ambiguity in the question surfaces as a confident answer to the wrong reading, not as an uncertain answer. Keep the distinction in the answer type, or state it in the question, and validate the wording against controls.

Store the raw judgments. The gate becomes a WHERE clause, the reason for every drop survives, and when a merge looks wrong six months from now the diagnostic Nouls are there to read.

Jev is not a replacement for either of the other extractors. GLiNER2.5 produced 170 candidate edges in fourteen seconds on a CPU with no key, and a System One model cannot do that. What it did was answer several hundred questions about those candidates that neither a span model nor a threshold could, in typed form, in under a fifth of a second per request.

Resources#

• extraction-knowledge-graph-experiments on GitHub - the repo: notebooks 01 through 14, the kgx library, and the synthetic corpora with gold labels
• The five notebooks behind this post: typed judgments, selection, not generation, what contradicts what, escalate on confidence, the boundary of the no-LLM position
• TypeSafe documentation - System One, the three primitives, confidence, and the Python SDK
• Knowledge graph entity alignment cookbook - the Score-with-three-levels shape used for pair adjudication
• Confidence-gated routing and composite scoring - the two TypeSafe patterns this post leans on
• GLiNER2.5 - the span extractor that produces every candidate in these experiments

The business-news, shopping and customer-service corpora are synthetic, written for the repo with gold labels attached. The companies, people and products in them do not exist.