HyDRA: contracts as the control mechanism for knowledge-graph construction

HyDRA is joint work with Adrian Kaiser, Claudiu Leoveanu-Condrei, Ryan Gold and Markus Hofmarcher at ExtensityAI. Adrian Kaiser led it; I am the fourth of five authors. The paper is on arXiv as 2507.15917 and the code is at ExtensityAI/ontology-hydra.

The problem

Ask a language model to turn a pile of documents into a knowledge graph and it will happily produce triplets. What it will not reliably produce is a graph. The failures we targeted are structural rather than factual: disconnected subgraphs, which the paper calls isolated islands, and the conflation of abstract classes with concrete instances, which quietly destroys the distinction the graph exists to encode. Underneath both sits entity resolution, the question of when two differently worded mentions are the same thing, which gets harder when the graph is built one batch of documents at a time.

The consequence is that the automation saves nothing. Somebody still has to read the output and repair it, which is the work the pipeline was supposed to remove.

The idea

HyDRA treats construction as a verifiable multi-step process rather than an unconstrained generative one, and takes its control mechanism from Bertrand Meyer’s Design by Contract. A contract here has four parts: preconditions, postconditions, invariants, and a verification function realised through neuro-symbolic operations. Stages compose, so the postcondition of one becomes the precondition of the next. That gives an end-to-end lineage from high-level requirements down to individual triplets.

The pipeline starts from a plain-language domain description rather than from the documents, deliberately, so the schema does not overfit whatever sample happened to arrive first. HyDRA generates a prioritised list of stakeholder groups, then personas per group, with more personas for higher-priority groups. Personas are shuffled into heterogeneous groups, each group writes a scope document, and those documents are recursively merged in batches until one remains. From the scope document and the persona groups come competency questions, the natural-language queries the ontology must be able to answer.

Competency questions are the interesting move. Prior work treats them as informal documentation for humans. Here they become the root scope contract, and their identifiers are embedded in the generated ontology’s metadata, so any class can be traced back to the requirement that justified its existence.

Ontology generation then runs in batches. Each batch of candidate concepts is merged and checked. Simple constraints are syntactic: naming conventions, uniqueness, referenced classes existing, property domains and ranges resolving. Complex constraints are structural: no circular inheritance, no self-inheritance, single inheritance, a single root class. A violation is not a terminal failure. The model is re-prompted with both the violations and its own previous output, and a batch is merged only once it is clean. Because constraints are checked against the ontology as it stands, validation gets progressively stricter as the graph grows.

Populating the graph works the same way against the finished ontology, over several epochs so that facts from distant parts of a corpus can connect. One detail worth keeping: an entity may be reassigned to a subclass of its current class but not to an unrelated one, on the assumption that later documents refine a type rather than contradict it.

Why it matters, and what it does not show

The headline number goes the wrong way, and the paper says so plainly. On the 143 question-answer pairs of the biomedical-engineering subset of MedExQA, the ontology-free baseline scores between 95.1% and 97.9% depending on the model, while HyDRA scores between 42.0% and 61.5%.

Our diagnosis is that the benchmark cannot see what an ontology buys. Most of those questions resolve in a single relational hop, and an unconstrained graph that mirrors the source text almost verbatim is close to ideal for single-hop retrieval. An ontology deliberately abstracts, trading exactness against the source for consistency across sources. The evaluation is also LLM-in-the-loop, so it cannot separate the model’s own knowledge from the quality of the graph it was handed.

To make the point concretely we wrote three propositions requiring two or three node traversals, as Neo4j Cypher queries. The graph answers all three correctly. An o4-mini given the same graph as context does not: it confirms the first relationship and then fails to validate the second.

So this is a negative result on the benchmark we chose, together with an argument that the benchmark is the wrong instrument. The proposed replacement, loading the graph into a database and translating benchmark questions into queries whose traversal depth can be dialled up, is explicitly unfinished. Generating accurate queries automatically has so far proved harder than we expected.

Where it applies

The setting HyDRA is actually built for is the one the benchmark does not test: building a single consistent graph from many sources over time, where a later document must attach to what earlier documents established rather than sit in its own island. A regulatory or clinical corpus that grows month by month is the obvious case.

Two smaller artefacts transfer immediately. The constraint set is reusable, and it got shorter with use: an early version allowed multiple class hierarchies and needed a separate repair phase to merge them, which we later replaced by enforcing a strict hierarchy during generation. And some violations do not need a model at all. When a triplet arrives with subject and object swapped, swapping them back is deterministic, and doing so avoids discarding an entire batch of otherwise valid output.