RAG systems reach a structural limit at scale exactly when modern LLM models context window is not enough and you need RAG. Loading thousands of pages of rules into a vector store — complete legal systems, regulatory frameworks, comprehensive policy libraries — degrades inference quality. Chunks concentrate around the same positions in vector space, and the model finds too many equally-plausible matches to distinguish between. Metadata filtering and distance tuning provide temporary relief, not permanent solutions. This isn't a configuration problem. It's a limitation of vector similarity for dense, interconnected content.

We experimented with existing solutions — Neo4j, Microsoft's GraphRAG, common graph RAG libraries — and found them suboptimal for enterprise use cases we had in our mind. Constraints around hosting environments, operational complexity, and configuration flexibility made them impractical for the contexts we work in. We built our own implementation, inspired by the original Microsoft GraphRAG concept but independent of existing tooling. The result gives us full control over deployment across different cloud environments, with inference quality matching or exceeding the original tools.

The proof came from testing against the Spanish legal code — the complete corpus of Spanish law. We wanted to know whether an agent could reliably answer complex, edge-case legal questions from a dataset of that scale and density. With the right graph configuration, it can. Not retrieval — reasoning. The system navigates interconnected rules, finds relevant precedents, handles exceptions, and produces reliable answers even for questions that would defeat vector-based approaches.

Graph-based systems introduce their own implementation challenges: hosting architecture, query performance at scale, governance, security. We're working on these in parallel with the core inference research.

Context Graphs

This research connects directly to the synthetic employees work. Context graphs — the structured representation of how knowledge, decisions, and exceptions relate to each other — are how we encode operative knowledge into systems that need it.

An employee who's been in a role for years knows which policies apply in which situations, where the documented exceptions are, how similar cases were handled before. That's not retrievable from documents. It lives in the connections between facts, in the precedents that govern edge cases — what Jaya Gupta and Ashu Garg recently described as "the exceptions, overrides, precedents, and cross-system context that currently live in Slack threads, deal desk conversations, and people's heads" (https://foundationcapital.com/context-graphs-ais-trillion-dollar-opportunity/). Context graphs capture this structure. They make it available to agents that need to reason the way experienced employees do.

We believe knowledge graphs and context graphs will become essential infrastructure for enterprise AI in 2026. What's currently treated as an optimization — an enhancement to RAG — is becoming a requirement for agents operating in complex domains.