Your LLM does not need your graph database

Suppose you want an LLM to answer a deceptively simple question:

Who is working on what?

That sounds like a database query until you draw it.

Alice owns the checkout incident. Bob reviewed the pull request that probably caused it. Priya is on-call for payments. The payments service depends on the identity service. The identity service has an open migration. The migration is blocked on a customer-support escalation. The escalation is attached to a Jira ticket, but the fix is in a GitHub branch, and the deploy is tracked in Slack.

Congratulations. You have a context graph.

The obvious next question is where that graph should live. Should it be a graph database like Neo4j or FalkorDB, or an embedded/local graph index such as a LadybugDB-style store if that is already part of your runtime? Should it be a GraphQL API? Should it be REST?

This is where many designs go wrong. These are not three versions of the same thing. They sit at different layers.

The storage layer, query interface, and agent tool surface are separate architectural choices. If you collapse them into one choice, you either give the model too much power or give the product too little expressiveness.

My bias: for an LLM-facing organization context graph, GraphQL is usually the best contract. Not always the best database. Not always the best operational tool. The best contract.

Use a graph database when traversal and graph indexing are actually your workload. Use REST when you want bounded, boring reads and commands. Put GraphQL in the middle when the agent needs to move through relationships without getting a raw database login.

the category error

A graph database answers:

How should I store and traverse connected data?

GraphQL answers:

What typed relationship graph am I willing to expose to clients?

REST answers:

What resources or commands should be available through stable endpoints?

An LLM tool surface answers:

What operations can a probabilistic caller safely perform?

Those questions overlap, but they are not interchangeable.

If you expose Cypher directly to the model, you have built a very clever SQL injection demo with better marketing. If you expose only REST endpoints, you will eventually create get_team_member_projects_with_recent_incidents_and_open_prs_v3. If you expose GraphQL without shared business-logic authorization and query limits, you have built an elegant denial-of-wallet machine.

The boring version of the architecture looks like this:

Sources of truth stay where they already are: GitHub, Jira, Linear, Slack, PagerDuty, incident docs, CI, deploy metadata, org charts. A graph or index layer may materialize relationships. GraphQL exposes the relationship contract. REST or MCP-style tools package the most common and risky operations into small named actions.

That split sounds pedantic until the first time a model asks for "all related context" and discovers payroll notes, private HR escalation threads, and every customer name in your ticketing system.

the example graph: who is working on what

For this essay, keep the graph concrete. We are not modeling the whole company. We want an agent to answer questions like:

• Who owns this bug?
• Who touched the related code recently?
• Which team is overloaded?
• Which issues are blocked on this service?
• Who should review this fix?
• What context already exists before I burn tokens investigating it?

The data is naturally connected:

• Person works on Project.
• Person owns Issue.
• Issue affects Service.
• PullRequest changes Service.
• Deploy includes PullRequest.
• Incident references Deploy.
• Team owns Service.
• Issue duplicates or blocks another Issue.

You can store that in Postgres. You can store it in Neo4j. You can store it in FalkorDB. You can keep it in memory for a small local product. The important question is not "is it a graph?" It is:

Which layer needs graph expressiveness, and which layer needs restraint?

option 1: graph database

Graph databases are good at what they say on the tin. Neo4j's property graph model is built around nodes, relationships, labels, and properties. FalkorDB exposes graph queries with OpenCypher-style syntax and is positioned around fast graph workloads and GraphRAG-style retrieval. If your core workload is relationship traversal, graph storage can be the honest tool.

For organization context, graph databases shine when you need:

• multi-hop traversal: "show issues within three relationship hops of this service"
• path queries: "how did this incident connect back to this deploy?"
• graph algorithms: centrality, communities, dependency impact
• relationship-heavy indexing: code owners, reviewers, services, deploys, incidents
• local embedded graph context: fast lookup inside a bundled agent runtime

This is the upside people understate. A graph database is not just a prettier join syntax. It can make relationship queries feel like the native workload rather than a pile of recursive CTEs and application joins.

But the same expressiveness is why I do not like raw graph databases as the LLM tool boundary.

An LLM that can generate arbitrary Cypher can also generate expensive Cypher. It can ask broad questions. It can stumble into sensitive paths. It can run semantically valid queries that violate your product intent. "Find everyone connected to this customer" might mean support owner, account executive, incident commander, private escalation, billing contact, and the engineer who pasted logs containing an email address.

Database permissions help, but they are the wrong granularity for many agent interactions. You usually want rules like:

• This user can see issue ownership, but not private people metadata.
• This agent can traverse from issue to service to owner, but not from person to all private tickets.
• This model call can fetch context for one incident, not enumerate the company.
• This tool should return evidence, not raw comments with secrets.

You can implement those rules near the database. But the policy language of the product often lives above the database.

My rule: use a graph database when you need a graph database. Do not confuse that with giving the model a graph database shell.

option 2: GraphQL

GraphQL is unfashionable in some backend circles because it was overused for frontend convenience and under-secured in the usual ways. Fair. A bad GraphQL service gives every client a shape-shifting endpoint, hides N+1 behavior, and makes caching less automatic than simple resource-oriented GET endpoints.

For LLM context graphs, though, GraphQL has become more interesting.

Why? Because the agent's job is often to follow typed relationships:

query ContextForIssue($id: ID!) {
  issue(id: $id) {
    title
    status
    owners { name team { name } }
    affectedServices { name owningTeam { name } }
    relatedPullRequests { title author { name } mergedAt }
    blockers { id title status }
  }
}

That is almost exactly the shape of the mental model. The LLM does not need to know whether owners comes from Linear, GitHub, Okta, a graph database, or a cached Postgres table. It sees a typed graph of what the product permits it to see.

GraphQL also has a useful property for agents: it exposes affordances. A schema tells the model what entities exist and which relationships are legal. With tool calling or an MCP wrapper, you can expose a few trusted or persisted GraphQL documents as tools instead of asking the model to invent queries.

This used to be expensive. GraphQL was the thing the one API specialist on the team maintained while everyone else quietly cursed resolver performance. That cost curve changed. LLMs are now decent at drafting schemas, resolvers, DataLoader batching patterns, test cases, and query examples. They do not make GraphQL free, but they reduce the ceremony enough that the tradeoff deserves a fresh look.

The strongest argument for GraphQL is not "let the LLM write arbitrary GraphQL." I would not start there.

The strongest argument is:

Define the context graph as a typed API, then expose a small number of allowed query shapes to the agent.

GraphQL gives you:

• a schema the model and humans can inspect
• field-aware authorization through resolvers and shared business logic
• natural relationship composition
• one interface over many source systems
• trusted or persisted documents for common context bundles
• cleaner evolution than endpoint proliferation

It also gives you familiar footguns:

• N+1 resolver explosions
• query depth and breadth abuse
• HTTP caching that needs deliberate GET, document-hash, or application-cache design
• schema sprawl if nobody owns the domain model
• authorization bugs when resolvers bypass the business layer

Those are real issues. They are not reasons to avoid GraphQL. They are reasons to treat it like an API surface, not a query toy.

option 3: REST

REST is the most boring option, and boring is often correct.

If the LLM needs to perform a narrow operation, REST is excellent:

• GET /issues/{id}/context
• GET /teams/{id}/workload
• GET /services/{id}/owners
• POST /issues/{id}/summaries
• POST /investigations/{id}/assign-reviewer

You can document it, cache the read endpoints, test it, rate-limit it, log it, and explain it to the security team. HTTP caching semantics are mature, especially for GET resources with explicit cache headers. URL-level authorization is easy to reason about. Audit trails are straightforward. If the model calls list_blocked_work, the logs say list_blocked_work.

That is why REST is still my default for dangerous actions. Mutations should be boring. Commands should be boring. Anything that writes to a tracker, changes ownership, comments on an issue, pages a human, or updates a customer-visible field should usually be a small explicit tool.

REST becomes awkward when the user asks graph-shaped questions:

Give me the bugs this engineer is likely to know about, excluding stale work, prioritizing issues connected to services they touched recently, but include only incidents in the last thirty days unless they are duplicates of this bug.

You can build a REST endpoint for that. Then another. Then another. Eventually your context API becomes a museum of product questions from last quarter.

REST is great for stable, named workflows. It is weak as a general relationship exploration language.

the decision matrix

Here is the practical comparison:

The table version:

Criterion	Graph database	GraphQL	REST
Modeling	Best for relationship-native storage and traversal	Best for typed relationship contracts	Best for resources and commands
Querying	Excellent for multi-hop and path queries	Excellent for permitted composition	Excellent for known shapes
Security	Powerful, but risky as direct LLM surface	Good if auth lives in resolvers/business layer	Easiest to bound per endpoint
Speed	Strong when indexed for graph workloads	Resolver-dependent; needs batching and limits	Strong for fixed access paths
Deployment	Extra operational component unless embedded	Usually part of app/API service	Lowest footprint
Caching	Internal/query cache patterns	Trusted documents, GET queries, and object caches	HTTP-native caching is simplest for reads
Agent fit	Too much freedom if raw	Best schema affordance	Best reliable tool calls
Debuggability	Specialized	Good with tracing and cost controls	Easiest

No row has one universal winner. That is the point.

modeling and querying

Graph databases win the modeling argument when the domain is relationship-first and the queries are not known in advance.

For example:

Find the shortest credible path from this incident to a recent deploy, but prefer paths through services with recent ownership changes.

That is a graph query. You can implement it elsewhere, but the graph database is not pretending.

REST wins when the model is simple:

Who owns service X?

This should not require a graph traversal language. It should be a boring endpoint or a tool.

GraphQL sits in the middle:

Give me issue context, with owners, services, blockers, recent PRs, and the active incident thread.

That is a composed relationship read. It is not necessarily a graph algorithm. It is a context bundle. GraphQL is very good at context bundles.

The trap is letting the existence of a graph-shaped domain force a graph database decision too early. A lot of "context graph" products can start with Postgres, good indexes, and GraphQL resolvers. You materialize a graph store later when query shape and scale prove you need it.

security and access control

This is where people talk past each other.

One camp says: "Just self-host the model, then security is not a problem."

I mostly buy the direction and reject the conclusion.

Self-hosted or vendor-private models reduce one major risk: sending sensitive data to a third-party inference endpoint. That matters. For enterprise agents, I expect more workloads to run in a customer's VPC, on reserved capacity, or behind strict data controls.

But self-hosting does not solve authorization. It changes the blast radius.

The model can still be tricked. The user can still ask for data they should not see. The retrieved context can still contain secrets. A prompt injection in a Jira comment can still tell the agent to pull unrelated private context. An internal agent can still become a confused deputy: it has permissions the human caller does not have, and it uses them on the human's behalf.

So the question is not only:

Will the model vendor see this data?

It is also:

Should this model call see this data at all?

For graph databases, fine-grained authorization is possible but hard to align with product semantics. For REST, authorization is easy to place at the endpoint boundary. For GraphQL, the authorization story can be good if you are disciplined: resolvers call the same business services as the rest of the product, every field has a permission model, and broad traversal is constrained by query policy.

The GraphQL.org guidance points authorization back toward business logic rather than scattering it through presentation code. That is the right instinct. The mistake is building GraphQL resolvers that quietly bypass the permissions your REST API already got right.

For LLMs, I would add four rules:

1. Prefer allowlisted operations over arbitrary query strings.
2. Enforce depth, breadth, cost, and result-size limits.
3. Redact or summarize sensitive fields before the model sees them.
4. Log the user, tool, operation, variables, result size, and policy decision.

Self-hosted models make this easier to sell. They do not make it optional.

speed, indexes, and query planning

Graph databases are strongest when you repeatedly ask relationship-heavy questions and can index the nodes and relationships that anchor those traversals. Neo4j has explicit documentation around indexes and constraints for improving lookup and enforcing data integrity. FalkorDB similarly exposes graph indexing and OpenCypher-style query patterns.

But "fast graph database" does not mean "fast agent."

The slow part of an LLM workflow is often not the database lookup. It is over-fetching, serial tool calls, giant JSON payloads, resolver fanout, summarization, and model latency.

GraphQL can be fast or slow depending on resolver design. If each field performs its own database call, you get the classic N+1 problem. If resolvers batch by entity type, enforce query limits, and return context-sized payloads rather than raw archives, GraphQL can be very efficient.

REST can be extremely fast for fixed shapes. A precomputed /issues/{id}/agent-context endpoint backed by Redis will beat a flexible query system most of the time. The tradeoff is that every new context shape needs a new endpoint, version, or parameter set.

My performance rule:

Cache stable context bundles. Query live relationships only where freshness matters.

For "who owns this service," cache aggressively. For "who is currently on-call," fetch live. For "what did the agent already learn about this incident," read from the investigation store. For "what related issues exist," a graph index may be worth it.

deployment footprint

Neo4j is a real database. That is good when you need a real graph database and bad when you just wanted to answer three relationship questions. You now have backup, restore, migrations, monitoring, auth integration, memory sizing, cluster decisions, and local development setup.

FalkorDB is attractive when RESP/Redis-compatible connectivity, Docker deployment, OpenCypher-style graph access, and graph indexes fit your stack. FalkorDBLite-style embedded/file-backed stores are attractive for development, tests, local agents, and desktop tools because the operational footprint is smaller. For less-public embedded stores such as LadybugDB, treat them as product-local indexes until you have verified their persistence, transaction, concurrency, and migration guarantees.

GraphQL usually runs inside your existing API service. That is operationally boring, which is a compliment. The cost moves into schema ownership, resolver performance, query policy, and testing.

REST is easiest to deploy because it is what you are already deploying.

The deployment question I would ask:

If the graph layer is down, what stops working?

If the answer is "the whole product," operate it like a primary database. If the answer is "agent context gets less rich," maybe it should be an index derived from source systems, not the source of truth.

For LLM context, I strongly prefer derived indexes at first. The canonical ownership still lives in GitHub, Jira, Linear, Okta, your HR system, or your service catalog. The graph layer accelerates and shapes context. It should not become the surprise source of truth for the company.

caching and performance

REST has the cleanest caching story for reads because HTTP already knows what a resource is. GET endpoints, ETags, cache-control headers, CDN behavior, and surrogate keys are boring in the best way. POST can be cacheable only with explicit freshness information, but most teams should treat command-style POST tools as uncached unless they have a very specific reason not to.

GraphQL caching is not exotic, but it needs design. GraphQL.org points out that GraphQL can be cached like other parameterized APIs, and that GET query operations plus hashed persisted documents can support HTTP caches and CDNs. You can also normalize entity objects, cache resolver sub-results, or push caching into the backend. The trick is to avoid arbitrary ad hoc queries as the only interface. Trusted documents make GraphQL much more operationally friendly for agents because the shape is named, reviewed, measured, and cacheable.

Graph databases have their own internal cache and index behavior, but that does not replace API-level caching. The LLM tool should not retrieve the same "team ownership context" ten times in one session because each prompt worded the question differently.

For agent workloads, cache keys should include:

• user or actor identity
• tenant or organization
• operation name
• arguments
• permission scope
• freshness requirement
• redaction policy version

That last one matters. If you change what the model is allowed to see, stale cached context can become a data leak.

agent ergonomics

This is the under-discussed criterion.

An LLM is not a normal client. It is both useful and untrustworthy. It can read a schema and still make a bad call. It can follow examples and still over-fetch. It can be influenced by the data it retrieved. It can confuse "available field" with "appropriate field."

Raw graph query languages are poor tool surfaces for that caller. They require too much judgment at call time.

REST tools are excellent for agent reliability:

get_issue_context(issue_id)
list_blocked_work(team_id)
find_review_candidates(pull_request_id)
summarize_service_ownership(service_id)

Those are stable handles. They can be documented in the tool description. You can test them. You can add rate limits. You can return exactly the shape the model needs.

GraphQL is excellent one layer below that:

Tool: get_issue_context
Implementation: trusted GraphQL document IssueContextForAgent

The agent sees a narrow tool. The backend gets GraphQL's composability. Humans get a schema. Security gets a reviewable query. Operations gets a named unit of performance.

This is the architecture I would ship.

what I would build

For a first version of "who is working on what" for an LLM, I would not start with Neo4j.

I would start with:

1. A small domain model: Person, Team, Project, Service, Issue, PullRequest, Incident, Deploy.
2. Source-specific ingestion from the systems you already use.
3. A GraphQL schema that exposes only the relationships the product endorses.
4. Trusted or persisted GraphQL documents for common context bundles.
5. REST or MCP tools that call those trusted documents.
6. Strong query limits, field-aware authorization through shared business logic, audit logs, and redaction.
7. A boring relational store first unless traversal performance proves otherwise.

Then I would add a graph database when one of these becomes true:

• Relationship queries dominate latency or complexity.
• You need variable-depth traversal as a product feature.
• You need graph algorithms, not just relationship joins.
• You want a local embedded graph index inside an agent runtime.
• You need to combine symbolic graph traversal with vector retrieval for GraphRAG-style context.

At that point, put Neo4j, FalkorDB, an embedded graph index, or another graph store behind the GraphQL layer. Do not let the storage migration leak into the agent contract.

The public contract should survive backend changes. That is the point of a contract.

when I would choose each

Choose a graph database when:

• the graph is large enough and queried often enough to deserve specialized storage
• multi-hop traversal is core product behavior
• graph algorithms matter
• the team can operate the database properly
• the graph is not just a fashionable name for "some joins"

Choose GraphQL when:

• clients or agents need graph-shaped reads
• the schema itself is valuable documentation
• multiple source systems need one typed facade
• you can enforce resolver authorization and query limits
• you want to evolve context without endpoint sprawl

Choose REST when:

• the operation is narrow, high-risk, or mutation-heavy
• caching and auditability matter more than flexibility
• the model should call a verb, not compose a query
• you want the simplest possible production surface
• the question is stable enough to deserve a named endpoint

The mature system usually uses all three.

the opinionated answer

If your context graph is "who is working on what," do not start by asking whether to use Neo4j or FalkorDB.

Start by designing the graph contract you are willing to expose.

The LLM should not be spelunking through your company graph with arbitrary query privileges. It should be calling tools with clear intent. Under those tools, GraphQL is a strong fit because the domain is relationship-shaped, the schema is readable by humans and models, and the implementation cost has dropped now that LLMs can help with the boilerplate.

Graph databases are still excellent. REST is still excellent. But they are excellent at different layers.

My default architecture:

Source systems -> relational/graph/index storage -> GraphQL context API -> narrow LLM tools

The graph database is an implementation choice.

GraphQL is the context contract.

REST is the safety rail for commands.

That is less pure than picking one winner. It is also how I would want this system to behave at 2 a.m. when an agent is trying to decide who should be pulled into an incident.

references

• Neo4j documentation: graph database concepts
• Neo4j documentation: search-performance indexes
• FalkorDB documentation
• FalkorDB documentation: persistence on Docker
• FalkorDB documentation: FalkorDBLite for Python
• GraphQL.org: authorization
• GraphQL.org: security
• GraphQL.org: caching
• GraphQL.org: performance
• MDN: HTTP caching
• MDN: REST
• MDN: GET request method
• MDN: POST request method
• Model Context Protocol: tools