LocalLens: Conversational AI Agent

A LangGraph multi-agent system that answers questions about places by searching, reading reviews, and fact-checking its own summary.

Private · write-up

Answer grounding: verified

The problem

Asking a model about a local business gets you a confident answer assembled from training data that may be years stale, with no way to tell which parts came from anywhere real. The answer needs to be grounded in sources fetched at question time, and the grounding needs to be checkable.

Architecture

A LangGraph orchestrator runs a five-stage graph — intent, geolocation, multi-source search, review and sentiment, grounded summary — rather than one prompt doing all five badly.
Results stream to the client over SSE, so a multi-second graph traversal shows progress instead of a spinner.
Fetched pages are cached to disk, so re-asking a question during development does not re-scrape the internet.
A final fact-checking pass validates the summary against the retrieved source data before it is returned.

What I chose, and what I rejected

Splitting this into a graph rather than one large prompt was the decision that made it work, and the reason is about debuggability more than capability.

Why a graph, not a prompt

A single prompt asked to infer intent, resolve a location, search multiple sources, weigh reviews and write a grounded summary will do all five. It will also do some of them badly, and there is no way to tell which. The output is a paragraph; the failure is somewhere inside a black box; your only lever is to rewrite the prompt and see if the paragraph gets better. That is not engineering, it is bargaining.

With explicit stages — intent, geolocation, multi-source search, review and sentiment, grounded summary — a wrong answer is traceable to a stage. If the summary is about the wrong restaurant, geolocation failed and no amount of summarisation prompt-tuning will fix it. If the sentiment reads as glowing when the reviews are mixed, that is one stage with one job and one place to look.

That also changes what you can test. A stage with a defined input and output is testable in isolation with fixtures. A monolithic prompt is testable only end to end, against a live internet, which means it is effectively not testable.

The cost is latency and orchestration complexity — five sequential model and network calls instead of one — and that is what the streaming below is for.

The fact-checking pass is the part I would defend hardest

A grounded summary that is never checked against its sources is just a summary with citations attached. Citations that nobody validates drift from what they actually support: the model retrieves five sources, uses two, and cites all five, or asserts a detail that appeared in none of them and attaches the nearest-looking link.

Running an explicit verification step — does each claim in this summary actually appear in the retrieved source data — changes the failure mode from confident fabrication to refusal or omission. Those are not equally bad. A system that says “I could not confirm the opening hours” is usable. A system that invents plausible opening hours is worse than no system, because it destroys the user’s ability to trust the answers that were correct.

This is also where multi-agent architectures earn the overhead. The checker having a different job from the writer is what makes the check meaningful; the same call that wrote the summary asked to also validate it will validate it.

SSE, because a five-stage graph feels broken otherwise

A graph traversal that makes several sequential calls takes multiple seconds. Behind a spinner, that reads as a hang, and users refresh — which starts the whole traversal again.

Streaming the stages over SSE turns dead time into visible progress: resolving location, searching, reading reviews. The wait is the same length and feels entirely different, because the user can see the system working and can tell where it is. It also means a slow stage is visible to me in production rather than hiding inside an average.

SSE rather than WebSockets specifically because this is one-directional server-to-client streaming over plain HTTP. A WebSocket would add a connection upgrade, its own reconnect logic and a stateful server for no benefit — the client has nothing to say once the question is asked.

Disk caching, which turned out to be a correctness feature

I added caching for development speed: re-asking a question while working on the graph should not re-scrape the internet, both for iteration time and for not hammering the sources.

It became something more useful. Once responses were reproducible, it was possible to tell whether a change to the graph actually improved the output or whether the web had simply returned something different that day. Without a cache, every evaluation is confounded by live data, and you end up tuning a system against noise — convinced a prompt change helped when a review site merely reordered its results.

That is the underrated part of building on live sources: the environment is non-stationary, so reproducibility is not a convenience, it is the precondition for knowing anything at all.