From Embeddings to Ingress: The Complete Architecture Behind a Production RAG System

I built Arch Review around a very ordinary engineering problem: architecture knowledge rarely lives in one clean place.

It is usually spread across README files, source code, Pulumi stacks, Kubernetes manifests, migration scripts, dashboards, and half-remembered operational decisions. When a new engineer joins a project, or when a reviewer wants to understand the shape of a system, the question is not just “can I search the repo?” It is more like: can I ask a direct question and get an answer that is grounded in the actual project material?

That is the idea behind Arch Review: a cloud-native architecture review assistant for technical documentation and source code.

It is a RAG system, but I did not want to build a thin chat wrapper around an LLM. I wanted to build the whole path: intake, validation, chunking, embeddings, retrieval, answer generation, citations, a usable interface, persistence, deployment, and observability. The application matters, but so does the way it is operated.

What the application does

The product surface is intentionally small.

A user uploads technical files from the browser: Markdown, Python, TypeScript, YAML, TOML, JSON, or plain text. The backend validates them, splits the content into retrieval-friendly chunks, generates embeddings, stores the data in PostgreSQL with pgvector, and mirrors chunk nodes into Neo4j.

Once the material is ingested, the user can ask questions through a chat interface. The backend embeds the question, retrieves similar chunks, builds an evidence-constrained prompt, and returns an answer with citations pointing back to the source files and chunks.

The goal is not to make the model sound confident. The goal is to make the system useful when confidence has to come from evidence.

A typical question might be:

How is this system deployed?

or:

Which services are involved in storing retrieval context?

or:

What trade-offs does this architecture make?

Those are the kinds of questions that often take longer than they should, especially when the relevant information is split between application code and infrastructure code.

Backend shape

The backend is built with FastAPI, but the request handlers are deliberately thin.

I split the code into bounded contexts:

• intake handles documents, chunks, validation, chunking, embeddings, graph enrichment, and persistence.
• chat handles retrieval, prompt construction, answer generation, and citations.

Inside those contexts I used a ports-and-adapters style. The domain services and use cases do the application work; PostgreSQL, Neo4j, FastAPI, and LLM providers sit behind infrastructure adapters.

That separation is not ceremony for its own sake. It lets me test the ingestion and retrieval behavior without needing every test to spin up the entire platform. It also makes it easier to change implementation details later: a repository can move from in-memory to PostgreSQL, or a model provider can change, without rewriting the use case.

The ingestion flow looks roughly like this:

Upload file
  -> validate extension and size
  -> create Document entity
  -> split content into chunks
  -> optionally build hierarchical summaries
  -> generate embeddings
  -> create graph nodes
  -> persist document and chunks

The chat flow is similarly explicit:

User question
  -> embed question
  -> retrieve similar chunks from pgvector
  -> build evidence-only prompt
  -> call the LLM
  -> return answer plus citations

I find this kind of structure useful because it keeps the application honest. A RAG project can become a pile of glue code very quickly. Here, I wanted the retrieval pipeline to be something I could reason about, test, and improve.

Chunking is part of product quality

One thing this project reinforced for me is that chunking is not a boring preprocessing detail. It directly affects retrieval quality, latency, and failure modes.

Arch Review uses language-aware chunking where it makes sense: Markdown, Python, TypeScript, and JSON each deserve different treatment. For other structured text formats, the fallback should still be careful enough not to produce useless fragments.

This matters in small ways that become very visible. A package.json should not be split into chunks like {, }, and },. Those chunks are bad retrieval units, and they can also create strange downstream behavior when passed through summarization or embedding models.

The current direction is to keep chunking strategies explicit and format-aware:

• Markdown is split as documentation.
• Python and TypeScript are split as code.
• JSON is split structurally.
• YAML and TOML are handled as structured configuration rather than anonymous text.

That gives the retrieval system better raw material. The model cannot recover context that the pipeline destroyed earlier.

Retrieval and citations

For retrieval, I use PostgreSQL with pgvector. Each chunk stores its embedding, and chat queries are embedded and compared against stored vectors.

The answer generation step is constrained by retrieved evidence. The prompt tells the model to answer from the available context and mention the filenames used. The API returns citations containing the document id, chunk id, filename, snippet, and similarity score.

I like citations because they make the interface less magical. A reviewer can inspect where an answer came from, and a wrong answer becomes debuggable. Was the wrong chunk retrieved? Was the right chunk retrieved but interpreted badly? Was the source material itself incomplete?

Those are different problems, and a useful system should let you tell them apart.

The interface

The frontend is a Next.js application built around a practical review workspace.

It has two main areas: document intake and chat. That layout is intentional. Architecture review is usually iterative: upload a few files, ask a question, inspect the answer, realize another file is relevant, upload that too, then ask a better question.

The UI uses React, TypeScript, Zod, Tailwind, Radix primitives, and @assistant-ui/react. The chat adapter talks to the FastAPI /chat endpoint, while the intake form handles upload status and chunk counts.

I did not want the frontend to feel like a demo landing page. It should feel like a tool an engineer could leave open while investigating a system.

The architecture around the application

The second half of Arch Review is the platform work around the app.

The local environment includes:

• FastAPI backend
• Next.js frontend
• PostgreSQL with pgvector
• Neo4j
• OpenTelemetry
• Tempo
• Prometheus
• Loki
• Grafana

The deployment path includes Docker Compose for local development, Kind for local Kubernetes, Pulumi for infrastructure as code, Traefik as ingress, dbmate for migrations, and RustFS as a local S3-compatible backend used by the development infrastructure workflow.

That may sound like a lot for a personal project, but it is the kind of surface area I wanted to exercise. Many projects work in a notebook or a single container. Fewer projects show how the same idea behaves when it has to be deployed, observed, and reproduced.

The Makefile is part of that story:

make install
make dev
make test
make lint

Those targets are not glamorous, but they are important. A project that cannot be started, tested, and inspected reliably is harder to trust.

Observability as a design feature

I included observability from the beginning because RAG systems fail in ways that are not always obvious from the final answer.

The backend emits traces and metrics around ingestion, retrieval, LLM answering, database access, and HTTP calls. Prometheus tracks counters and histograms such as documents ingested, chunks created, chunks embedded, context chunks retrieved, and questions answered. Logs are structured and correlated through the observability stack.

This helps answer questions like:

• Is ingestion slow because of chunking, embeddings, graph writes, or database persistence?
• Are we retrieving too many chunks or too few?
• Did a file fail validation, enrichment, persistence, or graph processing?
• Is the answer slow because retrieval is slow or because the LLM call is slow?

I care about this because production engineering is often the art of knowing where to look next. Instrumentation is not decoration; it is how a system explains itself.

What this project is meant to show

Arch Review is partly a RAG application and partly a platform engineering exercise.

On the application side, it shows that I can build a full retrieval workflow: file ingestion, domain modeling, chunking strategies, embeddings, vector search, answer generation, citations, and a usable frontend.

On the architecture side, it shows the surrounding engineering habits I value: clear boundaries, testable use cases, infrastructure as code, reproducible local environments, migrations, observability, and deployment automation.

The interesting part is the combination. A lot of AI prototypes stop once the chat box answers something plausible. I am more interested in the next layer of questions:

• Can the system explain where the answer came from?
• Can I inspect and test the pipeline?
• Can another engineer run it locally?
• Can it be deployed in a repeatable way?
• Can it tell me what went wrong when it fails?

Those are the questions that move a project from demo to engineering.

Closing thoughts

Arch Review started as a way to make architecture knowledge easier to query, but it became a compact playground for the kind of engineering I enjoy most: building useful software while keeping the operational reality in view.

The AI part is important, but it is not the whole story. The value comes from the system around it: the ingestion boundaries, the retrieval design, the persistence model, the UI choices, the infrastructure, and the feedback loops that make the application observable.

That is the kind of work I like to do: connect product usefulness with engineering depth, and make systems that are understandable after the first happy-path demo is over.

This article was written with the help of artificial intelligence, as you might expect.