A homemade AI chatbot with Symfony, RAG and OCR
A chatbot that understands a municipal website, self-hosted, for about $3 a month in API costs. The interesting part isn’t any single trick. It’s how the pieces fit together.
The requirements
The chatbot answers questions about terlicko.cz, the website of the municipality of Těrlicko. The constraints were simple: a maximum of $50 a month for the OpenAI API, self-hosted, and no fine-tuning, which would have been too expensive to run on our hardware. In practice it costs around $3 a month and handles roughly 100 conversations a day.
Why build it instead of buying a SaaS chatbot? The data stays on our server, SaaS pricing is per message, we control the rules and the prompts, Czech needs custom handling, and building it is the best way to understand how this actually works.
RAG instead of fine-tuning
Retrieval-augmented generation means giving the model your documents as a cheat sheet. The model doesn’t know your data, so for every question you find the relevant pieces and hand them over as context. With dynamic CMS content and a small budget, RAG wins easily.
A few terms the rest depends on:
- Embedding: text turned into a vector of numbers, where similar meaning gives similar numbers. OpenAI’s embeddings give 1,536 dimensions per piece of text.
- Chunking: documents are split into pieces of about 1,000 tokens, so search finds the right paragraph rather than a whole PDF.
- Ingestion: extract, clean, chunk, embed, store. It runs on content change, not per question.
- Moderation: user input goes through OpenAI’s moderation check before anything else. Cheap insurance against abuse.
The architecture: two pipelines
Everything is one Symfony application on PostgreSQL with pgvector, plus Redis for caching. Content comes from a Strapi CMS, and the AI parts are OpenAI APIs: chat, embeddings, vision and moderation. The work splits into two pipelines.
Ingestion runs offline as a console command on cron, getting content into the system:
CMS pages ─┐ PDF files ─┼─→ extract text ─→ clean ─→ chunk ─→ embed ─→ pgvector images ─┘ (OCR if needed) ~1000 tokens
The query pipeline runs per request:
question ─→ moderation ─→ embed ─→ vector search
│
stream answer ←─ LLM call ←─ build context
The schema keeps documents, chunks and embeddings in separate tables on purpose. Separate chunks mean you can re-chunk without re-fetching. Separate embeddings mean you can swap the embedding model without touching the rest.
Hybrid search
Retrieval is a plain pgvector query on cosine distance, with a 0.7 threshold that filters out weak matches:
SELECT c.content, c.source_url, (e.vector <=> :vector) AS distance
FROM ai_embeddings e
JOIN ai_chunks c ON c.id = e.chunk_id
WHERE (e.vector <=> :vector) < 0.7
ORDER BY distance
LIMIT :limitPure vector search misses exact keywords, and keyword search misses meaning. So every question is searched twice: once as typed, and once after an LLM normalises it. The two result lists are merged with reciprocal rank fusion, which combines them by rank position rather than raw score:
public function dualQueryHybridSearch(string $query): array
{
// 1. Search with the original query
$resultsOriginal = $this->repo->findSimilar($this->embed($query));
// 2. Search with the LLM-normalised query
$normalized = $this->normalizer->normalize($query);
$resultsNorm = $this->repo->findSimilar($this->embed($normalized));
// 3. Merge with RRF (score = 1/(k + rank), k=60)
return $this->mergeWithRRF($resultsOriginal, $resultsNorm);
}Czech gets extra care on top: stemming, stop words and diacritics.
OCR for scanned documents
Municipal PDFs are sometimes real text and sometimes scans. Ingestion first tries plain text extraction with PdfParser. If it finds no text, the page goes to OpenAI’s vision model instead, and pure images go straight there. Rate limits are handled with exponential backoff.
Context and streaming
The matched chunks are packed into the prompt as numbered sources, up to a 2,000-token budget, so the answer can cite where it came from. The response streams back token by token with plain server-sent events: a Symfony StreamedResponse that echoes each chunk and flushes. No Mercure needed.
$response = new StreamedResponse();
$response->headers->set('Content-Type', 'text/event-stream');
$response->setCallback(function () {
foreach ($this->ai->stream() as $chunk) {
echo "data: {$chunk}\n\n";
flush();
}
});Lessons learned
- It’s mostly search. RAG is smart retrieval plus text generation.
- Start simple, then iterate. Ship the MVP first.
- Chunking matters a lot. Experiment with the size.
- Cache everything. Query normalisation is cached for 24 hours.
- Czech needs love: stemming and diacritics.
- Test with real questions early. Users ask weird things.
- Limit it. People will try to abuse it.
Users can rate answers, which closes the loop: failed questions show where chunking needs work, where content is missing from the CMS and which distance thresholds need tuning. The chatbot gets better over time. The obvious next step is going agentic, from answering questions to doing things like booking a room at the town hall, through tool use and MCP.
The whole thing is open source: github.com/JanMikes/terlicko.cz.