EasyRAG

A compact, local-first Retrieval-Augmented Generation (RAG) prototype built as a learning project for exploring embeddings, vector search, document chunking, retrieval, and local LLM inference.

The project keeps the pipeline intentionally small and explicit: ingest text documents, embed them with Sentence Transformers, store vectors in Qdrant, retrieve the most relevant chunks, and pass that context to a local llama.cpp server.

How it works

data/docs/*.txt
      |
      v
  ingest.py
      |
      v
all-MiniLM-L6-v2
      |
      v
    Qdrant
      ^
      |
   query.py <--- user question
      |
      v
 top-3 chunks
      |
      v
    ask.py
      |
      v
llama.cpp / OpenAI-compatible API
      |
      v
 streamed answer

Features

• Local document ingestion from .txt files
• Sentence embeddings with all-MiniLM-L6-v2
• Cosine-similarity search in Qdrant
• Simple chunking for larger documents
• Basic metadata extraction from document headers
• Top-k retrieval before generation
• Local LLM inference through the OpenAI-compatible llama.cpp API
• Streaming responses in the CLI
• Small retrieval test set for quick sanity checks

Tech stack

• Python
• Qdrant
• Sentence Transformers
• llama.cpp
• Docker Compose
• Qwen3-4B Instruct GGUF in the provided local setup

Repository structure

EasyRAG/
├── data/
│   └── docs/              # Sample knowledge base
├── infra/
│   └── docker-compose.yml # Qdrant + llama.cpp
├── mini-rag/
│   ├── ask.py             # CLI + generation
│   ├── ingest.py          # Chunking, embedding and indexing
│   ├── query.py           # Vector retrieval
│   ├── tools.py           # Embedding helper
│   ├── test.py            # Retrieval sanity checks
│   └── requirements.txt
└── README.md

Quick start

1. Clone the repository

git clone https://github.com/Mefgner/EasyRAG.git
cd EasyRAG

2. Create a Python environment

python -m venv .venv
source .venv/bin/activate

On Windows PowerShell:

.venv\Scripts\Activate.ps1

Install the Python dependencies:

pip install -r mini-rag/requirements.txt requests python-dotenv

3. Add the local model

The provided Docker Compose configuration expects:

infra/models/Qwen3-4B-it.gguf

The current llama.cpp service is configured for NVIDIA/CUDA.

4. Start Qdrant and llama.cpp

docker compose -f infra/docker-compose.yml up -d

This exposes:

• Qdrant on http://localhost:6333
• llama.cpp on http://localhost:8080

5. Ingest the documents

Sample documents are already available in data/docs/.

python mini-rag/ingest.py

Note: ingest.py currently has TEST_MODE = True, so the mini-rag Qdrant collection is recreated when ingestion starts.

6. Ask questions

python mini-rag/ask.py

Example:

You: What is RAG?

Type exit to stop the CLI.

Retrieval checks

After ingesting the sample documents, run:

python mini-rag/test.py

The script contains a small set of expected document matches and prints retrieved files and simple hit/precision-style checks. It is intended as a lightweight experiment rather than a full RAG evaluation suite.

Current scope

EasyRAG is intentionally a small educational prototype, not a production RAG framework. The current implementation favors readable, direct code over abstraction and includes several areas for experimentation, such as better chunking, richer metadata, reranking, retrieval evaluation, configurable models, and more robust context construction.

License

See LICENSE.