Turn a pile of documents into a private knowledge base you can question

The sequence

1. 1. Turn documents into clean text

   Scanned PDFs and heavily formatted Word files break here first. Parse them, then spot-check a dozen pages to confirm tables and headings survived. Most bad retrieval traces back to this step.

   

   MinerUOpen source★ 82.2KAccurately convert PDF and Office documents to Markdown and JSON, suitable for direct use with AI models.

   

   doclingOpen source★ 69.3Kdocling is a document parsing tool that converts PDF, Word, and other documents into AI-friendly Markdown or JSON formats, making it easy to feed to large models.

   

   markitdownOpen source★ 191.5KConvert various office documents to Markdown format with one click, convenient for writing and organizing.

2. 2. Build the index and retrieval layer

   This decides whether one question finds the right paragraph. Chunk size and whether you keep heading structure matter more than which model you pick.

   

   ragflowOpen source★ 92.6KRAGFlow is an open-source retrieval-augmented generation engine that enables large models to answer questions accurately based on your private documents.

3. 3. Test retrieval before tuning generation

   Write 20 questions whose answers you know the page of, and check only whether the right passage was retrieved. Wrong retrieval defeats any model; right retrieval lets an ordinary model succeed. This step gets skipped, and then everyone blames the model.

   

   ragflowOpen source★ 92.6KRAGFlow is an open-source retrieval-augmented generation engine that enables large models to answer questions accurately based on your private documents.

4. 4. Put a usable interface on top

   Connect retrieval to a chat UI and constrain the system prompt to answer only from retrieved passages, and to say so when nothing matches. Without that constraint the model starts inventing.

   

   open-webuiOpen source★ 156.4KOpen WebUI is an open-source AI chat web interface that connects to Ollama, OpenAI, and other large models, giving you a ChatGPT-like experience when self-hosted.