Create and search a vector database from a wide variety of file types and get more reliable responses from an LLM. This is commonly referred to as "retrieval augmented generation."
| Tool | Purpose |
|---|---|
| 🪟 Microsoft Windows | Only for Windows but open to pull requests |
| 🐍 Python 3.11–3.13 | Run the application |
| 🌿 Git | Clone / manage the repository |
| 🧲 Git LFS | Handle large model files |
| 📄 Pandoc | Document parsing support |
| 🛠️ Visual C++ Build Tools | Required for compiling dependencies |
What happens if the compiler is missing?
Correct SDK options to select during installation
Or you can run these commands in Powershell on Windows:
winget install Microsoft.VisualStudio.2022.BuildTools --silent --accept-source-agreements --accept-package-agreements --override "--wait --quiet --add Microsoft.VisualStudio.Component.VC.Tools.x86.x64 --add Microsoft.VisualStudio.Component.Windows11SDK.22621"Test-Path "C:\Program Files\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC"
python -m venv .
.\Scripts\activate
python setup_windows.py
| 📂 Ingest | 📄 .pdf, .docx, .txt, .html, .csv, .xls, .xlsx, .rtf, .odt 🖼️ .png, .jpg, .jpeg, .bmp, .gif, .tif, .tiff 🎵 .mp3, .wav, .m4a, .ogg, .wma, .flac |
| ⚙️ Process | 📝 Extract text from documents 🖼️ Generate descriptions from images 🎧 Transcribe speech from audio |
| 🧠 Store | All processed content is embedded and saved into the vector database for searching. |
| ❓ Ask | ⌨️ Type or 🎙️ record a question |
| 🧠 Retrieve | Relevant chunks are pulled from the vector database |
| 🤖 Generate | Sent to an LLM (Local Model, Kobold, LM Studio, or ChatGPT) |
| 💬 Respond | LLM returns an answer based on the context you provided |
| 🔊 Optional | Text-to-speech can read the response aloud |
Note
Instructions on how to use the program are being consolidated into the Ask Jeeves functionality, which can be accessed from the "Ask Jeeves" menu option. Please create an issue if Jeeves is not working.
Feel free to report bugs or request enhancements by creating an issue on github and I will respond promptly.
I welcome all suggestions - both positive and negative. You can e-mail me directly at "bbc@chintellalaw.com" or I can frequently be seen on the KoboldAI Discord server (moniker is vic49). I am always happy to answer any quesitons or discuss anything vector database related! (no formal affiliation with KoboldAI).
