A robust, secure and scalable system: local users, personal workspaces, data connectors and isolated RAG for every account.
PDF, TXT, Markdown and audio are automatically chunked. Manual uploads and connector snapshots land in the right workspace, with source metadata tracked.
Vector similarity on dedicated Chroma collections per workspace. Optional Source Diversity or MMR selects better candidates before BGE local, Regolo remote, or custom reranking. Namespaced cache prevents leakage.
Contextual chat with auto-summary, namespaced by user and conversation. When history exceeds threshold, older messages are compressed into a summary.
Upload audio → STT (OpenAI-compatible) → automatic indexing. TTS for spoken responses. Supports MP3, WAV, M4A, WEBM, FLAC.
Local users with admin/user roles, password hashes, scoped API keys per workspace, encrypted connector SecretStore, rate limiting, input validation and XSS sanitization.
Gunicorn async with thread pool, real-time NDJSON streaming, LRU retrieval cache, embedding caching, retry with backoff on provider failure.
/api/v1/query, /api/v1/health, /api/v1/files, /api/v1/audio, /api/v1/tts, /api/v1/models
Token-by-token streaming in NDJSON. Meta events with model info, source references and token usage.
API keys with scopes: query, ingest, speech. Every key always resolves to one user and one workspace.
RAGuardian scales from a single Gunicorn worker to a multi-process runtime with Redis, keeping user isolation, the same public APIs and zero state loss.
Rate limiter, retrieval cache, conversation memory, job state and active lock move to Redis. In-memory fallback for dev. No changes to public endpoints: only the storage layer changes.
Index rebuild, PDF/audio upload and transcription become async jobs (RQ on Redis). User queries are never blocked by ingest. Admin shows status via polling; API returns 202 + job_id with ?async=true.
Production default: workers=1, threads=16. Advanced profile: 2-4 workers, 8-16 threads after load testing. Multi-worker safe only with Redis: no lost conversations or jobs.
Internal VectorStore interface with Chroma PersistentClient implementation. Ready for Qdrant, Chroma HTTP or managed vector DB when load tests require it. Zero refactoring at switch time.
Locust suite: 50 concurrent users on queries, streaming, ingest and parallel rebuild. Metrics: error rate, p50/p95/p99 latency, queue depth, job time, CPU/RAM, separate Chroma and LLM latency.
/api/v1/health includes: redis_ready, queue_ready, queue_depth, active_jobs_count. Your orchestrator always knows if the service is ready and overloaded.
Chroma local stays as the initial vector store. Switching to Chroma server, Qdrant or managed vector DB only happens after real load tests identify the bottleneck. No premature optimization, only data-driven decisions.
Read the code on GitHub, explore API documentation and plan deployment.
RAGuardian