- Status
- Fertig
- Zeitraum
- 03/2023 to 08/2023, Hyderabad
- Rolle
- Machine Learning Intern
- Technik
- RAG
- Milvus
- GPT
1,000+
beta users
80%+
fewer off-topic answers
500+
test cases
Problem
The tutor had to stay on topic. Incorrect or off-topic answers were the gap I measured.
Approach
Treat the off-topic rate as a number. Evaluation pipelines flagged unsafe or off-topic outputs, and 500+ test cases showed whether a change helped.
Implementation
- Evaluation pipelines that catch unsafe or off-topic outputs.
- GPT models fine-tuned with guardrails.
- A RAG pipeline in front of the model. Documents are split with semantic chunking and stored in Milvus.
- An ingestion path that covered 10K+ documents.
What I measured
- 80%+ fewer incorrect or off-topic answers across 500+ test cases
- 35%+ higher contextual accuracy after adding the RAG pipeline
- 40% lower retrieval latency after semantic chunking and Milvus ingestion
These figures come from internal evaluation during the internship in 2023. The test sets are not public.
Current status
Shipped to 1,000+ beta users during the internship, which ended in August 2023.
What I learned
A guardrail is only real if I measure it. The 500+ test cases were how I knew the off-topic rate had dropped.