Building & Evaluating a RAG Chatbot
Migrating Prompts to Open-Source Models

- Setting up a batch evaluation pipeline to run the prompt on both the original model (e.g., GPT) and the new target model (Mistral), while diffing the outputs.
- Analyzing the results, including accuracy scores, cost/latency metrics, and string output diffs, to assess the impact of migrating to the new model.
- Seamlessly updating the prompt template to use the new model (Mistral) if the migration is beneficial.