Botium Fact Checker is a testing tool that checks if a bot is providing true answers or not. This is particularly useful for Large Language Model (LLM) based bots.
What is it?
- The tool creates a test set based on an uploaded source of truth with possible questions extracted from the source of truth document.
- When the test is run, the bots' answers to these questions are compared with the uploaded source of truth, and the status of either correct or incorrect is provided.
- The source of truth is a PDF document(s) (for now) that is provided by the user and will be used to generate test cases that will check that the answers the bot gives matches this source of information.
How to use it?
- Connect to your bot
- Create an Fact Checking Project
- Upload a source of truth and click save
- Click start new test session
- Review results, click on the results link to view the reasoning behind the
provided status.
