Skip to main content
  1. Botium Support
  2. Test Sets
  3. Test Set Insights

What are Test Set Insights?

The Test Set Insights in Botium provide a rough estimation on the quality of your training data. A Static Analysis is done, which means that the data is not actually evaluated with an external chatbot engine like IBM Watson or Dialogflow, but just with an integrated state-of-the-art generic language model. Nevertheless, it provides some good first insights what might be wrong with your data.

Test Set Insights vs Botium NLP

Test Set Insights provide:

  • static analysis against an in-memory generic language model

  • quick evaluation

  • provides metrics on the data itself, the keywords and the similarity

Botium provides:

  • evaluation against your chatbot engine of choice

  • possible long-term execution sessions required

  • provides metrics on how the data behaves in real-life

Test Data vs Training Data

Generally spoken, you should use all the training data available for getting the most out of the test set insights. Doing this kind of evaluation on test data only will not yield good results.

Enable Botium Worker

You have to enable the optional component Botium Worker for using the test set insights. In docker-compose.yml or docker-compose-standalone.yml:

  • Uncomment the environment variable COACH_WORKER_ENDPOINT

  • Uncomment the coach container section to enable the container

Tip: If the worker is running with API Key protection, add the environment variable COACH_WORKER_APIKEY

Using Test Set Insights

The test set insights are available in the Insights section of the test set. You can see descriptions of the available metrics and the history there.

The Test Set Insights are automatically recalculated every time the test statistics are updated on the Test Set Dashboard, or by clicking the Update Insights with Latest Test Set Data button in the Insights section.
Tip: If you're not interested in the insights, you can skip the implicit calculation with the Skip Auto-Calculation for Test Set Insights switch in the Test Set Settings.

Was this article helpful?

0 out of 0 found this helpful