Skip to main content
  1. Botium Support
  2. Test Suite
  3. NLP
  4. NLP Quality Metrics

Intent Confidence Deviation Risks

The confidence deviation is a measure for the bandwidth of the predicted confidence score for all the user examples of an intent. The top 10 risks are presented in the chart. Continue reading to learn more.

Deviation risk is calculated as standard deviation of the confidence scores.
Note: Issues with the CLARITY of the test results will be visualized here.


Alerts

A high deviation of confidence scores typically means a mixture of strong and weak confidence scores within the user examples of an intent. A standard deviation of 0.2 - 0.3 usually is acceptable. Everything above should be handled.

Actions

  • Move the average confidence score to a higher range by adding more specific user examples or by removing the less specific ones

  • Remove the user examples with the weakest confidence score or replace them with more specific instances

Was this article helpful?

0 out of 0 found this helpful