Parent topic: Reference: NLP Quality Metrics
Purpose
The confidence deviation is a measure for the bandwidth of the predicted confidence score for all the utterances of an intent.
It is calculated as standard deviation of the confidence scores.
Alerts
A high deviation of confidence scores typically means a mixture of strong and weak confidence scores within the user examples of an intent. A standard deviation of 0.2 - 0.3 usually is acceptable. Everything above should be handled.
Actions
-
Move the average confidence score to a higher range by adding more specific user examples or by removing the less specific ones
-
Remove the user examples with the weakest confidence score or replace them with more specific instances
Comments
0 comments
Please sign in to leave a comment.