Parent topic: Reference: NLP Quality Metrics
The confidence deviation is a measure for the bandwidth of the predicted confidence score for all the utterances of an intent.
It is calculated as standard deviation of the confidence scores.
A high deviation of confidence scores typically means a mixture of strong and weak confidence scores within the user examples of an intent. A standard deviation of 0.2 - 0.3 usually is acceptable. Everything above should be handled.
Move the average confidence score to a higher range by adding more specific user examples or by removing the less specific ones
Remove the user examples with the weakest confidence score or replace them with more specific instances
Please sign in to leave a comment.