This charts shows the intents of your test set with the weakest average confidence score:
All test cases which resolved to an intent are collected, together with the confidence score
For all intents, the average confidence score is calculated
A low average confidence score is an indicator that
either there are single utterances with very low confidence score having high impact on the average, due to the lack of training data for an intent
or for a large amount of utterances the NLU engine is not able to clearly decide for an intent
If the average confidence score falls below a threshold (depending on the NLU engine typically between 0.4 and 0.6), this is a sign that the training data has to be balanced and/or sharpened for the intents under question.
Investigate the details for the intents with low average confidence score.
In case there are single utterances having a high impact on the average, there is either additional training data needed for this intent, or the utterance is not specific enough and can be removed from the training data
Otherwise it is an indicator that the training data in general is not specific enough for the NLU engine, and it should be sharpened and enlarged by providing additional user examples for all weak intents.
Please sign in to leave a comment.