Welcome to the Botium Knowledge Center

This chart shows utterances with a high confusion probability based on confidence score. This is only eligible for NLU engines returning alternate intents with alternate confidence score.

• Utterances with low difference between primary and alternate predicted intent are classified as risky

• Probability is not meant in the mathematical sense, but it is simply 1 minus the confidence score difference between primary and alternate predicted intent

• The higher the value the higher the risk of mismatches

It basically is a measure of intent uniqueness: if mismatches happen, then either the training data is not tailored to the intents in question, or the intents are from a design view not really different. For example, when having “buy product A” and “buy product B” modeled as two different intents (“buy_A” and “buy_B”), the NLU engine may not be able to clearly decide between the two. In this case it would be a could decision to redesign the intent structure (having a “buy” intent and entities for “product A” and “product B”).

In general, there should be a significant difference between the confidence score of the primary and the alternate predicted intent. What is a significant difference depends on the application.

• The more user examples for an intent, the more acceptable is a low difference

• The more balanced the intents are expected to be, the less acceptable is a low difference

• In general, the problematic utterances should be adapted or removed from one of the intents.

• In case intents are overlapping from a content point of view, redesigning the intent structure is usually the best option (see above).

The Intent Mismatch Probability Risks show the intents where there are a notable number of mixings

• If Intent 1 is expected for an utterance and Intent 2 has been predicted, the pair Intent 1 / Intent 2 will be shown in this table

A special low-level analytics is possible with the Intent Mismatch Probability Risk, Alternative Intents table. This table lists:

• Intent pairs which often appear in the alternate intents list of each other

• If Intent 1 is very often part of the alternate intents of Intent 2 and ranked very high in the list, it will be shown here

• In addition, the worst utterance for the intent pair is shown - the mismatched utterance with the highest confidence