Skip to main content
  1. Botium Support
  2. Technical Reference
  3. Botium Asserters

NLP Asserters

Natural language enabled chatbots are using some kind of NLP engine in the background to recognize intents and entities for the user input. This information is not shown to the user directly. It may make sense to assert for the recognized intents and entities instead of the text response of the chatbot - or you can even use it in parallel (assert text and intent confidence for example).

Some NLP engines, such as Microsoft Luis, do not manage conversation flow and only provide NLP analysis (like intent and entity recognition). For these engines, you can't verify the response content (like text or buttons); you can only check the NLP analysis using NLP Asserters.

You can use NLP Asserters to gather statistics by comparing expected results with the actual responses from the NLP engine.

Note: Not all Botium Connectors can use NLP Asserters; it depends on whether the chatbot technology exposes the necessary information to Botium. For instance, the Dialogflow and IBM Watson connectors do support these asserters. You should check the connector documentation for details.

NLP Intent

INTENT (arguments: intent name to look out for), used in #bot section, will assert that bot answered with the specified intent.



NLP Intent Confidence

INTENT_CONFIDENCE (arguments: minimal accepted confidence, like “70” for 70%), used in #bot section, will assert that bot answered with at least the specified minimal confidence.

Tip: The INTENT_CONFIDENCE asserter can be used as global asserter to make sure the recognized confidence is always higher than a defined threshold.

NLP Entity

ENTITIES (arguments: expected entities like “from|to”, or minimal entities like “from|…” ), used in #bot section, will assert that bot answered with the specified entities.

NLP Entity Values

ENTITY_VALUES (arguments: expected entity values like “2018|2019”, or minimal entity values like “2018|…” ), used in #bot section, will assert that bot answered with the specified entity values.

NLP Entity Content

ENTITY_CONTENT (arguments: entity and expected values like location|Budapest|Vienna)

  • One ENTITY_CONTENT asserter checks only one entity. Use more asserters to check more.

  • Does not fail if the response has more values as specified in arguments.

Example Botium Scripts

Imagine a chatbot taking orders for pizza delivery. It has a well-defined inventory of possible pizza sizes and toppings. The recognized intent, entities and the confidence should be asserter:
#me
please send me two salami pizza

#bot
INTENT I_ORDER_PIZZA
INTENT_CONFIDENCE 70
ENTITIES E_PIZZA_TYPE|E_FOOD
ENTITY_VALUES salami|pizza
Please select the size of the pizza
Using ENTITY_VALUES asserter can be confusing sometimes. This assertation will be valid:
#me
I want to travel from Berlin to Vienna.

#bot
Im happy to hear it. And where are you now?
INTENT travel

#me
in Münich.

#bot
So you are in Münich, and want to travel from Berlin to Vienna?
You will travel to Berlin on your own?
INTENT travel
ENTITY_VALUES Berlin|Vienna|Münich
But maybe it is not what you want. You can be more specific using ENTITY_CONTENT asserter:
...
ENTITY_CONTENT FROM|Berlin
ENTITY_CONTENT TO|Vienna
ENTITY_CONTENT LOCATION|Münich
Note: This example works only on Dialogflow; it aggregates entities.

Using the Intent Confidence Asserter globally

A very common use case is to use the Intent Confidence Asserter as global asserter, to make sure to filter out the weakly resolved intents. To make all conversation steps fail where the intent falls below a confidence of 80, add this section to your botium.json:
{
  "botium": {
    "Capabilities": {
      ...
      "ASSERTERS": [
        {
          "ref": "INTENT_CONFIDENCE",
          "src": "IntentConfidenceAsserter",
          "global": true,
          "args": {
            "expectedMinimum": 80
          }
        }
      ]
    }
  }
}

Was this article helpful?

0 out of 0 found this helpful