(Previously QBox) Evaluate your Chatbot's NLP (Natural Language Processing) model comprehensively, and identify areas where intents overlap or concepts need reinforcement. This includes in-depth analysis of your NLP model performance.
Tip: To make this process quicker it is considered
best practice to setup your provider instances first before you create a test. If
you have not already done so, please do so now to make the test creation process
simpler - Add a New
Provider Instance
To create a test, do the following:
-
Navigate Using the Left Pane: Navigate to
Test Suite > NLP > NLP Advanced
-
Create: Click the Create test button to open up the test
configuration page.
-
Project Name: Enter a new project name in the field provided or,
alternatively you can select an existing project.
Tip: You can choose to edit permissions here if you wish. User Visibility and access to Direct Connections can all be configured here.
-
Test Name: Choose a name for your test. This will be grouped with all
other tests under the project you have chosen
-
Test Type: Choose what type of test you are performing. Your options
here are;
-
Automated test data: This is the quickest way to achieve a good
base level performance. This will create a test dataset from your chosen
training data.Tip: This is simply choosing a training data file in the next step. This is an export file which you can get from your NLP provider (Dialogflow, IBM Watson, etc...). Usually these are in the form of a JSON file or a .zip file.
-
Upload test data: Use this is you want to measure performance
against your own test dataset (from real user-interactions, UAT testing
etc.).Tip: This is your own training data which will upload in the next step.
-
Automated test data: This is the quickest way to achieve a good
base level performance. This will create a test dataset from your chosen
training data.
-
Training Data: Upload a file, or select a model from your connected
providers
Tip: If you choose to use a direct connection you may/may not have access to all connections. Check with your administrator if you do not see the connection you need. You can also setup a new direct connection under Account and Settings - Add a New Direct Connection
-
Optional: Click the Show advanced options link if you would like
to add a list of intents that you wish to exclude from testing. When doing so
you should separate each intent with a comma, or press the enter key.
Remember:Some things to remember if you choose to exclude intents:
- Any intents you list here will be excluded from the validation data but not from the training data. These exclusions will be remembered in subsequent tests for the same project.
- To exclude intents that follow the same naming convention, you can use wildcards; e.g. bt_* will match all intents starting with bt_.
- If no wildcard is specified then it will be treated as an exact match. All matches are case insensitive.
-
Results Output: You may select up to 3 providers here to run your test
against.
Tip: You may need to configure a new provider instance if you do not see them highlighted here. This is a quick and easy process. Just follow the link to setup a new provider instance.
