Best Practice: Humanification of Test Sets

Humanification in Botium stands for simulation of human behaviour or habits. Read on to know more.

Why Humanification ?

BotiumScript is a scripted, pre-defined or pre-recorded conversation, which makes it is easy to verify the chatbot’s ability to follow a conversation flow. It is an important step to recognize the need for automation in this area. But in real world, you cannot expect human users to act like a computer script:

For more information about the script, see BotiumScript.

typographic errors are introduced
different typing speeds
sausage finger syndrome
…

With Botiums Humanification layer it is possible to evaluate how your chatbot deals with typical human typing habits.

Note: Humanification is an additional layer on your existing test cases in Botium. Our recommendation is to use your own, clean test cases for automated testing, and from time to time use the humanification layer to verify your chatbots ability to handle those common input situations.

Prerequisites

Botium Enterprise Plan
Clean Test Set

Instructions

Step by step

Add Humanification Layer to Test Set

First Open the Test Data Wizard section of Botium and select the Humanification Transformer
Here you will see all available Test Sets that can be transformed, you can filter the list with the search box in the top right corner.
Select a Test Set to continue.
Go to the Transformation Wizard an open the Humanification Transformer
You can set a percentage of how many of the utterances will undergo the humanification by adjusting the slider bar or entering a percentage value in the box on the right.
Select from the list of Humanification Algorithms you want to apply (see below for details)
Click on Clone Test Set and Apply Humanification to clone the test set and apply the selected humanification algorithm(s).

Note: All the humanification algorithms will only be applied to utterances, not convo files.

Evaluate cloned test set

The test set has been cloned to a shared folder in your default folder location. It can be used as any other test set now. If you want to apply additional humanification, you will have to do the clone process again.

Example

Here is an example for a clean utterances list:

This is how the same utterance list looks with the humanification layer applied

Available Algorithms

Typing Habits

Case sensitivity - will convert everything to lower case
Duplicate space - will randomly duplicate usage of space key
Emojis - will randomly insert some emojis
Missing punctuation mark - will randomly remove punctuation marks

Common Typographic Errors

Double character - will randomly duplicate a character
Flip character - will randomly flip some nearby characters
Homophones - will randomly replace with some similar sounding words
Mishit - will randomly replace with a key located nearby on the keyboard
Missing character - will randomly omit some characters
Mixing character - will randomly mix up some characters
qwertz vs qwerty keyboard - will randomly replace mishits from different keyboard layouts
Sausage fingers - will randomly insert additional key strokes nearby on the keyboard
Shift characters - will randomly assume the shift key has/has not been pressed at the beginning of a word

Other Humanification Options

Botium provides additional humanification options to make your test cases appear human.

Delay user input with PAUSE

PAUSE is one of the integrated logic hooks, which will just wait for a defined amount of time.

See here for details.

Call Me Captain
A simple Test Case

#me
hello
PAUSE 5000

Note: When using this, your test cases will run in real time and will be significantly slower.

Simulate human writing speed

The capability SIMULATE_WRITING_SPEED simulates human typing speed: the longer the text to send to the chatbot, the more time it will take - just as if a human is typing.

Note: Average typing speed is about 290 ms/keystroke.

Summary

As you can see in the examples provided above, catering for a wide variety of different human interactions in your Chatbot is essential for it to function correctly. For a summary on how Botium can help humanify your Test Set data, watch the video below.

Welcome to the Botium Knowledge Center

Search

Why Humanification ?

Prerequisites