Regression testing involves comparing a new version of a system against a previous
benchmark to ensure no functionality has been lost. With Botium, you can simulate typical
human conversations with your chatbot or virtual assistant, using these transcripts as test
data (test cases).
Running these test cases automatically helps identify flaws in your conversational AI's
dialogs. Botium's regression testing checks your chosen assertions, such as text,
buttons, and cards, against the bot, and the test fails if any assertion fails. Botium’s
wide selection of asserters aids
in creating more complex test cases. It may seem simple, but it's crucial for ensuring
the smooth operation of your bot. Figure 1. Botium Asserters Figure 2. Regression Test - Failed results
What does it test?
A regression test set should cover the bot's "happy path", which refers to the ideal,
intended flow of a user interaction. This ensures that the functionalities work
correctly under typical user scenarios.
Why do it?
Regression testing is essential
for several reasons:
As bots evolve with updates and new features, regression testing ensures that
previously working functionalities continue to work as expected. It helps in
maintaining the overall quality and reliability of the bot.
Changes made to the bot codebase or its environment can introduce new bugs or
unintended behavior. Regression testing helps in detecting these issues early,
preventing them from reaching users.
Bots often integrate with various systems and services. Regression testing
ensures that these integrations are functioning correctly even after updates or
changes to those systems.
Users expect consistent behavior from a bot. Regression testing helps in
ensuring that the user experience remains smooth and uninterrupted despite
changes made to the bot.
If the bot relies on external data sources or databases, regression testing
helps in verifying that data integrity is maintained after updates or
changes.
Note:Botium
Crawler can simplify the creation of your regression test set. This tool
can help you automate the process of generating test cases for your bot's "happy
path" scenarios.
Who benefits from this?
This testing process
is particularly useful for roles like QA testers and conversation designers who are
responsible for ensuring the bot's functionality and user experience.
