Best Practice: Setup a Process for Test Case Development

One of the questions coming up in the beginning of every project: Where should I keep my test case scripts ? In an Excel file ? In a Git repository ?

Fortunately, this question is quick to answer: Store your test case scripts in a Git repository!

Test cases should be handled in the same way as source code, and the ***state-of-the-art method for storing source code is Git. ***Connect your Git repository to Botium and use it for versioning your test cases.

In this article, we describe the recommended Botium structure for test case development.

Local Repository vs Linked Repository

Botium comes with an integrated local repository: test case scripts and excel files are stored within the Botium database.

Botium also allows to connect to linked repositories: test case scripts and excel files are stored in a remote location linked to the Botium.

Currently supported linked repositories are:

Git Repositories, for example Bitbucket, Gitlab or Github
Shared Folders connected to the Botium server
Download Links to retrieve test case content from an external HTTP server or Google Sheets

Note: You should always keep the test cases in a linked repository, preferred is Git, as the local repository doesn't have many options for making backups, versioning etc

We will follow best practices from software development:

There is a development environment which will be used for developing the test cases
There is a production environment which will be used for running the finished and approved test cases
Snapshots of the test cases will be transfered from development to production environment on certain events (end of sprint, before go-live, …)

Preparing the Test Case Development Process

We will introduce a suggestion for the test case development process (yes, you should handle and call it like a software development process):

You are using the Live Chat, the Botium Crawler or the Convo Editor for composing your test cases
There is a Git repository for holding the test cases

We will create two test sets, one to place the test case scripts during development phase, and another one to use for the regular Botium test runs.

First Test Set: Development

This test set is for developing purposes. Make sure to select Use Test Cases from Botium only in the Content Selection Type of the Test Set Settings.

Connect your Git repository in the External Connections section.

Note: You should point to your Git main branch here. Later you can choose to not directly commit your test case scripts to the master branch but to separate feature branches.

In addition, create a test project pointing to your development chatbot and this test set.

Second Test Set: Ready2Test

Create a second test set. This time, select Use Remote Repositories only, and again connect your Git repository.

In addition, create a test project pointing to your chatbot under test and this test set.

Implementing the Test Case Development Process

As usual, this development process is running in cycles - you can and should repeat it as often as required.

1. Working on Test Cases

In the Development test set, you can use all Botium tools to create and edit your test cases, you can even combine all of them.

Use the Convo Editor or the Script Editor to manually compose your test cases step by step
Use the Test Case Wizard to generate test cases from your conversation model (works with IBM Watson, Dialogflow, QnAMaker and more)
Use the Live Chat to record your conversations with your chatbot and save it as test case
Use the Botium Crawler to automatically detect your conversation branches

Using Botium Excel sheets for setting up this process is not suggested, as Git features like version tracking and pull requests work best on plain text files.

2. Verifying Test Cases

You can use the Development test project to run test sessions against your development chatbot to verify that the test set itself is complete and working. As you selected Use Local Repository only above for this test set, only the local repository with your own test set will be used for the test session, not the test cases in the linked Git repository.

!!! note You can consider the Development test set as the sandbox where the unpublished test case scripts are placed, before they are commited to Git and published to other project members.

As soon as you consider your test set to be complete, it is time to publish it to Git.

3. Export Snapshot to Git repository

Use the Export function and select … to Git Repository to export a snapshot of your test set to the Git repository now.

The Git checkin screen will look familiar to software developers:

You can choose a branch to commit your test cases to (and your Git credentials if required)
There should be a checkin comment describing the changes you have done

We recommend to not push directly to the main branch but instead push to a feature or development branch and use the established Git workflows of Pull Request / Review / Merge for bringing the test cases to the main branch.

The test case scripts are exported to the Git repository as files, and the existing test case scripts will be overwritten.

Note: You can use all Git features in your Git repository now: feature branches, pull requests, revert changes, version history, ...

4. Setup Continuous Testing Pipeline

In your Ready2Test test set, you should now use the Update Test Case Statistics function, which will pull the main branch of the linked Git repository content to Botium and verify the test case content.

Now you are ready to use this test set in your Continuous Testing pipeline - Botium will pull the all changes on test cases from Git automatically on each test execution.

Conclusion

You now applied state-of-the-art software development methods to test case development with Botium.

Welcome to the Botium Knowledge Center

Search

Local Repository vs Linked Repository