Welcome to the Botium Knowledge Center

The scripting language behind Botium test cases is called BotiumScript and supports several file formats:

• plain text

• JSON

• YAML

• Markdown

• CSV

• Excel and Google Sheets

While it might seem tempting to use Excel for writing your test cases because everyone out there is familiar with using spreadsheets, this quickly becomes hard to oversee and maintain when there are hundreds of test cases spread over multiple worksheets.

The clear suggestion is to use one of the text-based formats and setup a Git integration with Botium.

This is maybe a no-brainer. The suggested naming conventions are:

• Utterance names should start with UTT_

• Partial convo names should start with PCONVO_

• Convo names should start with anything you choose for naming your test cases, for example TC04711_…

• Consider splitting up your test cases in convo files and utterance files, in case you already didn’t do so. There are several benefits:

• A clear separation of concerns between the conversational flow and the actual conversation content

• Extensive NLP testing with Botium NLU/NLP can only be reached with massive amount of user examples, and this is only doable with separate utterance files

• Some of the awesome tools in Botium are working exclusively on utterance level

  • Test Set Insights

  • Utterance Analytics as part of Botium NLU/NLP

  • Humanification

  • Test Set Translation

  • Test Case Wizard to upload/download conversation models to/from IBM Watson, Google Dialogflow and more

• It makes it easier to build up multi-lingual test scenarios (see below)

With partial convos it is possible to reuse parts of a convo in multiple places. For software engineers, partial convos provide modularization and separation of concerns. Use them at least in any of these cases:

• When having a common greeting / goodbye section in all of your test cases, you can have two partial convos to be included in the “real” test cases.

• If you have to setup conversation context before starting some test cases, the setup process should be extracted in a partial convo. A typical example is a user login process.

The awesome thing is that in partial convos, you can do everything that you can do in full convos, including utterance expansion, using scripting memory, assertions and logic hooks and more.

In Botium it is possible to define dependencies between test sets. When running test sessions, all content from dependent test sets is loaded as well. Possible use cases:

• When working on multiple chatbot projects, you can have basic test cases for greeting, goodbye, smalltalk etc in a shared Smalltalk test set and set this as dependency for the specialized test sets

• When working on multi-lingual chatbot projects you can have multiple test sets

  • one shared test set holding the convo files with the conversational flow of the test cases - only utterance names are used here, no text content

  • for each supported language on test set holding the utterance files for this language used in the convo files, and maybe some additional language-specific test cases

  • the language test sets are dependent on the shared test set holding the conversational flow

    

Use the Botium Crawler for detecting uncovered branches and updating your test sets accordingly. A suggested setup is similar to the suggested development workflow:

• Run Botium Crawler for the first time to detect all conversation flows

• Copy the resulting test scripts into an empty test set and do a first-time export to the main branch of a linked Git repository

• At time, re-run the Botium Crawler to detect new conversation flows

• Copy the resulting test scripts into another empty test set and export to the same linked Git repository as before, but into a feature branch

• Use Git pull requests to review the detected changes in the test set