Skip to main content
  1. Botium Support
  2. Test Suite
  3. E2E

E2E-Testing with Botium and Selenium/Appium

This article explains how to set up Botium as an End-to-End-testing environment connected to a device lab or browser lab.

Your chatbot needs testing, you know that — otherwise you wouldn’t read this article. Apart from the conversation flow, which is best tested on API level (see Appendices for a list of supported APIs and SDKs), you want to make sure that your chatbot published on a company website works on most used end user devices out there. Botium helps by connecting your BotiumScript test cases with your chatbot widget using a Selenium server, the current industry standard for testing web applications.

Step 1: Connect to Device Lab or Browser Lab

Botium supports cloud providers, but you can connect to your self-hosted Selenium server as well:

  • Saucelabs.com cross browser and real device cloud

  • Perfecto.io cloud

  • Experitest Mobile App & Cross-Browser Testing End-to-End

  • Local, self-hosted Selenium server

  • Headless Chrome



Note: The Selenium Endpoint has to be filled with URL of the Selenium server. For cloud providers, consult the documentation or contact the cloud provider support. You will have to enter username and password or access key as well. For local installations, consult your developers.

  • For Saucelabs, the Selenium Endpoint typically is https://ondemand.saucelabs.com/wd/hub, the Username and the Access Key you can find on the User Settings section in the Saucelabs dashboard

  • For Saucelabs Real Device Cloud (former TestObjects), you have to create a Test App first, switch to the Appium view and go to the Setup Instructions — you will find the Selenium Endpoint and the API Key there

  • For Experitest, you can find the Selenium Endpoint (typically https://cloud.seetest.io/wd/hub) and the Access Key in the Seetest Dashboard in the Automation view

Tip: See here for a guide how to setup your own Selenium Grid and connect it to Botium.

Step 2: Compose Device Sets

Next Step is to select the devices for running your test cases. Botium comes preconfigured with reasonable default device sets.
Tip: There are several guidelines available what devices you should select for your use case.



Tip: For cloud providers, the available devices and browsers are available for selection by clicking on the arrow underneath the Device Provider drop-down menu on the left.



  • For Saucelabs and Perfecto, the device list is retrieved in real-time from the Saucelabs API

  • For Experitest, the file EXPERITEST.json in the resources folder is used to show the selection of the available devices and browers

  • For a local selenium grid, the file LOCALSELENIUM.json in the resources folder is used to show the selection of available devices and browsers

See this guide for instructions on adding available browser configurations from your local Selenium Grid to Botium.

Step 3: Prepare Botium E2E UI/UX Testing Connector

To finish the configuration process, you now have to tell Botium how to connect to your chatbot. Make sure to select E2E UI/UX Testing as chatbot technology. Botium Tools & Settings > Chatbot Connections > Create new Chatbot



  1. If you ever worked with Selenium, you are aware that writing an automation script usually is a time-consuming task. Botium speeds up the development process by making reasonable assumptions:

    • There maybe is some kind of click-through to actually open the chatbot

    • The chatbot has an input text field, or some buttons to click (however it looks)

    • The chatbot outputs responses in some kind of list within a window (however it looks)

    All those assumptions can be parameterized for adapting it to your actual chatbot website.



  2. Screenshot Test Cases on Chatbot Widget: You can instruct Botium to take a screenshot of every test case on each device to get an overview of the visual appearance of the chatbot widget when running your test cases.



  3. Configuration for Common Chatbot Widgets: For some common chatbot widgets and websites, Botium provides out-of-the-box Selenium scripts (Microsoft Bot Framework Webchat, IBM Watson Assistant, and more to come …) — Botium will be able to detect the widget on the website automatically, based on Selenium selectors. Select the technology in the Webdriver Script field.



  4. Custom Configuration with Selenium selectors: With some know-how with Selenium Botium can be adapted to almost every webchat out there. The first thing to do is to analyze your chatbot widget to find out about the Selenium selectors to use — here is a guide what to look out for. The common Selenium flow in Botium is:

    1. Open the Start Url as given in the Botium configuration

    2. Do some clicks to open the chatbot window (if it is not opened automatically)

    3. Use the Input Element Selector to send some text to the chatbot

    4. Use the Output Element Selector to detect the responses of the chatbot

    Note: For rather simple chatbot widgets, the input and output selectors can be entered in the respective fields in the Botium configuration screen. If there are dynamic actions required to open the chatbot window (Point 2 from above), you have to do some Javascript coding and register the scripts in Botium.
  5. Custom Configuration with Javascript Snippets: There are several extension points where you can inject your own Javascript snippets. In most cases, the extension point you will have to adapt is the WEBDRIVERIO_OPENBOT extension point: it is responsible to open the chatbot widget and make it ready for Botium to start the conversation.

    Here is an example:

    • it waits for the cookie banner and clicks it away

    • it waits for a chatbot button to appear (#StartChat) and clicks it

      module.exports = async (container, browser) => {
  const ccBtn = await browser.$('#onetrust-accept-btn-handler')
  await ccBtn.waitForClickable({ timeout: 20000 })
  await ccBtn.click()  const startChat = await browser.$('#StartChat')
  await startChat.waitForClickable({ timeout: 20000 })
  await startChat.click()
}
      Tip: You can find more samples in the Github repository for the Botium connector.

  6. Adding your Javascript Snippets to Botium: Place the snippets in the corresponding section of the connector settings.



Step 4: Run Test Cases

When running test cases, you have to select the device set(s) to use. Apart from that, it behaves as any other test project in Botium. Click on Save and Start Testing to start the first test run.

Step 5: View Test Report

The generated test report shows every single detail of the test run, including the transcript, the expected outcome and, probably one of the most interesting information items, the screenshot(s) taken by Botium.



Selenium FAQ

Selenium Frequently Asked Questions

Was this article helpful?

0 out of 0 found this helpful