Skip to main content
  1. Botium Support
  2. Test Suite
  3. Performance

Create a Performance Test

You can imagine performance testing as a simple loop, which repeats a conversation to simulate human behavior.

Warning: Additional cost may be involved!

Please note that the performance tests are generated based on the Setup Test Parameters. The number of requests made to your chatbot can increase or decrease as these parameters are changed. Please pay attention to the 'max conversations' alert shown at the bottom of the screen, which will advise you on the number of requests that will be generated.



Create a Load Test

The slightest increase in Processing Time can be critical for a long running System like a Chatbot. This can be tested by sending a Constant Load. Botium generates Constant Load by simulating a constant number of parallel Users.

For these tests, every user is repeating a simple 'Hello' conversation. To simulate human interaction speed a convo is repeated max once per second, which is usually much more load than an average real life user generates.

Steps:

  1. Navigate Using the Left Pane: Navigate to Test Suite > Performance and click the + CREATE TEST button to create a new project.
  2. Select Test Mode: Select the LOAD TEST option.
  3. Test Project Name: Give your project a meaningful name.
  4. Select a Chatbot: Select your chatbot technology from the dropdown menu.
  5. Namespace: Optional - Add a namespace to associate with your project. For more information on namespaces see Botium Namespaces

  6. Setup Test Parameters:

    1. How long should the test run (in minutes)
    2. How many parallel users should be simulated (max 1000)
    3. What is the required percentage of users



Create a Stress Test

This is an example setup which can be used for first time users.
Note: We have two goals with these settings;
  1. Getting a first impression about the max parallel users the chatbot is able to serve
  2. Discovering how the chatbot breaks when under extreme load
In production your chatbot could have much more load than expected, so you have to test it.

By default all parallel users are having a very simple conversation with the bot, saying just ‘hello’ for example. They wait for an answer and repeat the conversation until the end of the test is reached. Any response coming from the chatbot is accepted. So if the chatbot answers sometimes with ‘Hello User’, and sometimes with ‘I don’t understand’, it has no impact on the performance results. The main goal here is to get an answer, while a http error code, or lack of answer is interpreted as failure of course.

Steps:

  1. Navigate Using the Left Pane: Navigate to Test Suite > Performance and click the + CREATE TEST button to create a new project.
  2. Select Performance Test Mode: Select the STRESS TEST option.
  3. Test Project Name: Give your project a meaningful name.
  4. Select a Chatbot: Select your chatbot technology from the dropdown menu.
  5. Namespace: Optional - Add a namespace to associate with your project. For more information on namespaces see Botium Namespaces
  6. Setup Test Parameters:
    1. How long should the test run (in minutes): Set the duration for the stress test.
    2. With how many Parallel Users should the Test start: Set the initial number of parallel users to begin the test with.
      Tip: Every User is repeating a simple 'Hello' conversation.
    3. With how many Parallel Users should the Test end: Set the maximum number of parallel users
      Tip: The number of parallel users will increase after each test step until the desired number of parallel users is reached.
    4. Required percentage of Successful Users: If the failed responses exceed the required percentage of successful users, the test will stop.
Note: In this stress test example, we simulate an increasing number of users in the pattern 1, 3, 5, 7, 9, 11, 13. Starting with 1 initial user, the number increases at each test step for a total duration of one minute, with each step lasting 10 seconds. This results in:
  • 6 test steps
  • 2 new users per step (every 10 seconds)
  • 11 maximum users



Was this article helpful?

0 out of 0 found this helpful