Skip to main content
  1. Botium Support
  2. General
  3. FAQ

I want to run tests in parallel - how do I do this?

It is common to shorten the feedback cycle by running tests in parallel. This article explains how to achieve parallel test execution using Botium Agents.

What are Botium Agents ?

Botium Agents are the work-horses that actually run the test cases. The number of agents depends on the Botium plan you have:

  • Free Botium Mini Plan: Shared pool of agents available to all users.
  • Paid Plans: Dedicated agents that allow parallelization.

Each agent will run exactly one test case at a time.

Parallel Test Execution

What’s happening behind the scenes is:
  1. There is a queue with test cases to process.
  2. Each agent processes the test cases it receives from the queue as quickly as possible.
  3. As soon as an agent completes a test case, it immediately pulls the next one from the queue.

Example Scenario:

As an example, let’s consider a scenario where 5 Botium Agents are available, and you have two test sets, a BIGTESTSET with 1000 test cases, and a SMALLTESTSET with 100 test cases.

By default, without any further configuration, the behaviour in Botium is to equally distribute the load onto all available agents by creating batches of test cases. So let’s say you start to tests in BIGTESTSET, and right afterwards in SMALLTESTSET.

  1. The queue is filled with five batches of 200 test cases out of the 1000 ones taken from the BIGTESTSET

  2. The workers begin to process the test cases - each worker takes one of the five batches

  3. In addition, the queue is filled with five batches of 20 test cases out of the 100 ones taken from the SMALLTESTSET

Outcome: The outcome in this scenario is:

  • All agents are first working on test cases from the BIGTESTSET in parallel

  • Afterwards, the test cases from the SMALLTESTSET are processed

So, although there is parallel execution, you have to wait for the BIGTESTSET to finish until the test cases from the SMALLTESTSET are processed.

Parallel Jobs Count

To avoid waiting for one test set to finish before starting another, you can define the degree of parallelism. This is done in the Test Project Settings section under Test Execution by setting the Parallel Jobs Count field.



This number gives the number of batches that Botium will push to the queue for the test set.

Let’s extend the scenario from above - for the BIGTESTSET, the Parallel Jobs Count is set to 3, for the SMALLTESTSET, it is set to 1.

Extended Scenario:

  • When launching the BIGTESTSET, there are three batches created (334/334/332)

  • Three of the agents begin to process it, two agents are still waiting for something to do

  • For the SMALLTESTSET, there are two batches created (50/50)

  • The two idle agents are processing them

So now the test cases from two different test sessions are processed in parallel.

Summary

  1. Single Botium Agent: No parallel test execution.
  2. Multiple Botium Agents: By default, test cases for a test session are equally distributed for parallel execution.
  3. Multiple Test Sessions: Specify the degree of parallelism using the Parallel Jobs Count field to run test cases from multiple test sessions in parallel.

Was this article helpful?

0 out of 0 found this helpful