Security Testing with Botium

Introduction

Botium is providing you the option to test your chatbot against common security issues. OWASP Top Ten is a list of the current security threats for web applications and therefore also apply to chatbots endpoints hosted on HTTP(S) servers (which is nothing else than a conversational web application). This list is continuously updated by thousands of active community members. Based on this list the OWASP Zed Attack Proxy (ZAProxy) is developed and continuously adopted to the current threats.

Important: Security Testing in Botium is in no way comparable to a full security audit. It is able to highlight red flags and shows an entry point to serious security testing.

Note: Security Testing is part of the paid plans.

Security Testing in Botium consist of two components:

Continuous Security Testing: We built the ZAProxy into Botium Box, to continuously scan for security issues of your chatbot and solving them with the helpful information provided by ZAProxy.
Security Test Set: There is a test set available representing common hacking techniques for a chatbot exposed on a website (SQL Injection, XSS Injection).

Continuous Security Testing

With the standard installation, no additional setup is required. If you already have your own installation of ZAProxy, you can link it to Botium Box.

From a technical perspective, the ZAProxy is used as an HTTP(S) proxy and sits between Botium and your chatbot endpoint. This setup only makes sense when you are actually hosting your own HTTP(S) endpoint. It does not make sense to use continuous security testing for hosted 3rd party endpoints such as IBM Watson or Google Dialogflow.

These technologies use other HTTP(S) proxy settings than linked by Botium
The big SaaS providers already have minimum security vulnerabilities
- and if they have one, Botium and ZAProxy are not the right tools to find them

Activate Continuous Security Testing

As soon as you have defined your chatbot connection (most likely with the Generic HTTP(S)/JSON connector), test sets and a test project, enable Security Testing in the Features tab. When running the test project, the ZAProxy will be active.

After each test session, the identified security issues are shown in the Security Scan tab of the test results.

Note: When running it for the first time, no scan results will be shown - see below, Troubleshooting.

Setup your own ZAProxy

ZAProxy is reachable from Botium or a running internal ZAProxy.

This is how you would start ZAProxy in a docker-compose file:

zaproxy:
    container_name: zap
    image: owasp/zap2docker-stable:2.7.0
    command: [
      "zap.sh",
      "-daemon",
      "-host", "0.0.0.0", "-port", "8080",
      "-config", "api.disablekey=true",
      "-config", "api.addrs.addr.name=.*",
      "-config", "api.addrs.addr.regex=true"
    ]

In Botium, the environment variable ZAP_ENDPOINT is set to the ZAProxy endpoint. How to set environment variables depends on your installation.
```
ZAP_ENDPOINT="http://my-zap-server:8080"
```
Restart Botium

Security Report

Risk/Confidence

Risk is describing the impact of the security issue and Confidence is measuring the accuracy of the security threat.

Alert

Alert is describing the root cause of the security issue. In this case X-Content-Type header is missing.

Solution

Is a possible solution described to solve the security issue.

Reference Links

Those links are a references to blog articles or wikis, which are describing the security issue in detail.

Proposed workflow

Once you reached the state to test your chatbot against security threats, simply turn on the security check in the test project. Then you are running the tests → solving the security threats from the report → run tests → solving new appeared security threats → … till the security check is not finding any security threats anymore.

Security Test Set

The OWASP Top Ten is a list of the current security threats for web applications. Some of them are very content-specific and should be covered by conversational flow tests:

Broken Authentication and Broken Access Control can be addressed by a custom “Authentication & Authorization” regression test set
Sensitive Data Exposure is very application specific and should also be addresses with custom test sets
If there is XML processing involved, XML External Entities (XEE) should be part of a custom test set as well (for example, when using AIML)
Insecure Deserialization can be detected as part of UI tests

Other points from the list are convered by the Continuous Security Testing with ZAProxy as shown above:

Security Misconfigurations are typcially detected as part of security scans and spiders
Same applies to Using Components with Known Vulnerabilities

There is one point in the list, Insufficient Logging & Monitoring, which cannot be covered by tests at all, at it fully depends on the application.

Finally, there are points on the list which can be detected by writing custom test sets like a malicious hacker would try:

For Injection (SQL, NoSQL) an attacker would send malicious input typically interpreted by SQL or noSQL databases. If there is no data store involved, most likely you are safe.
For Cross-Site-Scripting (XSS) an attacker would send input interpreted by the web browser to inject malicious functionality into the website. If you chatbot doesn’t allow text input,most likely you are safe.

For these two, attached to this article are two test sets full of malicious input for XSS Injection and SQL Injection:

You can import those files with the Import to Local Repository function in Botium and let it run against your chatbot.

The XSS Injection makes most sense when running E2E tests in a web browser:

The SQL Injection should be run on the HTTP(S) endpoint of your chatbot (see Generic HTTP(S)/JSON Connector).

Troubleshooting

1. The first time I run a test with security checks enabled I don’t get any information displayed?

This happens just the first time after you start ZAProxy. The reason for that is that it needs time to build up the folder structure in the zap container and most of the time botium is to fast in executing tests. With the next scans zap already setup the folder structure and you get the alerts/warning immediately.

Welcome to the Botium Knowledge Center

Search

Introduction