Tests with Locust

Installation (local tests only)

Skip this section if you are using Docker.

Install Python3 and (optionally) pipenv.

pipenv is recommended for keeping your local Python3 environment clean. You can also use alternatives like virtualenv, venv, or pyenv.

Install locust with: pip3 install locust

Configuration:

You need to provide config.json file and put it in this directory.

It must contain your user credentials for accessing the PatentSight Web UI.

This is how it should be formatted. Simply copy/paste it and insert your credentials and folderId.

{
    "accounts": {
        "default_username": "[email protected]",
        "default_password": "my-pwd"
    },
    "databases": {},
    "servers": {
        "master": "https://login3.patentsight.com/master/webapi",
        "caching": "https://login3.patentsight.com/BRANCH/WEBAPI/Caching"
    },
    "runtime": {
        "test_against": "master",
        "folderId": "ADD_FOLDER_ID"
    }
}

Program logic and structure

To prevent redundant calls and to keep the folders clean, the tests should inherit from PsUserBehavior class.

This class contains a few methods that take care of user logons and token management.

The test logic is split between ps_searches.py that contain JSON representations of queries and ps_locustfile.py that contains several classes for client-server communication and execution of tests.

Adding new tests

To add a new test you'd have to provide a new JSON query in ps_searches.py as well as a new call with @task(NUMBER) decorator in a class that inherits from PsUserBehavior.

One such class is SavedSearchTasks from ps_locustfile.py. It of course depends on the test context if you are going to extend this class or develop a new one.

In any case, this class should inherit from PsUserBehavior, because you will need certain environment data coming from PatentSight servers.

Example: a test that creates a new saved search

1. Add a new method for creating JSON queries in PsSearchBundle class from ps_searches.py:

    def create_my_saved_search(self, search_name, folderId):
        return {
                    "description": "",
                    "folderID": folderId,
                    "name": search_name,
                    "searchFilter": {
                        "familiesType": 0,
                        "folderObjectID": None,
                        "name":"Filter name",
                        "options":{
                            "isFamilyOptionsExpanded": True,
                            "isSheetSliderShown": False,
                            "version": "2.0"
                        },"parts": [],
                        "reportingDates": [21],
                        "reportingDate": 21,
                        "userCreated": "",
                        "userLastUpdated": "",
                        "clientID": "1",
                        "objectType": 7,
                        "lastAvailableData": None,
                        "searchMode": 1,
                        "searchSyntax":"Owner=(Apple)"
                    },
                    "shareWithReferencesRequest": {
                        "confirmShareReferences": False,
                            "referencedSharingObjects": [],
                            "sourceSharingObjectSettings": []
                        }
                }

If your search is more complex, adapt the above JSON accordingly.

2. Add new test in SavedSearchTasks class from ps_locustfile.py:

@task(10)
def my_saved_search_test(self):
    payload = self.env.searchBundle.create_my_saved_search("MyUniqueNameForSavedSearch", "MyFolderId")
    response = self.client.post("/savedsearches/createAndShare", json=payload, name="My Task Group")
    # save responses for later processing, if needed
    if response.status_code == 200:
        self.env.add_search("MyNewSearch", "My Task Group", response.json())
    else:
        print(response.text)

In the above example I am using an environment object called env. This is an instance of the PsEnvironment class that contains data which should be persisted between different calls.

The decorator @task(10) is mandatory as it instructs the Locust framework to execute a method. The number is the weight of this test.

The higher the number the greater the chance that it will be executed more frequently.

Running tests

You can either run tests from console with locust or with Docker.

Local testing

Type locust -f ps_locustfile.py.

You'll see an oputput like this:

PS C:\Projects\PsTests> locust -f .\ps_locustfile.py
[2020-06-22 12:03:13,717] T470-Harris/WARNING/locust.main: System open file limit setting is not high enough for load testing, and the OS wouldnt allow locust to increase it by itself. See https://docs.locust.io/en/stable/installation.html#increasing-maximum-number-of-open-files-limit for more info.
[2020-06-22 12:03:13,718] T470-Harris/INFO/locust.main: Starting web interface at http://:8089
[2020-06-22 12:03:13,727] T470-Harris/INFO/locust.main: Starting Locust 1.0.3

Now you can open locust's web interface at http://localhost:8089 and run the tests.

Docker testing

You'll have to install Docker and Docker Compose first. There are variants for Windows, macOS and Linux available.

After you have started your Docker engine, enter this in your shell: docker-compose up --build

This command will kickoff the build and booting procedures. At the end, one Locust master instance with three workers will be running.

You'll see an output like this:

Successfully built 0209587e4ce3
Successfully tagged pstests_worker3:latest
Recreating pstests_worker1_1 ... done
Recreating pstests_worker2_1 ... done
Recreating pstests_master_1  ... done
Recreating pstests_worker3_1 ... done
Attaching to pstests_worker1_1, pstests_master_1, pstests_worker2_1, pstests_worker3_1
worker2_1  | [2020-06-22 11:42:38,120] f6b9471e438a/INFO/locust.main: Starting Locust 1.0.3
worker1_1  | [2020-06-22 11:42:38,123] 52bfeb1f7d60/INFO/locust.main: Starting Locust 1.0.3
worker3_1  | [2020-06-22 11:42:38,127] cf093a3d3c65/INFO/locust.main: Starting Locust 1.0.3
master_1   | [2020-06-22 11:42:38,141] ddc3882869ad/INFO/locust.main: Starting web interface at http://:8089
master_1   | [2020-06-22 11:42:38,163] ddc3882869ad/INFO/locust.main: Starting Locust 1.0.3
master_1   | [2020-06-22 11:42:38,324] ddc3882869ad/INFO/locust.runners: Client 'f6b9471e438a_fe9cd47a11664d6681318cd2151c4bb1' reported as ready. Currently 1 clients ready to swarm.
master_1   | [2020-06-22 11:42:38,325] ddc3882869ad/INFO/locust.runners: Client '52bfeb1f7d60_d1013619d30347b8b022f45b924c7f62' reported as ready. Currently 2 clients ready to swarm.
master_1   | [2020-06-22 11:42:38,326] ddc3882869ad/INFO/locust.runners: Client 'cf093a3d3c65_3ef2c831d7f6448592929f24db250410' reported as ready. Currently 3 clients ready to swarm.
master_1   | [2020-06-22 11:42:49,427] ddc3882869ad/INFO/locust.runners: Sending hatch jobs of 1 users and 6.67 hatch rate to 3 ready clients
worker2_1  | [2020-06-22 11:42:49,429] f6b9471e438a/INFO/locust.runners: Hatching and swarming 2 users at the rate 6.66667 users/s (0 users already running)...
worker3_1  | [2020-06-22 11:42:49,434] cf093a3d3c65/INFO/locust.runners: Hatching and swarming 1 users at the rate 6.66667 users/s (0 users already running)...
worker3_1  | [2020-06-22 11:42:49,435] cf093a3d3c65/INFO/locust.runners: All users hatched: WebsiteUser: 1 (0 already running)
worker1_1  | [2020-06-22 11:42:49,428] 52bfeb1f7d60/INFO/locust.runners: Hatching and swarming 2 users at the rate 6.66667 users/s (0 users already running)...
worker1_1  | [2020-06-22 11:42:49,580] 52bfeb1f7d60/INFO/locust.runners: All users hatched: WebsiteUser: 2 (0 already running)
worker2_1  | [2020-06-22 11:42:49,580] f6b9471e438a/INFO/locust.runners: All users hatched: WebsiteUser: 2 (0 already running)

Now you can open locust's web UI at http://localhost:8089 and run the tests.

Docker Screenshots

Locust Master Instance

Docker Node Statistics

Docker Node Console

Web UI Screenshots