PERFORMANCE TESTING SHARING
Contents
I. Running JMeter in command line mode..................................................................................2
II. Apache JMeter Distributed Testing Step-by-step.................................................................4
1. Setup SSL for RMI...............................................................................................................4
2. Setup JMETER_HOME path...................................................................................................5
3. Run distributed test with GUI mode.....................................................................................5
4.1. Terminology......................................................................................................................6
4.2. Step-by-Step to run a distributed test................................................................................6
4.3. Starting the Test.................................................................................................................8
4.4. Start a single clients...........................................................................................................8
4.5. Start all clients...................................................................................................................8
4. Run distributed test from a CLI mode Client.......................................................................9
III. Some important note.............................................................................................................9
1
PERFORMANCE TESTING SHARING
I. Running JMeter in command line mode
Running JMeter using command line in non-GUI mode is very simple.
1. Open command prompt
2. Go into JMeter’s bin folder
3. Enter following command:
jmeter -n -t "ProjectTest\Simple Test.jmx" -l ProjectTest\testresult.jtl
Argument note:
-n: It specifies JMeter is to run in non-gui mode
-t: Name of JMX file that contains the Test Plan
-l: Name of JTL(JMeter text logs) file to log results
-j: Name of JMeter run log file
View test result:
Once your test is complete, you will see the following display:
2
PERFORMANCE TESTING SHARING
After completing the test, a JTL file will be generated at jmeter2.12/bin folder
Follow below steps to view test results from the resultant JTL file.
Open JMeter in GUI mode.
Add any listener Eg. View Results Tree.
Click Browse button of the file name field in listener.
Open testresult.jtl file.
You should be able to see the result in listener now.
3
PERFORMANCE TESTING SHARING
II. Apache JMeter Distributed Testing Step-by-step
This short tutorial explains how to use multiple systems to perform stress testing. Before we
start, there are a couple of things to check.
the firewalls on the systems are turned off or correct ports are opened.
all the clients are on the same subnet.
the server is in the same subnet
Make sure JMeter can access the server.
Make sure you use the same version of JMeter and Java on all the systems. Mixing
versions will not work correctly.
1. Setup SSL for RMI
The simplest setup is to use one key/cert pair for all JMeter servers and clients you want
to connect. JMeter comes with a script to generate a keystore that contains one key (and
its corresponding certificate) named rmi. The script is located in the bin directory and is
available for Windows systems (called bin/create-rmi-keystore.bat) and Unix like
systems (called bin/create-rmi-keystore.sh). It will generate a key-pair, that is valid for
seven days, with a default passphrase of value 'changeit'. It is advised to call it from
inside the bin directory.
When you run the script, it will ask you some questions about some names it will embed
in the certificate. You can type in whatever you want, as long the keystore tool accepts it.
That value has to match the property server.rmi.ssl.keystore.alias, which defaults
to rmi. A sample session to create the keystore is shown below.
$ cd jmeter/bin
$ ./create-rmi-keystore.sh
What is your first and last name?
[Unknown]: rmi
What is the name of your organizational unit?
[Unknown]: My unit name
What is the name of your organization?
[Unknown]: My organisation name
What is the name of your City or Locality?
[Unknown]: Your City
What is the name of your State or Province?
[Unknown]: Your State
What is the two-letter country code for this unit?
[Unknown]: XY
Is CN=rmi, OU=My unit name, O=My organisation name, L=Your City, ST=Your State,
C=XY correct?
[no]: yes
Copy the generated rmi_keystore.jks to jmeter/bin folder or reference it in property
'server.rmi.ssl.keystore.file'
4
PERFORMANCE TESTING SHARING
The defaults settings for RMI should work with this setup. Copy the
file bin/rmi_keystore.jks to every JMeter server and client you want to use for your
distributed testing setup.
2. Setup JMETER_HOME path
3. Run distributed test with GUI mode
Once you've made sure the systems are ready, it's time to setup remote testing. The tutorial
assumes you already have JMeter installed on all the systems. The way JMeter works is one
controller node initiates the test on multiple worker nodes.
In this tutorial we use GUI Mode just for demonstration. In real life you should use CLI mode
(NON GUI) to start your load test
5
PERFORMANCE TESTING SHARING
One controller node with multiple worker nodes
4.1. Terminology
Before we dive into the step-by-step instructions, it's a good idea to define the terms and make
sure the definition is clear.
Controller Node
the system running JMeter GUI, which controls the test
Worker Node
the system running jmeter-server, which takes commands from the GUI and send
requests to the target system(s)
Target
the webserver we plan to stress test
4.2. Step-by-Step to run a distributed test
1. Run Jmeter sever (Slave)
On the worker nodes, go to jmeter/bin directory and execute jmeter-server.bat (jmeter-
server on unix).
By default Jmeter server will start at port 1099 and one of your current IP randomly.
6
PERFORMANCE TESTING SHARING
If you want to open on the specific port:
jmeter -Dserver_port=<PORT> s -Djava.rmi.server.hostname=<Your Specific IP> -s -j
jmeter-server.log
EX:
jmeter -Dserver_port=1099 -s -Djava.rmi.server.hostname=192.168.1.191 -j jmeter-
server1.log
jmeter -Dserver_port=1098 -s -Djava.rmi.server.hostname=192.168.1.191 -j jmeter-
server2.log
2. Run Jmeter Master
On controller node acting as the console, open windows explorer and go
to jmeter/bin directory
Open jmeter.properties in a text editor
Edit the line remote_hosts=127.0.0.1
Add the IP address. For example, if I have JMeter server running on 192.168.0.10,
…, 192.168.0.15, the entry would look like this:
remote_hosts=192.168.0.10,192.168.0.11,192.168.0.12,192.168.0.13,192.168.0.14
Start JMeter.
Open the test plan you want to use
Simple test plan
7
PERFORMANCE TESTING SHARING
4.3. Starting the Test
At this point, you are ready to start load testing. If you want to double check the worker nodes
are working, open jmeter.log in your editor. You should see the following in the log.
Writing log file to: /XXXX/XXXXX/bin/jmeter-server.log
Created remote object: UnicastServerRef [liveRef: [endpoint:[192.X.X.X:XXXXX]
(local),objID:[-6a665beb:15a2c8b9419:-7fff, 3180474504933847586]]]
If you do not see this message, it means jmeter-server did not start correctly. For tips on
debugging the issue, go to the tips section. There are two ways to initiate the test: a single system
and all systems.
4.4. Start a single clients
1. Click Run at the top
2. Select Remote Start
3. Select the IP address
Start a single worker node
4.5. Start all clients
1. Click Run at the top
2. Select Remote Start all or use Ctrl + Shift + R
8
PERFORMANCE TESTING SHARING
4. Run distributed test from a CLI mode Client
GUI mode should only be used for debugging, as a better alternative, you should start the test on
remote server(s) from a CLI mode (command-line) client. The command to do this is:
jmeter -n -t script.jmx -r
or
jmeter -n -t script.jmx -R server1,server2,…
Other flags that may be useful:
The first example will start the test on whatever servers are defined in the JMeter
property remote_hosts;
The second example will define remote_hosts from the list of servers and then start the test on
the remote servers.
The command-line client will exit when all the remote servers have stopped.
Ex:
jmeter -n -t "ProjectTest\Simple Test.jmx" -l ProjectTest\distributed_cli_testresults.jtl -r
jmeter -n -t "ProjectTest\Simple Test.jmx" -l ProjectTest\distributed_cli_testresults.jtl -R
192.168.1.191:1099
III. Some important note
1. CPU utilization:
It is the percentage of CPU capacity utilized in processing the requests.
=> In all test scenarios, the CPU correlation graph at test time is recorded. Need to adjust the
source code to block if any occurs is at the time when CPU and RAM reach the threshold
2. Memory utilization:
This metric measures the utilization of the primary memory of the computer while processing
any work requests.
=> In all test scenarios, the CPU correlation graph at test time is recorded. Need to adjust the
source code to block if any occurs is at the time when CPU and RAM reach the threshold
3. Response times:
It is the total time between sending the request and receiving the response. Better the response
time, better the performance of website/application.
9
PERFORMANCE TESTING SHARING
If your server response time is under 100ms, it’s excellent.
Between 100ms and 200 ms is considered good.
Between 200ms and 1 second is considered acceptable, but should be improved.
10 Seconds, It is a maximum limit after which response time goes beyond the acceptable limit.
However, in today’s time, if response time exceeds 6 seconds, the user will leave that site or quit
the application.
Generally, response time should be fast as possible in the interval of 0.1 – 1 second.
However, people can adapt a slower response times, but they will never be happy with a
response time greater than 2 seconds. Lesser the response times, better is client satisfaction,
lesser costs, higher customer satisfaction.
4. Average load time:
This metric measures the time taken by a webpage to complete the loading process and appear on
the user screen.
=> The ideal website load time for mobile sites is 1-2 seconds.
In a recent study, the average load time for a web page was 3.21s.
Technology sites take on average 6.8 seconds to load.
Most pages should load in 2 seconds, some special pages may allow 6 seconds to complete
5. Throughput:
It measures the number of transactions an application can handle in a second, or in other words,
it is the rate at which a network or computer receives the requests per second.
Ensure that Througput increases relatively linearly, which is seamless and less broken
10
PERFORMANCE TESTING SHARING
6. Bandwidth:
It is the measurement of the volume of data transferred per second.
=> In all test scenarios, the correlation graph of Bandwidth is recorded at the time of test,
ensuring that the system must ensure enough bandwidth to work. Bandwidth is also a factor
causing system congestion. If the system is deployed in a cloud environment with unlimited
bandwidth, this factor can be temporarily ignored. If the system is self-hosted on premise, it is
necessary to ensure that the bandwidth purchased from the network operator is sufficient.
However always consider payload for each API should not exceed 600kb max
7. Requests per second:
This metric refers to the number of requests handled by the application per second.
8. Error rate:
It is the percentage of requests resulting in errors compared to the total number of requests.
Perform stress testing to find the maximum threshold when the system starts to fail.
Stable maximum load will count as 80% of peak load
If the system is configured to auto scale-out. Make sure that Error is not found at
system scale-out times.
9. Transactions Passed/Failed:
It is the percentage of passed/failed transactions against the total number of transactions.
Make sure that the DB connection can handle concurrent loads. Each DB has a limited
maximum number of connections depending on DB type and DB configuration. If the
requests concurrently query the DB, there will be a connection congestion with the DB.
In parallel with the performance test, it is necessary to record the transaction volatility
chart for comparison and to detect DB bottlenecks. Besides, it is necessary to apply more
techniques and architectures to ensure that this connection will not increase linearly with
system load
- Apply cache to limit reading DB
- Apply queues for processing that do not require realtime
- Apply cqrs pattern
- If the DB hardware enhancement cannot meet the load requirements, consider DB
cluster options such as cassandra.
11
