Mobile App development does not stop when you build your app and have a binary ready to be installed on the device. Regardless of how good your code is or how much unit and regression testing you performed, there are elements that need to be tested under different circumstances, for example, data traffic, the number of users, location, and high latency in the mobile network.
At this point, what you want to test is app performance, which means “responsiveness” of your app, and one way to test is Load Testing. The testing will cover the different variables or circumstances unique to mobile apps.
Another good reason to test your app performance or responsiveness through load testing is the user engagement factor, it is well documented that the attention span of mobile app users is very short, and many mobile consumer and enterprise apps are only used once and never used again because the performance was disappointing.
You probably don’t need much more convincing, but here a few more good reasons:
- The cost of failure, the cost of an unsuccessful app launch, think about that for a second.
- Find the limits of your app, the maximum work capacity before performance degradation.
- How is it going to perform on anticipated peak load conditions, for example, holiday sales that increase the app usage?
- Identify scalability enhancements.
- Improve architecture to remove performance bottlenecks
Once you perform load testing and address any identified issues, you can complete your mobile app development and be confident to release your app in production.
How to load test mobile apps?
You can spend time and effort on writing complex scripts and tooling to generate load testing or use one or a combination of load testing tools, some of them open source.
Here are some questions to ask when selecting a testing tool:
- Can you load test with scripting on different programming languages and protocols?
- Do the tools support your desired mobile platform (iOS, Android or Windows) or protocols?
- Can you load test both client app and backend integrations?
- Can you also include on-device testing?
- Do you get log files, graphs, charts or other measurements?
- Can you load test with different network bandwidths?
- Can you integrate your load testing with CI/CD tools?
How do you prepare for load testing?
You can make load testing an ongoing task throughout your development lifecycle. Best practice is to involve not only your Developers and Testers but also your Operations team. In the case of hosted mobile platforms like Red Hat Mobile Application Platform, we highly recommend you provide advance notice to the Red Hat Cloud Operations team so they can monitor and be ready for potential traffic spikes and computing capacity limitations.
Plan and decide what you want to test. It could be at different levels, for example, an increase of individual users/sessions accessing the app and/or repetitive use of the same function or service; individual performance of features and/or individual performance of backend integrations and API calls. All these tests will help you to narrow down and quickly identify performance issues or degradation.
You also have to decide on what you are going to measure and how you are going to record those measurements, think beyond Response Time and Throughput, look for memory and CPU impact, memory leaks, and the performance from an app user’s behavior perspective.
Load testing tools
Here are some options recommended by developers doing load testing on mobile apps.
JMeter (open source)
One of the most used open source tools to load test mobile apps is JMeter. You can build JMeter definitions that simulate a user logging and then interact with the backend. It can be used to simulate a heavy load on a server, network or objects to analyze overall performance under different load types. It can also be used to perform functional tests on web apps, databases, LDAPs, web services, etc., and reports can be added to the GUI, even record, and playback scenarios. A popular feature is the ability to record JMeter scripts with a mobile device by connecting the device to a proxy that you set up inside JMeter. Once we set up this proxy, you can then connect the mobile device to it in an effort to test various iOS and Android mobile applications.
Locust (open source)
Locust is an easy-to-use, distributed, user load testing tool. It is intended for load testing websites (or other systems) and figuring out how many concurrent users a system can handle. Locust is completely event-based, and therefore it’s possible to support thousands of concurrent users on a single machine. In contrast to many other event-based apps, it doesn’t use callbacks. Instead, it uses lightweight processes, through gevent.
BlazeMeter is a self-service, on-demand, cloud-based load testing tool. You can write tests and code in a domain specific language (DSL) to generate and instantly run JMeter tests. It allows you to record real mobile device traffic and playback at any scale from any geographic region through real mobile network conditions. It integrates with JMeter, Selenium, Gatling and Locust.
SOASTA Digital Performance Management platform provides key quality indicators across multiple devices with each version of your app. You can also automate mobile performance testing with SOASTA CloudTest and cover use cases using the same test across a range of real devices, Android or iOS, and see device-side metrics from build-to-build while applying load on the backend.
LoadRunner is a load testing tool licensed by HPE, one of the key features is that it can create and handle thousands of users at the same time. LoadRunner includes different tools: Virtual User Generator, Controller, Load Generator, and Analytics.
NeoLoad by Neotys offers an unlimited number of tests, HTTP/S & SOAP protocols support, pay-per-use cloud platform load generation, Continuous Integration Plugins, and monitoring. Providing a scriptless design and visual programming, and recording of mobile apps native or hybrid.
Perfecto offers a cloud-based, mobile enterprise application performance and monitoring solution. They allow access to many mobile devices in a secure and governed environment. You can analyze visual reports, logs, and video recording of test runs. Testing environments include real devices under real user conditions including battery constraints, memory shortage and switch-offs between Wi-Fi and cellular.
Apica enables load testing by device type (iOS, Android or Windows), large volume tests, user scenario recording with HTTP/S support and custom network speeds. Apica mobile load testing network is built on 50+ co-locations around the world with cloud cluster.
Sandstorm supports protocols for the Web, Mobile, and Email. It facilitates test from multiple locations, across browsers and simulates cache with a rich extensible framework for modification of test cases by using JavaScript and core Java APIs.
It provides a User Interface for handling dynamic parameters and automated cookies. It also integrates resource monitoring for most of the popular Web applications and Database servers.
LoadImpact is an online load testing tool used to test the performance of websites, web and mobile apps, and APIs under load and across various platforms. The tool enables you to conduct tests online instead of having an offline platform in place. While the testing process is fast, the user can also repeat the tests easily and within a shorter time span.
Monkey is an Android only program that runs on an Android emulator or device and generates pseudo-random streams of user events such as clicks, touches, or gestures, as well as a number of system-level events. You can use the Monkey to stress-test applications that you are developing, in a random yet repeatable manner.
Hockeyapp is a set of tools created by Microsoft to manage and report iOS/Android/WinPhone/OSX app crowdsourced/beta testing. It integrates the open source HockeyApp SDK for Android, iOS, Mac OS X, Win, Unity, Xamarin and then uploads the app’s build to HockeyApp.
It reports metrics presenting which devices were tested, by whom, how long the app was used, the language tested, and more.
Red Hat Mobile Application Platform is available for download, and you can read more at Red Hat Mobile Application Platform.
Last updated: October 31, 2023