Software Testing Trend: Performance Engineering is replacing Performance Testing

Amrit Poudel

Software Testing Engineer

Odoo image and text block



Performance Testing which comes under the umbrella of testing is a type of testing that simulates the realistic end user load and access pattern in a controlled environment in order to identify the responsiveness, speed and stability of the system.It is different from usual functional testing stuff. It is also not a type of automation testing where test scripts are created using a tool and automated test runs are scheduled. Here, test coverage is the most important factor and test accuracy is most essential part. The successful performance testing can be obtained only when the realistic simulation of end user access patterns both quantitatively and qualitatively is taken as key factor during testing. 

The main objectives of performance testing are to identify the bottlenecks on Application under test. The process of identifying the bottlenecks can be done with diagnosis in different layers of Application under test such as presentation layer(web server), business layer(Application Server), data layer(Database)This testing doesn't have any specific metric to measure but they usually requires reporting of system performance metrics like transaction response time, concurrent user loads supported, server throughput etc along with additional metrics reporting the software and hardware layer-specific performance metrics like browser performance, code performance. 

The basic straight forward analysis and experience-based analysis can be performed by a performance tester. But if the findings require deep dive analysis, say a specific transaction or method reported to have high response time, it will be the responsibility of the performance engineer. The performance engineer performs performance engineering. The term performance engineering is completely different than performance testing. Performance engineering is a discipline that involves systematic practices, techniques and activities during each phase of software development life-cycle(SDLC) to meet performance requirement.

Odoo text and image block

Performance engineering though is a board set of processes, and it’s also a culture. Performance engineering is an art based on years of observation that have led to proven practices. These practices define a culture that enables teams to deliver fast, efficient and responsive systems architected for large-scale populations of customers, employees, regulators, managers and more. But moving from performance testing to performance engineering is not an easy process. The team must be ready to move from simply running a checkbox performance test script and focusing on scripts to studying the way that how all parts of the system are connected and work as a single unit.

Odoo image and text block

 

So, a road map is necessary for making a trip from performance testing to performance engineering. For this, first of all, the organization should define a culture to enable individuals to collaborate on such environment that will inspire the formation of cross-functional teams and logical thinking. Then, a performance engineering team should be built that contains technology, business and user representatives which can focus on performance nature of everything they are working on and figure out together. 

Now, this team should identify the key metrics, that matter for organization and offer a good way to judge results against the desired outcome. Once, these metrics are captured, the team should look and ask about the indirect metrics within the business that would show results related to revenue, customers(attraction and retention),  competitive advantage and brand value. Now the team should start gathering information and sharing initial prototypes for the type of results, reports and dashboards that are necessary to show to stakeholders at a a required frequency. Once the SDLC progresses, the team should conduct proper retrospectives, track process with metrics. Now the team should highlight the opportunities for improvement and show the wins and losses. The team can also set up the gamification of performance engineering itself at a team level to encourage, a little healthy competition within team.

As the team begin to elaborate on requirements, stories and features, it is important to remember that whole team is working to define the what, why and how of each item. Also, while the team continue down the engineering path, the team will learn from each other’s domain expertise.

However, the performance engineering will be fruitful and successful, when the team starts thinking about it from the beginning. The earlier the team begins addressing performance in the product life-cycle, the more likely the final system will run quickly, smoothly and efficiently. But if it can’t be done from the very beginning. It’s still possible to add the process to the redesign and reengineering work done to develop the next iteration or generation of a product.