RAC: Performance testing the new digital platform

The Client

RACWA is a motoring club and mutual organisation offering motoring, insurance, travel and financial services for its 840,000 members with a revenue of more than $670 million in 2016.

Like many organisations, RACWA faces pressure to increase the quality and breadth of services while containing costs. Digital platforms are seen as a key enabler to deliver new services and higher customer engagement at a lower operational cost. But the RACWA has issues with legacy systems and processes and faces a comprehensive challenge to modernise its application suite.

The flagship digital platform at RACWA is the myRAC website. This is the primary interface for all digital customers at RACWA. It brings together a number of legacy platforms into a single user experience and upgraded underlying technologies, like the web Content Management System (CMS).

The Problem

RACWA engaged Mechanical Rock to validate the performance of the new myRAC platform in production.

This included the new Azure PAAS based architecture, substantial content and UX changes, and some back end process and application modifications.

Rather than supply a one-off performance test, Mechanical Rock created a standalone performance testing solution for the myRAC website – leveraging the initial investment and providing a much better return over time.

Further RACWA opted to have Mechanical Rock supply performance testing over the next 2 years as a cloud based managed service.

The Mechanical Rock solution used the Blazemeter platform at a fraction of the cost of high-end performance tools. Blazemeter is based on jMeter, an open-source performance testing framework, thus avoiding any problems of proprietary lock-in, a common problem with specialised performance testing solutions.

Blazemeter benefits included:

• Lower initial investment.
• Greater return through reuse.
• Continued optimisation of user experience.
• No proprietary lock-in.
• No dedicated infrastructure for cloud based PAAS.
• Convenience and cost-effectiveness of a managed service.
• Increased fidelity and accuracy due to cloud-native Performance testing in real world conditions.

The Solution

Based on the constraints of the project, Mechanical Rock suggested the following test configuration:

1. Shakedown test - to validate the performance test approach and configuration
2. Typical Load- evaluate system performance under an expected user load profile
3. Soak Test - to evaluate the system’s performance under load over a sustained period of time ( 24-48hrs)
4. Stress test - evaluate system performance under increasing load until it fails or performance is seriously compromised.

The delivered solution utilised BlazeMeter’s cloud platform which can run massively scalable, open source performance tests and NewRelic’s APM solution which offered detailed insights into application performance.

RACWA also opted to have Mechanical Rock deliver ongoing support of the platform as a managed service, with the option of bi-monthly performance test on upgrades and changes to the myRAC platform.

The Benefits

• Multiple performance opportunities identified in:
  • Application configuration
  • Azure platform configuration
  • Configuration issues with IIS and SiteCore
• Memory leak identified within SiteCore product - subsequently logged and resolved with SiteCore product team.
• Realistic baselines for normal operations and peak capacity established.
• Platform resilience proved via 48 hour soak test
• Entire solution delivered as an on-demand managed service with minimal ongoing costs