Modern data strategy

The Client

As a top tier resources company, our client uses data as a key enabler of operational and business performance.

They've invested in a modern cloud-based data warehouse and tooling to speed the flow of value.

However demand is outstripping the capacity of their service and is continuing to grow as operational business units find new opportunities and mature their own data products.

This is a pattern that is repeated across the industry – the uptake on successful data platforms exceeds the economic sustainability of supporting it via a traditional centralised team.

According to McKinsey, only about 7% of Australian organisations are effective at reaching their primary objectives on data.

The Challenge

Currently, much of the data infrastructure depends on manual processes that are not repeatable and subject to error.

• Provisioning of resources uses some scripting but still relies on manual configuration changes in the console.
• Administration of users, data and access is manual rather than using code driven access patterns.
• Security policy is implemented manually with inconsistent results.
• While business teams are becoming self sufficient, the sprawl of data patterns and sources is becoming unmanageable, affecting discoverability, compliance and reliability.

Wardley Map for visualising data strategy

The Solution

Our solution recommended a complete strategy to enable data agility and unlock the flow of value in the organisation :

• Domain Driven or Data Mesh architecture.
• Automated DevOps pipelines for Infrastructure-as-Code management.
• Better compliance through policies-as- code, applied at the domain level.
• Organisational design based on Team Topologies platform, stream-aligned and enabling teams.

The enabler to move to higher levels of service is to employ modern software engineering techniques (DevOps/DataOps) to automate processes across the data value chain.

Wardley Map for visualising data strategy

The Benefits

Our client’s data platform will be able to scale to accommodate the projected future growth of its operational business.

• The Total Cost of Ownership (TCO) for the data platform should be reduced as processes and services are optimised.
• Automation makes processes scalable, repeatable and reliable.
• A successful platform allows seamless self service for consumers (no more service tickets).
• Self service reduces waiting time and allows business customers to iterate problems quickly.
• Speed of iteration leads to better experiments and the delivery of real business value from data driven decisions.