Data is everywhere in our organisations. Yet, how many of us truly interact with it in meaningful ways? Instead, we stare at static dashboards, are buried under piles of quarterly reports, and manually wrangle spreadsheets in search of elusive insights. The tools we use to manage information may be called data-driven, but are they really helping us make better decisions?
I experienced this reality first hand when trying to optimise our LinkedIn content strategy. Each month, I'd export our performance metrics into spreadsheets, create charts to identify patterns, and then attempt to extract insights that might improve future content. The process was cumbersome. It's disconnected from the actual publishing workflow, and worst of all: insights often came too late to be actionable. By the time I understood what was working, the opportunity to capitalise on a trend had often passed.
The typical data workflow in most organisations follows this same predictable pattern:
- Business users identify a need for information
- They submit a request to a data team
- The data team builds a dashboard or report
- The business users receive this static output
- They attempt to make decisions based on what they see
- If they have questions or need adjustments, the cycle begins again
This process has several fundamental problems. It's slow. It creates bottlenecks. It separates the people who understand the business questions from those who can access and manipulate the data. And perhaps most importantly, it treats data analysis as a product rather than a process - something to be consumed, rather than something to be engaged with.
Where Databricks Is Nudging Us: Interactive Apps
Unlike static dashboards that limit you to pre-set reports, interactive apps give users the power to ask their own questions, dive deeper into data, and quickly adapt to new trends. This immediate interaction leads to faster, more insightful decisions, something static reports just can’t provide.
This shift mirrors what we've seen happen with consumer technology. I like to blame the zoomers, but in truth we all have become accustomed to swiping and tapping our way through mobile apps with instant feedback. It seems obvious that this expectation has been brought to the workplace. Why shouldn't business tools offer the same level of interactivity?
To explore this capability and test its potential, I built a LinkedIn content optimisation application for a presentation at the Perth Databricks User Group. The app was designed to ingest our LinkedIn post metrics, analyse engagement patterns, and enable direct scheduling through the LinkedIn API.
What began as a demonstration project actually became genuinely useful in my own workflow. As the only person managing our public content (Hamish's controversial posts aside), I discovered that the app was particularly valuable for understanding optimal content scheduling times - a surprising insight that emerged from the data analysis. The app revealed specific time windows when our audience was most engaged, a pattern I had missed through manual analysis.
However, building these apps still requires software engineering effort. Databricks hasn't eliminated the need for technical expertise, they've just provided a framework that makes it easier for engineers to build data-intensive applications quickly. Not every marketing team has someone comfortable with both marketing analytics and software development.
What Databricks Apps do provide is an environment where engineers can iterate quickly. The framework handles many of the complicated aspects of building data applications, allowing developers to focus on solving business problems. This significantly reduces the time from idea to implementation, even if it doesn't eliminate the need for technical expertise entirely.
The evolution toward interactive data tools is just the beginning. As we look to the future, we’ll see AI take on an even more transformative role in shaping how we interact with data, and ultimately, how we use it to drive business outcomes.
The AI Generated Future
While Databricks hasn't (yet) explicitly stated this as their end goal, their trajectory seems clear: we're heading toward a future where generative AI will create data applications on demand.
Consider what's already happening with AI in other domains: generating images from text prompts, writing code from natural language descriptions, creating content from simple instructions. Apply this pattern to data applications and the outline of something revolutionary emerges.
I imagine a future where I could simply say, "I need a tool that analyses our LinkedIn post performance, identifies optimal posting times, and lets me schedule content directly." Provided I had my source data in order, generative AI would simply build this application without requiring me to write any code.
In hindsight it seems obvious, but what's particularly interesting about my experience with the LinkedIn app is that the most valuable insight: optimal posting times, wasn't what I initially built the app to discover. It emerged organically from the data. This highlights how AI might find non-obvious patterns that even experienced professionals might miss.
Without fully drinking the AGI Kool-Aid, its easy to see how this capability would fundamentally change how organisations interact with data. Maybe even the humble spreadsheet - mainstay of business computing for decades - would finally meet its match.
This represents (dare I say it...) true democratisation of data capabilities - not just the buzzword version we've heard for years. AI would handle the translation from business requirements to technical implementation. A marketing manager could simply request a sophisticated customer analysis tool without understanding database queries or interface design.
Preparing for This Future
If this vision of the future resonates with you, you can begin preparing now:
- Experiment: Begin by identifying a specific business challenge that could benefit from interactive data tools, like optimising your content strategy.
- Iterate for Impact: Build and test prototypes. Use real-world feedback to refine your tools and make them truly useful.
- Prioritise Data Quality: The power of AI-generated tools lies in clean, reliable data. Make sure your data infrastructure is up to the task.
- Cultivate a Culture of Feedback: Foster an environment where continuous improvement is the norm. Feedback loops will be essential when working with AI.
Looking Forward
As we stand on the cusp of this revolution, to me the question isn’t whether AI generated data apps are coming - it’s whether your organisation is ready to embrace them. The future of data is interactive, intuitive, and responsive to the needs of the humans in the business, and it’s time we start preparing for that shift today.
As always, you can reach out to us to discuss anything Databricks, Data Apps, AI (or anything else really) - we would love to chat.