For all we hear about AI changing the landscape of the tech sector, very little is mentioned about AI in analytics. The truth is that the analyst’s role is changing, but not in the way that most people think.
The most valuable analysts in the age of AI wont be those who can write the best prompts. They’ll be those who can combine human judgement, domain expertise and AI effectively.
1. The Prompting Trap
Today, much of the conversation around using AI effectively centres on good prompting. This is essentially a rehash of the old IT adage ‘Garbage in, garbage out’ – in other words, the quality of your AI outputs will only be as good as the quality of your prompts.
But is this all there really is to AI in analytics, and is becoming an AI-enabled analyst really all down to good prompting? The real question is: What does the role of an analyst look like when AI becomes part of everyday work?
2. What AI Changes
As analysts we’re used to going through a fairly linear process:
- Source the data
- Import it into your platform
- Explore the story you want to tell
- Transform the data into a usable format
- Load it into your reporting tool
- Build the product
- Present it to stakeholders
- Gather feedback and make changes.
There’s nothing wrong with this process, but it can be difficult to know where AI fits in it. The answer isn’t in looking at the process and working out which parts we can automate. That makes the assumption that analysts are somehow replacable, at least in part. Rather, we should look at how AI can augment our processes at each stage, to make us better analysts. AI can help analysts work faster, but it doesn’t remove the need for analysts.
3. AI Getting Sloppy
A big problem for AI in analytics is one of trust. How do I know that the outputs I get are going to be valid enough for me to put my name on them? A few weeks back we got a new AI hoover, which scans our lounge at night and cleans automatically. We diligently cleared our floor of cables and left for the evening. All well and good – until our cat missed his litter tray at night and we come down the following morning to perfectly geometric lines of poo smeared into our carpet!
AI is probabilistic, and sometimes, despite our best efforts, the outcomes of its use create more mess than if we’d done the job manually ourselves. But the answer isn’t to go back to doing jobs ourselves. Rather, we can analyse what went wrong, optimise the process and try again. Just because we sometimes get sloppy outputs doesn’t mean that we can’t get clean ones, if we’re willing to put in the work. As the experts in analytics, we’re the ones who can do this most effectively.
Some practical tips
Becoming an AI enabled analyst is not about finding tools to replace your existing skills – it’s about using AI to enhance them:
1. Use AI to accelerate, not automate, your thinking
A good question to ask yourself when using AI is: “Given enough time, could I do this job myself?” If the answer is ‘no’, then you’re limiting your ability to curate the content it produces. But you can still use AI to speed up tasks by getting it to fill in the yawning void of the blank page you start with. It may produce a rubbish first draft, but at least then you get to see what’s rubbish and make it better!
2. Focus on judgement and communication
No-one wants to be left behind, finding out too late that their skills are now obsolete. Although it’s doubtless that AI will change the skills which will be important for a future analyst, for every skill it diminishes in value, another is enhanced. Perhaps I no longer need to be able to write Python from scratch, when Hermes (my coding agent) can do it for me. But I still need to be able to read what it writes and understand how to change it to suit my needs. Furthermore soft skills, like thinking critically or communicating effectively, will always be important, no matter what new technology comes along.
3. Verify before you trust
This is so important I’m going to say it again: Verify an AI’s outputs before you trust its contents. We don’t know what’s going on inside the black box of the model, just in the same way we can’t read people’s minds. We may have some idea of how the process works, but that doesn’t nullify our responsibility to check the outputs before presenting them. AI is much better at sounding right, than it is at being right. Don’t be lured by its conversational fluency into thinking that means it knows what it’s talking about.
Final thoughts
With AI tools advancing so rapidly, it’s easy for fear and anxiety to set in: ‘Am I still going to have a job in 5 years?’ – or if we listen to recent doomsayers from OpenAI and Anthropic in recent weeks ‘Is AI going to kill us all?‘
Much smarter people than me have weighed in on this question, on both sides of the argument, and I’m not going to add my voice to the masses. I think it’s important to note that just because there may be real societal dangers to advancing frontier AI models too quickly, that doesn’t mean we can’t (or shouldn’t) be using the tools at our disposal right now.
For analysts, even the current models can do wonders for writing first drafts, writing code, exploring data or summarising long documents. This sounds really basic, but I believe this could have genuine benefits for how we do our work. It’s not that our tech skills will suddenly become obsolete – rather we need to rethink how we apply our skills, augmenting them with AI tools, and focus on developing the softer skills that we humans have always been best at.
Steve Parsons
Advanced Analyst & Researcher
Suffolk Office of Data & Analytics