THE INSIGHTS INDUSTRY NEEDS REINVENTION, NOT JUST ACCELERATION

NEW TECH OFTEN FUELS POINT SOLUTIONS BEFORE DRIVING SYSTEM CHANGE

New technologies tend to go through two phases of adoption. First, people use them to improve the existing way of doing things. It’s only later, when we use a technology to reinvent how things are done in the first place, that its real value is realised.

The economists Ajay Agrawal, Joshua Gans and Avi Goldfarb describe this as the shift from ‘point solutions’, which slot a new technology into the existing system, to ‘system solutions’, which redesign the system around it.

The first electric factories swapped the steam engine for an electric motor and changed nothing else. The first personal computers in offices were typewriters and ledger paper on a screen. The first home microwaves were used for scratch cooking. The first company websites were brochures, posted online. The first mobile internet was shrunk-down websites.

Then came system reinvention. Factories rebuilt around electricity. Offices reorganised around networked computers. A whole new world of convenience microwave food. E-commerce. Uber and Instagram.

Economists have been tracking the pattern for more than thirty years, from Paul David’s work on electrification to Erik Brynjolfsson’s on computers, and now AI. They found that often the real gains of a new technology came when it started to drive system change and reinvention.

When it comes to the insights industry and AI, it feels like the biggest focus is still very much in the first phase: point solutions accelerating the existing research process.

There’s nothing wrong with using AI to speed up research as we know it. It’s probably an essential foundational step in the journey of change. But we shouldn’t stop there. We also need reinvention: new models of robust, evidence-based decision making. We need to rethink how the whole system works.

That doesn’t mean abandoning everything we know. Reinvention can complement existing methodologies like survey research, qual and social analytics, fill the gaps around them, push past them and open up things they were never designed to do.

THE INSIGHTS INDUSTRY SEEMS MOSTLY FOCUSED ON POINT SOLUTIONS & ACCELERATION

Right now, most of our industry’s energy is going into using AI to speed up what we already have: research automation, synthetic respondents and digital twins as a stand in for real people and AI-moderated interviews for running qual at a scale, along with AI-assisted analysis, from coding open ends to text analytics, which is now one of the most common uses of AI across the industry.

These are real gains, and I wouldn’t argue against any of them. They’ll keep running side by side with whatever comes next, pushing the industry forward.

But all of this makes the existing research process the finish line. Much of the time the very best these innovations can be is a faster, cheaper version of something we already have. More efficient, but still a copy of something that we know has limitations anyway. Their ceiling is set by their benchmark.

So why is the focus more on point solutions and acceleration than reinvention?

For a start, it’s probably the easiest place for any industry to begin, reworking what we already know. Traditional research is known and trusted. For practitioners, they’re a comfortable place in which to innovate. And for clients, it’s easier to adopt. It’s also a well-documented pattern: as Rebecca Henderson and Kim Clark showed, established players find it much harder to see and act on innovations that reconfigure a whole system than on ones that improve its parts.

And there is an attractive commercial opportunity in making the existing systems faster and more efficient. Many startups are raising huge investment sums and achieving spectacular valuations on the back of the promises of automation and acceleration. It makes sense that these opportunities will be pursued simply because there is money to be saved and money to be made.

But commercial expediency and advancing the cause of the insights function aren’t always the same thing. What opportunities are we missing when, as an industry, we focus only on acceleration?

To answer that question, we should take a step back. Ask not what we do as an insights industry, but what we are for. What we do is insight work. What we are for is helping guide better evidence-based decision making. We shouldn’t just ask, ‘how can we do research quicker and more efficiently?’ We should ask, ‘how can new technologies help us show up in decision making in new and better ways?’

BETTER DECISIONS ARE OUR BENCHMARK, NOT TRADITIONAL RESEARCH RESULTS

When evaluating the potential of a new application of AI, people often ask questions such as ‘is it as accurate as a survey?’ or ‘does it give us the same depth as qualitative research?’ In an acceleration mindset that’s the sensible thing to do. The danger is that this mindset keeps us focused on point solutions and acceleration and stops us moving to true reinvention. Traditional research is held up as the gold standard of truth by which everything else is measured.

But we know that traditional research results aren’t in themselves the truth. ‘The map is not the territory’ (Alfred Korzybski). Every research methodology has its weaknesses and limitations as well as its benefits and strengths. Traditional research methods are embraced widely not because they are the truth, but because their limitations and weaknesses are known and accepted, alongside their benefits.

Quant survey research gives us the confidence of a hard number but very little understanding of the context behind it. Qual gives us richness and depth but not proof of whether or how those findings apply at scale.

Businesses are used to using traditional research methods to support the decisions those methods excel at: often one-off, big gate decisions. However, decision making is more complex and nuanced than that. Yes, there are the big set-piece decisions: the go/no-go gates, the strategic turning points, the major optimisation rounds.

Often the most important decisions are the ones in between the big set-piece decision gates. The myriad small moments of judgement made step by step, turn by turn, through the strategic and creative journey of brand building. Small decisions that compound to make a big difference. Death by a thousand cuts if you get it wrong. Success by a thousand nudges if you get it right.

What if we made better decisions our true benchmark of success, not existing research results, and developed new methodologies to support more decisions, earlier decisions and better decisions?

DECISION INTELLIGENCE SYSTEMS WILL BE KEY, BUT WHAT FEEDS THEM?

The insights industry isn’t evolving in a vacuum. AI isn’t only changing how research gets done. It’s changing how brands are built, how marketing is done and the basis on which business decisions get made.

Gartner predicts that by 2027, half of all business decisions will be augmented or automated by AI agents for decision intelligence. Whatever the exact number, the direction is clear. Most businesses will soon run on some form of decision intelligence system, and many insight teams are well on that path already.

But a sophisticated decision system doesn’t in itself make the invention of new insight methodologies redundant. Arguably it makes them more important than ever. Decision intelligence layered on top of existing research sources and data sets will work those resources hard. But it doesn’t magically make up for their deficiencies or fill the gaps between them.

We need a comprehensive answer to the question: ‘what is feeding the system that makes our decisions, and how do we make it better?’

There are vast information sources out there that are still relatively untapped, and the ones we do use are often too siloed. Data that can only offer limited guidance on its own can support decision making with real confidence when it’s combined and worked in the right way.

THE MISSED OPPORTUNITY: WEIGHT OF EVIDENCE

We now have access to more real-world evidence about what people think, say and do than at any point in history. How can we harness that in new ways, not just use it to replicate survey research? The answer is perhaps embracing a new kind of evidence, one that fills the gap between quant and qual rather than replicating either.

Quant gives you confidence. Qual gives you depth.

Weight of evidence gives you the depth and the confidence to act.

Weight of evidence is built from the breadth and independence of real voices across many sources: how many independent people, in how many different places, are saying or doing the same thing, and how much evidence stands against it.

You need to know when that evidence is strong and be clear when it’s thin but directional. Weight of evidence can become the foundation of confident decision making, not sample size.

It’s an approach designed to support decisions, not to report statistics. Something that can sit alongside quant and qual, complementing them and, in the right situations, replacing them.

What does that look like in practice?

We’re all at the start of the reinvention journey, so who knows for sure. There are so many possibilities. For now though, there are key principles underpinning how we are approaching this with Living Studio, our own reinvention method we are developing.

KEY REINVENTION PRINCIPLES

1. Layering sources (online content at the heart)

No single source tells the whole story, so we don’t rely on one. We layer them: open discussion, reviews and forums, expert and trade coverage, search demand, what LLMs are saying, and primary qual and quant. Each layer covers for the blind spots of the others, and we always show the mix, so you can see exactly what a conclusion rests on. That means decisions built on a broad base of evidence rather than one view of the world.

And any existing research investment keeps working harder, because it becomes one layer in a much richer picture.

2. Structured sampling

The loudest voices online aren’t always the most representative. So before anything is collected, a sampling frame decides where to look: across the different angles that matter to the question, and across different kinds of voices, from customers and experts to retailers and hard data. That keeps the picture balanced by design, rather than shaped by whoever happens to be shouting loudest.

3. Curation, not volume for the sake of it

More data isn’t necessarily better data. A thousand posts saying the same thing from the same corner of the internet might just be one piece of evidence repeated a thousand times. The right data used in the right way is the most powerful.

That means curation. Where you look for data is shaped by the brief, and every piece of evidence is judged on its quality, relevance and independence, and no single voice is allowed to dominate. The result is a read you can trust, because confidence comes from breadth and independence, not from volume.

4. Observations as common currency

Data comes in all kinds of forms, from more qualitative to hard quantitative data. To connect them most effectively you need a common currency of knowledge: observations. It’s not about merging databases, it’s about translating your knowledge sources into single units of observable fact: a written statement of what has been learned, whether the source is more qualitative or hard numbers.

5. Semantic analysis & triangulation

Every observation gets a full semantic read, not blunt analytical instruments like word search, pattern matching or clustering. And a finding only stands if it holds up across independent sources.

It’s about looking across sources and observations for meaningful connection, identifying the signals and weighing the evidence that supports each one, the evidence against it and the evidence that adds context. And stress testing it: a ‘green team’ looking for supporting evidence, a ‘red team’ arguing the opposite case and seeking out alternative narratives.

What survives comes with an honest level of confidence attached. Leaders know which findings are rock solid and which are promising but early and can decide how hard to back each one.

6. Soft signals – depth and confidence

Some of the most valuable learnings are the ones that are hardest to measure and count. An early shift in behaviour. A frustration nobody has solved yet. A new need that’s just starting to be voiced.

This is what traditional methods struggle to capture at scale. A survey number alone is not enough. An evidence layer of observations from multiple sources can be triangulated to identify these soft signals. They help explain the meaning behind the hard numbers, adding depth, richness and nuance, while weight of evidence gives you the confidence to act on them.

7. Laddering & transparent evidencing

Evidence is only useful if it leads somewhere. It needs to be laddered up, step by step, to the findings a business can act on: the questions it needs to answer, the opportunities worth going after, and briefs ready to hand to a team.

It spans the gap with a clear and measurable bridge between “what the evidence says” and “what we should do next”, backed by a clear and auditable evidence trail: from curated sources to observations to signals, to answers to questions, to opportunities and decision guidance. Every conclusion can be traced back to the exact evidence behind it across sources. When someone asks, “how do we know that?” you can show them.

IT’S ABOUT CREATING A LIVING SYSTEM (AI + HUMAN)

All of this knowledge, learning and decision guidance sits in one place: a living brain with an evidence layer behind it. It’s not necessarily about being always on. It’s about being there when you need it: a resource that adapts to your questions and decision challenges as and when they arise.

That means an evidence layer that can be refreshed whenever a new question or decision comes up. Soft signals that emerge, strengthen, fade and are retired as things change. And a single source of truth that can answer any growth question, uncover fresh opportunities and provide ‘living briefs’ that guide the important decisions people make every day (not just the big boardroom calls).

AI isn’t the answer on its own. There’s no magic model that simply “knows” your customer, your brand and your business.

AI is the smart operating system that makes this way of working possible. It gathers evidence from many sources and curates it, reads every observation, organises it, triangulates it and acts as the integrated ‘brain’ that can quickly adapt to knowledge and decision-making needs.

Human expertise is focused on what people are best at: framing the right question, shaping the approach, challenging the evidence, judging what matters and driving action.

REINVENTION LIVES ALONGSIDE TRADITIONAL APPROACHES

Approaches based on the principles of Living Insight Systems open up opportunities to support evidence-based decision making beyond what traditional consumer research can do, rather than replicating it, whether that traditional research is ‘real’ or synthetic.

Brand owners can make more decisions, better decisions, sooner and with more confidence, because they have a blend of soft signals and hard measures with a clear and auditable evidence trail back to multiple sources.

None of this reinvention means the end of survey research or other primary research methods. Far from it. When a decision rests on a hard number, a well-run quant study can still be the right tool, and it should be run brilliantly.

But existing research approaches and Living Insight Systems can work and evolve side by side. Quant surveys measure what matters with precision. Living Insight Systems find the questions worth asking, spot what’s emerging and keep the picture live.

Each makes the other more valuable, and together they give brands a far more complete view than either could alone.

IN CONCLUSION

The technology shifts that matter most end the same way: the move from acceleration to reinvention.

The biggest gains for the insights industry as a whole will come when we rebuild around new technology, not when we use it to do the old job faster.

So, let’s not stop at asking how AI can make what we already have faster. Let’s ask what we can build that’s genuinely new, and judge it by one simple test: does it help brands make better decisions?

That’s the future we’re building with Living Studio at Radius and ONE Strategy Studio.

We hope you agree!

SOURCES

Ajay Agrawal, Joshua Gans and Avi Goldfarb, Power and Prediction: The Disruptive Economics of Artificial Intelligence, Harvard Business Review Press, 2022

Paul A. David, “The Dynamo and the Computer: An Historical Perspective on the Modern Productivity Paradox”, American Economic Review, 1990

Erik Brynjolfsson and Lorin M. Hitt, “Beyond Computation: Information Technology, Organizational Transformation and Business Performance”, Journal of Economic Perspectives, 2000

Erik Brynjolfsson, Daniel Rock and Chad Syverson, “The Productivity J-Curve: How Intangibles Complement General Purpose Technologies”, American Economic Journal: Macroeconomics, 2021

Rebecca M. Henderson and Kim B. Clark, “Architectural Innovation: The Reconfiguration of Existing Product Technologies and the Failure of Established Firms”, Administrative Science Quarterly, 1990

Shoshana Zuboff, In the Age of the Smart Machine: The Future of Work and Power, Basic Books, 1988

Gartner: half of business decisions to be augmented or automated by AI by 2027