The pitfalls of building your own workplace analytics tool

Every so often, we encounter organisations whose in-house data teams are adamant that “we can do what Audiem do.” They argue that they have the expertise and knowledge to build their own version of Audiem, which would of course, save the business money and maintain full control over their data.

Sounds like a convincing argument, doesn’t it?

Well, perhaps in theory, but in practice we find that the same organisations come back to us a few months later, frustrated that the in-house project hasn’t really gone anywhere; either because they misunderstood what Audiem does, because their data team was pulled onto other priorities, or a blend of the two.

We’ve even watched organisations work hard to develop something in-house and compare the results to those from Audiem, only to realise that Audiem does much more, much better. In the meantime, they’ve wasted months on developing a tool that doesn’t end up doing what they wanted.

So why does this happen?

I think there are three factors at play. The interplay between these factors varies from business to business, but all play a consistent role.

The first reason is the proliferation of AI software since the launch of ChatGPT back in 2022. Many of these products are essentially “AI wrappers”: lightweight user interfaces that work by making calls to an OpenAI (or similar) large language model (LLM). This gives the illusion of intelligence, but doesn’t embody any domain expertise or proprietary technology.

If you remember the adage, ‘if all you have is a hammer, everything looks like a nail’, what’s happening here is that OpenAI (etc.) have become one huge, generic hammer!

As I explained in a previous article, Audiem isn’t just a thin veneer over a generic LLM. It’s been built from decades of workplace knowledge and experience, and rigorously trained text-classification models. So, if your data team is confident they can build you your own version of Audiem, you need to be 100% sure that they have the required workplace knowledge to back up their claim.

A second reason why organisations try to do things in-house is that they think it will save money. But in-house development work also costs money – including the opportunity costs of other work not done – and these costs will be ongoing if the tool is to be continually maintained and improved.

There’s a classic bias here around optimism in projects. Even if a data team can commit to releasing an initial version, what they’ll almost certainly learn along the way is that it was harder and more resource intensive than they anticipated, and that they just can’t justify the level of maintenance needed in future.

When buying a product like Audiem, you’re benefiting from pooled R&D costs, and years of prior work and problem-solving – we’ve been wrestling with this on the workplace sector’s behalf for a long time! You also benefit from being part of a community of active product users. For instance, one client may suggest a new product feature that other clients will all benefit from.

The third reason is that sometimes the drive to do it yourself is driven less by cost-saving and more by internal politics – teams jockey for attention, sponsors want to keep ‘control’ in-house, or there’s an unspoken fear of relying on external vendors for something that “we should be able to do ourselves”.

These agendas all make complete sense for the people involved. They can also be hard to spot, often hidden behind other arguments. But there’s a real risk that, a few months down the line, priorities will change, attention will shift, and your in-house project will lose momentum – leaving you stuck waiting for your employee feedback to be analysed the right way.

What I’ve described here isn’t new – it’s the classic ‘make-or-buy’ dilemma that organisations have always faced. But just because the make-or-buy dilemma is an old one doesn’t make it any easier to resolve, particularly when organisational politics are involved.

If you’re facing this dilemma, here are some questions to ask your in-house data team:

‍

1. Do you really understand what Audiem does and how it does it?
2. Will the tool you develop do everything Audiem does?
3. Do you have the required domain (workplace) knowledge to develop the tool?
4. Will you have the time to develop, maintain and improve the tool?
5. What will it cost to develop the tool, and when will it be ready?

‍

If they don’t provide you with convincing answers to all these questions, I would be very sceptical that they can deliver what you need when you need it.

I get that this can be hard to hear — but we’ve seen it play out time and again, and we’d rather help you avoid the same pitfalls.

If you want to find out more about what Audiem does and how it does it, then get in touch at hello@audiem.io - we’d love to chat.

‍