Why People Analytics and AI Fail Without Data Foundations

Hi everyone, and welcome to season six of CultureClub X, powered by CultureMonkey. I'm your host, Darcy Mehta. CultureMonkey is an AI-powered employee engagement platform that helps people leaders listen to their employees and build stronger workplace cultures. CultureClub X is our global thought leadership forum where CHROs and people leaders share insights, discuss emerging trends, and exchange practical strategies for building thriving, future-ready organizations. Today we are so excited to host Ian O'Keefe, founder and CEO of Ikona Analytics, a people analytics leader with more than 25 years of experience across human capital consulting, HR transformation, and workforce analytics. Ian, welcome. It's wonderful to have you here with us.

Great to see you, Darcy. Thanks for having me.

Ian is a globally recognized HR executive, people analytics leader, and AI entrepreneur with over 25 years of experience helping organizations use workforce data to make better business decisions. He has built and led people analytics functions at Amazon, JPMorgan Chase, Google, and American Express. Most recently, as Head of People Analytics at Amazon, he led a multidisciplinary team that developed data products and analytics solutions supporting more than 400,000 corporate employees worldwide. In 2024, Ian founded Ikona Analytics, a company that helps HR organizations prepare for the AI era by combining consulting, data science, and technology to modernize HR transformation. He also serves as an HR venture advisor, teaches HR analytics at New York University, and is a frequent speaker and author on people analytics and AI. Your experience building people analytics functions inside some of the world's most influential organizations gives you a very unique perspective on why workforce analytics succeeds in some companies and struggles in others. We're excited to have you discuss our latest topic: data without direction, why analytics and AI fail without the right foundations. Before we dive into our questions, can you share a little bit about your own leadership journey?

Of course, thank you. You summed up quite a bit of it in that intro, which I appreciate. My journey covers consulting, in-house product development, and then in my most recent turn, starting up my own company. All of that comes together nicely when you think about the way that HR organizations today are looking to understand patterns and trends externally. They're looking for objective perspectives, and experienced and seasoned perspectives, not just from those who understand what's happening in the industry, but from those who have sat in the seat and built and shipped. It's a natural progression of having done all those things, and I'm fortunate to do that with a couple of amazing co-founders, Fritz Rosnow and Benna Voorhees, who are part of the leadership team, along with a lot of seasoned experts in residence at Ikona who are seasoned practitioners as well. A really cool confluence of practitioners, academia, consulting, and scalability through software and AI.

That's the dream, isn't it? To take everything you've done and all your amazing experience into now doing your own thing. Congratulations on that.

Let's dive into our first question. You've built people analytics functions at Amazon, JPMorgan Chase, Google, and American Express. What is the single biggest foundational mistake organizations make before they even start measuring anything?

One of the biggest pitfalls that I've seen, and frankly have been guilty of at some points, and you learn from your mistakes, is starting with data and tools and the outputs, and not looking at what precedes those outputs: the decisions that need to be made by leaders, and everything that's leading into what you typically control with analytics. Rushing to stand up reports and dashboards, as a simple example, is something many people analytics leaders are tasked with doing, or fixing something that's not working well. We can't get the data we need to make basic headcount and workforce planning decisions, as an example. The report is wrong. Fixing the report is not the issue. Fixing everything that's upstream transactionally is the issue, typically. That involves going deep into production systems, workflows, the architecture, the point at which transactions are generated by HR teammates and by leaders and managers. That tends to get pretty wide and out of reach pretty fast, but not acknowledging that ecosystem, and just working at the point where data's consumed, is a bit of a trap.

That makes perfect sense. It's not just that the report is wrong, but why. What led to that?

A really good grounding question, when you're being asked as a people analytics leader to spin up a study or a model or a product, is the questions that the leaders would answer and the decisions that they'd make with those answers. What would you do differently, Darcy, if you knew the answer to that question and I created that report for you? Would you change a process or policy or procedure, or an approach that you have with this business or that business? That's usually a great way to frame and reprioritize, and maybe reset some of the expectations. So decisions, and then the data and the infrastructure and the metrics that would support those decisions, is a good place to start.

At JPMorgan Chase, you inherited fragmented HR data across lines of business and functions, a common starting point for most organizations. What is the first thing a people analytics leader must do when the data foundation is broken?

There's a couple of things. One, with respect to the vision of what you're building, is to resist the urge right up front to fix everything in all directions at all times, and consolidate everything, and maybe confuse reorganization as a solve for some of the data fragmentation that's out there. The other is to understand data as an asset, and who owns it, all the way back to where it's originated and generated. That's something I've seen many orgs not quite get right when it comes to the recruiting, comp, talent, L&D, and operations functions. Everyone has a role to play in the data ecosystem, because if you're hitting submit, or you control a system or a technology platform in your COE, you're a data generator, and that data goes somewhere. Hopefully it goes somewhere common that can be looked at and analyzed and then productized. If it doesn't, that's going to create gaps for you on the analytics side, and that will express itself as data that is incomplete, or a foundation that is a little cracked and broken.

So, back to what we started with: understand what those key decisions are that you need to make from a leadership perspective, the data and the metrics that you need for that, and then work backwards into the ecosystem to understand the most critical workflows and data flows you need to land and shore up, and foundationally get right, with boring but important things like data definitions and data quality scanning routines. Think big, but start small, and really prioritize what's absolutely necessary rather than doing an ocean boil.

That's great advice. Think big, have the broader picture in mind, start small. And I really like thinking about it as a data ecosystem, because in an ecosystem everything affects everything else.

Most HR leaders confuse reporting with analytics. What is the real difference, and why does conflating the two quietly kill the credibility of the entire people analytics function?

Generally speaking, you can think about reporting as looking back at what happened, and analytics as why it happened or what probably will happen. Reporting is often lumped in with analytics because it's a step away; they're close cousins. But I would argue that reporting is really just an extension of your tools, your technology, your processes, and your transactions. The number of candidates that enter your ATS, the number of hires, how you're counting human beings on headcount reports, what you're paying people. Basic reporting of transactions is hard to do, very hard, and it speaks to the operational complexity of processes and programs keyed into the technology platforms you have. Humans at a company have data tagged to them across HR from all different COEs. They were once a candidate, they're getting paid, they're leaders, they go through talent. There are many domains with different types of tools that need to be joined and reconciled, and that's the challenge for reporting. Then you throw in procurement and contingent workers, and how finance counts humans at work as opposed to HR, and it gets definitionally gnarly pretty quickly. That lives in foundational data capabilities related to governance, data definitions, and core common platforms.

Analytics is asking what will happen, why did something happen. You're testing hypotheses, building models, doing A/B experiments and research, and you need rock-solid data to do that, or you need to know where the gaps are to do it well in a way that's credible. There's been a pattern over the years of reporting being separated out from people analytics, and it depends where the org is in its journey of understanding its overall data foundations. If you have a people analytics leader who has reconciled and consolidated reporting and knows how to put smart, prioritized data science and research initiatives into motion, it makes sense to group the two functions together. The trap is that if your ecosystem is not as known and diagnosed as you might believe, you risk having really talented, high-priced researchers or data scientists getting pulled into the data reporting reconciliation quagmires that a lot of orgs face, and that's not a good use of their time and skill set.

It had me thinking, at its most basic level, reporting is the what, what happened, and the analytics is the why.

Let's talk about Amazon. You led analytics solutions supporting more than 400,000 corporate employees through centralized and federated operating models. How do you build a foundation that scales without losing data integrity or decision-making speed?

The technology platforms you use, and the governance models and mechanisms and processes you put on top, are part and parcel. That's key. Amazon and many tech orgs are similar in that you have pretty much any kind of analytic tool you might need at your disposal, whether it's to organize data within the infrastructure and architecture, pull basic reports and dashboards together, or do model training and deployment and predictions. The key, and I'm going to say this many times on this chat, is to make sure you understand the origins of data, and create accountability loops for what needs to be inspected and root-caused and repaired if data is showing up wrong downstream. The Amazon team I led saw a lot of that and had a lot of those routines and sweeps in our data engineering team and within our reporting and analytics.

But then, having practitioners and teams embedded in the lines who had access rights to published data sets sent out from center: here's the access path, go in and inspect and root-cause and remediate. At that size and scale you can't do all of that centrally. You need to allow the right access patterns and permissions into data outside of your one team, for many others, with the right training and governance protocols in place to make sure nothing crazy is happening with respect to unwarranted data access. Having your governance, security permissions, and architecture access patterns matched up to the humans doing analytical work inside the org, whether they're on your team or not, is very key. You want to strike that balance.

Makes perfect sense. Have everything mapped out and matched up, so it's not just centralized.

You've written about how analytics is disrupting HR and algorithms are entering talent decisions. What does every HR leader need to put in place before AI can safely and ethically sit on top of their workforce data?

We've had AI in HR for a long time, although in November of '22, when OpenAI dropped GPT-3.5, that was a moment, and the start of the arc we're in now. AI writ large is aimed at language models, things like GPT and Anthropic. How are they disrupting and redefining how HR operates, and how companies operate, frankly? I'll start by acknowledging that we are having an internet moment. It's probably bigger than the internet moment we experienced in 1999 and 2000. How is this new technology, this new pattern of being able to do what we can do with language, affecting the way work will get done? That's pretty similar to what companies were asking themselves in '99: how can we redefine the way things get done with this new technology? That sounds silly to talk about now, but it was pretty hard to get your head around thirty years ago.

For many HR leaders it's an inflection point, and understanding how work gets done in HR is not an easy thing. We would contend at Ikona, and I've seen this over 20 years, that much of the way work gets done by people is not encoded into systems. It's not written down in SOPs and guidelines and protocols. Some of it is, and some of that results in the configuration of systems. But so much of how companies operate, and especially HR organizations, is tribal knowledge. It's tacit knowledge in the minds of people sitting in the seat. That's why so many teams have so many people doing programmatic things that are out of product and manual. Ian goes to system A, downloads it, emails it to Darcy, Darcy sends it to so-and-so. Those are clues that there's human effort doubling for what technology could do; we just haven't wired it that way for one reason or another. Those clues sometimes point to things AI can do quite well.

It's not to say that humans writ large will be replaced in their roles, but some things we all do could and probably should be automated and augmented, with faster ways to get information, consolidate it, look for a pattern, and then send that signal somewhere. That's something AI does quite well. For HR leaders, one of the first things to do is think about the kinds of activities and tasks that comprise the way we work, and where we can use AI to augment and accelerate, not replace. There's a lot of buzz: AI is taking jobs, AI is not taking jobs, where are we? There's evidence that AI is taking some roles. Oracle filed an SEC filing saying AI is responsible for twenty-one thousand roles, and that's creating a huge ripple. There's also plenty of evidence to suggest AI is replacing tasks and not the entire human in a role. That's where we stand: it can replace some of the minutiae and the drudgery, and create speed and acceleration and unlocks. Understanding the tasks, and the context that surrounds how work gets done, is really key to getting a return on an AI investment, and not just relying on the data and the guidelines and what's already written down. Much of what AI needs, rules, context, judgment, is up in people's heads, and we have to unlock that.

That's a fascinating topic in and of itself. I like the words you used, augment and accelerate. When the internet age first happened, you almost realize we started doing these things because we had this new internet. If we can take those loops away again, it frees up humans from those mundane tasks.

Companies and teams get really good at the manual forty-seven step task, and those aren't fake numbers. We see workflows inside some of the orgs we work with where there's an outrageous number of workflow steps and process derivatives, and you wonder, how did we get here? Taking that and lifting and dropping it into an AI environment, or just putting AI on top of it to read and learn, is a mistake. Rethink what we are trying to get done, what the inputs are, what the outputs are, and whether the current setup helps us or hinders us. If you have many people spending hours and hours getting something done, that's a decent starting point to look at and say, I wonder if there's a better way.

Your playbook blends rigorous data science with an operator's feel for change management. Why is change management the most underestimated element of building a successful people analytics foundation?

Because people analytics isn't as much about the analytics as it is about the people that generate the data you conduct analytics on. Understanding how people in a place we call work come together to get things done is the name of the game. Using the data that's available once that happens, to generate reports and build models, is what a lot of PA teams do. As a practitioner, you start to see that only using the data coming out of your core HRIS or your talent system or your ATS gets you a lot, but you will run into instances, whether in data definitions or competing metrics or historical views, where you don't have the data you need, or something is wrong, and you need to understand why. The downstream outputs of data are the endpoint of a lot of things happening upstream in the ecosystem, with your operations and your technology and the people involved, and inspecting that is enormously important.

Practitioners have their war stories and battle scars they talk about between sessions at conferences. Those are more than badges of honor; those are inspection patterns you can nowadays scale with the right types of prompts and rubrics with LLMs. The way we think about this at Ikona is by having human-to-human conversations with leaders and subject matter experts and technical experts inside the organization about the things that are not encoded, not programmed, not currently supported by tools and technology. What's happening out of product, what's happening manually, where are you spending time, what are your biggest pain points? That sounds simple, but when someone starts telling you a story for ten minutes, they'll hit upon a dozen data capabilities, history, and a lot of contextual information that AI just won't have access to and systems won't capture.

That's where we now have the ability to capture that type of information, digitize it, and make it accessible and readable by machines to do something about. The next level for a lot of PA teams is turning what otherwise might be considered water-cooler talk into a data asset you can use to improve a workflow or guide an AI contextually.

It's very interesting. I can see how your mind must work: if someone tells you a story, you see all the data things you can do with it.

If you and I were having a diagnostic conversation and I said, Darcy, tell me about the data definitions and the integrity and quality of data, that's a boring exercise. Instead: Darcy, talk to me about where you spend your time, the tools and systems you use, and the way the data you need may or may not show up well. Just walk me through it. How you describe that will hit on a number of things that translate technically, and we can work that out on the back end. We want to know the experience you're having. You're going to say some things we could have researched: you pull this from Workday, you pull that from Greenhouse. But why do you do that? Where does it go wrong? Who do you call? That never shows up anywhere, and that's the glue that I think is missing in a lot of these conversations.

What is the one non-negotiable step every CHRO must take in the next ninety days, before investing another dollar in analytics or AI workforce tools?

A non-negotiable step, I would say, is governance, ownership, and understanding of your data ecosystem. Do not expect AI to fix it for you. If we walk backwards from the AI moment we're in now: I can remember vividly, five or seven years ago, let's just build a machine learning model to fix that. Backing up from there, won't big data fix this? Backing up from there, how can cloud help us with this? Backing up from there, how can the internet help us with this? If you're putting new technology on top of old problems, you're gonna amplify old problems. And AI will not only amplify, AI will confidently and eloquently be wrong and incorrect in convincing ways that are hard to look past.

So before piling in and learning the hard way that your data foundations and your context layers, and some of the governance and ownership that comes in any scalable, high-impact data program, aren't there, take a good hard look at where you are with your understanding of data foundations, governance, and ownership across the entire data ecosystem in HR. From there, pick and choose smartly where you're going to get higher-value returns from the application of a small AI use case, to accelerate decision-making that needs data coming from a solid foundation. That's a loop.

That's great, very actionable advice.

How can real-time employee listening platforms like CultureMonkey strengthen the people data foundation, rather than becoming just another disconnected data source?

Listening, and I'll broaden it to say listening and sensing, is enormously important. In people analytics, teams have either owned or been heavy stakeholders and shapers of the listening agenda inside organizations for a long time. Take open-ended survey comments. For a long time that was the bane of many practitioners' existence, because what do you do with that? That unstructured spoken word is where I would argue the truth lies, more than a quantified value in a lot of ways. If you can have very intentional ways to capture the spoken word, the shared experience of people, of teams, of orgs, digitize that, synthesize it, analyze it, and then put new types of metrics and pattern-spotting and trending on it, that was out of reach until somewhat recently, and now it's table stakes. Survey providers that aren't moving in that direction ought to be. Platforms like yours, and others moving down that path, are going in a way that helps a lot of orgs transform, and not just understand as they did in the past.

And it's listening and doing it regularly too, because that's how you can see patterns. I agree about those open-ended questions: what is most pressing on someone's mind is what they write about. Ian, I really enjoyed our conversation today. Your insights have highlighted a challenge many organizations face: having more data than ever before, but not always having the right foundation to turn that data into meaningful action. From data governance and analytics maturity to AI readiness and change management, today's discussion reinforced that successful people analytics is not just about technology. It's about building the systems, processes, and organizational discipline that allow data to create value. Organizations that invest in strong foundations first are far better positioned to scale analytics, adopt AI responsibly, and make smarter workforce decisions. Before we end, how can our listeners connect with you to keep this conversation going?

Thanks, Darcy. For listeners in or near the East Coast, particularly the DC area, that's where I hang my hat, and we're up and down the conference circuit pretty regularly. You can probably see us next at People Analytics World in the fall.

That's great to know, and I definitely encourage everyone to connect with Ian on LinkedIn, and try to see him at one of those events. To all our listeners, thank you so much for being here. Don't forget to follow, share, and subscribe. That's a wrap for this episode of CultureClub X, powered by CultureMonkey. Until next time, I'm your host, Darcy, signing off.