Why Is Standard Responsible AI Broken? Reid Blackman on AI’s Ethical Nightmares

Andreas Welsch: Clearly, there’s no shortage of doom and gloom and hype in the media about AI and agentic AI, but it’s also creating a lot of ethical nightmares and challenges. Today, we’ll talk about that and what it means. So let’s jump right in.

Andreas Welsch: All right, welcome back to What’s the BUZZ, where leaders share how they have turned hype into outcome. Today, we’ll talk about how you can address AI’s ethical nightmare challenges. And who better to talk about it than someone who has just written a new book about that, Reid Blackman. Hey Reid, thank you so much for joining.

Reid Blackman: Thanks for having me. What an exciting intro. The video, the music. So professional.

Andreas Welsch: Thank you. You’ve been on the show at the end of 2022, which seems like a lifetime ago.

Reid Blackman: Definitely, in AI terms.

Andreas Welsch: But maybe for those of our listeners and viewers in the audience who are not familiar with you, can you share a little bit about yourself, who you are and what you do?

Reid Blackman: Yeah, sure. So I’m the founder and CEO of a company called Virtue. Back in 2022, I came on because I had just published a book called Ethical Machines with Harvard Business Review. For the past six-plus years I have been advising organizations, mostly in the Fortune 500, on how to avoid what I would now call the ethical nightmares of AI, or the ethical, reputational, regulatory and legal risks of AI, that sort of thing. So I mostly advise those companies, do a lot of keynote speaking, and wrote this new book. The other thing is that, by way of background, I’m not a technologist. My PhD is in philosophy. I was a philosophy professor for 10 years specializing in ethics, so my background is philosophy, not data science or computer science.

Andreas Welsch: Awesome, wonderful. If I think of the term having a spiky point of view, quite honestly you’re one of the first people that come to mind. So I really appreciate you bringing this topic to people’s minds and to the forefront with a little bit of an edge. I’m excited for our conversation today.

Reid Blackman: Yeah. Sharp edge.

Andreas Welsch: Awesome. Part of it.

Reid Blackman: Yeah. Hey, that’s what you need these days to stand out.

Andreas Welsch: So folks, if you’re just joining, drop a comment in the chat where you’re joining us from. I’m always curious to see how global our audience is. And if you want to find out how you can prepare your teams for the next generation of AI readiness, consider picking up a copy of my book called The HUMAN Agentic AI Edge. That’s on Amazon and everywhere else you can get your books online, as paperback, ebook and audiobook. Now, with that out of the way, Reid, should we play a little game to kick things off?

Reid Blackman: Let’s do it. We got the button.

Andreas Welsch: Okay, so here we go. Exactly. There’s the button. When I press the button, you’ll see the wheels spinning, and when they stop turning, I would like you to answer with the first thing that comes to mind and why, in your own words. And to make it a little more interesting, you only have 60 seconds. Are you ready?

Reid Blackman: Let’s do it.

Andreas Welsch: Okay, here we go. If AI were a sports car, what would it be? 60 seconds on the clock.

Reid Blackman: Oh, man. It would be a Lamborghini without brakes.

Andreas Welsch: Why that?

Reid Blackman: Well, because we know how to build these things. We know how to stitch these things together, but we don’t really know how to control them very well. So a car that can go extremely fast, but we don’t necessarily have the requisite safety equipment, like brakes on a car.

Andreas Welsch: I like that. It’s very easy if you go down the quarter mile, just straight, but we know life in business has a lot of curves and turns.

Reid Blackman: Turns. Yeah.

Andreas Welsch: So we need to know how we can brake and steer, which helps us steer the conversation. A lot has happened since you were on the show last time in November 2022. It feels like the industry keeps riding the AI hype cycle like a roller coaster, up and down and super fast. And it’s no longer just AI everywhere, now we’re talking about agentic AI everywhere. Again, kind of like my book, too. But what ethical challenges do you see that introduces now for organizations and even for society as a whole?

Reid Blackman: Yeah. Okay. So there’s obviously a lot to say here. One thing is going to depend on how you think about AI agents, what they are. So I’ll give a very quick articulation of how I think about them. Two simple conditions. Usually it’s an LLM, typically at least now, at the center that you interact with. The first condition is that it’s connected to other tools. It could be the internet, it could be other narrow AIs or other generative AIs, it could be connected to enterprise software, databases, and so on and so forth. And then the second condition for being an agent is that instead of you saying, “Hey, listen, I want you to go to narrow AI database number three and put it through narrow AI number five and then also consult with GenAI number seven,” instead of you giving it step by step, you just say, “Here’s what I want to do.” And it, quote unquote, figures out by itself, autonomously so to speak, how this should go, and then it goes and does the thing. So you don’t give it step-by-step instructions. You give it the goal. That’s how I think about an agent: an LLM hooked to a bunch of tools, with some degree of autonomy in how it pursues the user-specified goal. Okay, lots of places for things to go wrong. First of all, we already have all the stuff that besets narrow AI and generative AI. So you still have things like biased outputs, privacy violations, including IP and copyright violations. You still have hallucinations. All of that still applies. You also have some other things, because we’re getting really complex and messy here, because we’re connected to all these different things. So I’ll highlight three places where things potentially go sideways, new nightmares. Number one, you have what you can call cascading failures, or a snowball effect. We’re talking about big systems here. So this thing does something wrong and then it passes the information to the next thing, which amplifies the error, and then it goes to the next thing and amplifies the error again. With all these systems interconnected, if one of them gets the wrong answer and goes sideways ethically or otherwise, it’s going to echo throughout the system until you get really big, bad stuff. I actually don’t know exactly how a stock market crash like the flash crash works, but you could have, if you like, an ethical flash crash, where something goes wrong over here and then it resounds throughout the system. So that’s one source of errors, called cascading failures. A second kind of problem is systemic or emergent risks. The way I like to think about this is, suppose you’ve got a bunch of basketball players and they’re each good. No problems with any of them. They’re not making errors, let’s say. But there’s something about the way they play. This one plays a really fast game. This one plays a really slow game. This one is all about passing. This one is not. And for whatever reason, even though they’re individually good players, they don’t play well together. They don’t make for a good team. You can have the same thing with agentic AI, where all the tools to which it’s connected are on the up and up ethically, reputationally, brand and legal. But something about how your agent stitches them together doesn’t make for a good team. And so you wind up with these emergent risks. And then the last thing that I’ll say is this autonomy bit. Autonomy is not a risk unto itself. It’s that it exacerbates all the other problems. When AI creates biased outputs, privacy violations, hallucinations, when there are cascading failures, systemic or emergent risks, it does it at tremendous speed and scale. And if the AI is autonomous and you say, “Go do the thing,” and you walk away and come back, and it has been doing the things because it’s autonomous, and it turns out that one or more of those errors, ethical, reputational and so on, have occurred, you’re in trouble. So autonomy sort of pours gasoline on the other ethical and reputational risks.

Andreas Welsch: It also sounds like if you have increasing levels of autonomy, there are fewer humans in the loop or aware of what is actually happening or able to step in. So to your point, it multiplies, or exponentially grows, the risk of failure in that system. Not just additive, but multiplicative.

Reid Blackman: Exponential. Yeah, it becomes exponential, and the human-in-the-loop strategy breaks down. A human in the loop, as your audience probably knows, is meant to stand between the AI outputs and the outcome. The human takes the outputs and reviews them. Let’s say it’s a very simple broken-bone identifier for radiologists. The radiologist is the human in the loop. She says, “Yeah, that’s legit, that really is broken,” or, “Actually, the AI is mistaken here,” and then prescribes whatever she prescribes, and that’s the outcome. It already begins to be difficult with generative AI, but with agentic AI, let alone multi-agentic AI, when you have agents talking to each other, the human in the loop really begins to break down. Imagine the mountain of data and decision points made by the AI, and then you go to Sam in accounting and say, “Hey Sam, validate that, is that cool?” Sam can’t do that. Sam can’t handle that amount of data, that quantity of decisions, the speed at which they were delivered and the speed at which Sam is supposed to validate them. It’s not possible. So one thing we see with agentic AI and multi-agentic AI is that the human-in-the-loop strategy for risk mitigation really begins to completely fall apart.

Andreas Welsch: So what’s the answer then?

Reid Blackman: Oh, there’s lots of things to say here. We still need some kind of oversight, some kind of validation. People use other acronyms now, and they have for a while: human over the loop, human in charge of the loop, human behind the loop, ahead of the loop. There are all sorts of places. The short of it, though, is that I don’t think there’s one generic answer of here’s what you do. It’s for this particular generative AI solution, this particular agentic solution, this particular multi-agentic solution: how do we identify how things could go sideways, what can we do now to decrease the likelihood of those things going sideways, and what can we do now to monitor the thing once it’s in the wild, so that if it does start to go sideways, we can intervene in the right kind of way. And the answer to who should be doing what, when and how must vary on a per use case basis. It can’t be decided ahead of time, because of the context in which you could use an agentic system.

Andreas Welsch: So what do you do then?

Reid Blackman: Yeah. So I don’t think there is the thing to do. With all these different kinds of agents and all the different contexts of deployment, there isn’t the way you avoid the risks that a human in the loop might address in, say, narrow AI. The issue is that there are lots of different ways that things can go sideways. There are lots of different things we can do to decrease the likelihood of it, and lots of different things we can do to monitor the AI. So to me the question is not what’s the silver bullet if it’s not a human in the loop. Rather, it’s for this particular use case, for this particular context of deployment, what’s the set of things we can do to decrease the likelihood of really bad outcomes. In some cases that might be a data scientific solution. It might be someone downstream of the data scientist, like a marketing person or a person in HR using the tool. Usually it will be some combination of that and more. So I just think that when it comes to human in the loop, it’s not a panacea. It’s not the thing that’s always going to fix your problem by any means. In many cases it’s just going to heap a level of responsibility on someone that it’s not fair to give them, and then things will break down.

Andreas Welsch: So it sounds like, definitely take a deliberate approach. Look at the risk: what is the task, what is the risk involved anyway, and then when you layer agentic AI on top of it, how does that risk change, and is it even feasible to automate it? Is it feasible to have a human in the loop and make decisions?

Reid Blackman: Yeah, that’s exactly right. And even more generally, I think the key thing in the era of agentic AI is what it looks like to do that kind of problem solving at scale. I think that’s the question we have to wrestle with. And I’m sure we’ll talk about this some more, but it’s not just a matter of doing the thing. It’s: in each particular agent that you’re developing, let’s solve the problem of what the various things are that we can do to decrease the likelihood of various kinds of nightmares occurring.

Andreas Welsch: So I’ve obviously been following your content for a number of years, and I really love the perspective that you bring to that topic, not just of responsible AI, but really the ethical piece of it. I’m sure you purposefully call it ethics, as opposed to this broader term of responsible AI. Companies have been standing up ethics committees, they’ve been standing up responsible AI programs, but from what I see and keep hearing from you, it seems that something’s broken. What are you seeing? What’s really changed when it comes to AI ethics and governance and responsible AI?

Reid Blackman: Yeah. Okay. So there’s tons to say here. There’s a whole chapter of my book where I tear apart what I call the standard approach to responsible AI or AI ethics. We could talk about the difference between AI ethics and responsible AI, but it’s all in the same kind of mix. You can also call it AI governance, trustworthy AI, that sort of thing. Anyway, there’s a standard approach that I think is totally broken, and there are a couple of things going on here. The standard approach to responsible AI is very top down and very policy driven. It roughly looks like this. You get some executives together. You create something like an AI ethics statement, where you talk about being committed to privacy and fairness and transparency and accountability and that sort of thing. Two things happen to those values. Number one, they get translated into procedures. What does it mean to be committed to fairness? Well, we’ll check for bias in our models throughout the AI life cycle. What does it mean to be transparent? Well, if we use an AI in a customer interaction, we’ll disclose to them that an AI was used. So you translate it all into procedures. And the other thing is that the values get embedded, one way or another, in your enterprise risk policy. Now, one of the big problems with this is that it requires an enterprise-wide policy. That’s the approach. And if you talk to Fortune 500 companies, this is going to take forever. A year is the fastest you’re going to get an enterprise-wide AI risk policy passed. And once it’s passed, great, you can celebrate for half a second. Now you have to implement, because no one cares about your policy. It’s not like your employees wake up with a thirst to read the new policy. So now you’ve got to implement that policy, and that’s going to take at least six months for a department. And if you’re talking about organizations that span hundreds of markets and dozens of countries, there’s a long implementation roadmap. Meanwhile, the technology is leaping ahead. So you pass your AI policy on narrow AI. Great. Oh no, GenAI just came out. You finally pass your policy on GenAI. Oh my god, now it’s AI agents. You considered AI agents but not multi-agentic AI. Uh oh, another one, what do we do now? And especially because the technology keeps changing, your policy might ban something in one instance one year, and they might not want to ban it the next year because the technology has changed such that it’s now safer and we don’t need the ban anymore. For instance, there are monitoring agents, there are techniques and software that don’t exist now that will exist next year, and it’s going to be way safer to deploy them next year as opposed to now. Your policy can’t really speak to that except in an extremely general, generic way. So you take forever to pass this policy. Then you’re trying to implement it, but the policy is kind of stale and dated and dead. So you implement the policy, but the policy is out of date. So you get the C-suite back in and the board back in and update the policy. This is just a chaotic nightmare where you’re never actually getting to robust nightmare avoidance. That’s one big problem, the timeline. That’s one reason why it’s crumbling. The other reason is that you translate all the values to procedures, and given the complexity of agentic systems especially, half of those procedures don’t apply anymore. Half of them do, and there’s another 10, 50, whatever procedures that you do need to go through with this new agentic system that are not mentioned in the policy. So that never works. And the other thing is that you never know if you’re successful with this procedure-first approach, because what does success look like? Did you achieve fair outcomes? Well, I don’t know. We did the things. We did the procedures. But did the procedures make for a fair outcome, or a privacy-respecting outcome, or a transparency-respecting outcome? I don’t know. We did the things. You told us the value is just our procedures. We did the procedures, and so that’s it. It’s not outcome oriented. You don’t know what success looks like. It’s too slow to implement. The whole thing’s a mess.

Andreas Welsch: Yeah. And it’s always hard to put a price tag on something that hasn’t happened. Risk avoidance is a lot harder to estimate than something that materializes. So it sounds like there’s a lot of talk, a lot of red tape, a lot of back and forth between all the different departments and stakeholders and players, then finally you roll it out, and by the time you do, the world has moved on. You’re always in this iterative mode of catch-up, and then, hey, here’s something new, we need to think about this and roll it out.

Reid Blackman: Yeah.

Andreas Welsch: Now, you talk about this very purposefully in terms of nightmares. Maybe if you hold up your book, we can all get a look at the cover of it. That’s what it looks like, right? The Ethical Nightmare Challenge. It’s been out for a couple of weeks now, so I’m super excited to get some time with you to talk about this. But why specifically do you talk about this as nightmares?

Reid Blackman: There’s a lot of reasons. I have a chapter called “Why I Like Nightmares and You Should Too.” When you talk about values, like I just said, they’re very abstract. If I ask you to tell me what heaven looks like, we’re not very good at explaining what heaven looks like. If I ask you what some hellscape scenarios are, you can do those. We’re very experienced talking about nightmare scenarios. We have all of history, we have the news, we have books and film and TV and so on. So we have a deep familiarity with nightmares. This means that we can make them concrete and specific and vivid. They’re also emotionally engaging, so they’re motivating. They carry a level of urgency that values don’t. If you don’t live up to your values, or not fully, you say, “Well, we’re doing the best we can. It’s hard.” But if you run into a nightmare, you don’t say, “Oh well, you’re going to run into nightmares sometimes.” It’s, “Oh my god, no, we really got to avoid nightmares.” So I like nightmares because they are concrete specifications of outcomes, specifically outcomes that you want to avoid. You can build strategies and tactics around avoiding those nightmares in a way that you don’t really build strategies around abstract values like fairness. They’re very sharable. I can tell you a nightmare of mine and you’re going to get it in a way that you might not get if I just say I’m for fairness. When you say you’re for fairness, what do you mean? Nightmares are much more relatable in that way. They’re also relevant to organizations. You could specify what’s organizationally relevant. I’m not asking for your personal nightmare, but rather, for the organization for which you work, what are some ethical and reputational and legal nightmares that your organization can run into? That’s pretty graspable. And the last thing I’ll say, and then I’ll shut off, is that everyone can speak the language of nightmares. Maybe you’re a data scientist, maybe you’re in HR, maybe you’re in marketing, maybe you’re in product. If I say, “Tell me some kinds of nightmare scenarios that could result if this goes sideways,” everyone gets that. And if everybody gets that, now the data scientist can talk to marketing and HR fruitfully in a way that they couldn’t necessarily earlier if they said, “Well, we’re for fairness,” and then the data scientist asks, “What is that? What do you mean by that?” I think the language of nightmares is a touchstone that everyone can relate to, so they can have conversations with each other, which is the kind of thing that enables collaboration.

Andreas Welsch: But at a time when there’s so much talk about opportunities and an ideally bright future on one end, and on the other the super dystopian everything-is-going-downhill and it will erase humankind, doesn’t the term nightmare, ethical nightmare specifically, fuel this angst, this fear of something going wrong? Isn’t it leaving out the opportunity of what the technology can actually do?

Reid Blackman: A couple of things. I’m not a pessimist. I don’t think the sky is falling. I think it can fall if we don’t do things the right way. So this is me saying let’s face the problems head on. I don’t think we get to avoid the nightmares and get to the promised end of all this opportunity if we just close our eyes to the nightmares. “No, nothing bad here, nothing to see here, let’s just think about the opportunity.” That’s a really bad way. That’s a way for things to go really bad. The other thing is that the book is focused on making sure that the nightmares you specify are organizationally relevant, which means that you could actually do something about them. If you’re worried about AGI, artificial general intelligence, taking over and extinguishing humanity, well, look, you work at an insurance company. There’s nothing your insurance company is going to do about it. There’s nothing your CPG company is going to do about it. There’s nothing your health care company is going to do about it. If that’s going to happen, it’s beyond the purview of the organization’s nightmares. That’s humanity’s nightmares as opposed to the organization’s nightmares. Another thing to say is that I think people are already anxious and fearful, not just of the complete doomsday scenarios, but of lots of middle-range bad things happening: discriminating against people at scale, creating reports for clients that contain lots of hallucinated material, things going sideways in some massive way that you didn’t see coming at all and didn’t have the tools to see. I think people are anxious about that stuff. And the answer is not to say, let’s not focus on the anxiety-producing stuff, let’s just focus on the good stuff. That’s silly. I think we justifiably quell anxiety by facing the problem head on and then problem solving. We can solve this problem. We can reduce the likelihood of catastrophe here. I’ll say one last thing. Suppose someone said, “Hey, listen, we’re going to engage in developing nuclear technology. But listen, we can blow up humanity.” And someone would say, “Oh, don’t focus on that. Focus on all the opportunity that nuclear has.” It’s like, are you out of your mind? You need safety. That’s obviously an insane view.

Andreas Welsch: So you need safety.

Reid Blackman: You need safety, and you need to attack the problem head on so that you can take advantage of the opportunity. So this is not about eliminating all risk by pursuing AI. That’s not what I’m suggesting at all. Pursue all the cool things. Make sure you don’t blow up things, whether it’s people, your brand or both. In the book I specify a method for doing such a thing, and so on. But that’s my short answer to why focus on the nightmares. I think it’s compelling in the right kind of way.

Andreas Welsch: Yeah, I can absolutely get behind that. Personally, I see myself as a technopragmatist. Yes, there’s lots of opportunity, but there are risks. The truth is somewhere in the middle, and we need to find a way to make the technology practical without exposing everybody else to unreasonable amounts of risk or unmitigated risk. Now Reid, we’re getting close to the end of the show, and I was wondering if you can summarize the key three takeaways for our audience today. We covered a lot of ground about ethical nightmares. What are the three things that stick?

Reid Blackman: I’d say one of those things is that the AI risk landscape is getting insanely complex, diabolically complex, as I call it in the book. The move from narrow to generative to agentic AI is a big leap in the AI risk landscape. And when you get to multi-agentic AI, we’re really talking about, in the book I call it an AI risk cluster.

Andreas Welsch: Yeah.

Reid Blackman: So that’s one big takeaway: the AI risk landscape is getting crazy. Takeaway two, the standard approach for dealing with AI risk, the responsible AI approach, etc. The kind that I’ve developed over the past six-plus years, the kind that the Big Four have developed. I’ve advised three of the Big Four, so I know exactly what they’re doing. Any consultancy has been doing the same kind of approach. I don’t think me and my colleagues and competitors deserve blame. We just didn’t know what was coming. But it’s broken. It’s not the right way of doing it anymore. And anyone who’s trying to sell you that standard way of doing responsible AI, I think they’re setting you up for failure. The third and last thing is less a takeaway, because we didn’t talk about it too much, but it’s what I try to articulate in the book. Whatever the solution looks like, and I have my own, the ethical nightmare challenge is my solution, it has to be something that can be rapidly implemented and scaled. If you can’t rapidly engage in ethical, reputational and legal nightmare avoidance, then either you’re not going to innovate at all because you’re going to be too scared, or you’re going to innovate totally recklessly. The ethical nightmare challenge is my solution for rapid, scalable governance of responsible AI, AI ethical management. But even if you don’t use mine, you’ve got to find something that can move at the pace at which AI evolves and is implemented and scaled.

Andreas Welsch: To me, that’s really the big piece, the last one. It’s moving incredibly fast, blazing through all these different innovations, and it’s not just a matter of keeping up, but really also keeping pace with it. So thank you so much for sharing your expertise with us. It’s always a pleasure talking to you and learning what you’re seeing around ethical AI, responsible AI. Folks, if you’re in the audience, make sure to check out Reid’s book, The Ethical Nightmare Challenge. That’s out now. And thank you so much for joining, Reid. Really appreciate it.

Reid Blackman: Yeah, it was my pleasure. Thanks for having me.

Andreas Welsch: Wonderful. Folks in the audience, see you next time for another episode of What’s the BUZZ. Bye-bye.

Blackman said agentic AI still has biased outputs, privacy and copyright violations and hallucinations. It adds cascading failures, where one error is amplified through connected systems, and systemic or emergent risks, where individually sound tools do not work well together. Autonomy then multiplies all of these at speed and scale.

Blackman said a human in the loop is meant to review AI outputs before they become outcomes. With agentic and multi-agentic AI, the volume of data and decisions and the speed they arrive at are more than one person can validate. He said it can also put an unfair level of responsibility on that person.

Blackman said the standard approach is top down and policy driven. Passing an enterprise-wide policy takes at least a year in a Fortune 500 company, and implementation takes years more, while the technology keeps changing. He also said it is procedure based, so it does not show whether outcomes were fair.

Blackman said values such as fairness are abstract, while nightmares are concrete, vivid and emotionally engaging, and people carry a sense of urgency about them. He said they can be specified for an organization, and that data scientists, HR and marketing can all discuss them, which enables collaboration.

Blackman said he is not a pessimist and does not think the sky is falling. He argued that ignoring the nightmares is a way for things to go badly wrong, and that facing the problems head on is how organizations pursue the opportunity without blowing up people, their brand or both.