Is generative AI still in a hype phase or already in disillusionment?

Andreas Welsch: Today, we’ll talk about cutting through the generative AI hype and finding out who’s ahead. And who better to talk about it than somebody who’s actively working on that. Mark Beccue. Hey, Mark. Thank you so much for joining.

Mark Beccue: Hi, Andreas. How are you doing?

Andreas Welsch: Doing well. So excited to have you with us. And I thought maybe you can tell our audience a little bit about yourself, who you are and what you do.

Mark Beccue: Yeah, sure. I’ve been an analyst for a long time. I won’t tell you how long, but it’s long. I had been working on AI as a market research analyst. I started following, if you remember the old Slack back in 2015, that started coming out. People were like, what the heck is this? And they had all these little bots that would run on that. And I started writing some research about that, which quickly turned into a virtual assistant focus. It was the early days of AI, early thoughts about AI. And what that turned into was working for Tractica, which was a small market research firm that became a specialist in AI. So I’ve been focused on market research around AI since 2015. I still do that today. I’m a principal analyst with ASG. So that’s what I do.

Andreas Welsch: Awesome. Wonderful. I know we’ve worked quite a bit at SAP and that’s why I’m so excited to have you here for an independent perspective and independent conversation today on the topic of Gen AI. So folks, for those of you in the audience, if you’re just joining the stream, drop a comment in the chat where you’re joining us from, because I’m always curious to see how global our audience is. Now, Mark, what do you say? Should we play a little game to kick things off?

Mark Beccue: Let’s do it. Perfect.

Andreas Welsch: All right. So this one is called In Your Own Words. And when I hit the buzzer, the wheels will stop spinning. And here we are. So I would like you to answer with the first thing that comes to mind, and why, in your own words. If AI were a fruit, what would it be?

Mark Beccue: I’m going to say Gen AI is a dragon fruit.

Andreas Welsch: Okay, why is that?

Mark Beccue: Because it’s exotic and bizarre and not many people have tried it yet. Maybe they want to, and they’re a little afraid to try it. They’re intrigued by it, and they want to try it. It just sounds fun.

Andreas Welsch: Okay, that sounds good. It can be pretty sweet too, right? I remember having a dragon fruit juice in Asia when I was traveling there. Sometimes it’s also the right mix and the right place to get it from that you need. Awesome. Great answer. With that out of the way, why don’t we talk about some of the questions that we’ve discussed up front that I see many people, and many leaders, are asking about. Just two weeks ago or three weeks ago, something like that, Gartner released their latest hype cycle for AI. We see Gen AI is moving towards the trough of disillusionment. There are some reports coming out. I think it was Morgan Stanley the other week where they’re saying maybe we’re missing a trillion dollars in that promised economic potential that we forecast last year. So I’m curious, are we still in a hype phase or are leaders already getting disillusioned? What are you seeing?

Mark Beccue: It’s interesting to me that it’s following a typical cycle for technology disruption, in a couple of different ways. And you can stop me if you want when I get too crazy on this. We were introduced to a more economic way to bring models to market. If we go back, before Gen AI you had to have a data scientist, you had to have your own data, more or less, you had to have a lot of expertise, and you had to have your own data to do anything. We were trying to do something you couldn’t. And then this drops out of almost nowhere, and all of a sudden it becomes feasible and plausible that you might not have to have all this data to make an AI model work. And you might not need a very specific data scientist to make it work. This drops in, and there’s a lot of hype, obviously, a lot of promise. I think that’s still there. What we’re running into, and I’d like to hear your thoughts on it, actually, is that it makes some things easier, but it also makes some things the same. That doesn’t change the fact that this isn’t a completely baked solution. There are no completely baked solutions. There’s the promise of this. It’s very raw, it’s so new, and we watched how the models have really just morphed dramatically over a very short amount of time. And I just think we’re at the front end of really operationalizing Gen AI, and that’s causing, so is it disillusionment? In a way it is, but I look back to what I originally said, which is technology disruption: nobody hits a home run immediately. It doesn’t work like that. People are figuring it out and we’re going through that sorting process.

Andreas Welsch: So do you feel the industry, the VCs, the big and small companies in the software industry have over-promised? Or is it just a natural excitement because we want to be innovative? We need to be innovative. We know that we can be innovative because of this technology and thereby, first of all, deliver more value for our customers, but also make more money, and from a startup perspective, attract more money. To me, it’s almost like this flywheel effect that comes together here.

Mark Beccue: It does. I don’t know what we’re really calling the first iteration of AI. I call it legacy AI. That kind of happened around 2015, as I mentioned earlier, when compute got cheaper and it became feasible to run a lot of these pieces. What happens is there’s a learning process: the life cycle of experimentation, what things can do and how to put guardrails around things. We were getting there when Gen AI showed up. All of these companies that had been down this road were moving at a pace I’d call legacy AI. It wasn’t lightning speed, but it was moving forward and it was a lot of investment. And what happens is when people all of a sudden go, oh my God, what can I do? They don’t have any experience, so they were trying to create something from scratch. When Gen AI came in, it broke up the models, the business model around AI. It was like, we can make our own thing. What do you want to make? And you and I have talked many times about what you are trying to solve. If you think about where we are in market adoption of these things, the people that were going to solve something for someone haven’t done it yet. I’ve got some tools. I think you’ve said shovels. We’ve got shovels, but we don’t have the end result. Here’s something you can use to go try and figure it out. That’s where we are: there haven’t been a lot of what I call end-to-end solutions for AI. So if you were Corporation B and you asked, how do I fix this? Nobody can come to you. They can give you compute and they can give you models, but it’s not an end-to-end solution. With some exceptions, and stop me if I go crazy, some of the SaaS players have built solutions that you could plug and play and try to do something.

Andreas Welsch: Yeah, and I’m also wondering for you in the audience, if you have any questions, feel free to pop them in the chat, we’ll take a look. And I think just lately we’ve seen so much coverage when it comes to this question of whether there is disillusionment in the market. First of all, I think we’re just building the foundation. If you look at NVIDIA, if you look at Microsoft with Azure, Google and others, you first need to build that platform, you need to build that foundation for others, for higher value services and offerings to come in and make those available. Maybe that’s a good segue, because I know you’ve been covering the adults in the AI room for quite a bit. I’ve already mentioned a few just now, but who are they from your point of view? And what do they do well, or what do they do extremely well?

Mark Beccue: Yeah, so you’re talking about a little segment I used to run, called the “Adults in the Generative AI Rumpus Room.” You had some companies running forward with ideas and technology that I would call not so responsible. We’ve been there. Some of them start with O and some of them start with S and end with AI and things like that. But when you throw software into the wild and just let it percolate there without any guardrails, things happen. If we go back and look at when ChatGPT first came out, one of the first things that happened was an assault by students of all sorts, high school students to college students, starting to write their papers with plagiarism and things like that, and you had to scramble. Schools and universities were like, oh my God, what do we do with this? These people are cheating. That’s irresponsible. So we go back to adults. The adults in the generative AI rumpus room are the ones that I think have put guardrails for themselves and responsible use of AI in place for themselves and their customers. There’s a group that I call AI pioneers, that were AI pioneers in the legacy realm. They are some of the usual suspects that you might think of. Some of the hyperscalers are adults. They write research, they look for ways to build guardrails around things that they have access to. I think of Google, AWS, Microsoft, IBM, and I will name several SaaS companies that I think have been very good: SAP, Salesforce, Adobe come to mind. I think some of the chip folks have been good, or what’s called on-prem people like Dell, Qualcomm, Intel and AMD. All of those kinds of companies have been thoughtful about what they do. Then there are lots of small ones I’ve named, but what they’re doing is saying, before you go, here are things you need to consider. I think they’re all generally, in many forms, thinking about responsible use, whether it’s transparency or looking at how to reduce inaccuracy that you have with some models, those kinds of things.

Andreas Welsch: There’s one question in the chat by Ashish that goes in that direction, and he’s asking about AI guardrails: is there a concerted effort coming together or forming, or is each player putting out their own version or interpretation of how we can govern and safeguard against the perils of AI?

Mark Beccue: Yeah, that’s good. I think it’s moving on multiple fronts. I don’t know what you think, Andreas, but there are things like NIST, which people have started to look at. If you think of the AI Act and the impact it’s going to have, from a regulation standpoint, there are some guardrails in that sense moving forward. We look to these really big regulatory players like the EU. In the U.S., I think NIST has put out some really good papers and some frameworks around responsible use. That’s not really a law, but there are some frameworks. I’ve seen a lot from the Responsible AI org. They have some interesting uses. But it’s a mishmash of different things, where you have some companies, early ones like SAP, that wrote some guardrails and some frameworks. So did Salesforce. I think they’re one of the ones I look to that wrote some very pragmatic ways to approach it. I’ll just talk about that for a second. They were one of the ones that I first heard talk about responsible use, not just in the iteration of the idea, but how you manage it over time. You’ve talked about this before many times, having oversight committees, playing into your other guardrails like security and those kinds of things. To me, it’s a mishmash. What do you think?

Andreas Welsch: So to me, I think it’s a really good sign that over the last years vendors have been investing in that topic. And especially that many of those have been investing beyond lip service. That’s the important part. It’s easy to say, yes, we’re responsible and we do this, or we feel we need to do this because the market seems to expect it. But really seeing that become a real practice, and something that is actually adopted by large and leading organizations, I think also can be a lighthouse or a beacon of light, if you will, and a good example for others to follow, especially smaller companies who might not have all the resources to put these programs together themselves. If there’s something they can orient themselves around and say, to your point, Microsoft, Salesforce, SAP, whoever else is doing this, maybe there’s something in there that we can adopt on a smaller scale for our mid-sized companies.

Mark Beccue: This is a slightly different route. Microsoft published their AI safety, was it the UK safety conference or something in the summer? Remember that? And it was important to me because their partner was OpenAI. If you read through it, it was really very detailed about how they were building these guardrails around how they use OpenAI models, which was different from when OpenAI does it by themselves.

Andreas Welsch: Yes.

Mark Beccue: And I thought that was really well thought through. It doesn’t necessarily give you a path if you’re an enterprise, but I felt the market got confidence from using Copilot and those kinds of things based on what Microsoft had built in such detail in these documents that they published, which was great. It’s nice that you could see that from a big player.

Andreas Welsch: Yeah. And it also gives good assurance to the market and to companies looking to use these systems if there’s more transparency and more explanation, more thought behind it overall. So great, perfect. Good question. Please keep them coming in the audience. We’ll take another look in a minute or two. Now, reassurance that you’re on the right path, that it’s a solution to invest in regardless of the vendor, is to me also tightly coupled with the topic of cost and pricing. I know you talk to many different vendors, and I’m sure you also talk to many different buyers, but we know that buyers prefer cost transparency and plannability. I want a fixed bill, so I know how much I’m paying every month. And I don’t like the spikes because they’re too unpredictable. And you have folks in both camps, to be fair. I’ve seen that myself as well. But I’m wondering, what commercial models are you seeing that resonate with B2B buyers around Generative AI? Is it the $30 per user per month, or is it pay what you use, calling card, phone card type of model?

Mark Beccue: This goes back to your first thoughts around disillusionment, and you mentioned that very well. The point is that we’re at such an early stage of making Generative AI real in the enterprise. First, it’s what are the costs, and what are you buying? That is really the first question. What I’ve noticed is a difference. Looking at earlier iterations, with legacy AI, we saw a big bunch of venture capital money put into what I call end-to-end solutions. I got the sense that the vast majority of the market in the legacy AI trends before late 2022 was toward solutions, not tools. What I see differently right now, in this early stage of the market, is that enterprises are not looking for end-to-end solutions. They’re looking for tools and platforms, and it’s almost like do it yourself. They want to invest to build their own expertise. That goes back to your point about cost, because it means you have to figure in a whole lot of other costs, like the compute cost. That might be one of the reasons we’re seeing this hesitancy: we didn’t know it costs that much to run cycles on foundation models. What does it cost for compute? What does it cost for that development? What does it cost for securing it? They’re going through all this, where they have to think about the complete life cycle, and they didn’t. I think that’s a little backwards. And it’s something you’ve mentioned many times, and I’m a big believer in it: I think enterprises are forgetting their basic business school training, which is what am I trying to solve? What am I good at? What do I want to solve? Should I build, buy, or partner? It’s all those basic questions. And then you get to what I want to pay for something. I know I’m answering that in a weird way, but what are you trying to buy? Because then you’ve got to put it all together. At the end of the day, it has to make economic sense, either that you’re saving money or making money. Is it going toward cost savings as an enterprise? Is it going toward revenue generation? All of those answers are very much in flux. I’m not sure I answered it, except to open more of a can of worms. What do you think?

Andreas Welsch: Yeah, I think it really depends on the organization and their preference. So I can see it both ways. One is, change management is tough enough in itself if you introduce a change process into the way people work. So why not try something that’s built in, where you get a head start and you don’t need to build everything from scratch in your entire stack? But I also see that where it’s more differentiating to your business, again coming back to your point of really understanding what is that business problem that I’m trying to solve, if it really differentiates my business, my operation, if it gives me an edge in my industry to be a leader or become a leader, then I think investing more on that platform level and looking for those opportunities that you might build yourself makes perfect sense. But the change management around it is still going to be there no matter what.

Mark Beccue: Which is why, and I’m interested in what your thought is here, this might cycle back to where we were before with the emergence of SaaS. Enterprises will always say, what am I good at? What is the focus of my company? Is this something I outsource? Is this something I want to own? I think when you want to leverage your data, that’s something you might want to own more than outsource. That being said, you look at some of these SaaS players and they’re giving you the opportunity to buy. And to your point, I’m buying this as part of something that accelerates what I already do with that company, whether it’s supply chain management or collaboration tools with Zoom or whatever. How much more am I paying if I’m just getting a call summary from Zoom? I don’t have to do it. They already did it. It’s all baked into the price that I already bought for this collaboration service. So a little bit of a bump up, sure. If I can see that, I think there’s a lot of experimentation with that.

Andreas Welsch: I think that’s a good segue to our last question for today. We already established that you obviously talk to many different vendors and you have a front-row seat at what’s happening in the industry. It also goes nicely with Jason’s question that he was asking in the chat. I’m wondering, what’s your prediction? Where do you think things are going in the next six months on Generative AI? Maybe one of the questions here from Jason: will we see a convergence of LLMs, of conversational interfaces? Is it one of the things that you see in your crystal ball?

Mark Beccue: Yeah. All these different vendors. We could talk for days, couldn’t we? Two things. I think over the next year, from now and just keep moving forward, there’s this struggle to bring data to the AI or bring AI to the data. I think at the end of the day, the differentiator over time is not going to be the technology that we’ve seen. It’s not LLMs. The models are going to become models. They’re going to be used for different reasons. Enterprises might use a bunch of different ones, and they’ll use them for different reasons. But at the end of the day, it’s how do you leverage what’s unique to you as a company, and that generally ends up being data. And there’s a struggle. This has been a struggle for a long time. Remember when we used to all talk about big data and Hadoop and all those things? That hasn’t happened yet. To Jason’s point, I think one of the great hopes and promises of Generative AI is that maybe, with that help and the economy of scale we get with these foundation models, we can tap and unleash the data that companies have so that they can use it for multiple reasons, for whatever it is. One of them is just that idea that you’re going to tap into your data to use it for whatever purposes you want. I think that’s one of the big things. I don’t think there’s going to be a convergence of LLMs. I think they’re becoming more task oriented. You’d use a big one for certain things. You’re using smaller ones for certain things. And because of cost, you’re balancing that when you need it.

Andreas Welsch: Thank you so much. Now look, we’re getting close to the end of the show today, and I was wondering if you can summarize the key three takeaways for our audience.

Mark Beccue: You asked a tough question up top, which is, are we in disillusionment? I wouldn’t call it disillusionment. I’d call it the pragmatic phase. We’re getting into the pragmatic parts of what enterprises need to do to operationalize generative AI. For the folks that are listening, there’s a lot of upside to this. Some of the predictions seem a little heavy to me, for it’s going to do this, it’s going to do that. It’s going to take time. It always does. When we really back up and look, be patient with Generative AI. Take your time. Be pragmatic. Think about what you want it to do. Start with the problem first and then see if it can help you solve those things. And be pragmatic about whether you need to build, buy, or partner based on your needs.

Andreas Welsch: Thank you so much. Also thank you so much for joining us today and for sharing your expertise with us. It was wonderful having you on the show and learning from all your insights from talking to so many different vendors and being in the space for such a long time. Thank you so much, Mark.

Mark Beccue: Oh, thanks, Andreas. Appreciate it.

Andreas Welsch: Wonderful. And thanks to those of you in the audience for joining us live.

According to Mark, generative AI is following a typical cycle for technology disruption. He says it is disillusionment in a way, but he prefers to call it the pragmatic phase, in which enterprises operationalize generative AI, and notes that nobody hits a home run immediately.

Mark uses the term for companies that put guardrails and responsible use in place for themselves and their customers. He names hyperscalers such as Google, AWS, Microsoft and IBM, SaaS companies such as SAP, Salesforce and Adobe, and chip and on-premises vendors such as Dell, Qualcomm, Intel and AMD.

Mark says that before late 2022 the market leaned toward end-to-end solutions, while enterprises now want tools and platforms to build their own expertise. He adds that buyers must then budget for compute, development and securing the solution.

Mark does not expect a convergence. He says models are becoming task oriented, with big ones used for certain jobs and smaller ones for others, and that cost drives the balance. He sees the lasting differentiator in how a company leverages its own data.

Mark recommends starting with the problem the business wants to solve, then asking what it is good at and whether to build, buy or partner. The answer must make economic sense, either as cost savings or as revenue generation.