How can businesses build AI agents with a mixture of assistants?

Andreas Welsch: We’ll talk about building AI agents for business with a mixture of assistants. And who better to talk about it than someone who’s actively working on that? Jeremy Ravenel. Hey Jeremy, thank you so much for joining.

Jeremy Ravenel: Hello, how are you?

Andreas Welsch: Doing really well. Thank you so much for being with us. Hey, why don’t you tell our audience a little bit about yourself, who you are, and what you do?

Jeremy Ravenel: It’s a tough question to start with, but I’m going to start to make it short. I’m French. I grew up in a nice countryside in Brittany. I’ve always been super fascinated about the world around me, connecting music, art, science, nature. And I started my journey in the professional world at 18 years old. I took my backpack, went to India, traveled Asia, started working for a French company, doing an internship over there, came back, and went to business school, forced by my dad, in 2009. So during the financial crisis. And at that time I had to take a loan from the bank, and it was a financial crisis at that time also. So everything was about: is the money mine? How am I going to be able to use that cash for what I want? So I deep dived into the world of corporate finance, cash flow management, cash flow forecast. And I started being specialized in that area. And when you work for a corporation, you have to do cash flow forecast. You also have to connect everything to everything. So my little child brain, super curious, asking so many questions, got a bit of substance to work in that area of cash management and cash flow management. And so it was really great to start with this. I did a lot of Excel, VBA, PowerPoint presentation. So I turned a bit data mercenary in a way, for every C level, CEO, CFO. I worked for two companies, mainly in software prototyping and in oil and gas. And it was always the same thing: how to answer the questions from business managers, obsessed by how to answer the questions, how fast you can deliver those answers to the business. I was slowly going from Excel, VBA, PowerPoint to Python notebooks and orchestration of pipelines and data and AI eventually. And it was really wonderful at the time when I started using those technologies, because it was also the advent of open source libraries. You could do a lot with those open source libraries, write scripts, build pipelines, and I got fascinated by the world of open source at the same time. And we created, with my brother and my associate, a company at that time, a consulting company, that helped us build a product. And this product is now called NAS, for Notebooks as a Service. Essentially, what we do is we use templates, open source templates that we’ve been creating with our community, more than 3,000 templates. And we use them as components to create data products, data and AI products. And one of those products is what we’re going to talk about today, which is this ABI, Artificial Business Intelligence, mixture of assistants project. And it’s really interesting to be able to talk to you about it today because we have been working on it in an applied research mode with design partners that helped us build the first version of this project. And I’m super excited and super grateful to be here and chat about it.

Andreas Welsch: Thank you so much for being with us. It’s great hearing about your journey and how going to business school, finance, corporate finance, now led you to delving more into the world of AI and what you can do with it for business. For those of you in the audience, if you’re just joining the stream, drop a comment in the chat where you’re joining us from, because I’m always curious to see how global our audience is. So should we play a little game to kick things off? What do you say?

Jeremy Ravenel: Yeah, let’s go.

Andreas Welsch: All right. So let’s see. This game is called In Your Own Words, and when I hit the buzzer, the wheels will start spinning. And when they stop, you’ll see a sentence. And I’d like for you to complete that sentence with the first thing that comes to mind and why. In your own words. To make it a little more interesting, you only have 60 seconds for your answer. Now, if you’re in the audience, please put your answer and why in the chat as well. Always curious to see what you come up with. Jeremy, are you ready for What’s the BUZZ?

Jeremy Ravenel: I’m ready.

Andreas Welsch: Awesome. Then, here we go. If AI were a song, what would it be?

Jeremy Ravenel: Oh, I think it’s a song from Bob Dylan, Everybody Needs to Serve Somebody. It’s saying this story about how you could be anyone, but you need to serve somebody. And I think that the first thing that comes to mind is this song from Bob Dylan. I could take my guitar and sing it to you if you want, but it’s a very good song. It’s all about serving someone, being in service of something. And I guess that this is what is completely lacking today. It’s a general technology. So who do we serve, really? Who is the person, the public, the audience? What’s the outcome? Why do you serve? I’m always in this mindset of serving somebody.

Andreas Welsch: That’s awesome. I love it. That’s a fantastic answer. We’ve had a number of different answers before. When people were asked that, the question about serving somebody I think is really key. And also when you think about AI in business, who are you serving? And why is it AI? What should it actually do other than just serve? Chase a shiny object.

Jeremy Ravenel: Yeah, the song is fantastic. It’s taking all these personas all the way, and if you’re this, you end up needing to serve somebody at some point.

Andreas Welsch: I’m looking at the chat. So we have folks joining from the UK, from Houston, from Toronto in Canada.

Jeremy Ravenel: And hi everyone.

Andreas Welsch: Hey, so thanks for joining. And looking at the answers: If I had a hammer, In the Morning, and Yesterday from the Beatles. So some really good suggestions there as well as some classics. Now, Jeremy, you mentioned you’re building artificial business intelligence. And you said you defined it as a pragmatic alternative to AGI, Artificial General Intelligence, for businesses. So I’m wondering, what’s the story behind it? You already shared a little bit about it, but why do businesses need a specialized kind of intelligence?

Jeremy Ravenel: If I’m totally natural and honest with you, I think that AGI is really about a lot of hype, and it’s something that I don’t think I can see in the business setting. So at the beginning, when we started doing these jokes internally with my team, “Hey, AGI this, AGI that,” we said, “Hey, no.” It was an internal, private joke. This is something that is not really useful. And we had been working with my team for a lot of years in the world of business intelligence. And this totally makes sense. The business intelligence, the reports, the data that you bring to the business, how you serve someone at the management level with proper, trustworthy data, our world was really turning around BI and how to automate, how to go fast to the reporting and to the knowledge of how a business works. So eventually, AGI turned ABI, because artificial business intelligence. And we found out that this idea of BI was a bit too connected to Google, dashboards and reports. So how do you bring back the core idea of business intelligence, which is essentially mapping the whole flywheel of your business? And then we started thinking about all those different data products that we have built over time, very specific data products. And this mission that I also have, since I’m out of the corporate world, to be able to explain the cash flow. So when you want to explain cash flow, you actually need to connect everything in the business. It’s not only about the cash. You need to explain how to generate top line revenue, but for generating the top line revenue, you need to explain the awareness, the marketing, so how many people do you reach on your website, how many people convert, how many people that convert become potential deals, how those deals become eventual activities for your sales team, how those sales activities become conversations, tasks, projects, contracts, and so on. There is a lot of flywheel effect to different data that traverse the flywheel over time. And so we started thinking about this flywheel as a “content to cash” flywheel. And I must give a hi to one of our design partners. We have been working with Vin Vashishta for a long time. We also thought about this idea that we need to have a flywheel “content to cash” that actually makes sense to be able to map all those different tables together to really serve the holistic understanding of key business essentials. And so I guess that our main idea with this ABI is to really be lean around a few engines, content, marketing, sales, operation, finance, and open data, and be able to operate tables that will aggregate most of the data that a business is working on. So that’s the main thing. And then you need to answer questions and distribute those outputs to a chat-like interface. Because most of the questions that you have, you want to ask them through a chat. A business dashboard is here, but you need to have other questions. You need to further the process. And I think that the two technologies, the classical BI reporting tools and the chat, are actually two sides of the same coin, which is bringing business intelligence to executives so they can take the right decision and basically drive the car in the right direction.

Andreas Welsch: Now, you also mentioned earlier that it’s about a mixture of assistants. And I know, if folks in the audience are familiar with LLMs, there are some ideas on a mixture of experts, right? How does that relate? Or where’s the agentic AI component in this when it comes to these?

Jeremy Ravenel: Yeah, sure. It’s a good question. So the idea of mixture of assistants came up with the observation of the current state of the market. So you have LLMs trained on external data. This external data is basically everything that we see on the internet, and this is called mixture of experts. And that reflects the current way an expert is. It has a lot of knowledge about the world, the external world, but it doesn’t know so much about you. When it’s about you and your business, you have an assistant. You have an assistant that knows your business, knows your workflow, knows your data, knows your tools. And that makes sense. In the context of a business, an expert is always someone that is coming from the external world and comes to your business environment to connect with your assistant, essentially the person that knows best how your company works. So we found out that this MoE, mixture of experts, and mixture of assistants were probably two sides of the same coin. And then when we started thinking about it, the question was, how do you actually merge the two? And how do you have the external knowledge of the world merge with your internal knowledge, your workflows, and have this entity? So I often have the image of a yin and yang kind of unity of those two sides. And when those two fuse together, they actually create your business AI system. So that’s the mental framework that we started developing. And the agentic part of this is really about the fact that now you have LLMs and business data that is structured as knowledge, and we’ll talk about it. It’s inherently directed to this kind of infrastructure where you want to operate a personal AI assistant that accesses those different assistants for content, growth, sales, operation, finance, and open data. And so we crack those six assistants, and the idea is that your personal Andreas assistant will call out this layer of ABI mixture of assistants that will be able to communicate and explain to you: the cash flow is going down or is going up because we did less of these tasks, or because we did fewer contacts, fewer deals, and basically the new ad that we ran didn’t work. And so all this link, and how to go back to the source of the explanation of the actual cash flow data, is really how we want the orchestration and the whole experience of the user to be.

Andreas Welsch: That brings up several interesting points. Now for those of you in the audience, if you have a question for Jeremy, please put it in the chat and we’ll take a look in a minute or two and pick some of those up. Now I think what you shared is really intriguing. On one hand, having assistants that can tell you more about individual parts of your business. What you mention around sales, marketing, why are certain things working or not working, and then go and get the data and bring it back. The other part you mentioned, I think, is also key, especially in business and enterprises. And that’s the part of how do I actually know that the information that I’m getting back from my AI is trustworthy, right? How can I trust it? And so on. So I’m curious there, how does your artificial business intelligence framework actually work? And you mentioned the knowledge graph underneath, how does all of that play together in it?

Jeremy Ravenel: Yeah, so it’s really important to state first that LLMs shouldn’t be trusted. They hallucinate. And so the whole game is how to ground, to make this fusion of the LLMs and the knowledge that you have internally. And so the main thing that we did was to create pipelines, essentially data pipelines, good old ETL pipelines, that you connect to a tool, you extract the data, you structure it, you curate your data. So that’s why I think that this whole story about removing the data engineers and the data scientists and everything, you need those people. They’re bringing the curated data set to you. But the question now is, when they bring that data to you, that curated data set by domain, with subject matter experts, with data engineers and data scientists that can work to bring those big data sets, you need to structure it in a certain way. And structuring it in a certain way means also getting back to the domains: content, marketing, sales, operation. You need to break this down to another level, which is big tables. So you have a content table, an ID table for the content engine. You have a people and organization table for the growth marketing engine, because it’s all about people. You have a deal table and an activity table for the sales. And so those big tables, those OBTs, which is a pretty standard modeling technique, right? That is going to be your foundation. And those foundations, those big tables, you need to transform them into a knowledge graph, which is not the same structure, but it’s the same data. And you need to structure this knowledge graph in terms of nodes and relationships so that the LLM can understand and retrieve data when it comes back to you in a chat. You can’t really feed the tables as such to an LLM. You need to use this new structure that is known for a long time. Google, Amazon, Facebook have been using knowledge graphs for a long time to serve us targeted ads. And so we need to use the same technology to serve ourselves targeted outcomes that will come from our data. And so the real big challenge is to create this schema, this ontology that is going to be the reference point of your human to AI interaction. So that’s the big part where there is so much yet to be done, but we have constrained ourselves in this project. It’s an open source project that you can look at in our repository, where you’re going to see that there are different models, different engines. Those different engines break into templates, which are components of a pipeline. Those pipelines create tables, and those tables create knowledge graphs, eventually creating a plugin that is going to be your final interface in the chat. So that’s an architecture that is pretty novel and that has to be designed from the ground up, without forgetting about the good old and the new techniques of data modeling that we’ve been doing all the way. Because bringing that curated data into the knowledge graph is one thing. The knowledge graph is probably the new part where data engineers and teams need to adapt to craft those ontologies and knowledge graphs. And then you have the distribution aspect, the RAG. So I see really the ETL and RAG and the knowledge graph on top, and you climb upstream and downstream, basically. And so when you do that, at the end of the RAG level, it’s all about understanding the questions. Because we could take the problem the other way and start with the questions to model the knowledge graph that will model the data. And so those two worlds can reconcile at some point. But what we see is that the good old data modeling is very useful, but now we need to really think in terms of questions, not so much in terms of reports and graphs, but more in terms of conversation and how I want the experience of conversation with the AI to be. So it’s another level of interaction that you need to craft, another level of experience that you need to craft. And it’s completely a data ops topic. I see that there are comments about data ops. It’s totally about data ops and data operation and how to serve in a constrained environment, so that you can run tests. Because once you have the questions, you can run tests. And you can verify that for a certain scenario, you will have a certain output. So if you extrapolate to the other scenario, if nothing changes whatsoever, you can confirm that your test of questions always outputs the right answer. The right structure, the right number, the right data. So that’s the whole way and journey that we are into. It’s cracking all the questions, cracking all the pipelines, making sure we have an ontology. So there are many moving parts. But we are lucky to have really amazing design partners to do that.

Andreas Welsch: That’s awesome. And like I said, those are all important, meaningful problems to solve.

Jeremy Ravenel: That’s for sure. Yeah. Because if you think about it, it’s all about where do we stand right now? We had the beginning of the internet. We had CMS, content management systems, WordPress, websites, all that stuff. Then those brought up leads. So CRM needs to be here to catch the leads and transform them into deals. Those CRMs, now that the deal is closed, I need some enterprise resource planning, some ERP, to actually do this stuff. But what does the CMS connected to the CRM, connected to the ERP, really do, and how are those agents all going to work together? We don’t have any clue right now. So I think the technologies have been additive, from CMS, CRM, and ERP. And I see this ABI, whatever, mixture of assistants or agentic system, whatever we call them, those agentic systems come on as an additive technology on top of those three to serve the end user eventually in a natural way, which is what we have all been thinking about: the Jarvis Iron Man mindset, where you want to be talking to your assistant as a business owner, an AI assistant or something that you can trust that has access to all your knowledge and that you can really challenge with questions, to build different simulations, different scenarios to anticipate, because the world is moving so fast right now that long-term planning and stuff is not going to cut it. You need to be able to dynamically move, and I think those systems are here to do that.

Andreas Welsch: Now, I heard you mention earlier, if you’re a data engineer, maybe if you’re in the audience, or if you’re listening to the recording, you can sleep easy tonight. Jeremy said you’re still needed and very much needed and valued. Don’t be afraid, right? There’s more work to be done than others might like to admit. Now, on the other hand, if you’re a leader, if you’re a technology leader and you hear about these new architectures, you hear about certainly large language models, right? You’ve done your own experiences and experiments with it. You know about knowledge graphs, the architecture that you mentioned. How can technology leaders get started with all of these different things and things moving so quickly? What’s your recommendation?

Jeremy Ravenel: I think it’s all about data modeling. It’s all about modeling your business. So thinking really carefully about what makes your business successful today and what the connections are. What part do you take in this flywheel? I think that’s the most important and underserved topic. It’s how to do high-level data modeling, understanding the whole business, understanding the value creation, how data is used. It’s connected to one point and creates value at the other point. And that’s why, with this framework that traverses content to cash, we think that we can provide at least a blueprint. And all the topics, everything that we do right now, when we talk to leaders, because those are the primary persons that we address our technology to, it’s like: you need an AI system for your business. Do you have an AI strategy? What does it look like? Do you want to serve your end users? This is a completely different thing, a topic on its own. That is your product and how your product is going to integrate AI. But for your back office operations, how do you handle everything? What is the AI system that will support your core value proposition to your users? So I think that everything around this back office and front office needs to be carefully put into two different boxes. You do your front office operation to serve your customer, and you have your back office operation to handle everything. And ABI is more something about the back office operation. We’re not going to do a front-end chatbot on your website. That’s not what we think needs to be done. But my first advice would be having those two things and then understanding the flywheel. Understanding the data models and, yeah, start capturing the relationships between your data and not the data itself.

Andreas Welsch: That sounds like a real great call to action, and especially starting from the data.

Jeremy Ravenel: Yeah. And understand the relationships between the parts of the business. It’s all of the moving parts. It’s all about the supply chain and how to create value at the end.

Andreas Welsch: Perfect. Now, we’re coming close to the end of the show, and I was wondering if you can summarize the key three takeaways for our audience today.

Jeremy Ravenel: It’s a good one. One, LLMs shouldn’t be trusted. You need to be part of a bigger ecosystem. And so you need to have the ecosystem that supports those LLMs. They are very useful, but they are just bringing you all the wealth of knowledge of the world into a small, tiny model that you can pair and synchronize with your internal knowledge. So number two is to map your internal knowledge and start looking very carefully at knowledge graphs and how this knowledge of your business, this unique, authentic way of you doing the business, needs to be modeled. And the third thing is mostly, I think, know who you serve. If you are doing a project, you either do a project for your front office operation, everything that relates to your customers, or your back office. And these are two different kinds of things that you need to think of. Maybe some of your back office operation can be monetized at some point and become products. But concentrate on those two aspects one at a time. Do incremental, continuous upgrades. It’s a journey. It’s not a destination. So you need to start with small pieces that you handle well, and then craft your AI system, bit by bit, workflow by workflow. Very important: break everything down to your workflows, because that’s your unique way of doing business. Your workflows are the gold of your business. No other business operates like you do, doing stuff like you do. This is why your customers come to you. And I think that understanding this internal gold that we have, and why we are successful today, is something that needs to be captured into a system. And I believe AI and agents and all those technologies that are going to emerge in the coming years are going to be instrumental in capturing that value, so that tomorrow every company can become a technology-oriented company. The balance sheet of that company can embed the value of the asset that you create through your AI system, and it can eventually grow the pipe for the whole global economy, I think.

Andreas Welsch: Jeremy, thank you so much for summarizing that and for joining us. Thank you for sharing your expertise with us.

Jeremy Ravenel: Yeah. Thank you so much for having me. It was a great pleasure. And I’m glad to have had this conversation with you.

Andreas Welsch: Wonderful. And for you in the audience, thank you so much for joining us.

Jeremy describes artificial business intelligence (ABI) as a pragmatic alternative to AGI for businesses. He says AGI is mostly hype in a business setting, while ABI maps the whole flywheel of a business through engines such as content, marketing, sales, operation, finance and open data.

According to Jeremy, LLMs trained on external data work like a mixture of experts: they know the outside world but little about your business. An assistant knows your workflows, data and tools. A mixture of assistants fuses both sides into a business AI system.

Jeremy says LLMs shouldn’t be trusted because they hallucinate. He grounds them in internal data through ETL pipelines that curate the data, big tables by domain, a knowledge graph of nodes and relationships, and RAG for retrieval in a chat interface.

Jeremy explains that you can’t feed tables to an LLM as such. Transforming them into a knowledge graph of nodes and relationships lets the LLM understand and retrieve the data, and the schema or ontology becomes the reference point for human to AI interaction.

Jeremy’s first advice is data modeling: think carefully about what makes the business successful, split front office from back office work, and capture the relationships between your data rather than the data itself. He also tells leaders to start small and break the business into workflows.