How can your business get ready for the EU AI Act?

Andreas Welsch: Today we’ll talk about how you can get your team ready for the EU AI Act, and who better to talk about it than someone who’s helping others do just that, Andrea Isoni. Hey Andrea, thank you so much for joining.

Andrea Isoni: Thank you for having me.

Andreas Welsch: Wonderful. Why don’t you tell our audience a little bit about yourself, who you are, and what you do?

Andrea Isoni: My name is Andrea Isoni. I’m the Chief AI Officer at a company called AI Technologies. All we do is AI and machine learning development for other companies, both private and public. That means I touch a variety of different projects in many different industries, in all the countries possible, with data, multimodal as we say nowadays. Apart from that, I am also on the committee board for disclosure, because I guess we are going to touch on ISO certification for AI. For the new ISO certification, I am a board member who helped to approve and draft the policy.

Andreas Welsch: Wonderful. I’m excited to have you on the show and looking forward to learning from you what’s top of mind, not only for the EU AI Act, but also what’s happening around other certifications, standardizations and so on. For those of you in the audience, if you’re just joining the stream, drop a comment in the chat where you’re joining us from today, because I’m always curious to see how global our audience is. Now, Andrea, should we play a little game to kick things off?

Andrea Isoni: Sure.

Andreas Welsch: Alright. This game is called In Your Own Words. When I hit the buzzer, the wheels will start spinning, and when they stop, you’ll see a sentence. I’d like you to answer with the first thing that comes to mind and why. In your own words. To make it a little more interesting, you only have 60 seconds for your answer. Andrea, are you ready for What’s the BUZZ?

Andrea Isoni: Yes.

Andreas Welsch: Perfect. And those of you in the audience, if you would like to participate as well, put in the chat what your answer would be. So here we go. If AI were a month of the year, what would it be? 60 seconds on the clock.

Andrea Isoni: If it was a month, it would try to predict the next month until it got to the end, and then it would turn out a mistake. It would hallucinate, and after December it would get to banana, member. Something like that.

Andreas Welsch: Awesome. Definitely captures the state of AI in many areas today. Great answer. Let’s come to the questions that we’ve prepared. We know the EU AI Act is coming. It was announced at the end of last year. I think just before that there were a couple of things that extended the process. I know Germany and France were in the mix there, and they gave additional input and feedback. I think it has evolved quite a bit from its original draft. But I’m wondering who is affected by the EU AI Act? Whom does it actually aim to protect?

Andrea Isoni: First of all, it’s a question of principle. The first thing to remember is the wording of the act and the policies: safety. Safety of beings, and above all, human beings. That’s the first thing to remember. It’s nothing about a business per se. Obviously it affects businesses that deal with consumers. But the goal is to make AI safe, safe in usage for whatever it is. There are different tiers of risk. When I say risk, I don’t mean business risk. I mean risk to human life, or hindering life in some way or some form. So there are different tiers, from unacceptable to acceptable. The acceptable tier will be just GDPR, like we know it today. Unacceptable, we can go a little bit into that. In between are the important ones, regulated, and when you are regulated, you need to put governance in and make sure the system is compliant with certain regulations. It’s interesting to understand from a regulatory point of view what is not allowed anymore in a practical sense. Biometric identification in public spaces, including cameras, will not be allowed, apart from the police or an institution with a specific procedure. Another one that is very important is social scoring. Scoring a specific individual, not a group of people but a single individual, based on emotion or racial background, will not be allowed. I want to point out that this regulation is not completely disaligned from other regulations. Think about airlines. Everyone knows this. You go on the web to find the best price, and then you come back five seconds later and the price has risen. What do you think? They targeted me, because I watched it before. I can tell you that was already illegal before, because any algorithm, for airlines or anything else, cannot target the individual. It can only target a group of people. If they raise the price for you, they have to raise it for everyone else. The new regulation on AI is just aligned to what was there before. I think that is important to understand. It’s not something out of the blue that they invented.

Andreas Welsch: Thank you for sharing. That’s an important part as well, how it aligns to other regulations. Out of the EU, the other landmark regulation is certainly GDPR, introduced a couple of years ago. On one hand it protects individuals. On the other hand there are concerns that it adds additional burden or might stifle innovation, that you’re over-regulating. But what you mentioned about the risk-based approach, whether something is high risk, medium risk or acceptable risk, is a classification that you can make in your own business with a good degree of certainty. That’s a good north star to say these are the things that are allowed and these are not. From your perspective, if you’re in an AI role, if you’re an AI leader in a business today, what are the three things that you need to know and that you need to do now?

Andrea Isoni: If you are a business that is using AI somehow, and when I say AI it can also be an API from other providers, the first thing to check is the worst potential outcome that can happen to your clients, your employees, or third parties that use your service, which usually means clients. Once you identify the worst potential harm, the second step is to check your cybersecurity exposure, because if anything, this regulation will increase the cybersecurity budget. The reason is, think about a chatbot. In theory, the worst that can happen is that it insults someone. You have a chatbot that, for whatever reason, starts to insult the client, or tries to mislead them into buying something they shouldn’t. Obviously this is not happening, because you are controlling it. However, imagine a hacker who breaks into your system and tries to mislead your client. In that case, the first thing is understanding the worst potential harm. The second is increasing your cybersecurity barriers so no hacker can make it happen. In my example, the chatbot is exploited by an external party and now you are committing something against the rules. The third thing is governance. You need to have the right governance in place. If you remember the GDPR regulation, you had data stewardship, champions, the data protection officer, all these roles. As a rule of thumb, translate these roles into AI roles. The data protection officer becomes an AI officer. Data stewardship becomes AI stewardship. The data champion becomes an AI champion. You can start your governance in the same way you did for data under GDPR. So essentially three things. Identify the worst potential harm. Check your cybersecurity exposure towards that harm, because a hacker could exploit it. And third, set up a governance that resembles what you did for data under GDPR.

Andreas Welsch: I’m sure there’s a lot more detail that one can go into on each of them. I like how that aligns with some of the previous topics we’ve covered on the show, like the OWASP Top 10 for Large Language Models, which already outlined some of the things that you mentioned. In a couple of weeks, I’m looking to have someone on the show to talk more about your security exposure because of generative AI. On one hand there are hackers exploiting AI to attack you and your systems, and on the other hand there is what you can do within the company. So a great connection there as well. The AI Act has been hotly debated for the last couple of years. Some say it stifles innovation, that it completely misses the point, that it’s over-regulating. At the same time, I think governments do have a responsibility to protect their citizens. I empathize a lot with the intent that the European Union has put forward with this regulation. But the AI Act has been so hotly anticipated in the industry for that period of time, so I’m wondering what is really new. What has changed since the initial discussions? Obviously high risk, medium risk and low risk classifications are still there. But with the introduction of generative AI, ChatGPT and so on, what would you say has changed that people need to be aware of?

Andrea Isoni: As we experienced, we are trying to push as much sophistication into the model as possible. As you mentioned, large language models are coming into place now, with billions of parameters, actually trillions. What that means is that the more sophisticated a model is, the less intelligible and less explainable it is. All of this is going to change when this regulation comes into place, because not explainable means you are going to be even more affected by the regulation. You need to pay even more for governance. The end result is that the number of parameters of models will go down. Why? Because the higher the number, the more you need to spend on governance, and businesses try to minimize cost. Now, where did the EU AI Act start? It started with safety, which is still here. The other aspect, which is only marginally here compared to when we started, is copyright law, essentially protecting people’s intellectual property. There was way more in the regulation originally, but it was even more difficult to get all the parties to agree on an actual intellectual property law. So I would say there is a hint of what was originally in the regulation. It’s also important to say, when you mention France and Germany, yes, they are still discussing. But this regulation is almost approved, in a way. I’m not a legal expert, but all I know is that the final piece is when the EU Parliament and the EU Council ratify the Act. That didn’t happen yet. From that moment there is a transition period, and the transition will not be for all the tiers together. The highest risk will be within six months, and then slowly until 24 months, which I think is for the last acceptable risk. That moment will come when the European Parliament ratifies it, which as far as I know has not happened today. Frankly, I don’t know the date.

Andreas Welsch: Maybe one of the obvious questions, if you’ve looked into the AI Act or read it, and to make it very explicit: even though it’s called the EU AI Act, it doesn’t only apply to companies in the EU. Can you talk a bit about that and what that means?

Andrea Isoni: First of all, what’s the status in the world? The EU is the most advanced, but I have to say even China has regulation as well. I wouldn’t say similar, I don’t know enough to say. It’s funny that the EU and China are leading the pack in this, while the US has a very light touch, because the US is afraid to lose competitiveness for its own businesses. Probably the EU, actually, this is the main reason. And for the rest of the world, I think there will be adoption of this, or at least alignment. I’ll give you an example of what I mean by alignment. There is an international organization, the OECD. They made a definition of what an AI system is, and that definition is used in today’s EU AI Act. All I’m trying to say is that, and that’s my hope, all these regulations, although in different continents or parts of the world, will somehow be transferable and compatible. That’s very important to me. I really believe we need worldwide regulation with no distinction. Otherwise you will get too many imbalances. A hacker in a non-regulated country can easily attack you in a regulated one. So in a regulated country you can end up with the worst of both worlds. You are regulated, so your company is limited, but at the same time you are affected by those outside. The only way to avoid that is for everyone to play the same game. If you’re not playing the same game, a certain country may exploit that against you, even if you think you are safer. Which is funny, but that’s the way it works.

Andreas Welsch: That’s a very important point as well. Now I’m wondering, you mentioned that because of the regulation you anticipate models will get smaller, with fewer parameters, and be easier to explain. Does it mean there will be more fine-tuned models, more domain-specific models, overall smaller models that the large vendors will also provide because they’re easier to explain? Where do you see this having the most impact? Is it in your own business, if you’re building your own model?

Andrea Isoni: There are two different approaches here. One is what the business needs. Most of the time, a business needs a very simple automated solution. For a simple automated solution, frankly, even a neural network architecture is probably too much. So what the regulation is forcing businesses to do is to take this decision: can you do it with logistic regression? Please do it. Why? Because logistic regression is very explainable. I need to do little governance on it, and it costs less, so it’s better for the business. If you start to use increasingly complex models, then you need to count on more regulation. You need to do more tests, safety checks. Because, frankly, nobody, at least today, if you use a large language model, even assuming you know all the data, and if you are using an API from OpenAI or Anthropic you don’t even know the data they are training on, which is not a problem, but definitely you don’t know the path of the neurons that are activated. So this is very difficult to explain today, and you will need even more tests to check the security and safety of the model and the governance. That’s one part. And second, we don’t really understand how they work anyway. Until these two things are mature and under control, it makes no sense to scale even more parameters.

Andreas Welsch: I remember when GPT-4 was announced, it was all about how many parameters it has, or if OpenAI is working on GPT-5, how many more parameters it has and how it grows. But if the direction is smaller models with fewer parameters, also for the purpose of explainability and interpretability in the context of regulation, that makes a lot of sense. We touched on it in the beginning, the EU AI Act is a landmark regulation here. But there are also other regulations, compliance efforts and standards when it comes to AI. What are some of the really important ones that people need to be aware of but might not be as familiar with?

Andrea Isoni: There is the International Organization for Standardization, ISO. They have policies for quality assurance and information security: 27001 for information security, 9001 for quality. You’re probably familiar with them. Many companies take these standards to certify that they follow certain quality and information security standards. That helps them to be less exposed and to have the right governance in place. In exactly the same way, one or two weeks ago, ISO published a new standard. It’s called 42001, and it is about AI. With this standard, you create your own governance around your AI model. Not the data, the model. That means how to train for accuracy, how to build the data pipeline right, how to check for hallucinations, using the right metrics and correlations, in a governance-safe way. That is in tune with the EU AI Act. The problem is that nobody knows when the regulation will effectively be approved. The important thing is, how do I know I have the right governance? You follow a standard. There’s no guarantee on anything, that would be a fallacy. But at least you can show you are following acceptable worldwide standards for your governance, like you do for quality or information security.

Andreas Welsch: It’s great to see that there is some standardization being introduced, and that you and your business can, and potentially should, adhere to it. I would recommend taking a look at that as well. I’m sure you’re quite active in that area. We’re getting close to the end of the show. Could you summarize the key three takeaways for our audience today?

Andrea Isoni: First, understand the potential harmful impact of the AI model you are using. Once you understand that, understand whether there is any cybersecurity exposure or leak that can be exploited. And third, try to understand which kind of governance you can have around the models. That’s for business people.

Andreas Welsch: Awesome. Thank you so much, Andrea, for joining us and for sharing your expertise. It’s great having you on. And for those in the audience, thank you for learning with us today.

The EU AI Act aims to make AI safe to use, with the safety of human beings first. According to Andrea, it affects businesses that touch consumers, and it sorts AI systems into risk tiers from unacceptable to acceptable. Risk here means risk to human life, not business risk.

Andrea names two examples: biometric identification in public spaces, and social scoring of a single individual based on emotion or racial background. Targeting an individual with a different price, as an airline site might, was already illegal before the act.

Andrea gives three steps. Identify the worst potential harm of your AI use to clients, employees or third parties. Check your cybersecurity exposure to that harm. Set up governance by translating GDPR roles such as data protection officer and data steward into AI roles.

Andrea notes that the EU and China lead on AI regulation while the US takes a lighter touch. Andrea expects other regions to align, as the OECD definition of an AI system already appears in the act, and argues for compatible worldwide rules so that regulated companies are not left with the worst of both worlds.

ISO 42001 is a new standard about AI that Andrea says was published a week or two before the episode. It guides how a company builds governance around its AI models, including training for accuracy, the data pipeline and checks for hallucinations, and it fits with the EU AI Act.