Alice Drahon & Sarah Oury - IA & Green : Il existe des méthodes pour faire cohabiter IA et frugalité

[00:00:05] Hello. So, you can hear me well? Perfect. Well, hello, hello. Welcome to this talk. So we're going to talk to you about environmental impact and AI issues. If that can work together.
[00:00:26] So in the first part, we're going to try to present to you the main impacts. We will also adapt according to what we learn from you, we'll ask you a few questions. After, in the second part, we will talk to you about the evaluation of the measurement, because working on environmental impacts is above all knowing how to evaluate them. And after, in the third part, we will more globally talk to you about systemic approaches
[00:00:54] that allow us to transform ourselves to try to make frugal AI. There you go. So, who are we?
[00:01:05] So, my name is Sarah, I'm an AI and backend developer at Sopra Steria, and I'm part of the Boa Vista association, which I joined about a year ago, and I'm also active on the machine learning group.
[00:01:20] And Alice Drouin, I am also in the Boa Vista association, in the Machine Learning group, and I joined the board of directors not long ago, and I am an independent consultant for a little less than a year on sustainable AI projects. That is, I do project management, but really focusing on mastering environmental impacts and human rights. There you go.
[00:01:52] So, about Boa Vista, so Boa Vista is an association that has existed since the Covid period, around 2020, and which builds commons that allow to make an environmental impact assessment of digital. So the first common that was built
[00:02:14] it's a common that you can either download or access via API, which allows globally to do a multi-criteria evaluation of the environmental impact of one of your services. There you go. And then, as it has had a lot of success, it's a large community, there are other topics that have been launched, and notably this question of the environmental impact of artificial intelligence. We will talk about it more later. And also the way we are organized is that every two weeks there is a presentation of tools, methods, there is really a community that is very active to share the know-how that is evolving very quickly at the moment.
[00:03:10] So now that we've introduced ourselves, we're going to try to get to know you better to adapt our presentation a bit. Uh, can you raise your hand if you have already done your carbon footprint?
[00:03:23] Individual. Quite a lot, many even. Um, and have you already carried out an environmental assessment of an IT service? It's more specific, there are fewer hands raised, but there are some. And do you know what an LCA is? So a life cycle analysis of an IT service? There are a few hands too.
[00:03:45] Okay. Yep.
[00:03:49] So, today, why are we talking about AI? Uh, as you know, it's a topic that we hear a lot about, there were also several topics around AI during the Flocon, and uh it's a topic that is pushed a lot from all sides, and there are many issues related to this very rapid deployment of AI, notably ethical, societal and environmental aspects, and so it's important to take them into account and not rush in headfirst.
[00:04:17] And I would just add a small point, which is that with the arrival of generative AI, it really allowed to democratize the understanding of AI and its uses, and that's what also creates a lot of an acceleration phenomenon with a whole ideology that also comes from Silicon Valley, which is to say, well, we have to go, we have to go, we have to go. Hence this diagram that I think is great because, well, we are also here at Flocon, we ask ourselves questions about what we do and how we do things well.
[00:04:47] And, well, there's really this problem at the heart of the issues around AI and especially because of the environmental impacts.
[00:05:00] So we're going to talk more specifically about the impacts on the environment. So AI, like any digital service, uh, it has many impacts on the environment, uh, which are linked to the infrastructures on which they are based. So to have an AI service, we need to use servers, the network, and user terminals. Uh, and so to do this, we need to manufacture them, which has many impacts on the environment, and also during all their phases of use, we will need electricity consumption. And this electricity has impacts at production.
[00:05:35] So, notably on greenhouse gas emissions, uh, which have impacts on global warming, uh, via the combustion of fossil energy. Then, to manufacture the components, especially in the context of AI, we need many quite specific chips, like GPUs. To manufacture these GPUs, we need minerals.
[00:05:55] that we go looking for in mines that are gigantic and which therefore have enormous impacts on the reduction of biodiversity and the availability of abiotic resources. And besides, these mines are really exploited in conditions, in human conditions that are really disastrous. And finally, the third resource that is most affected is fresh water.
[00:06:19] So it can be polluted or evaporated during the electricity production cycles, but also and especially in data centers to cool the data centers. And during the manufacturing cycles of electronic components.
[00:06:36] If we focus a bit on electricity consumption, we can see that since 2023, electricity consumption in AI-related data centers has started to become non-negligible. Um, and so we have this report from Deloitte that makes estimations and projections on how this impact will evolve in the coming years. Um, and it used two different scenarios, a standard scenario which leads to a multiplication by four of electricity consumption in data centers by 2050. And uh a second scenario of strong adoption which multiplies by nine the electricity consumption by 2050. Um, it should be known that the standard scenario considers that we will have a rather reasoned development of AI, only in rather simple and profitable cases. And so considering a growth that will be lower than what we have seen between 2020 and 2024.
[00:07:39] And uh the high growth scenario, it considers that the growth rate will remain the same as the one we observed in 2000 between 2020 and 2024. Which is very probable given the arrival of generative AI, uh, which in 2023 exploded with ChatGPT and therefore text generation, but since then we also have audio, video and image generation which are even more energy-intensive. And this increase in electricity consumption is linked to the manufacturing of many new data centers, uh, to, well, precisely to be able to deploy more AI models. Uh, and uh we are building many of them in companies but also at the country level. We can see it for example in France at the AI Summit in February, uh the president announced the creation of a new data center with investments of 50 million euros to create a data center with a capacity of 1 gigawatt-hour and create one of the largest AI campuses in Europe.
[00:08:39] But this, this increase in the number of data centers also poses problems, we can see it for example in Ireland, where there are so many new data center installations very quickly that we have conflicts over the use of electricity. And besides, this hyper-rapid consumption, it also makes the energy transition complicated or even non-existent, because all the new renewable energy infrastructures that are developed, they are supposed to be there to replace the old, the old installations that are much more polluting, but as the energy demand continues to grow, well, in fact, instead of replacing the old infrastructures, they just add up, and so we're not talking about energy transition at all, but about even more global impact.
[00:09:29] Um, and so, faced with this increase in the growth of electricity, many hyperscalers have made commitments, notably on their reduction of carbon emissions. The problem is that there are different ways to calculate these carbon emissions. So from the same quantity of gigawatt-hours, for example, we can estimate a different quantity of equivalent CO2 depending on the methods used. There is the location-based method. which is therefore based on, which is based on the energy consumed in the region in which one is located, so for example for a data center in France, we use the electricity mix of France to convert the energy into quantity of equivalent CO2. And the market-based, it is based on the energy that is purchased by the company. And so, the company can in a way offset its real CO2 emissions by buying renewable energy certificates, also called carbon credits. Except that these certificates, they do not guarantee that we are going to create new infrastructures.
[00:10:33] It can be based on existing infrastructures and in addition it does not really reduce, it does not encourage companies to reduce their energy consumption and their carbon footprint. And we can see that in addition the correlations between location-based and market-based are really weak, it doesn't at all represent the reality of companies' emissions. And so we see a lot of hyperscalers who, well, announce that they are already net zero carbon and that they have reached their objectives. But in fact, behind that, it's by using the market-based method which is very largely contested by numerous organizations, including Boa Vista.
[00:11:14] And finally, if we look directly at the carbon footprint of a particular model, uh we can start with the example of ChatGPT, which is the most known, we could say. Uh, with the training of GPT-3, which is the model on which ChatGPT was based at its release. Uh, we hear a lot that it's the training of the models that is enormous and has an enormous impact, and that inference is ultimately negligible because if we look at the scale of an inference, the consumption of energy, water, etc. may seem negligible.
[00:11:45] Um, it's true that training really has an enormous impact when we see it there, 552 tons of equivalent CO2 emitted just for the training of GPT-3, knowing that after we went to 3.5, 4, 4.0 and there are plenty of other models from other providers. Uh, but in fact the impact on inference is also enormous because users don't just make one request in the year, and there are millions of users who use ChatGPT every day, which means that the carbon impact for one month of ChatGPT inference is already 18 times higher than that of training. So all that multiplied by the other models that exist and the other application cases, it makes a really colossal impact.
[00:12:31] So, we're going to do a little zoom on the methods of environmental impact assessment. Uh, as Sarah told you at the beginning, there are numerous impacts. So already, in general, if we want to be rigorous, we do multi-criteria analysis. We're going to look at CO2 as well as water, well, energy consumption, but we can also look at pollutants and pollutant emissions in the air, there are a lot of indicators. And the choice of these indicators
[00:13:05] uh, depends a lot on the business of the company that wants to do the evaluation. That's one point, the multi-criteria aspect.
[00:13:13] The second point is that we're going to look at the entire life cycle. Because using only, measuring consumption on usage is not sufficient. Studies show that, uh, consequently, all the mining extraction, manufacturing part is a huge part of the impact. So, we're going to measure this mining extraction, manufacturing, transportation of equipment.
[00:13:45] distribution, usage, so utilization, and finally also everything that is end of life. So our ability to recycle or uh, well, to decompose materials.
[00:14:00] There you go. So why do we do that? Precisely because, uh, digital seems to be completely abstract, but in reality, there are big physical impacts that we don't see. Because, at least for now, we don't have a data center next to our house. Maybe soon, but for now, no. So in fact we don't see these enormous data centers, the servers, all the equipment that allows us to access these digital services. So today, the manufacturing of equipment, so the extraction of minerals and the manufacturing of equipment, is the largest part of the environmental footprint. And proportionally, this part is decreasing, but why?
[00:14:46] Because in fact, with the deployment of data centers, usage has increased so much that finally, uh, the environmental impacts of data center usage make it a larger proportion of usage, but that's not necessarily very good news.
[00:15:05] To give a little bit of equivalences, uh, we consider in France, uh, that the management of, the consumption per year per French person of digital equipment, so that's all equipment beyond
[00:15:24] even beyond AI, it's all equipment, it's 1000, 1700 tons, 1.7 tons, sorry, 1.7 tons. Knowing that for the Paris agreements, we are all supposed to go below 4000. So already, if we have that part on digital equipment, we see that we are not quite on proportions that allow us to meet our commitments.
[00:15:52] And finally, on the last part of the life cycle analysis, on the recycling part. You should know that today, we hardly know how to recycle equipment. So there's really a whole sector to build.
[00:16:10] And that to this day, still for example in France, we know how to recycle 22% of e-waste, then it's collected and recycled, that's it, 22%. So it's very little.
[00:16:28] So, why? Already because, for example, to make a microprocessor, so in this example, a GPU chip, there are more than 80 metals.
[00:16:38] They are cut into very small pieces, uh, so that's one of the reasons why, uh, we have trouble recycling it. And in addition, there is also a phenomenon that worries people who need material. It's that we're entering a period of resource scarcity because the yield of mines is increasingly poor. So to extract these minerals, we need to dig more and more, so that also increases environmental impacts in terms of air and soil pollution, all deforestation. since we're going to look for new, new spaces, new mines. And here we focused on the environmental part, but as Sarah said, uh, the mining industry is one of the dirtiest industries in the world, and uh in terms of respect for human rights, uh, well, there are wars, there are working conditions that are close to slavery, finally, it's there are real systemic and geopolitical consequences. So, what does the European Union want to do in the face of this observation, and also in the face of this geopolitical dependence? Well, so there is an action plan that has been put in place, and the objective is to already open mines in Europe, because until now, in fact, we externalize our environmental impacts.
[00:18:13] So, we'll see if that can work or not. Uh, the idea is to try to develop the recycling of metals, but we saw that it was very complicated and we have, even if we have regulatory constraints, we have difficulties to do that. And uh develop eco-designed objects and the fourth axis, that's really the one we want to talk to you about today, is to prioritize sobriety because in reality it's
[00:18:42] it's the only point that we know we have a very high environmental return on investment and that works well.
[00:18:52] And uh so to finish on this impact part, uhm.
[00:18:59] Who has already heard of the rebound effect?
[00:19:03] So, thank you very much. So, the subject of environmental impacts and our way of consuming is that we are told, we do a lot of optimization. So we are told, ah well, we are going to optimize, for example, we have really managed to optimize the energy consumption of screens, uh well, that's great, but suddenly, in fact, we have changed all the screens, we have much larger screens. So the rebound effect, uh, that's it.
[00:19:28] It's that now we're starting to see statistically that all the optimizations we make in terms of environmental impacts, uh, finally have negative impacts because systematically, we increase our uses behind. And there we are right in it with uh with chat the AI, generative AI, there are a lot of uses that we can optimize, but at the same time, there are such rebound effects that globally, if we look globally at CO2 emissions, well, they are on the rise and they are not a small rise. So, in fact, we changed all the screens, we have much larger screens. So the rebound effect
[00:19:27] that's it, is that now we are starting to see statistically that all the optimizations we make in terms of environmental impact ultimately have negative impacts because systematically we increase our uses afterwards.
[00:19:43] And here we are right in the middle of it with uh with uh chat AI, generative AI. There are a lot of uses that we can optimize, but at the same time, there are such rebound effects that globally, in fact, if we look globally at CO2 emissions, they are rising and they are not rising by a small amount.
[00:20:09] Okay. And so to continue, we're going to talk about measurement and evaluation, because precisely to make good frugality decisions and to know when it is necessary or not, useful or not, to create an AI system, it is really important to have in mind all the information, including environmental impacts, especially in cases where we want to create examples of AI for green. So AI that is supposed to have a positive impact on the environment, well in this case, we must necessarily estimate the negative impacts of creating this system. Uh, and so concretely, I'm going to show you tools that allow you to measure or estimate the environmental impact and therefore the energy consumption of AI systems. Uh concretely, it's complicated because just like for AI, it's even more complicated than for more traditional digital services, since we have an AI life cycle that is made up of different stages, from data stages, training, inference, all of which have impacts. Uh, that the infrastructures uh they are varied, data centers, networks and user terminals, and that as Alice explained, the life cycle of equipment is constituted not only by the usage phase, but also by their manufacturing, their end of life, etcetera. And so it's a long process to estimate, it's costly, it requires expertise, and there are still obstacles, but there are also many initiatives already created. Uh, notably energy consumption measurement tools. Uh, so if you have access and you manage your servers, you can simply use uh, in a fairly classic way, a wattmeter to measure the energy consumption of your servers. Uh, if you don't have access, uh there are other types of wattmeters that are more software-based, that allow you to make estimations and measurements, notably the code carbon library, which is a library that in only a few lines of Python code, allows you to start a tracker and stop it, and thus track the energy consumption of the CPU, GPU and RAM during the execution of these lines of code. Uh, and then we have a lot of estimation tools, uh each of which has its own use case and its own specificities. I'm going to go pretty fast, it's just to give you examples of tools to delve into in detail, but there are QR codes each time if you want to take a photo of it to explore it later. So "RIGA algorithm" is a web interface for simulating the use of CPU and GPU, so we describe an infrastructure and an execution time, and that allows us to estimate energy consumption and a carbon footprint. Uh, there's EcoLogit, which is one of the best known, I think, which is easier to use and is really specific to generative AI, so to text generation in particular, and so we describe which model we use and what type of prompt we generate, so how many words we are going to generate with the AI, and it gives us the energy impacts and also multi-criteria with the depletion of abiotic resources and CO2 equivalent emissions. For the usage and manufacturing phases of the material. Uh, then there's AI Energy Score, which is a model comparison tool, again, it's only on inference, so without taking into account the training of the models. We come to compare models for a specific task, so here for example, on capture, we are on the text classification models, and with a star system, it will tell us which models are the most efficient.
[00:23:40] And finally, Boavista API, which is therefore a tool that was developed by Boavista, which is not specific to AI for now, but which allows to estimate the multi-criteria impacts related to the use and manufacturing of different digital services.
[00:23:59] Uhm.
[00:24:02] Ah yes, if you have a break, if you have questions, we can take a short break before moving on to the BoAmps initiative.
[00:24:16] No, I have one.
[00:24:17] Okay.
[00:24:18] In the depletion of resources. For example, I was wondering if the ADEME study you showed, if it had hypotheses on the depletion of resources. Because I haven't seen any details on this. Are there already hypotheses on the depletion of resources, or not at all?
[00:24:37] Uhm. In this ADEME study, no. Uh, after in other studies, I don't know if you have any.
[00:24:42] So, actually, I thought it was in the ADEME study, but actually, there is another one done by ADEME and uh another firm that actually takes mineral by mineral. which explains the geopolitical issues and uh consequently the risks we have in terms, in fact, there's more of a flow management problem than a stock management problem. There you go. They explain, in fact, the slowdowns that might occur per, per mineral.
[00:25:20] So we're going to continue with the BoAmps initiative, so that's what we wanted to emphasize today. It's an initiative that we created at Boavista. Uh, in fact, it comes from the observation that all these existing tools are limited, so they have quite specific use cases, some are only linked to inference and not to training. For specific models that text generation for example, or that take into account only one type of infrastructure. And so since these models are based on data, we tell ourselves that if we have more data, we can improve all these tools, and so BoAmps is a standard format for sharing the measurement of AI system energy consumption. So concretely, uh, it's a model that allows you to describe electricity consumption associated with an AI task, also describing the characteristics of the mobilized servers. So it can be nourished by different measurement tools that I presented earlier, and it can also allow standardizing open databases that already exist, such as the AI Energy Score database.
[00:26:23] which for once compares different models but for a single infrastructure. So if we were to transfer this data to a standard model, we could compare it with other data sources. And concretely, having this open data base on the energy consumption of AI systems would allow us to have very concrete and quantified good practices at the level of data, models and infrastructures. Uh that would allow to improve existing tools that are based on data, like EcoLogit and Boavista API for example. And that could surely lead to new frugal AI initiatives.
[00:26:59] Yes.
[00:27:01] This is in relation to the description of the model, which describes the production of the model. Yes. What is the What does the model take into account? Does it take into account the electricity mix? Exactly.
[00:27:22] Uh we have this part in the model that allows to describe exactly the environment and the type of energy used and also to directly provide the electrical mix. But it's a part that is non-mandatory. Because the goal for us is really to compare efficiency, or rather to optimize both at the data model and infra level, and so this part, uh, well, of energy source, if we were doing carbon reporting directly, as we've seen, there are different methods that allow calculating carbon, and so we wouldn't be able to compare directly, whereas electricity consumption, it doesn't change depending on the location. But actually from electricity consumption, we can then calculate multi-criteria impacts, especially since we also describe the infrastructure and so, since we have the type of GPU, CPU used, etcetera, we can calculate multi-criteria impacts from the data that is inside.
[00:28:22] And so this project was launched by Boavista because there are real transparency issues in assessing environmental impact. And the more data we have on, well, the consumption of algorithms based on the technical capabilities they rely on, the more we can then feed the know-how and propose measurement and evaluation tools open to all, and also, well, make visible the service providers, we could say the GAFAM too, on the optimization work they do and on the environmental impact and bring a little another source of data and much more transparency.
[00:29:14] Uh concretely, the functioning of BoAmps is, we imagine that we have two organizations uh that have deployed AI workloads, that are able to measure it because they have set up the necessary tools, whether it's code carbon, PiJoule or any other tool. Uh, so with these measurement tools, they extract they are able to extract data uh that we are going to fill in the BoAmps format thanks to filling tools that we have created. So we have a tool in the form of a form on Hugging Face, a bash script and an Excel script.
[00:29:47] Uh, and then, uh, it transforms the data into a JSON that is in the standard BoAmps format. that we will come to publish on an open data repo where everyone can come to share data or consult all the data that has been deposited up to now. Uh this data model concretely, it is composed of different parts, uh some of which are mandatory and others are optional. Uh we have a more complex report schema JSON that is composed of sub-schemas. Uh, so for example, we're going to describe the task we're currently performing. This task can be composed of algorithms and datasets. So for example, I can describe that I am doing generative AI inference, where I used an LLM, LAMA 3, in a version of 8 billion parameters, that I fine-tuned it, that I sent a prompt of 30 tokens and that I received 200 as output. Then I will describe the measurement I made. So it can be a list of measurements because as we said, there are many tools that exist, and so we can even provide measurements made with different tools of the same workload. For example, if I measured both with a wattmeter and with Code Carbon, I can provide both measurements and that will allow us to have more data on the differences between these tools. And the last really important part is the infrastructure, so exactly describing what type of server I used, what type of component, the model, the quantity of GPU, etcetera. And so it's in the environment part also where we can optionally come and detail what type of energy we used and what electrical mix if we also want to directly give the consumption, finally the CO2 equivalent emissions associated. For the filling tools, uh we have this tool on Hugging Face which is therefore a form where we see the different parts that I presented before, so the header, the task, measurement, etcetera, and the different parts, some of which are mandatory, others are not. Uh, and so, concretely, where we are on the project, we are at the second version of this data model. We have already experimented with it once with the teams, members of the working group, and with students who worked on the subject.
[00:32:03] Uh so we updated it so that it is more exhaustive, easier to use, more complete. Uh, we had bash and Excel tools that we are currently updating to this V2. For the form on Hugging Face that I showed just before, it's already done. And uh the goal is therefore the open, the open data participatory repo is not yet completely created. Uh but to consolidate all this, it's now that we need concrete user feedback to either update this data model or improve the tools that already exist.
[00:32:36] So if you are able in your company or even on your personal time to deploy AI systems and you can set up measurement tools, well we are looking for feedback, uh both on the data model, on the fields, if there are things that are not clear or missing fields. Uh we need feedback and concrete use cases to be able to consolidate the model, because there are surely use cases that we hadn't thought of yet. Uh and uh don't hesitate to create issues directly on the repo if you see suggestions for improvement or encounter problems. And uh for now, on the Hugging Face repo, at the end, we have a button to download the data, and if we want to submit it to a public test repo. So you can directly submit your data there or uh by uh email to my email if uh it doesn't work or if you prefer to do it differently.
[00:33:32] And uh so the data model was made in a way that makes it very easy, actually, uh there are many fields that are not mandatory, the fields that are mandatory are not so difficult to obtain. Somewhere, data scientists know all these fields very well, they know very well what it is about. And uh so it's an action that doesn't take much time.
[00:33:54] uh which can be well valued within uh your structure because it is participating in a sustainable project. It can also be valued if your company is part of the Sustainable Coalition, which was set up after the AI Summit. Uh, that's it, it's uh, sometimes it's small actions. uh that potentially have a very high added value and also uh on the data scientist side, uh, data scientists like to optimize their work, they like to work well. So anyway, this optimization work, they do it. So it's also a way to value their work. So it's really a win-win for everyone.
[00:34:43] So there, we really put the emphasis on the evaluation and measurement part, which is a pillar, you know, of the work on the environmental impact of AI. But in reality, there is a systemic logic and good practices of frugality.
[00:35:05] So that's it. So it's always a bit of a joke when I say that I work on frugal AI. Everyone tells me, but frugal AI, does it work together? Especially when we look at the definition:
[00:35:19] who feeds on little and lives in a simple way. Already our digital services before, it's uh, that's it, it wasn't great, but now it's, as you understood, the architectures of AI are quite terrible in terms of environmental impact. So uh, the answer is to tell ourselves that in fact, a frugal AI is above all an AI that doesn't exist. There you go. You have to keep that in mind. Uh there are many, there are reference frameworks that have come out with a lot of good practices, notably the general reference framework for frugal AI. which was co-piloted by Afnor and the EcoLab of the Ministry of Ecological Transition. There are 150 experts, in fact, including companies you are part of, research labs that came together for 6 months to share their good practices and validate them in this document. which is now being studied at the European level, hoping that they will manage to create a standard that would be both good for the environment and also a factor of competitiveness, of differentiation. Uh, then there is all the work done by the Data for Good community where they made a booklet that is 360 which also takes into account ethical issues.
[00:36:56] There's the work of the Shift Project that has just come out with an analysis as they know how to do on where we are in our decarbonization trajectory and uh with what's happening right now with AI, well, what are the impacts actually on our trajectory.
[00:37:14] And how can we do with this ambition that was announced to build a lot of new data centers? Uh that raises very practical questions.
[00:37:29] of how are we going to connect to the power plant, how is what we call the grid going to work, that is to say what I understood, because I'm not an expert. But in fact, having a lot of electricity demands at the same time causes network saturation, so apparently we don't have a network that is still adapted. We have the example of Ireland, Sarah mentioned it earlier, where they saw that there was a real industrial challenge to build data centers, but they had to stop and implement a moratorium because they couldn't, they no longer had to supply the population with electricity. So we will also have to face this type of challenge. So the Shift Project, they are very good at doing the planning and presenting all this. Uh, then there's the INR, so it's the Institute of Responsible Digital, uh which publishes, uh, which co-piloted for example the implementation of the RGSN, the General Reference Framework for the Eco-design of Digital Services. I don't know if you know it.
[00:38:39] And uh so they made a list of good practices on AI issues, which also includes the technical part and the ethical part. And finally, there's a site called IA Frugal, which is updated by a journalist and which actually presents all the good practices, each time there are 31 resources that are decrypted and that allow access to the latest good practices and it's also good in terms of governance and technical. That's it to equip you.
[00:38:32] the general framework for the eco-design of digital services. I don't know if you know it. And uh so they made a list of good practices on AI issues, also including the tech part and the ethical part. And uh finally, there's a site called IA Frugal which uh which is updated by a journalist and which actually presents really all the good practices, each time there are 31 resources that are deciphered and which allow access to the latest good practices, both in terms of governance and technical aspects. That's for equipping you. So, I participated in the drafting of the IA Frugal spec, so I'm going to emphasize that a little bit because uh it really lays the groundwork for the IA Frugal approach. So, in the concepts, there are three main axes, three major levers of action to distinguish. The whole system part. So the system is really the tech, what we build, the service. When we work on uh the system, we're going to work on efficiency. That is to say, we are going to optimize the expected result with regard to resource consumption. And it's very interesting to make this distinction with frugality because when we work on frugality, we will work on the service. So we're going to ask ourselves the question of need, things you know by heart. Uh, of the need, of the use, is it, well, will my service really be adapted to uh the expected result, even for my business, for my users. There. So when we work on the efficiency of the system, we're going to go, as we said earlier, look at the entire life cycle. because when everyone started to take an interest in the environmental impacts of AI, everyone went to measure usage, which can be 30-40% of the impacts, after a while it's not enough. So really this notion of using the tool for and there are very good online tools that Sarah showed you, uh it really allows for evaluation based on your equipment etc. to at least have this life cycle approach in addition to the work you do on usage. So we're going to look at that over the entire life cycle and on the three segments, that is, the algorithm, the data because data collection and storage also have immense impacts. And so the hardware, well, the servers, uh the GPUs but also why not the screens, the smartphones, there, with rebound effects, there can be edge effects with the deployment of AI. with the deployment of AI. And uh so on the uh frugality part, uh we're going to do evaluation because to be able to make comparisons to be able to make choices, we're going to do an evaluation. And uh and also work on architectures uh to ask the question, is it really the AI I need or can I have another component that might be simpler. And uh and so there's also this idea of always looking for good practices and implementing good practices. We often hear that it's impossible to do ROI on AI, let alone on generative AI. Well, on the environmental impact, we can also have the same discourse which is, well, let's implement as many good practices as possible, in any case it will help. There. So, uh, these moments always go by very quickly. Do you have any questions? I still have a lot of slides but
[00:42:47] There. it might be good to iterate.
[00:42:51] I'm talking about Green Ops, Fin Ops, AI Ops for AI costs. What is the relationship between using a large model, not doing and of a model that you have to do like that. Is there a solution that is more rigid than
[00:43:07] No, I think the experts will tell you that there are differences but at this stage I think we can put them in the same basket.
[00:43:16] Hmm.
[00:43:31] So clearly, uh in the recommendations, uh we are about using specialized models, preferably. On controlled datasets. The more we use architectures of this type, the less environmental impact we will have.
[00:43:52] And we can clearly see it with the ecology tool where you can change the model and change the number of parameters of the model, and we see that for example, going from a 7 billion parameter model to models like Chat GPT with thousands of billions of parameters, it really has an enormous impact, and even going from 1 billion parameters to 7 billion, we see there's a real difference. So the size of the model really influences the consumption behind it. really influences the consumption behind it.
[00:44:18] It's not
[00:44:23] Now, I, I, I want to say, until the end, uh if you have any questions as we go, don't hesitate. There, we'll try to advance as much as possible, but well, I think there's a lot of information, I think don't hesitate to ask for clarifications or ask very concrete questions.
[00:44:42] So there, you have the life cycle of an AI service. So here, we are really, by the way, we are really on the algorithm. We don't look at the equipment or the data. Uh globally, there will be the entire project launch phase. So there, uh the good practices are to work on qualifying the need and the uses. After that, we have the whole design, development, verification, validation part if the model works well. So that's what we can call training. during this training period, you can clearly measure, evaluate, and especially integrate these criteria into governance.
[00:45:32] So I'm going to zoom in a little now on governance. Uh since you are experts in KPIs and so on. Uh so, uh integrating measurement, uh uh figures like that, for now it's very hard actually for users, considering all the constraints already on a project, to have this extra data and integrate it into arbitrations for decision making. it creates a lot of noise, it creates a lot of cognitive saturation. It's a topic we talked a lot about during the 2 days. So actually, uh I'm quite advocating for trying not to make environment another thing, I mean, social, working conditions, environment, it shouldn't be either one or the other, it should work together. So on uh the methods like that in terms of governance, with the CSRD, something has accelerated, it's the idea of really working on environmental accounting. So to rarely do measurement but to do it well uh and consequently to work on a budget and somehow to work uh on these environmental indicators at the same time as working on the budget. So I have a budget in euros. then right next to it, I have my budget for water consumption or for or for carbon, and in fact to build this budget, I'm going to collect the data in the same way. There. This is to tell you, it's a bit what, in terms of feedback, works best.
[00:47:22] Is there a difference between Fin Ops, Green Ops, AI Ops? For AI, we'll talk about AI Ops.
[00:47:32] Exactly, well, if so Finops, Green Ops, same fight. Frankly, typically I think as soon as what is also interesting, uh, is that as soon as we are looking for efficiency, optimization, uh very often, there are there are big convergences. The professions of architecture, the professions of development, the professions of financial optimization generally go in the same direction as the reduction of environmental impacts. And uh and so we're a bit in the jokes, I don't know if it's a joke, but among cloud providers, people who are very invested in working on the environmental part, they always say, when I have the project manager interlocutors and all that, they don't care at all when I present this part of the cloud's operation to them. But if I'm lucky enough to have the CFO, then he will really be interested and he will actually add the use of the environmental argument to be able to get his financial objectives passed.
[00:48:50] So there, you can see a little bit all the themes on which uh there are there are good practices that exist in the Frugal Spec. So, well, we find, uh, everything that involves working on governance, uh, there, the qualification of the need, of the use, the three main segments. optimize system performance, optimize data management, and then optimize equipment management. And finally, also uh advance on evaluation and on skills because uh well, all the good practices that exist are actually skills, skills that already exist at the core of the profession. or new skills that need to be integrated into the role, into the profession, so that it's not just another impossible-to-manage layer. So, where to start? Uh that, there, is an important point because we talked a lot about the impacts.
[00:49:57] There are the equipment, the usage, well, it would be necessary to be able to act a little everywhere. We can each act from where we are. So first, the first point that I often say is, since many of you, and that's great, have calculated your personal environmental impact. you've seen that there are many levers on which we can act, and somehow, I'm sure that in the room, people who want to act, no one has acted in the same way. So the first point is to tell yourself that somehow it's personal actually. And uh as soon as there are external norms that come and arrive in dissonance with our way of acting, it blocks the action. So already, the first point is to tell yourself, where I am, with my knowledge, my scope, where can I act?
[00:50:56] There. Next, in the good practices, so governance allowing to pilot frugal AI. So that's what I told you about earlier. Right now, we really see what works well is still having this holistic approach to environmental accounting. After that, well, there are all the good practices, and on that, project management, everything related to agility to adjust the need, uh that's, that's, that's very valuable. The whole part about the architecture, the more your architecture - and these are good enterprise architecture practices we've heard for 20 years - the more modular your architecture will be,
[00:51:37] uh and uh you'll be able to reuse your components. an algorithm that has already been trained, the more you use it, the more it is shared, etc., the more you will lower the unit cost of training.
[00:51:54] So, uh, behind that, there's also the idea of using open source algorithms, of sharing your algorithms; there are many good practices related to that. Uh so there, this architecture part is important. On the technical side, there, I put being in communities, sharing information in commons, but also sharing what you have developed. So algorithms, data. There are tons of datasets that are already open, of good quality, uh if you yourself clean a dataset and if you can, from a business perspective, share it, these are things that, on large volumes, can very quickly have a lot of impact. And finally, well, training, there, I've already talked about it.
[00:52:57] And then, well, this is to give you an idea uh of the good practices, the lists of good practices that are uh that are available in the documents we cited. Uh there you go, but basically, it's a good day of training, or even specialized sub-sections that allow to deal with all these good practices.
[00:53:21] So, we can perhaps finish on concrete good practices for the end-user side that can affect everyone a little. Uh so, there are several levers that you can address at your level. first, question the need, do I really need to create an image with AI for a slide I'll use once, and realize and also share the impacts of AI, because we're also not aware of all the impacts it generates when we send a prompt to ChatGPT. Uh and so prioritize lighter models as we said before, don't necessarily use the most generalist model just to ask a fairly basic question, just use a web search when necessary. Uh so it's really about rationalizing our uses, and concretely we can see that there are good practices that can have a big impact. For example, I don't know if you use GitHub Copilot, but by default, it has an auto-completion function that's activated when you use it, so every time you just type a letter. it's going to try to suggest lines of code to us, and so in fact it sends a lot of requests that are not directly solicited by the user. Uh and there's an article from a research by Tristan Coignon uh which tells us that on average, we have nine unsolicited requests per minute that are sent to the server behind GitHub Copilot to try to predict the code, and that among all these requests, only 10 to 15% provide answers that are accepted by the developer, and then 8% are kept in the final code. Uh so this function, we can actually remove it, put it back as we want with a keyboard control, and so it's an example of a good practice that can have very concrete cases and concrete reductions. Uh and it also showed that, uh, using Copilot with autocompletion was like using as much energy as a second laptop in parallel. So it's huge, and therefore it's important to only use it when we know we have repetitive tasks and it will really be beneficial, whereas other times it will just clutter up our screen because it's constantly writing things there and it's not useful.
[00:55:34] That's right. Thank you.
[00:55:37] Thank you very much.

Transcript (Translated)