Chloé Lemarié - Comment la méthodologie Shape Up permet de décliner la stratégie produit d'Evaneos ?

[00:00:03] Hello everyone. So when I arrived at Evanos in 2023, it was quite peculiar because there was no backlog, no sprint, no ticket. And yet the teams, well, they were moving forward, they were delivering, they seemed to be making decisions and having an impact. And in fact, I quickly understood that it was not the result of chance, but the result of a method that had already been in place there for some time, it's Shape Up. And that's what I'm going to talk to you about today. So, my name is Chloé, I've been in product for 8 years, I've been at Evanos for 2 years. So it's not me who implemented the Shape Up method, but I arrived when it was already running well, and I'm going to tell you a bit about what we've done and how we've iterated on it since then. So first of all, to give you a bit of context. Evanos was created in 2009, so it's been around for 15 years, it's a scale-up. Today we are 170 people. And the idea is to connect travelers, like you and me, to local travel agencies at destination, with the idea of co-constructing a trip together and that it be rather responsible tourism. What's important to keep in mind is that we don't have physical agencies. It's not like Voyageur du Monde or other players. Everything is done online. So in fact our product is digital and only online. And to give you a little idea of what our product is. So what we have is the traveler part that you can see, so on the internet a site and a mobile application. So the idea is to consult offers, itineraries, co-create your trip, get information during the trip and upon your return. And we have the whole phase on the other side for our local agencies, where in fact we've developed a travel suite. So a CRM for our agencies to manage their travel requests, a tool to create quotes, a quotation tool, payment, and so on. So that's a bit of today's picture, and to tell you how we implemented Shape Up, in fact, you have to go back five years. To something you all know, COVID. Well, when you're in the tourism industry, it hurts, because in fact all borders close little by little. We don't know when it's going to reopen, so trips are first postponed. Then cancelled. Finally, there's not much left to do. So there was a wave of voluntary departures, the company cut almost half of its staff, and suddenly that left time. Time to ask ourselves questions about what we want to do, how do we restart? So finally, it was a pretty propitious period of watch, well, I put product because I'm a PM, but also tech watch, to ask ourselves questions about our practices, what we want to do, how people do it elsewhere, how we could improve ourselves. And in the idea of saying to ourselves, we want to improve two things. First, how do we focus on the right topics and how do we have an impact? Because in fact, at that time, Evanos was a bit of a feature factory, there were features everywhere, they were coming out, but we probably focused more on what was coming out, the how, than on the why. And also, a somewhat related question, the idea of how we align ourselves. How do we ensure that teams can work, that they are not constantly dispersing themselves or diluting the effort? So in fact that's a bit what nourished the discussions, and in the middle of all that, there's a little book that emerged. This book, then, is Shape Up. So it's a methodology that was developed by Basecamp in 2019. So Basecamp is a company based in Chicago that publishes software for project management. Ehm. And so to explain a little, so this is a bit the big slide that summarizes everything, but then we'll go into the detail of everything. First important notion, these are 6-week cycles. So it's longer than a classic sprint, but it's not too long either. And so in fact we have six cycles per year, so it goes pretty fast, three per semester, and between each cycle we have cool-downs. So cool-downs are what you see on the screen, they are periods of 2 weeks between each cycle. And then there are some important notions to keep in mind: notions of shape and build. We'll come back to that later a bit better, but basically, shape, to schematize, is what is framed, doing its discovery. The build, we build. So we iterate, these two things we do in parallel. That's why we talk about dual track. And in the middle, we bet. So it's the betting phase, we bet on the topics. Before going into a bit of detail of what all that means, there are a few key points to keep in mind. It's that the teams are autonomous on what they shape, on what they build, on what they want to do and on what they propose. And in fact, it's really the betting phase where we collectively validate on the company and we bet on our topics. There is no backlog, we will come back to it later, but really it's not at all the way things are done, and we are not supposed to be interrupted for 6 weeks. So in fact we deliver what we committed to on the shape and the build, there are no bugs, support tasks or anything else. Ehm, so to tell you a bit in what context we implemented, I'm going to talk to you about our product organization at Evanos. So we are organized in impact team, we have seven in total, and in fact each time we have an objective to achieve. So we have a KPI to move and a North Star. For example, my goal is to improve the sales rate of local agencies. So I manage my perimeter with regard to this KPI. And then what's interesting also is that we've enriched a bit the way teams are composed. So you probably know this, the product trio, we often say that you need a tech profile, so the tech lead or the engineering manager, a product profile, PO PM, a design profile, and a team of developers.
[00:05:10] We have this base, this core, but in fact we have enriched it and we speak internally of Product Orchestra. So in fact we add two profiles, a data profile, a data analyst. who is not 100% dedicated to the Impact team, but who allows us to really dig into the data, analyze it, analyze the experimentation and actually enrich ourselves with all that. And another profile in the orchestra is a biz profile, which we call the Biz co-lead, and in fact, it's the one who has interests directly related to ours. So typically, I told you, my goal is to improve the sales rate of local agencies, I work with the person who coordinates the entire network of local agencies, because in fact we have the same objective at the end. We have developers, of course, and we also have business experts. So these are not full-time people in the team, but they are still people who dedicate a part of their time to us to participate in workshops, in brainstorming, to look at mock-ups, to validate with us, and really who follow the team over time and who infuse us with their, really, business expertise so that we make sure that everything goes in the right direction.
[00:06:14] So that's for our product organization. And now we're going to go into the detail of these big notions of Shape Up, so we're going to start with the first one, the shaping.
[00:06:24] So in theory, the shape is actually a really good idea to clear the ground. We have a topic, we want to explore it, so it's a framework or a discovery, and they insist a lot on the fact that you have to find the right level of abstraction.
[00:06:35] Which means that at the beginning the topic seems a bit obscure, it's very abstract, it's shaped, but the outcome of a shape is not a spec. In fact, it must remain a rather macro level.
[00:06:45] So the idea is to see a bit the problems, the gaps, the sticking points, but you shouldn't arrive at something too precise. In our case, concretely, we base ourselves on that, but we decided to enrich the shape side a bit to find our way around as well. So in fact we have different types of shapes that we associate with types of discovery, so we have three. The first is everything that is exploratory shape. So here the idea is really to identify new value-added opportunities. So it's in this phase that we do a lot of user interviews, we dig into the data, we analyze the product, the idea is really to map all our opportunities. And the deliverable at the end of an exploratory shape for us is an opportunity tree, where we really have all that mapped out with a prioritization order. We also have shapes that are a bit more conducive. So in fact on our opportunity tree, we're going to decide to focus on a precise opportunity, and we want to dig into that one and experiment and better understand it. So it's in a conductive shape, for example, that we do user tests on the mock-ups, that we really dig, that sometimes we do small POCs in production, to really experiment. And we have a last type of shape, which is more technical there, so the PM intervenes much less. But the idea is more to explore a technical solution, to prepare the technical aspects of a project. And to evaluate all that. Concretely, I think it's really powerful for the PM because in fact often, when we're between discovery and delivery, delivery is time-consuming, it takes a lot of time, it's operational, and it's hard to dedicate time to this whole discovery part. Here, the fact that we really have time that is instituted in it and that it is quite well structured, well it's really powerful for the PM, it allows us to really be in control of our perimeter, to have time to do benchmarking, to study, to understand and to have a bit of our product strategy for our team. So we have a second quite interesting notion, it's the build phase. So, build in theory is really the phase where we put a solution into production. So in fact it's our delivery phase. And in theory, they insist a bit, again I put a diagram for you, on the fact that when we start a build, we don't necessarily know exactly how it will be or what it will look like. That's what I was telling you just before, at the end of the shape it's still a bit, it's not abstract but it's not too concrete either, but as we build, we learn to see what we're doing and to deliver little by little.
[00:09:00] Similarly at Evanos, we took this notion, but we have two types of build. So we have a build when it's a new functionality. So generally it comes from a shape, we had an opportunity, we tested, we think it will have an impact, so here we industrialize it. So we industrialize it well with really the whole test chain and so on, and we have another type of build that is not dedicated to adding a new functionality. But rather to do refactoring. So to rework a bit the code of existing functionalities. To also maintain excellent quality. And what is important to know for us anyway, is that the technical teams are extremely autonomous in delivering the solution. I intervene very little. In fact, the PM at Evanos doesn't cut up and doesn't write PRDs. In fact, it's really on the shape part that I transmit that. We talk about it a lot together, of course, but it's the engineering manager who actually does the cutting. and who really manages the tech team, between back and front, and they are extremely autonomous on that.
[00:09:58] Ehm. So now we've talked about our cycles with shapes and builds. We're now going to talk about the inter-cycle period, so that's the cool-down.
[00:10:06] The cool-down, in theory, is a period between cycles that has no programmed work. So we take advantage of this period to fix bugs. So for example, you see here our Slack channel where we have our support bugs, we prioritize and fix them. And the idea also is to explore new ideas and it's time to prepare the next cycle and go into a betting table. Which I will explain to you just after. Concretely, we take this framework, but we tried to really structure our two weeks of cool-down to make sure we have time to do everything well. So the first week, we really finalize the cycle, and that's where we set our team rituals. So we go into a betting table, we also do a team retrospective, which only happens every 8 weeks, so it's much less often than in Scrum or an agile model. classic, and we do product demos to the whole company to also show what we delivered at the end of our cycle. And we fix the bugs, because there are still a few. And the second week, we try to spend time by expertise. So basically, the front-end developers are together, the back-end developers together. PMs together, designers together, and the idea is to harmonize practices a bit. To no longer just be on your impact team doing things, but to work collectively together, to contribute to cross-functional or related projects. The idea behind this is also to do a bit of watch, to take a bit of a step back, not to always have your nose to the grindstone. And there is still a bit of a desire to say, between the cycle and the cool-down, well, it's not a cool-down for nothing, so you also have to recharge the batteries a bit and be ready for the next cycle. In fact, I find that it goes extremely fast, it's really two weeks that we don't see pass. Finalize the cycle, yes, no problem. Take a little time by expertise, yes, do some monitoring, honestly, it's quite hard. In fact, it still passes really fast. And why does it go fast? Well, first of all, as I put it here, there's the betting table during the cool-down. And so we're going to talk about it, but it's still a ritual that gets prepared quite a bit. So in theory, the idea is that we arrive at a betting table, so it's a ritual, we have pitches. So in fact the team comes to pitch the topics they want to work on for the next cycle.
[00:12:09] And so the idea is to convince those at the betting table to actually bet and invest for those 6 weeks. Normally, in theory, we only bet on the builds. So on what the team is going to build and industrialize. And it is autonomous on the shapes. So on what it wants to frame or explore. And that's what I told you at the beginning, there's the idea that there's no backlog. So in fact the important ideas will come back. Which means that we can present a number of pitches to a betting table. They are validated, sometimes there is a bit of rework behind, some are retouched, we do it. And in fact at the next betting table, we re-propose pitches. So sometimes it's the logical continuation of those who were there, so a shape that can turn into a build. Sometimes it's retakes that come back, sometimes it's new ones because new ideas have emerged. So, concretely, the betting table, well, we decided to bet on the builds and on the shapes. Because in fact we told ourselves that it was also important to frame what we spend time exploring, so that it is done efficiently and there are still opportunities that can be more or less large. So in fact we bet on both, and that's what you see on this little diagram, so we write pitches each time. So a pitch for a discovery that is technical, conductive or exploratory, a pitch likewise for a new feature or for refactoring. For once, we iterated a lot on that. So now it's a 1-hour ritual per team every 8 weeks. So since it's at each cool-down, so with the product orchestra that I presented to you at the beginning. And then three roles that are in the steering committee, so CPO, CTO and a steering committee sponsor. So who depends on the team, where it is. And for once, it's very effective because we still have an hour to review the. to do a quick recap on the previous topics, the impact we had, what we want to do in the next cycle. And so everything is templated, we use Notion. And in fact we really use the same templates from one impact team to another to write our pitches a bit in the same way, so to be sure to talk about the problem to solve. of the solutions we want, of the KPIs, well, really of everything, and it also saves us time because we fill it out and it removes a bit of mental load. Knowing that in fact this preparation work for the betting table, it is still very much the responsibility of the PM. It's collective for the whole team, but he's really the one who carries this ritual. And in fact it's a ritual that is prepared. Really, because it's an hour of exchange, so you have to be sure to talk about the right topics, at the right level of granularity. and to anticipate the potential challenges of the management committee people we see very little and who don't necessarily always have the same problems as us in mind.
[00:14:38] Ehm. So that's for the betting table, and there's a notion that's quite important behind that, it's the appetite. Appetite, to explain it to you, in fact it's really the time that a team is willing to spend on a specific problem. So in fact the idea is not to say how long it takes to build a functionality, but rather how much time we want to spend on it. In fact, how much time do we want to give ourselves to it? And so the adjustment variable will be the perimeter. Because obviously if there's less time, well, you have to adjust. And so we talk in terms of weeks. So a cycle is 6 weeks, and in fact I arrive with my pitches saying I would like to spend 2 weeks of appetite exploring such a topic, 3 weeks building such a thing, and so on. So it's not a notion that is always very simple to understand, especially on the developer side. Because obviously you still have to make a bit of a ratio between the time you want to spend on it, but concretely, if we dedicate a week to it, can we get something out of it? There is still a balance to be struck between the two. And in fact what we really try to do is to link this idea of appetite to the impact, the impact we want to have ultimately on the product. And in fact often, roughly, to help us a little bit to materialize that, it's actually asking ourselves, as a team, if we have 6 weeks, let's say 6 weeks represents about 100,000 euros of cost, how do we want to invest them? In fact, what do we want to build, where do we want to go, how do we want to do it, to try to rationalize the effort and to oblige ourselves to think about ROI, what return on investment compared to the time we spend on these things.
[00:16:07] Ehm. So there, normally you have pretty much all the concepts of Shape Up. So if I recap, you have cycles with shapes, builds, cool-down phases during which we bet during the betting table on topics. And now I'm going to tell you a bit about what we learn from it at Evanos. Knowing that it's the fourth year. So we've had time to implement it, to iterate from one year to the next, to adjust and and we continue with it, so it's going pretty well. So our successes for that, well, first of all, we think we've gained a lot in impact and clarity. That's what I was telling you at the beginning, at the time of COVID, when there was a whole product watch, there was a bit of a feeling of dispersion, we were releasing lots of things, we didn't really know why, we were interrupted, we weren't asking the right questions, now it really makes a lot of sense. Each team has an objective and is autonomous to work on its shapes and its builds, and in fact behind it has had concrete results with metrics that have moved, so the engagement rate on the site, the conversion rates and so on. And that allowed us to discover real opportunities on which we capitalized, experimented, and behind it really worked. Another thing that is quite interesting is that we collaborate a lot with the business teams. In fact, the product is not separate in the company, it's a bit in the middle of everything because finally, since we all have business experts and a biz co-lead, in fact there are a lot of people who take part in the daily life of the impact teams.
[00:17:35] And who therefore interact almost daily with the developers, designers and PMs. So it's a pretty rich collaboration that ultimately makes everyone follow the product developments.
[00:17:45] whether it's on the traveler side or on the local agency side. And we really all move forward together. And another success is that on the PM side, it gives a lot of autonomy. That's a bit what I was telling you, so we are less on the delivery because there is really a great autonomy of the tech teams to really be owner of the solution, to cut and to work for. And so the PM focuses much more on the discovery aspect. So how he anticipates, in fact, how he tries to have a head start for the next cycle, what he wants to explore, what he wants to experiment and how he wants to do it. And so I think that's pretty powerful for once.
[00:18:23] Of course, not everything is rosy. So necessarily there are things that we had to adjust and on which we are still iterating, in fact, it's not necessarily perfect. So the first thing to keep in mind is that it's very much driven by the impact, what impact we want to have behind, what business impact. And in fact, as you saw in the cycles, there is not necessarily time dedicated to technical health or technical debt.
[00:18:45] In fact, during the cycles, we move forward, we iterate, during the cool-downs, we fix, but then we fix what's wrong, and there's not necessarily that time dedicated to improving the product.
[00:18:53] So it's necessarily one of the alerts that the tech teams can raise. And in fact, what we have introduced for a few months already.
[00:19:02] It's to say to ourselves, in fact we are going to allocate time of the cycle, so not only time of cool-down, but to the tech. And in fact, for once, on that, they are autonomous, they have about 15% of the cycle time. And it's them who organize themselves, who prioritize, and who decide in fact what they want to do to actually improve the product and maintain it. And another thing we've introduced is a Captain Interrupt role. So roughly the idea is to say, normally we are not interrupted for 6 weeks, we don't fix bugs. After that's the theory, of course sometimes in reality there are incidents.
[00:19:35] There are major regressions, so in fact we still need to be interrupted. So the idea is a bit to say by team, who is the person who will be interrupted, so who is the point of contact that will qualify, is it really worth interrupting or not? And if yes, who will do it? So if that person is enough, they will do it alone. So there is one Captain Interrupt per team, so in fact there are seven, that means all the time.
[00:19:57] Otherwise, they will send other people to work together on this. who actually decide what they want to do to improve and maintain the product. And another thing we've implemented is a Captain Interrupt role. So basically, the idea is that normally, we're not interrupted for 6 weeks, we don't fix bugs. Now that's the theory, of course, sometimes in reality there are incidents, there are major regressions. So in fact, we still need to be interrupted. So the idea is to say, by team, who is the person who will be interrupted, so who is the point of contact who will qualify whether it's really worth interrupting or not, and if so, who will do it. So if that person is enough, they'll do it alone. So there's one Captain Interrupt per team, which means there are seven all the time. Otherwise, they'll send other people to work on it together. Uh, another small point that we also had to adjust a bit, which is quite related to the first one, is what I was telling you, Shape up forces you to prioritize by impact. So it's not necessarily easy to include priorities that are not business-related and don't necessarily have an impact. So there are tech priorities, but there are also design priorities. Uh, typically if the UX is a bit aging, that user journeys could be optimized, it's not a subject in itself that we can necessarily pitch and get approved in a betting table. In fact, we'll get there if we integrate it into a build or a refactor, if we can prove that it makes us lose travelers, understanding of value, but otherwise these are not naturally subjects that will end up, let's say, at the top of the pile. So it's not always easy to prioritize either. And you also have to find the right balance between shape, build, between each cycle. In fact, these are 6-week cycles, it's quite rhythmic, because we have shapes naturally if it's validated, it goes into build and so on, but sometimes the timing may not be right. And so, there's an R&D mode. For once, that's in Shape up theory. And we exploit it a bit, the idea is to say sometimes we have a subject, there's a lot of uncertainty and we don't necessarily want to go through an exploratory shape cycle, a conductive shape, uh, and build. And in fact, what we do is we go into R&D mode and the idea is to de-risk. So it can be de-risking on the functional product side or de-risking technically, and often, in fact, after that, what we have is a PoC, which is disposable, normally we won't iterate and build on it, but it's just to show that, okay, we're capable of having a solution for it. We are able to find iterance, there is usage. In fact, it's really about proving to ourselves that it can work before going further and then industrializing it even more. And the last thing, which is also a bit of a friction area, is all about prioritization and backlog. As I said at the beginning, there's no backlog in Shape up. However, we spend our time collecting insights, whether by doing user interviews, analyzing data, encountering small bugs, small frictions. So in fact, there are still a lot of things to manage on your product. And in fact, you need a certain discipline to store your ideas, given that there's no backlog. So for that, there are several tools and all Evaneos teams don't necessarily do the same thing. Uh, some use opportunity trees a lot. So even if we know we're focusing on a certain part of our tree, we still note all the other opportunities we have over time. Uh, similarly, we have tools to store user feedback, typically Dovetail, which we try to keep super clean, so we know that if one day we change the subject or have the time, we can find all the user pains, all the friction points, to iterate on them. And in fact, we have a challenge to try to keep all this feedback material, which is super rich, super dense, but which is not actionable for the next cycle, in fact, or which will not be worked on in the coming months. Uh, so that's a bit about the adjustment areas. In fact, if I had to summarize the strengths and limits of Shape up a bit. Well, as I was telling you, prioritization is very business-oriented. So depending on what you want to do with it, it's very useful because it really forces you to... well, to question what we want to do. There are always a thousand things to do on a product, things to correct, to improve. Having an objective and telling yourself "I prioritize only that", well, it helps to do it. But as I told you, sometimes perhaps a bit to the detriment of tech or design. And after the 6-week cycles, some people find it too long, others find it too short. So it depends a bit on where you are in terms of your product's maturity. Typically, we have impact teams, but in fact, we have a team that moved out of this Shape up mode because they had to switch to PoC mode. And we wanted to test things with AI. And in fact, we said that in a 6-week cycle, it's going to be too long, the feedback loop is super long. We want something much faster, much more iterative. After a week, we're able to know, okay, we continue, we reinvest, no, we stop. So they are no longer in Shape up and it's assumed because in fact, we told ourselves that the model doesn't correspond to what this team wants to do at Evaneos.
[00:23:56] Uh, and so if you also want to get started with the Shape up method, here's a slide that lists the prerequisites. Uh, first thing, I think it's having a stable product because that's what we were saying at the beginning, for 6 weeks, uh, well, we're supposed to be focused on our subjects and not get scattered, not fix bugs or provide support. So that means a certain technical stability. Uh, which we really had, for once, there was a very good technical stack at Evaneos, and that's why we allowed ourselves to switch to Shape up. We saw at one point that perhaps it was less well maintained, that it was going to degrade a bit. That's why we decided to allocate 15% of the cycle time to that. But it's a prerequisite, if you don't have it today, I'd say it's perhaps a bit shaky to start with Shape up. Uh, the second point is leadership trust because what we've seen is that teams have a lot of autonomy. So indeed, a betting table, they validate what we're going to spend time on in terms of shape and build, but in fact, it's 1 hour every 8 weeks and we don't see each other that much outside of those 8 weeks, in fact, we see each other once every 8 weeks. So it requires preparation, trust and commitment behind it. So uh, well, for it to work, the management must necessarily be on board and a rather mature product organization. Uh, so that's what I was saying, since the 6-week cycles can be a bit long, it's not necessarily ideal if you have to launch a product that doesn't have market fit. Uh, but also a rather mature organization, I include the profiles behind it. Uh, typically our tech teams are all quite senior tech teams, in fact, uh, who are very used to self-organizing. Notably the engineering manager who actually takes on a bit of a PO role because he's the one who follows the delivery, who breaks it down, who makes sure that everything is done properly. Well, that means he's someone with experience and who didn't start dev that long ago. Similarly, on the PM side, there's a lot of autonomy, a lot of discovery. Uh, it's very KPI, ROI, business-oriented. So that means you have to be a bit used to all that and not necessarily be mentored, well, you have to be autonomous on your own, move forward, that's it. So a bit, I'd say, these three main axes if you want to try Shape up at home. Uh, and so in conclusion, well, for us, it's been 4 years that it's been at Evaneos. For once, it's been a real catalyst. It allowed us to correct, well, a lot of things, to move forward. We continue on it. As I show you each time, we have iterated a bit, there are teams that are leaving it. And so, if I had one point to make, it's that there's no perfect organization. I told you about it because I live it, it doesn't mean you have to implement it at all costs. And for once, a product organization, well, it has to evolve with its product, with the teams, so its seniority, whether it moves, whether it doesn't move, and it shouldn't be stuck in time. Uh, so if I had to do this talk again in 6 months or a year, I don't think I'd say exactly the same thing. Uh, and that's it, the journey continues. Thank you.
[00:26:54] And if you have any questions or comments in exchange, I'm listening.
[00:27:11] Uh, how many people in the organization in total?
[00:27:13] In total, we are 170, uh, 7 PMs, and so there must be about thirty developers. There are 4 developers per team.
[00:27:25] Yes, thanks for your talk. And I have a question about the 8-week discipline, are there other mechanisms where we inevitably fall into decision-making moments that are during holiday periods? So your betting table falls between Christmas and New Year's Day. Do we adapt, do we not adapt? How has that gone over these 4 years?
[00:27:41] Yes, that's a good question. Uh, yes, we adapt a bit. Typically in the summer, we don't do a 2-week cooldown, we do a summer cooldown that will last July-August. And actually, it's not bad, it allows us to correct a lot of things, to see the transversal experience, because in July-August, the teams are too fragmented, and similarly, we make sure that it never falls between Christmas and New Year's Day, the betting table. And then, there's a thing that normally the PM, since the dates are fixed, is not supposed to be on vacation during the betting tables because it's still an important issue. I confess, I missed one, so I had really briefed the team well and worked a lot upstream, but it's still a ritual where it's better to be there.
[00:28:20] Thank you, that was very interesting. In fact, I have two questions about the bets, the betting table. How are these bets decided? Is it really you, the PM, who decides or proposes and then it's refined? Or that's the first question. The second is continuous improvement, but really product-focused, because here we were talking a lot about tech and technical aspects, but how does it work if, for example, you have a scope for your project during the shape and the build, but then if it's finished with that scope, how do you decide to improve it if it goes beyond that? Does it come later? Is it a new proposal? That's it.
[00:28:55] Okay. Uh, for the first question, so yes, finally, the PM is really supposed to be the one responsible for the subject to propose in the betting table. But in fact, he doesn't do it alone, he really builds it with his orchestra. So the co-lead Biz, for example, has a lot of information to give, often a bit strategic, about what he wants to do. After that, of course, there's the engineering manager. So we co-construct, let's say, five of us. After what I didn't say much is that when I say it's being prepared, the challenge is that maybe all the members of the steering committee don't discover the betting table on the same day. So there's also a bit of me telling you a bit beforehand, what your challenges are, I may have pre-validated with someone, well, a bit of preparing the ground, let's say, so that it goes well. Uh, so that's it, and your second question was. Yeah. Well, we don't really do that much, but I think it's a bit of a bias to say, we're going to improve this metric. So we go for it. Once we decide it's satisfactory, yes, we could do more, we could do better, we could improve the product. But it's really a trade-off: is this where we have the most impact? Is it not on the thing next to it? And so we assume that, too bad, the solution may not be perfect. It's only at 60, 80% let's say of its potential, but there's another bigger issue next to it at the key stage. And sometimes we don't know, and I arrive at the betting table saying, we can continue on this, I think it brings us this, there's a design conviction that we should continue to iterate on it, we can also move on to something else. In fact, it's up to you to bet, it's your company. So in the end, tell us what you want us to spend more time on. And sometimes there are also exchanges between them because the CTO doesn't necessarily agree with the CPO and so on, but we leave the meeting all aligned on what we're doing.
[00:30:27] Hello. Can you hear me?
[00:30:29] Yes. Oh yeah.
[00:30:31] Uh, thank you for the presentation, very interesting. I have two questions concerning the role of the product owner. Uh, I didn't really understand how you operate in your organization. And the second question concerns user support or production issues and how you manage them, do you have a lot or not?
[00:30:53] Uh, so, there are no POs, only PMs, so only product managers and no product owners. Uh, it's a bit what I was saying, the product manager uh manages a bit the team's strategy where he goes, he doesn't do as much delivery as that, the tech teams are very autonomous on it. And yes, we have a support team, uh, so they respond to customers, phone, messages on Freshdesk, who give us our support tickets. And for once, the big news is that either it's blocking and we say it's a business risk and we correct it, or we say, well, it brings a little friction but there's a workaround and, so to speak, it's not serious and we don't correct it right away. So there's a bit of that, it's accepted, you know.
[00:31:32] Yeah, over here.
[00:31:35] Alright, uh yeah, thanks for the talk, very interesting. Uh, just I didn't quite understand how many teams there are. That's a first point, and then uh, have you noticed things that have become somewhat standardized in terms of how things work uh during these 6 weeks, either in shaping or in building, uh between the different teams or between the different PMs? Are there any new practices that have emerged?
[00:31:57] So we have seven teams, therefore, seven PMs and so on. Uh, we're quite autonomous in how we want to organize ourselves. So there are PMs who follow delivery more than others, we don't necessarily have the same tools. After that, we still meet for cooldowns to harmonize practices a bit. But we still have a lot of autonomy on that. So we do it because it feeds us and we want to, but in no case is it an obligation to have the same methodology during the cycle.
[00:32:26] Uh, thank you very much already for the talk. Uh, I had a question more about the product orchestra you were talking about, because I know the product trio well. So actually, how does it work more on a daily basis with this product orchestra? Are there events between the five of them? So I understand uh, how it works in the field.
[00:32:43] Yeah, we do a weekly orchestra of five, and after that I do a lot of one-on-ones. Basically, I have a one-on-one per week with the engineering manager, the designer, yeah, data and the others. So it's still me who is the point, let's say, between these five, but we meet together once a week in a weekly and once a week with the whole team. So tech, business expert, orchestra and everything.
[00:33:07] Hello. Thank you for the presentation. I have a small question about the cycle which is 6 weeks, 8 weeks in total and on the learning part. Because we are very much into iteration, learning fast and uh, and therefore uh adding iterations if necessary or stopping. And I'm wondering, well, how do you do it because I have the impression that it's quite long and that learning can therefore be perhaps over several cycles and uh and have a loss of learning or speed of learning.
[00:33:32] Yeah, well, sometimes when we're pretty convinced about a topic, we say we'll do a week of shape and in fact we're almost convinced it's going to be good, so we say we prepare time to follow up with 2 weeks of build, for example. Otherwise it's a question of cycle arrangement, but indeed, you still have to be quite strong, it's what I was saying about ideas coming back, about how we list all these insights, how we use them at the right time to bring them out again and not forget them. So you need a certain discipline nonetheless.
[00:33:58] Hello, thank you very much. Uh, I had two questions. The first is uh you were saying that a lot of maturity was needed, both on the product side and on the tech side. And uh you're not small, you're quite numerous. Is there, in your opinion, a scaling limit beyond which it no longer works? And the second question is, is it compatible with uh telecommuting? Do you make sure to all be on-site during the bets and so on?
[00:34:27] Well, that's a good question because after COVID, like everyone else, Evaneos went remote and decided not to necessarily come back to the office. So we either have a hybrid contract where we do what we want, we come, or a full remote contract. Developers, not surprisingly, are largely full remote, almost not all, but in reality, 30% of the company is full remote and it's almost exclusively tech teams. Uh, for example, all my developers are remote. After that, we have rituals, we meet on Gather, which is an online platform, a kind of virtual office where we talk every day, there are rituals, but it works very well and in fact, we have to come back 2 days on-site during the Evaneos Days, where the whole company collectively is there, has rituals all together and the rest, let's say, of the month, it's a bit self-organization. And your first question, sorry, was
[00:35:15] Can it scale?
[00:35:16] Uh well, during a cycle, we're really very focused on our team, its problems. It's a bit hard to see what others are doing and to be aware. So the cooldown is for that and after that I would say our leads are for that. We have a B2B PM lead, a B2C PM lead who also do a bit of this synchronization to have a uniform product experience and know who does what. So I think it could scale, but it would require even more rigor perhaps on these inter-team rituals. Today, we don't need to have that many in fact.
[00:35:45] Hello, thank you for the presentation. I have a question, I'm not sure I understood the PM's role during the 6-week cycle for the tech part, the build. Uh, let me explain, so, uh, the 6 weeks, it can create a tunnel effect for me. Especially if the PM is very focused on their PM role, marketing PM. And so, is it the weekly one-on-one with the EM who with the IM who mitigates this risk or do you see each other more frequently? Because otherwise, uh, there you go.
[00:36:05] Oh yes. Yes, yes, we see each other every week. And actually, 6 weeks, but it's not a tunnel in the sense that over 6 weeks, we often have two or three subjects. So ultimately, it can come back to sprints. And yet, that doesn't mean there's a delivery after 6 weeks. In fact, we're almost in continuous delivery, if it's ready, it's shipped, I check, I correct, I give feedback, and so on as we go. It's just that it's less my responsibility, in fact, it's typically not me who's supposed to test, I always do it because I want to see what it looks like and from time to time I see things. But in fact, the devs test themselves and often, when they're ready to dev, they've already done this phase, for example, of acceptance testing. And similarly, they ask me questions if they have questions about the feature, what wasn't clear. But otherwise they self-organize, let's say, if it's clear how to break it down and do it. But we have weeklys and we talk every week and then of course.
[00:37:06] Yeah, it was almost the same question and I wanted to delve deeper. Uh, because precisely regarding the 6 weeks, uh in terms of cycle and capacity for evolution, because you were talking, you give a direction, you potentially give KPIs. And so potentially how many iterations can you have on a purely business KPI, uh that you would like to have and what is the team of, well, what is the number of cycles that the team can try to have.
[00:37:27] Yeah. In reality, that answer will depend on the ROI we want to have. But for me, my KPI is to improve the sales rate. In reality, it's endless, I can improve it in many ways, I've spent a year on it, I'm starting a second year. After that, I had more precise proxy KPIs, typically to improve the attractiveness of a traveler quote. And for that, I spent 4-5 cycles and sometimes, precisely, we had a very promising subject of offering alternatives to travelers. We did a shaping cycle, we did an experimentation cycle, and I think in the next cycle, I will propose another week of appetite to improve the functionality because in fact, it is very powerful, it works well and I have identified points where if we correct them, well, it will work even better. But on the other hand, if we see that it doesn't work, that there's no interest, no usage, we don't overinvest in it.
[00:38:17] Hello, thank you for your presentation, it's very interesting. Uh, I was wondering about, it somewhat relates to the idea of scaling, but how do you ensure that there is product consistency at the company level between the different, you have seven teams, if I understood correctly, 170 people in total. Uh, how do we make sure during the cooldowns that you take the time to try to ensure uh
[00:38:37] Well, typically on the design side, they have a very unified design system, so normally, we could say there's a certain style, an Evaneos identity that is found everywhere. Designers do design reviews each time there are new mock-ups to ensure this consistency. On the tech side, at each cooldown, and after that we're supposed to talk a bit. But for once, it's true, typically, we have the Evaneos personal space where several teams interact because you have your personal space before the trip, during, after. And at one point, we all went there with our feature. We all added something on the same screen because it served my business objective, for some it served their objective. The page was ugly, it looked like nothing, and we took the last summer cooldown of this summer saying, we consider this personal space as a product in itself, what functionality do we want compared to the trip before, after. And there, typically, it's an example of, well, it cohabited badly if we were each on our own perimeter. So we had to do some work together and we try to identify when these issues arise to work on them in a cooldown or in a summer cooldown.
[00:39:34] Uh, you said that it was quite good for an already mature product because managing bugs introduces too many disruptions in the cycles. Uh, but it still seems more suited for developing a new product than for a very mature product where there wouldn't be many more features to add. I don't know, what's your point of view on that? On its place in a product's life cycle?
[00:39:54] Hmm. Well, I think at the very beginning, when there's nothing, when we want to PoC, it's a bit early. Because 6 weeks is actually long, well, you need to test quickly and get faster feedback. Uh, after that, if you're very mature, I think it depends on the state of the product and the technical stack. But yeah. Sorry, I don't know more.
[00:40:13] Uh thank you very much for your presentation, it was very interesting. I have a question, I wanted to come back to the organization during the shape and build phases. Uh my question was, is it that during the shape phases, only the orchestra shapes while the developers are always in build, or do the developers also participate in the shape and then the whole team builds? How do you share the two cycles?
[00:40:36] Yeah. So the idea is that it's more shape orchestra, let's say, and build developers. But sometimes we need technical resources on the shape. Or there are many developers who also want to contribute to this reflection in the workshops, and we try to anticipate it at the betting table. So in fact, that's what I showed there on our formatted pitches. We have a resource box and sometimes we say we'll need someone from another community, from another team, we'll need front-end time, we'll need back-end time, and we say maybe 2 days of appetite, well, roughly, but we take that time into account because there's the whole aspect of how you build a cycle. So, well, you have a lot of ideas, you have 6 weeks, so what goes in, what doesn't, with vacations, capacity, etc. to manage anyway.
[00:41:19] Yes, I wanted to know.
[00:40:32] uh, the shape is possible, then the whole team uh in a build, how do you divide uh the two cycles?
[00:40:38] So the idea is, uh, it's more Shape Orchestra, we could say, and build developer, but sometimes we need technical resources on the shape or there are lots of devs who also want to contribute to this reflection in workshops, and we try to anticipate it at the betting table. So actually, that's what I was showing there on our formatted pitches, we have a resource box and sometimes we say we'll need someone from another community, another team, we'll need front-end time, we'll need back-end time, and we say maybe two days of appetite roughly, but we take that time into account because there's the whole aspect of how you build a cycle. So, you have a lot of ideas, you have 6 weeks, so what fits, what doesn't fit with holidays, capacity, etc. to do anyway.
[00:41:19] Yes, I wanted to know, uh, you talked about several types of shape, uh, constructive, is that it, or I don't know. Exploratory and all, yeah. Uh, are these phases that follow one another, or are they simply different types of shape? Meaning, depending on the subject, we'll go for something more technical or more conductive, I think.
[00:41:38] Yes, that.
[00:41:39] Exactly. Conduc.
[00:41:40] Uh, it depends. Actually, let's say in an ideal classic cycle, we explore. So we have lots of opportunities, we say this one looks more promising, so we do a conductive shape. It's validated, we go into build, actually we don't exploit the technical shape that much, I admit. Uh, then sometimes, it makes the cycle a bit long, exploration, conductive, build. It happens that we have strong convictions, having already iterated since the product exists, so actually we only do conductive build. It also happens for small subjects to say, well, we're shipping, we're building, we're not going to do an AB test, it's useless. So yeah, really. And sometimes also the confidence our leads have in it and to what extent they are ready to go or not.
[00:42:32] two questions in one. does it happen that at the end of the betting table, you've thrown everything away, or rather, that all your ideas are rejected? and and it's a bit related, so if there was, even if it's not your North Star, but if we realize there are really two incredible ideas, can we allow ourselves to slightly change the team's perimeters?
[00:42:51] Yeah, uh, so throwing everything away, no. But typically for me, it happened, we proposed something very well-built, very good, and actually our leadership wanted us to explore something else because they felt co-construction was super promising, they wanted us to go for it, and we had a more reasoned attitude saying, well, we continued this, we want to iterate on it, we know that a little. And they challenged us a bit on whether we didn't want to switch now, it's the beginning of the year, explore, take risks, it's okay if it doesn't work, if there's no heroism, but we want to know. So, we kind of went around it and changed some pitches. And your other question was, oh yes, do we change? Well, actually, we work a lot by semester, so one semester is three cycles and the other semester is three. And typically last year, from one semester to the next, we all slightly pivoted. Because actually, we learned things as we went, so we said the second semester would be more tinged with travel, more this and yeah. And it's the same from one year to the next, we re-do the product strategy a bit, so even if I always have the same KPI to move, it's not necessarily the same angle behind to tackle it.
[00:44:05] Are there certain times when there could be imposed subjects, uh, coming from higher up, for example, a security subject that has to be done, nobody wants to because nobody likes it, but there's no choice. Or, uh, typically in my company, we're migrating to the cloud, well, that's a very strong business challenge, we've been on it for more than 6 months and we'll be for a long time still, uh, that's major compared to everything else, clearly all product subjects are cleared. Can that fit into this framework?
[00:44:36] So we have that, I didn't mention it, a platform team that handles ops, security, etc., who might take these subjects into account precisely so that we always have the right technical stack to do it. And after that there were subjects of big migrations of and all front-end, well, the they did it from cooldown to a cooldown. So actually I think it could have been done in maybe two or three consecutive weeks, but since it spread out over each cooldown because we wanted to have our cycles, it ultimately took them 3-4 months. Afterwards, I think it wasn't an issue, it's just that we had agreed it would take longer. But yeah.
[00:45:14] I have another question, has it ever happened that you missed a build cycle? at the end of the build you don't answer the problem or whatever.
[00:45:22] It happened to us not to put enough appetite given the technical complexity. Uh, to say, typically we had irritants and we wanted to correct them because we knew it was still complicated in the agent experience but we didn't want to dedicate too much time to it, it doesn't bring too much business, we had said two weeks. On the tech side, they quickly had risks saying, "Okay, it holds," but starting, we realized, no, it doesn't hold in two weeks. Uh, and so what we did was, we still put down our pens at the end of the cycle, but in the next cycle we bought back time, we said, well, we're sorry, we thought we'd get to this, well, badly scoped, we want to buy back, we want an extra week. So that's why I say, in fact, it's about appetite, it's about how much time we want to spend on it, but you still have to be aware of how much time it takes, it's a small balance to find.
[00:46:14] And so you've kind of answered, I was wondering if you cheated a bit in those cases, so you admitted that you could cheat a bit. Meaning that in this case, you brought back to the betting table a subject where perhaps it wasn't the one with the best ROI but you said we'll finish it in a week. And my corollary question is, do you sometimes feel frustrated about not having finished something precisely with this system where it's very business-oriented, your functional debt sometimes you never resolve it, how do you experience that?
[00:46:41] I don't really have that frustration, but I think the tech people sometimes have it a bit, so we had two or three weeks of appetite, we did it, it's good but we could have done it much better, we could have refactored more and so on, so the tech people have it and designers sometimes too, well, since we were constrained, the experience is good, we know that given what's being done today, there could be delight, it could be wow, it could be better. Uh, as a PM, since I have so many incoming issues and all, I tell myself if it's already good, I'm okay with moving on to something else and I mourn, let's say, maybe more quickly than others.
[00:47:18] Do you think it would be feasible to say that not everyone is in shape-up, and I might have teams that, and it can be rotating or whatever, but are perhaps more oriented towards processing user feedback or who are less focused on a very business objective and more on fine-tuning or that kind of thing?
[00:47:35] Do you think so? Well, I think what would happen is that maybe, I don't know if there are teams that would naturally want to only process user feedback, but then again, why not if it was imposed and all. For us, in reality, it runs well. Honestly, it was one of my concerns before joining Revenos, like, "Okay, the PM doesn't deliver, doesn't manage bugs, and there are bugs every 8 weeks, so how does it work?" Well, it works, I don't know the technical stack and so on, but it works well and I think it also pushes support teams to properly qualify bugs. If they're important, we fix them, and that's what I was saying, otherwise the product isn't perfect, but actually what doesn't work well are small things, we tell ourselves it's not a big deal. In fact, it doesn't prevent booking a trip, having a good travel experience, being safe and coming back. Actually, we're quite clear on what our main objectives are and the rest we mourn, it's not a big deal.
[00:48:25] We're tackling the last two questions.
[00:48:28] Thank you.
[00:48:32] So I have two questions, sorry. Uh, the first one is, are your developers full stack? And the second is, do you ever form ephemeral teams because you need particular expertise?
[00:48:47] So no, they're not full stack, whereas before I had worked a lot with full stack devs, we really have a front community, a back community, I think that's also why they are quite senior and very experienced in their field. But what that means is that if I have one or two back-end developers on holiday, they're not interchangeable, so it requires pre-cycle preparation to know what resources I have. And then the ephemeral team aspect during cooldowns, yes, very much so. In fact, if there are small projects like that, three or four days of development on a specific subject, small teams are formed, and all the devs know each other very well because during each cooldown, they all work together a bit or with different people, actually.
[00:49:24] So the last question is, what's the number of shaped subjects that end up in a build? Are there many subjects that are set aside, that are never, during the beta phase, nothing is put on them, or not?
[00:49:38] Well, it's certain we shape much more than we build, I don't have a ratio in mind. But let's say sometimes after a shape, I've detected 4 or 5 opportunities and actually we'll focus on one and it takes time so three cycles later I might be on the same one or the second one, while I always have two or three in stock. But normally, if I did my shape correctly, they were less important than the others, so too bad, so to speak. Uh, and then as for what really gets rejected at the betting table, normally that doesn't happen much because there's a bit of preliminary preparation work where you've already talked to your leads, your leadership, you've prepared them. In fact, the idea is also that sometimes you give them, we send a pre-read, so they also read the documentation beforehand to get an idea of which interview, which data, and why we propose this choice. In fact, the idea is that at the betting table, we don't re-explain why we want to propose this, but that they have already seen the documentation. So there's this pedagogy beforehand, yes.
[00:50:28] Thank you, Chloé.
[00:50:29] Thank you. Ah.

Transcript (Translated)