Sébastien Sacard
Transcript (Translated)
Good evening everyone, I will start since we are already 8 minutes, 10 minutes late, and I will try to keep to the schedule.
I am going to talk to you about a product management method that I have... That I designed. I will explain a little bit about where it comes from, as well as a little bit about where I come from, so that you understand a bit of the history behind this method.
I started my career in 1999, I was the first employee of the company Kelkoo, for those who still know it, it was a price comparison site that was acquired by Yahoo in 2004. I started as a webmaster, developer, engineer in the R&D team. Then I took on technical lead responsibilities. And when Yahoo acquired Kelkoo, I had the opportunity to go to England and learn a new profession, which is the profession of product manager. The product manager is a profession that doesn’t really exist in France, at least not in the digital or web sector here. It exists in publishing, and of course in other sectors outside of IT. But in France, it didn’t exist. And when I returned in 2007, I was looking for a product manager job and I didn’t find any, simply because there were no companies offering it. So I decided to become a consultant and evangelize this profession and its practices in different companies.
For 7 years, I have been a consultant and I work mainly for large corporations today. And I alternate between creating startups and supporting large corporations, to try to cross-pollinate the two worlds. That is, when I am in startup mode, I share a bit of what I learn in large corporations. And when I am in large corporations, I try to bring a bit of the startup spirit into... These teams.
Today, I have three complementary activities. You will understand why. The first is that I co-founded a startup in June.
We created a network with my partners, a network for mobile phone repairs at home and in companies. So, I handle the product and the technology. My partner handles sales and network development. And we developed it using the Lean Startup method.
And it works.
My second activity is that, as I told you, when I arrived in France, I looked for a Product Manager job and didn’t find one. So for the past two years, I have been promoting this profession in companies. So we created an association. Today, there are more than 600 of us in the group in Paris. In the active association, there are about 70 of us. Every month, we hold conferences on product management. We bring in foreign experts. The last one, in September, we brought in the product coach from Google Ventures in the United States, who coaches all the startups incubated by Google. We brought him in to talk to us about products. We brought in GoCoAd. You surely know them. We try to bring in people to talk about products and discuss somewhat new things and promote the profession of product manager.
And so I am the author of a method I call Lean Product Management, which combines aspects of Lean and Product Management, enabling large organizations to innovate as quickly as startups. That is the purpose of this method.
So, I will talk to you about digital transformation as an introduction to understand a bit the context in which we operate. Lean Product Management applies in the context of companies undergoing digital transformation. Forgive me.
And what is a digital transformation?
It’s a word that means many things to many different people. For me, digital transformation is a change in the business model. That is, a traditional company will radically transform its revenue models to shift to revenue models based on digital. Take, for example, Yellow Pages, which sells paper and has been shifting to digital for years. That is digital transformation. It’s when you change your revenue, when you change your model, you deeply change your teams and your production.
And what doesn’t work in these companies?
I have worked in many of these companies and I also consult a lot.
The main problem that comes up is that there is too much time between the idea—we would like to do this—and the moment it comes out the other side, when we finally manage to release the feature, the new product, etc. It can easily take two years. Oh, it would be nice if we made an app that... And then two years later, it still hasn’t been released, or it’s just starting to come out. This is also true for internal IT departments, when we have modifications to make to back-office systems, changes, new tools, new processes—it takes a very, very long time. The second point raised by the people I interviewed is that the concept of value is very, very unclear depending on who you talk to. And this is one of the issues that causes things not to move forward.
The third point is that there is often a big difference between what the boss says, the vision, and what we are asked to do on a daily basis.
And the fourth point is that innovation in these companies is not seen as a reliable source of revenue. That is, above all, in these companies, the goal is to protect revenue—that is, the current revenue generated by current sales and the current business model. And not so much to innovate, because innovating is dangerous. Innovating would challenge the company’s business model. So, it is often not seen as a potential source of revenue. Whereas in startups, innovation is the main revenue model. If you don’t innovate, you die. And so, you have no reason to exist. So, these are the main differences.
And that’s what doesn’t really work in digital transformations. And that’s why it’s also so slow. The alternatives these companies have developed, precisely to still manage to develop and take a little advantage of innovation, are several, some of which you know. They hold hack days. Even we did that at Yahoo, a big startup nonetheless, but we held hack days, meaning that for one day, basically, we had free rein to innovate. One day out of the whole year, even at Yahoo. In companies like Yellow Pages, or even at AXA, in other large companies, they hold hackathons, which are open or closed, but we innovate for 24 hours, 48 hours, to create things. And often, they come up with interesting things. In these hackathons, they will release a product. They will assign a team, a budget, and release a product. It happens once a year, for one product. It’s quite small. What we also notice is that often innovation is delegated to a department, and even to an office very far away, often in another city. And that is how they try to find a bit of innovation: they put people, often technical, in a corner and ask them to come up with things. Or it’s an innovation department that is supposed to spread innovation practices to other departments. In daily tools. The third way to do a bit of innovation in these large companies is to incubate companies. They will finance, open offices, bring in companies, take an interest in their projects, and possibly fund them, buy them, but there you go, they don’t touch the normal business.
They will also make external investments, invest in innovative companies, possibly acquire startups. But there you go, they don’t touch the current... business. And then it’s the same with digital transformation, innovation means many things to many people, but there is one truth that I discovered, especially in the startup world: innovation cannot be commanded. You don’t say, 'Let’s innovate.' And so these solutions of creating an innovation pot, things like that, can’t work because innovation, by definition, happens by mistake, by chance. So you can’t provoke chance. You can’t say, 'Let’s find innovation,' it doesn’t work. You have to do many things, you have to test many things. And it’s by testing many things that one day, maybe, innovation will happen. And it’s not just by doing many things, it’s by doing many things, observing results, and saying, 'Hmm, there’s a gap compared to what I thought.' Who was going to arrive during my experiment. And it's by digging into this point that we will discover something that no one had discovered or that no one had looked for until now. And that's where there is potential for innovation.
So, so-called 'programmed innovation,' innovation programs, are simply investment programs, in fact. We say, here we go, we're going to develop a new project, we allocate a budget, and eventually the project will come out. But it won't be any more innovative than that. That is not what will be disruptive in relation to a market. We won't discover or revolutionize anything in innovation programs. So, innovation in digital transformation doesn't work very well with these solutions.
So, how could a company deliver faster?
Insert innovation into daily processes in classic development. Ideally, have a single framework that would be shared by all departments of the company, to avoid having IT on one side, marketing on the other, innovation even further away, and so these people don't talk to each other.
And then, ideally, why not, we come out of the Cost of Delay workshop with Ron Renadsen, but how could we replace persuasion with experimentation? That is to say, today, all major decisions, the new products that are launched, It's the CEO, it's marketing that decides, let's do this, it's great, and off we go. But there are no more facts than that. And so the framework I propose is to replace all these product and feature selection decisions with rational decisions based on experimentation, rather than decisions that can be good, but
which can be wrong half the time.
So, my method comes from there. It comes from two things, from two worlds, in a way. From product management, I'll talk to you briefly about it.
And from Lean Thinking. So, it's not Lean Startup, it's a bit what there was before Lean Startup. It's about focusing on value, on the search for new values, on delivering value more quickly, in a more continuous way, etc.
And I don't know if you know this book, Lean Thinking, but personally, it has changed my life a bit over the past few years. So, I'm going to talk to you about product management and how it can change the organization of your companies a little, and how the implementation of product management will enable... innovation in your development programs. So, we're going to go beyond the product development you may be used to practicing, We're going to talk about products and management. A product can be seen in a fairly systematic, almost mathematical way. The idea will be to put measurement into it. So, we need to measure things. So, we need to rationalize many things. Which, a priori, is not very rational. A product... When we talk about a product, we think of marketing, we think of communication, we think of market, price, etc. So, it's not very rational. And yet, there is a way to rationalize all this, especially for me who is rather... technical, rather analytical, it has allowed me to rationalize many things. A product can be seen as a system, with an input, needs, customers, and then as output, value for the company,
and user benefits for the user. So, we clearly distinguish the two things. When we talk about business value,
it's value, it's money, it's euros, it's nothing else. So, if you're used to dealing with backlogs where we talk about business value, it's money. When we talk about user benefit, it's time savings, performance, anything you want. But a product does two things. We'll go from the simplest to the most complicated. A hairdryer, a product like any other, has a need, user benefits, and value. I left this one blank on purpose to get your opinion a bit, but what is the benefit of a hairdryer?
Pardon?
Yes, it can heat in winter.
Save time, there you go. Save time, not get sick, to not go out with wet hair, etc. In short, saving time can indeed be a benefit of a hairdryer. The value.
30 euros, that's money. It's quite simple. We're dealing with a standard product. I buy a product. I give money to the company. The company makes money. It's quite simple. Now, let's make it a bit more complicated.
A GPS.
A modern TomTom GPS like this one. Pardon?
Is that true? But you're right. The last TomTom campaign was about no longer arriving late or something like that. We were quite far from the benefit. But yes, why not, domestic peace, completely.
But the value is a bit more complicated. Because the value is not simply the purchase value. Because behind that, they will sell services, they will sell a subscription, cartographic, etc. So, the product itself, we begin to decouple the immediate benefit from the perceived value when we buy it. It's even more true for printers. You know, when you buy a printer, you don't pay the price of the printer. In fact, that's where they make money, on supplies, on the purchase of... Pardon?
Is that true? But you're right. TomTom's last campaign was about no longer arriving late, or something like that. We were quite far into the benefit. But yes, why not, domestic harmony, completely.
But value is a bit more complicated. Because value is not simply the purchase value, Because behind that, they will sell services, they will sell a subscription, mapping, etc. So the product itself, we start to decouple the immediate benefit from the perceived value when you buy it. This is even truer for printers. You know, when you buy a printer, you don't pay the price of the printer. In fact, that's where we make money, it's on the supplies, it's on the purchase of...
Yes, indeed.
That's not really the point, it's to introduce the notion of a product, but yes, Google takes it even further. Any so-called free Internet service is even further. We have an immediate user benefit, which is to find information.
The value for Google is zero. If you search and never click on the ads, it's worth nothing. However, the audience, the traffic you generate will create a derived value, which is that this value, this audience, they will sell to advertisers who, in turn, will pay real money for the actual impact, etc. So the more we go digital, the more complicated it is to link user benefit and value. And that's why we need to measure both differently. Since one can impact the other, but on fairly distant levels nonetheless. And so when creating a product, when developing a product, you need to ask yourself these kinds of questions, you need to have this notion of benefit and value, to then know what you're developing. And one way to do this in companies, to manage this, to manage value, to manage benefits, is to set up a product organization.
In a company. What is a product organization? It's that, as you see, as usual, marketing and engineering, in fact, a product organization has product managers between marketing and IT. who are not part of IT, who are not part of engineering. And these product people, so inside there are product managers, there are UX designers, there are product designers too, all these people work directly with the customer. To determine the real customer benefit expected by them. It's them who will seek out ideas, find innovative ideas, rationalize all that and turn it into a system, build a product that ultimately creates value for the company and benefits for the customers.
The role of the product manager is to work with marketing, engineering, and finance to create a profitable product. So they ultimately bring value. So that's their role. They have a role of maintaining the heroine of their product.
Often, in more traditional companies, this role is delegated, I take the example of Pages Jaunes, for instance, it's rather the sales department that manages revenues. It's not at all the product manager who is responsible for monetizing a product. So this type of organization is the type of organization you very frequently find in the United States. It's a type of organization you find in software publishers.
It's something you find very rarely in France, but it's starting to happen, it's changing. And so, roughly in order, the product manager is responsible for finding ideas, designing the product, at the same time checking technical feasibility, checking financial viability, and he will define the offer with marketing. Once the product is developed, he will work with marketing to launch the product, and then marketing will work with sales to prepare the message and help salespeople sell the product. That's in a more B2B company, which is why I put it in gray. It's a bit particular. In B2C, that's all there is. There are no salespeople. Why am I talking to you about Product Manager? It's because this type of organization and this type of role can drive innovation in a company, rather than a separate department. That is, the Product Manager has the responsibility to manage the company's product, but also to continuously inject innovation. Thanks to a framework that I will show you. The question I would like to ask you is: are you a product company, at least in the company where you work, do you offer a product as defined before?
None of you?
You are more of an IT company, meaning an IT department in a company that sells something else, is that it? And which would potentially be in a phase of digital transformation in the near or distant future, for example?
A type of company, not necessarily the name, but a type of sector,
For a bank, yes. Which are in full digital transformation, in this case, for several years, and it's accelerating.
Okay. So, what we call business projects are the products. For example, a bank, an airline company, or others, they have a real business, which is actually to grow money, offer financial services, that's their product. After that, digital products could be said to be for marketing, for communication, or to improve the work of agencies, etc. We could say it's a side business ultimately, that it's not what makes money. But sooner or later, all companies will do software and digital, and that will be their... Their real business, in fact. Their real product will soon be that. Today, you have all the online banks, 100% online. If they don't have a digital product, they don't exist. They no longer have branches, they have nothing left. So the product itself becomes very central. Other types of companies are media companies, newspapers, etc. It's not really a product. The software, so to speak, is a support. It replaces paper, but it's not a standalone product. But we can see things differently. We can see everything as a product, to a certain extent. And so we can introduce these notions of KPIs, benefits, values, to help manage projects as products. In IT, for example, I currently work with a client in the IT department, whose business is not to create products, but to do commerce.
The IT department, is asking themselves how to become a publisher, thus having a product position. And so we are looking with them at how to introduce the role of product manager internally, within the IT department, and to be more in a position of selling
a finished product, software, a platform, rather than receiving needs and doing projects. So it's entirely possible. We can see things, especially in IT, as being software publishers and thus publishing a product.
So the phases of product management and lean product management, that's how it works. And in fact, I couldn't tell you that I've used all these phases with one of my clients. With my clients in the past and in my adventures as an employee, I've done one or several of these steps. And it's very difficult to do the entire framework. I hope that in the future, we'll manage it. The earlier you take on the project, the more you manage it. But the company's organization must also be set up in such a way that you can have a product manager role and a framework like this. But there are different steps. The discovery phase, which we rarely do.
the design phase, development, launch, continuous improvement, and especially at the end, what is quite new, is that we will force ourselves to discard most of what we develop.
It may seem absurd today, but when I was at Yahoo, that's what we did. And even back then, Google did ten times more than us, discarded ten times more things than us. What does discarding more mean? It means we will experiment a lot, a lot, a lot. And we will only keep what works. But only what works very well. Not what works a little. And so by testing a lot, as we saw with innovation, we allow ourselves to find ideas we hadn't thought of, or slightly strange things we hadn't considered. So we bring innovation to the surface. That's how large companies... In the United States discover things—it's by testing a lot of things, and so, boom, innovation emerges, and they throw out 95% of it. When we talk about experimentation, it ranges from 'I launch a full product,' like Google... Wave, there you go. A major innovation launched in one country, Australia, and it didn't work. But it also ranges from 'I change the color of a link' or 'I move a button' or 'I add a small thing' and test it on a small group of people. A company like GULF runs between 3,000 and 5,000 experiments per year.
Lots of small tests like that. And we discard. And we discard everything that doesn't work.
Closer to home, in France, a company like Viadeo—I witnessed some of their tests despite myself. One day, on someone's profile page, a button appeared called 'Schedule a meeting' or 'Propose a lunch' or something like that, I don't remember exactly. I clicked on it, and a window popped up saying, 'Thank you, but actually, the button doesn't work yet.' However, we've noted your interest in the feature, and we'll contact you again, etc. For those familiar with Instartup, this is somewhat well-known, but I found it interesting that a company like Viadeo, which is a good ten years old, is starting to adopt this testing approach. And not long ago, I received an email from the Product Manager. Personalized, thanking me for clicking the link and saying they'd like to know more about my needs, etc. So they're adopting these testing approaches. So what's the point? It's to test lots of small things and go through all these steps systematically, every time. And to do it faster and faster. So there are three important focuses. Continuously find new opportunities, whether they're new business opportunities—through new features that generate new revenue streams—or completely new products, why not? We must allow ourselves the possibility of creating new products. I worked for a company whose business wasn't e-commerce at all. And they asked me to explore the e-commerce avenue. Based on their DNA, their assets, and their culture as well. It's not about launching a new business, but exploring possibilities for e-commerce for a company that wasn't doing it at all. And this can be done very quickly—in a few weeks, we can explore possibilities, see if there's potential, interesting things, by interviewing clients, conducting a small ecosystem study, seeing how it works—so it's entirely possible. And all of this involves iterations. We iterate. In the discovery phase, we iterate a lot. In the design phase too, we iterate a lot. So we iterate constantly. In the development phase, you're all familiar. We iterate, we use Scrum. The continuous improvement phase is the same.
We iterate constantly. The second focus is achieving what we call Product-Market Fit—really achieving market alignment, meaning we launch a very quick first version and give ourselves a timeframe, say 2-3 months, to improve it, improve it, until we reach a satisfactory rate. And we decide to keep it. Or discard it. Meaning, if we don't reach the performance target we set, we remove the feature; we don't look any further.
In a somewhat traditional company like you might find in France, this would be on a scale of 3 to 6 months, an experiment. We give ourselves a month or two for discovery, two or three months for... design and development, two months to improve the feature, then after two months, we stop. At any rate, we take a first assessment. And we set an evaluation: do we continue or do we stop?
But the key is here. The key is to accept from the start that the experiment we're putting into this flow is destined to disappear. This is where a major cultural change is needed.
The big difference, as you can see, is that the Product Manager is primarily along the chain. They steer this with marketing, UX designers, and developers. Depending on where they are in the flow, they'll work with one part of the team or another. So it's not about breaking down silos in terms of hierarchy; it's not about destroying the marketing department, engineering, or putting everything together. Everyone continues working with their specialty, with their colleagues, etc. But we work within the framework of a product.
So someone in marketing will be dedicated to product X, an engineering team to that product, someone in finance, etc.
And in fact, it's a flow. It's a stream.
That's why I found it interesting to talk about it here. Because... This is how I represent a Kanban. I don't know if that means anything to you. But basically, I had trouble visualizing the double columns. You know, to do, done, to do, done, to do, done. Actually, I prefer to present it like this. This is the to-do. And this is what's in progress, actually. We're working on it.
And so, in fact, it enters the Discovery system. For example, this phase where we're discovering things. So we have lots of things to discover.
That are in the discovery backlog. And when it's in the discovery phase, it's here. And once it's discovered, boom, it moves to the backlog of the next phase. And this allows us to visualize what's really in progress and what hasn't started yet, across the entire company. It allows us to see all the projects on standby at all levels of the company, whether in terms of discovery or things planned to be removed from the product. It allows for a fairly simple visualization of the entire process. Each of these processes can be detailed themselves. Meaning, here's the development phase. In the development phase, there's, say, the backlog or the Kanban of the development team. It's just a zoom, a way to see the company's activity in terms of product development. You can do one per product or one for all the company's products, regardless of the zoom level, but at least you see everything. And so, there you have it, it simply moves.
Each phase corresponds to a particular activity. That's why we're really in a flow. Here, it's a discovery phase. Here, it's a design phase. Here, it's a development phase. And we unstack. You're familiar with these concepts of flow, WIP, queues, etc. Well, I've actually adopted this. This system allows setting minimum and maximum limits to ensure we're always in a rhythm.
For developing features. So that's what we're going to manage, ensuring that for each of the systems, we're always in a mode where we have enough things to do in the pipeline, and we proceed. And here, what do we see? We see that if we've defined an arbitrary maximum limit, as I've drawn here, we see that there are too many things in discovery, too many things that need to be launched.
So there's a problem; we need to stop, we need to descale, or we need to reorganize the teams a bit, because it means that things are no longer flowing, so to speak, in the development process. That is to say, we're not delivering things fast enough. We are still in a rhythm of developing features, so that's what we will drive, ensuring that for each of the systems we are always
in a mode where we have enough things to do in the pipeline, and we proceed. And here, what do we see? We see that if we have defined a maximum limit, arbitrary as I have drawn it here, we see that there are too many things in discovery, too many things that need to be launched. So there is a problem; we need to stop, we need to descale, or we need to reorganize the teams a bit because it means that things are no longer flowing in the development process. That is to say, we are not releasing things fast enough.
The interest also is that we need to constrain these phases with arbitrary time constraints. That is to say, we need to set a limit, saying, for me, the discovery phase for such a project, such an initiative that I want to launch, I give myself three weeks, no more. We force ourselves to be constrained by time.
And we move forward like this for development; we will give ourselves two months. But after two months, we stop. So basically, we start with a project, we have two months to experiment with it, we break it down into sprints, etc. But after two months, we stop; we have to launch. So this forces us to move forward.
And of course, this can manage multiple teams. That is to say, what are we saying here? We are saying that there are four things, four features being developed, four experiments in progress. It can very well be four different teams. So it can actually scale quite well. For the client I am currently working with, there will be five teams in place that will be able to take on projects in parallel like this. We could very well have 7, 9, 10, 15; it poses no problem. Each team is completely independent and takes an experiment and launches it. When I say experiment, it's in the broad sense, meaning it can be as simple as changing the color of a button or launching a real feature.
Everything is an experiment, in fact. Even if we tell ourselves we absolutely need this feature, we know we will keep it. In fact, no, we don't know. We must assume that potentially, we will discard it because it might not work. Potentially, it will be a flop. So we must allow ourselves the possibility of removing it. So everything is an experiment. Having this mindset completely changes the organization. And so, that's for the entire company. This is a board that can be visible to the entire company. We see where we stand in our product developments.
So the key here is speed. That's why we constrain ourselves by time; we force ourselves to be constrained by time in the development phases.
Of course, we need to learn from the results in the improvement phase; we will realize things are happening, so we need to try to collect all the information we can and stay focused. That is to say, rather than working, as I often see, with a team of 25 engineers working on 20 projects at the same time, we force ourselves to work on only four projects but to complete them in a very short time and release them.
So you are familiar with the idea of breaking things down into sprints, etc. Here, the idea is that yes, we will break things down into sprints, but we give ourselves the possibility that if other projects come up, we stop. A project was planned to have six sprints. We will only do three, but at least the three sprints that were done truly delivered value, and we will be able to test it. We will be able to expose it to the market and see what happens. So this reinforces, so to speak, adding value to the sprints and ensuring that what we release truly has value. Because potentially, we will really stop there.
I will... There!
It was just a focus on the different phases.
The discovery phase is to say a bit who works in these phases. The discovery phase involves the product manager, the UX manager, and engineering as well. This is also one of the major differences: we involve development leaders very early in the discovery phases to participate.
In surveys, with clients, etc. We also involve finance, we involve BI, to have data, initial analyses, to gather information. And here, what's the idea? It's to do opportunity discovery, surveys, define the vision, the strategy. It's very, very broad. We hardly develop anything, but we will try to explore a bit what we could do, simply. So there are plenty of tools for that. Some that I have developed, others that I have taken from other methods, and I have made a coherent whole out of them. In the design phase, there is the product, the UX, and still a bit of engineering. And here, we will start prototyping, creating mock-ups, conducting interviews, and starting to show things.
In the development phase, you all know it, I won't go into detail, except that there is one key point that is quite important: the role of engineering in the development phase
is not just to develop the feature, the functionality, or the product. It is to continuously improve and develop the framework. That will allow launching experiments faster and faster and delivering—this is the integration of continuous deployment—but beyond that, it's building the components that will enable developing experiments later. At Yahoo, for example, we had a tool that allowed us, when developing a feature, to target a category of the population, to launch the experiment only on that category, and not to show it to others, and to gather analyses. And that was in 2004, even before. The company was built around this because we knew the importance of experimentation. So beyond simply developing features, we had developed an experimentation engine. And what we did, as I told you, at Yahoo, Google had done it ten times faster, ten times better, and ten times more powerfully. I am pretty sure that at Google today, experiments are conducted automatically. That is to say, robots are experimenting. There isn't even a product manager saying, 'Hey, I'm going to test the color of the button,' etc. " I am pretty sure that it can be done almost automatically. And if something interesting comes out, boom, it surfaces, and then a product manager looks into it and investigates. But I am sure and certain that it is done today by robots almost entirely. So the launch phase is very important; it is often very neglected. When launching a new feature, even in test, and especially in test, we must be very careful with communication. We need to launch it to a community or to a... If we don't have the capacity to launch it to only a small part of the user community, we must communicate clearly to say, 'Watch out, this is a test.' Otherwise, it's complicated. We need to be connected in advance with users to be able to gather feedback. Because there is nothing worse than launching a feature and not getting any feedback. It is absolutely useless. So all of this must be planned in advance. That's why we work with marketing before development. Because we prepare a launch plan with them. So there are also plenty of things to put in place here. Finally, the continuous improvement phase is the one that interests me the most. We must try to minimize what we are going to release. That famous MVP you may know from the Lean Startup: release that thing, launch it, make it available to the community. And only from there does the really interesting work begin because we will improve it based on feedback and analyze until we find the right balance. And if we can't find the right mix, we must remove the feature. We must leave room for the next experiment. It's not necessarily a failure, in quotes, to remove a feature. On the contrary, it can allow us to learn things about our customers, and it can also help keep the code clean, in quotes, to... No, it's the remove part, sorry, I made a mistake. On the improvement part, what is very important is to measure. And that's the quantitative and qualitative part. That is to say, it's good to measure. We must measure, we must have information coming back from the system, but we must always compare this with real observations. That is to say, when we launch a feature, we see, hey, there's a click-through rate that has increased, but right after, we must do user tests to see why people, in a small sample, why people click and get qualitative feedback. This helps protect against false measurements, misplaced measurements, poorly measured measurements, things we wouldn't be able to understand. We must not rely solely on numbers; we must rely on studies.
And finally, the removal part. It doesn't happen very often, but we've seen it recently on LinkedIn. LinkedIn, although they've already done it two or three times recently, they say they will remove such and such a part of the site because it's not working as well as they wanted.
It happens quite often. In France, we don't do it at all. Because the idea is to say, after all, we paid for it, we're not going to remove it. It's still silly. But what we forget is that keeping something costs money. So not only the... business, in quotes, that tells you we paid for it, we're not going to remove it, what they don't even know is that in fact, they continue to pay for this thing, and they don't see it, so when you tell them, but you know that by keeping it, it's costing you even more, often they are a bit more agreeable to removing it. and the idea of removing features from the code, really removing them, that is to say, not just deactivating buttons or hiding pages, but to remove the code, it makes the product more 'lean,' it's cleaner, it's more sustainable. And that too must be planned from the start. That is to say, when we develop a feature, we must plan to remove it at the code level. It must be designed to be removed. So that's also what needs to be considered. Upstream.
So there you have it, that's what I was telling you about the philosophy of...
Large groups in the United States that do this, it's to experiment more and more, but with the aim of also removing more and more things. It's not experimenting for the sake of experimenting. It's really about keeping only what works really well.
Finally, what are the conditions for this to work? For it to work, you must first have a product organization. That is to say, ideally, you must work in a company that sells products.
If you're in an engineering or IT department, it's more complicated, but it's doable. There are more and more IT departments positioning themselves as publishers. So it's entirely feasible. However, what is non-negotiable is that the CEO or the manager sponsors this type of thing. In the last conference we organized, we had feedback from a CEO of an online travel agency.
He said, 'It was me who pushed for a product organization.' It can't come from the engineering team or marketing. Simply because I tried to do it in a large e-commerce company in France. When I proposed this to the general manager, already four years ago. The marketing director told me no, no, no, you're taking away my prerogatives. And then the engineering director told me no, no, no, you're taking away my prerogatives. So neither marketing nor engineering wanted it, and the general manager didn't want it either. Because instead of having to deal with five people, it would have made six people to deal with. So he wasn't interested. Preferring to have a smaller, more tightly-knit group of people. Well, these are things that must come from the top, so you really have to do a lot of lobbying. I'm working on it, I've started talking to digital leaders, so I'm starting to be heard at that level, and at the same time, well, it's... I also think it's the right time.
So it's changing. It's coming. The product organization is coming. Now, integrating innovation into a product development process is happening, but very slowly. And a method is needed to facilitate understanding of all this. So, just to summarize everything I've explained to you, I haven't shown you all the tools, all the methods that are in each phase, because it would be far too slow and a bit boring, but for each phase, there are tools and methods. When I say tools, I don't mean online tools, I mean well-made Excel sheets, things, ways of measuring, ways of conducting interviews, tools you know like impact mapping, well used at certain times, it makes sense. It must be used at certain times in a... certain contexts. The Lean Canvas you know from Running Lean, in the Discovery phase, is very useful. It must not be used any which way or at any time. So what I've done is document this framework and process for myself because I'm actually using it from scratch at a large company. And so we're currently rolling it out.
I've managed to unify all these methods you know, I've managed to give them meaning and organize them in relation to each other so that it makes sense and we can coordinate them and not have turf wars between 'I do startup,' 'I do something else.' No, all of this goes in the same direction.
And then, I've started implementing it with a few clients. And so far, it's working. My goal for 2015 is to continue initiating this culture through the association I've set up, to initiate the product culture within companies. And I would like to... Finally succeed in taking the time to write all this in a book to be able to put everything I've learned in recent years and try to share it and then why not create a community of practitioners around it because it's like product management, I saw a big difference between when I tried to evangelize it myself in companies versus creating an association. I think that if you're interested in trying to test these different methods, trying to push them in your company,
This thing is open; it's meant to be. open and shared.
So my goal is to create a community around this tool. So there you go, if you're interested, write to me. I have a blog that I write in English about all these discoveries, everything I try to practice, and of course, a Twitter account where I try to be quite active.
There you go.