Estelle Garcet

My name is Estelle Garcet, I work at UX Republic. UX Republic is a user experience agency. I am what we call a UX evangelist. A UX evangelist is a UX designer, if we call ourselves evangelists. It's precisely when we are on assignment at the client's site, to evangelize UX and so we can communicate our method, in fact. I am also a master of the UX Lab foundation. The masters are working groups. My foundation is UX Lab. It is based on everything related to user research methods. So we have activities, we have a big project, and my role is to ensure we achieve our big project within a year, to hold conferences, to provide training in UX design, user testing, and in schools. That's my little introduction. Today, I'm going to talk to you about measuring user experience.
First, what is user experience? I don't know if you all have this concept clearly in mind, because often people ask me, 'But what's the difference with ergonomics?' Actually, you are ergonomists. Well, that's not quite it. User experience is about reconciling efficiency and the emotional aspect. So, what interests us is to know whether your tool or product is usable, whether it is easy to access. But also, what is the emotion of my user. Is there a feeling of frustration, fear, joy, or anxiety? We aim to maximize this user experience.
To do this, what interests us is to evaluate both the rational and emotional sides of the user.
So why measure user experience? First, to improve the overall experience, because generally, you always remember when you have a very good experience with a product or service, you won't question it, you will return to it spontaneously. Conversely, it's the same thing: if you have a very bad experience, it leaves a mark and you don't go back. So what we aim for is really to leave a mark on users. Reduce development costs, because the more you do research before development, the less expensive it will be. That's interesting.
Improve performance and thus retain users so they can actually come back and use your product or service.
So, how do we work? We have different methods. We use both analytical tools like Google Analytics, which I think you all know, and user research methods as well. So I will present both a little and explain how we use analytical tools, what information is important, and how we use it to improve the user experience.
So how do you choose your metrics? It starts with that. We don't start by saying, 'Let's look at Google Analytics, the results, what does it show.' No, when we want to set up metrics, we first need to ask the right questions. What do I want? What is my goal, actually? What information do I want to get from it? Because what improvements can I make to my product or service? Having data for the sake of having data is fine, but in UX design, it's not interesting. We want to collect data to be able to make improvements to products.
So how do you choose your metrics? As I told you, you need to ask the right questions, but there are also templates that can guide you. So there is the Earth Framework and the Art Framework that I will present to you.
We have the HURT template, which is a template created by Google. Why HURT? For Happiness, Engagement, Adoption, Retention, and Task Success. These are the main categories defined by Google. They assume that this can cover your entire user experience. However, they have defined several categories. You see that to get to the metrics, we still have to go through goals and signals. We don't go directly. We don't immediately say, 'I'm going to set this as a metric.' No. First, we start by asking, 'What is my goal?' Then, we ask, 'The signal is: by what behavior will my user achieve this goal?' And finally, from there, we will define, 'OK, so what metric will I put in place to measure this?' You see, it's a bit sequenced; it really encourages reflection.
In the categories they have defined, there is the happiness aspect, which is user satisfaction. So, this is often something we measure with satisfaction questionnaires, for example. We have everything related to engagement, which is the level of user involvement. I have users who are already on my product or service, and I want to know if they often return to my platform. What do they do more than other, new users? For example, it could be the number of photos uploaded by a user.
Then, we have the adoption aspect, which is about new users of a feature or product. So, here we can measure, for example, new registrations.
Return rate, so the existing return rate, how long they stay on my platform, on my product or service, how long they stay, and if they return, what is the frequency of return, for example, the number of purchases they make. And task success is actually achieving their goal, so it's the efficiency rate, defining the error rate. And so this can be anything related to download time or profile completion.
If we take YouTube as an example, YouTube, for example, in the Engagement section, could say they want users to enjoy watching videos and discover new video channels. Discover, yes.
So that is YouTube's goal. They say, 'Here, I want my users to discover new channels.' Their signal is, 'I want to know how much time they spend watching videos.' And only from there do I say, 'OK, so to know how much time they spend watching videos,' here are the metrics I will put in place. So, how much time users spend watching videos, the total number of minutes per day and per user.
So you see there is really some reflection behind it, saying, 'OK, I have all this information, potentially behind it I may have download issues,'
reactivity issues, so I will be able to identify that and make improvements if needed.
Another template that exists is the AR. It's a pirate template. So, this is more of a template used by startups. Maybe you already know it.
It's a little different, but we find some of the same information. For example, there is the acquisition aspect, so how your users find you. This is not something we find in Google's template.
Do they have a good first experience? Do they come back? How do I make money? And do they talk about it to other users? So, these are templates that can help you. Afterwards, you are not obliged to use them. We don't always use them, but they help guide the reflection, to ask what metric I want to put in place, what I will do with it, how they will help me improve my user experience.
So that was everything about templates. You should know that with metrics, generally, we don't use only metrics. We always try to combine them with user research. Because metrics give you quantitative information. How many users do this? How many go to this page? But in fact, you don't know why. And for us, in UX design, that's what interests us. Why didn't people stay on this page? Why did they click there? Why did they do that? That's why we always try to combine metrics with user research, and rather qualitative research.
Then, we have another type of use for analytics tools. We can use them as a problem indicator to investigate or for triangulation. When we use it as a problem indicator, We have a measurement plan that is fixed on the main objectives. You have a main objective for your site, so you will observe how my users manage to achieve this objective. You have all the desired actions. So the desired actions are more like, hey, it would be good if they went through this page to reach their main objective. So you will be able to verify if they really go through it. And so, effectively, you have your web metrics afterward.
Then, we have more of, if you wish, to investigate. When you want to investigate, it's because you have developed hypotheses. You have observed that there were indeed conversion problems. So, generally, we make hypotheses by saying, maybe users can't do it because, I don't know, the button isn't seen, because they don't scroll on the page and in fact, it's below the fold. We develop a lot of hypotheses like that. And you will go look at your analytics data to check if it can give me evidence for my hypotheses and enrich them or not.
So the information you can get from it is about traffic problems, technical problems, content, visual design, and navigation problems. You should know that today, there are analytics tools that are quite powerful. Here, you see that on the screen, on the image on the right, it's what we call a heatmap. I don't know if you're familiar with this, we use it a lot in eye tracking. And in fact, the areas that are the hottest, so red and yellow, are areas where people have clicked. So it gives you an indication of where my users clicked. Sometimes, in design, it can really provide you with additional information. If we take the example of Le Monde, Le Monde, their website, they have an image, they have a logo, Le Monde, they have a title, and they have text. Extensou. In fact, we could ask ourselves, okay, but where does the user click when they want to read an article? Are there people who will click on the image? Are there people who will try to click on the logo? Others on the title, others on the content? Because sometimes, we have... Page designs that mean the link will only be on a title, for example, only on a piece of text, but not on an image. And when you can analyze that, you will realize that in fact, maybe 80% of your users click first on the image before clicking on the text. Typically, that's a problem. To improve the experience, we will add a link to the image or maybe change the design according to the needs to really improve the experience and avoid a feeling of frustration by saying I click everywhere, then I'll see, something will open.
And the third use is triangulation. So triangulation is when you have done qualitative research. The advantage of qualitative research is that you know why people do that. However, you have a user panel that will be maybe 10 or 20 users. You can't meet 1000 people, 2000 people to know why they have this or that behavior. So you have a user panel of, let's say, 20 people who tell you, well, I tend to act like this. And you will be able to verify, from an analytics point of view, if it actually groups a majority of your users. If you realize that it actually only concerns 20% and that your sample wasn't good, it may not necessarily be a problem to consider that isn't a priority. On the other hand, if 80% of your users encounter the same problem as your users during your qualitative research, Then there is a problem and it really needs to be resolved. That's why we always try to combine analytics data and user research to find the best possible solution.
So that was everything about analytics tools, which we always combine with qualitative research. After that, we have all the evaluation methods. As I told you, in evaluation methods, what interests us is usability and emotion in the user.
When we start, it's the same as for analytics data, we always start by asking ourselves, OK, what do I want to do? What do I want to measure? What information do I want? What will I do with it afterward? So, we have this matrix that exists for research methods. So, you see that my main axis is the qualitative side. Why do my users do that and how do they do it? And there is the quantitative side. How many do it? We always try to combine these two parts. And we also distinguish between what users do and what they say. I'll show you an example later that proves very well that in fact, we tend not to necessarily do what we say, especially in user tests.
So that gives you this type of information. For example, when I'm doing qualitative research, I will meet 10 or 20 users. However, I will take time to discuss with them. I will know why they act in a certain way, what their behavior is, what their goals are.
So here, we are more on eye tracking or interviews. Because eye tracking is always combined with user tests. We take the opportunity to ask them questions. You clicked there. I saw that you were looking a bit for why. And for interviews, we have a defined plan where we really ask questions. We have time. For that. Then, on the quantitative aspect, we are more on questionnaires, so questionnaires that you can send via Twitter or social networks, so I think we've all answered questionnaires of this type. It's good, it allows you to have a large volume of data, but then this volume of data needs to be processed. And above all, it doesn't give you an answer to why. Why always comes back? Because you have information, but necessarily, you're missing a bit. Whereas it's true that during the interview, you can always bounce back on a question and say, 'Oh yes, but then why did you say that?' And why like that? "
And A/B testing, same, we can be on the quantitative side. These are things, either it's already developed, or there are sites that exist for that. And so, it allows you to have answers on a large volume of data. But you won't know why a person prefers interface A or interface B. So I'll explain that to you a bit more later. This allows us to have a large volume of data, but after that, this volume of data needs to be processed. And above all, it doesn’t give you an answer as to why.
Why does it always come back? Because you have information, but inevitably, you’re missing a little bit. Whereas it’s true that during the interview, you can always bounce back on a question and say, 'Ah yes, but then why did you tell me that?' And why like that? There you go. And A/B testing is the same; we can focus on quantitative data. These are things that are either already developed or there are websites that exist for that. And as a result, it allows you to get answers from a large volume of data. But you won’t know why a person prefers interface A or interface B. So, I’ll explain that to you a little later.
So, there you have it, we measure emotion and usability. I’ll show you how we can measure the emotional aspect. What will we get out of it? How do we also use the data we collect? And in terms of usability, what solutions also exist to evaluate that?
So, why measure emotions? What we want when we measure emotions is to detect points of tension. Because when you’re a UX designer, and even when you’re a designer in general, I think, we try to identify problems. That’s what interests us. What are the problems? At what moment does my user have a point of difficulty in their journey? Because our goal is to find solutions to these problems. So even at the level of emotions, when I have a user in a journey who tells me, 'Here, I’m not sure of myself, I hesitate to go ahead, to click,' we sense small moments of anxiety; we want to try to unlock these moments of anxiety and turn them into moments of satisfaction instead.
So for that, we have facial emotions. You should know that facial emotions are the most studied expressive manifestation in emotion. Paul Ekman, who is a psychologist, defined in the 1970s six fundamental emotions, which are disgust, anger, fear, surprise, sadness, and contempt. So these are emotions that are universal; it’s international, it’s research that was conducted internationally, so these six main emotions are found in all cultures. And he estimates that all other types of emotions are derivatives of these main emotions. From this, Paul Ekman defined what is called the FACS, the Facial Action Coding System, which is a method for defining muscular patterns to recognize what type of emotion is associated with that pattern. I don’t know if you see, but in each of the images, we see that there are two or three points on each part of the face. These are the muscular patterns.
For example, for anger, there are three. There’s one between the eyes, one at the level of the eyes, and one at the level of the mouth. So each time, he estimates for each of these points what type of behavior we find in each user to express anger. And from that, they developed solutions. We have solutions, for example, like FaceReader, So, I have a short video; I don’t know if it will work.
It doesn’t matter about the sound.
No, no, no. So, we see that indeed, with FaceReader, the person exaggerates the expressions a bit, but we do see which expression it corresponds to. You see, the small points of the muscular patterns are highlighted, so immediately, we can recognize which emotion, which muscle is used, and which emotion emerges from each of these movements. But as you can see, the expressions are exaggerated. In reality, for example, in my company, we mainly work on websites, applications, or business applications. I don’t think that when you go to buy a book on Amazon, you make faces like that. So for us, as UX designers, on products like that, it doesn’t add value. It’s a bit of a wow effect. In fact, I went to meet ergonomics labs, the elves and the goblins, I don’t know if you know them. And they use these tools a lot, but because they also do a lot of research in psychology. And they also do a lot of things, for example, on video games. So video games are things where you can have very strong expressions, very strong emotions as well. So typically, for them, it can be interesting. Whereas for us, in the end, we say, yeah, there’s a wow effect, but in fact, if I test on my website, I can have a lot of strong emotions that emerge. And ultimately, it doesn’t... It’s not useful. It’s a bit of a shame.
There are other types of measurements we can have, such as physiological measurements. So, it was the same at the ergodesign lab of Gobelins, the school of image. So there are electrodermal measurements that measure the sweat rate. We don’t see it very well, but next to it, there’s a Zephyr belt that will measure, for example, your heart rate. So again, these are very interesting things, but for a website or an application, it’s not useful. It’s interesting for projects like, for example, if you work on an airplane cockpit interface, where typically, if you have a slightly stressful scenario and your user needs to be able to control their plane, that’s information that will be interesting for you, because you need this information to know what their stress level is and when it occurs, and how you can improve it by giving them user feedback.
Whereas with a website, I might be a little stressed because I can’t find where to enter my credit card number, but really, I think it generally goes well. We don’t actually reach that level.
Here’s an example that was done, for instance, with a video game. We measured the sweat rate with a user playing against a friend. And when he scored a goal, we see that indeed, the emotional curve rises. Because there’s that slightly competitive side, I’m playing against someone. Whereas when he plays against a computer, it’s rather flat because the challenge is not the same at all. So there you have it, video games are something that evoke strong emotions and which, as a result, are easier to measure with this type of tool. But they’re not suitable for us.
You’ll tell me, that’s fine, but we want things that are suitable for us. That’s something else, electroencephalogram headsets. I’ll show them to you because when we visit ergodesign labs, it’s a bit like the playground. We see all these tools, it’s great, we say, it’s amazing, I want to use it. But in fact, the data takes so long to analyze, and for our products, the result would be so weak that it’s not useful. It’s extremely unfortunate. So for us, on the other hand, what we’ll use as a tool will be more things like this. So, PrEmo is an emotion measurement scale. They’re actually little cards. So here, we directly ask the user to tell us, according to their journey, which emotion they feel and to what degree of emotion they are. So it’s about the...
It’s the user who declares their own emotion. Afterward, it’s to be taken with a grain of salt, because one can also say, 'No, no, it’s going very well,' but in fact, as I said, there’s a difference between what I do and what I say. It’s a very interesting tool; it’s a tool that’s somewhat validated by the UX community because it’s validated by Donald Norman, who is somewhat the father of UX. He was one of the first to talk about user experience, and he also wrote a book on emotional design. He’s a method he has approved and uses, which is clearly recommended.
So, as I told you, it allows you to identify perhaps negative emotions. Moreover, if you’re doing a user interview or a user test, the advantage is that you can combine it with qualitative information. So you can ask them, 'Oh really, but here, why don’t you seem happy?' What happened? How could we improve this? So immediately, we can gather information from the user to then improve either their journey or the interface.
Another method is what we call UX scales. The advantage of UX scales is that they have been developed by ergonomics researchers. And as a result, they are standardized, tested, and approved. So, these are models based on Likert scales. Likert scales, I think you have all already answered questionnaires where you are given a statement and you say 'strongly disagree,' 'disagree,' 'somewhat agree,' 'agree,' 'strongly agree.' That is the principle of Likert scales. It means either I have five responses or I have seven responses.
For example, in the Attractive example, since we are dealing with UX scales, what we want to try to capture is really the emotional aspect. In Attractive, for example, in the first questions, there are two points. We use two words that are somewhat opposite. There is the human side and the technical side. We will ask the user to position themselves. Did you find that the interface or the language we used was more human or technical? And it is up to them to gauge where they stand. A bit as if they had variables each time with opposites. And it is up to them to position themselves, what do I think, at what moment, where do I stand. Now, this is interesting; it does not replace observation because there is always the question of why. Why do you find it more technical than human? Yet, we used playful language, the informal 'you,' we tried to create a... relationship, but we always want more information. And above all, what is important with this is that what is interesting about this type of tool is using it multiple times and over the long term. That is to say, if you do it as a one-off, just once, you will say, 'Okay, so they found the interface very technical, that it was not accessible,' well, they will give you information like that. " It will give you information, telling you effectively that you need to work on that, but not much information. Whereas if you are on a project that lasts 2 to 3 years, you will be able to conduct user tests more often and give this type of questionnaire more frequently. So perhaps you will have it two or three times in a year, and that will allow you to also track progress. Look, we have improved a bit on the human side, we are a bit less technical, but here we have not yet performed well; we need to work on that again. And each time, it allows you to iterate and improve your product.
So that was about everything related to the emotional aspect. But the emotional aspect is not what makes up the entire user experience. It is good, but it complements the evaluation of usability.
So, in usability, what are we going to try to evaluate? It is indeed everything related to task success, errors made. So, we are dealing with something very pragmatic. We want to know what the hesitations are, understanding of the interface, how long it will take them to complete their purchase journey, for example. There you go, we really want concrete information. And in this case, the emotional aspect, whether they are happy or not, does not come into play.
For this, we have usability scales. Now, usability scales are not like UX scales. With UX scales, we are really focusing on the emotional aspect. Whereas with usability scales, we ask for more concrete things. Did you find the interface easy to use? Strongly disagree, somewhat agree, strongly agree. We are really dealing with concrete things... The advantage is that you will have quantitative measurements. With your user panel, perhaps of 20 people, you will be able to say, 'I have 15 users who said they strongly disagree with the fact that the interface was easy to use.' " which usually, when you conduct user interviews, you do not have this kind of information. Because if you ask 20 different people to use their own words to describe their interaction, you will get 20 different answers. So concretely, it is not useful for you. You cannot group them; it is a bit complicated. You do not know exactly what they think behind it. However, this type of scale helps you a bit to frame and track evolution. Like UX scales, it is something we set up and that is iterative, which we will put in place several times throughout a year.
So that is an example, in fact. It is the System Usability Scale. Often, it is a questionnaire that we give at the end of a user test. At the end of the user test, we gave them scenarios to complete. There is a bit of an interview to understand what their behaviors were. And generally, we ask them to fill out a questionnaire of this type. So here, the advantage is that it is quite short, it is 10 questions, so it goes quite quickly. And the questions are quite simple. And as I told you, we are using a Likert scale of 5 to 7 points. So it will really give you a quantitative database.
Another tool we use is expert evaluation. Expert evaluation, you cannot use it alone. You really have to combine it with user research, either with tests or with interviews, but you cannot use it alone. I will explain why. Because these are often ergonomic criteria. We have the ergonomic criteria of Bastien and Scapin, Nielsen's heuristics, the ergonomic criteria of Colombo and Pascoe. Those are the most well-known. And in fact, in each, you have a list. For example, in the ergonomic criteria of Bastien and Scapin, they defined eight points. And within these eight points, which are detailed, they believe that if each of your pages, if we are on a website or an application, meets these eight points, then your interface is accessible and considered ergonomic.
So that is interesting. You should know that this also exists for video games, because video games are not handled the same way. And it is also starting to appear for virtual reality, because we are beginning to have more complex interfaces, so we do not handle them the same way.
If we take examples, we have in the criteria of Bastien and Scapin, to explain a bit what it looks like,
I forgot to tell you, but the advantage of expert criteria is that you can do it yourself. You do not need to ask a user to do it. That is to say, you can do a first evaluation, Did we think to provide... all the necessary user feedback in our project. So that is a first level for you to estimate whether your product is on the right track or not. So if we take an example, in the ergonomic criteria of Bastien and Scapin, we have the example of guidance. And so guidance, it says, is evaluating how the system supports the user. So it is everything related to autocomplete in search engines, like Google, and it is also help in completing forms. And indicating to the user the clickable elements. You know, when they tell you, well, on the link, actually, there should be a changing state, a hover, either the mouse that transforms, the color that becomes different. In short, there needs to be a changing state. Actually, that is why. It is really to give indications to the user, and it falls within the framework of guidance in the ergonomic criteria of Bastien and Scapin. Another example is explicit control. Explicit control means showing that every system action corresponds to an explicit user request. For example, it is everything related to buttons at the end of an action. The fact that your buttons have a clear label. For example, we will say 'send' and not necessarily 'validate.' But we say, when you click on this button, what will happen is that it will send your email. These are small things like that to take into consideration. For example, in the Amazon screenshot, it is having a back link to my search. That is to say, I had done a search... Emotional Design, and Amazon offers me to return to my search, rather than offering me to go to another list that does not match, or perhaps all the books by the author that are not at all what I am looking for. So there is this notion. In the criteria of Bastien and Scapin, for example, there are eight categories like that. Explicit control and guidance are each a category. So there are eight categories, and you can go and see for yourself whether you fit into each of these categories. But this comes in addition to user research, of course. Because you can have an interface that is very accessible from an ergonomic standpoint, but perhaps the elements are not well placed, or users don't see them, so that's a bit of a shame. A search for Emotional Design and Amazon suggests I return to my search rather than proposing I go to another list that doesn't match me or perhaps all the author's books, which are not at all what I'm looking for. So there is this concept. In the Bastien-Scapin criteria, for example, there are eight categories like this. So explicit control and guidance are each a category. So there are eight categories, and you can go and see for yourself if you fit into each of these categories. But this comes in addition to user research, of course. Because you can have an interface that is very accessible from an ergonomic standpoint, but perhaps the elements are not well placed, users don't see them, so that's a bit unfortunate.
And we have user tests. User tests are clearly what we use the most. This will help you evaluate a product or a prototype. What's important to consider with user tests is that they occur at all stages of the project. I'm always asked, 'But when do we do the tests?' They can occur at all stages of the project. That is to say, if tomorrow I arrive at a client's, I can very well suggest doing user tests right away to understand what isn't working on their interface. We can do user tests on paper. If I've done a sketching session, I have my screens drawn on paper, I have my sequence of screens, I can bring in a user and ask them for a first level of feedback. And the earlier you do your user tests, for me, it will cost you a lot in development. Because it's not once you've developed... Afterwards, it depends on how it works. In the Agile method, it's a bit different. And the earlier you do them, the more time you save, in fact. And what's also important when we do tests is to reassure users. Because we're not there to judge a user's skills, but we're really there to judge the product. And I say this every time because... I see it clearly when we do user tests, people always feel a bit judged. There's a bit of a notion of 'I hope I'll manage.' Sometimes, I have people who tell me that. Whether you manage or not, it doesn't matter. We're not here to judge what you did. We're here to judge the work we did and tell you, 'If you don't manage, it's because it doesn't work, there's a problem.' We'll rework it.
So when we do tests, we film the user's screen to get feedback afterward, but we also film via the webcam. As I told you earlier, there's a difference between what people do and what people say. And in fact, I find it's a very good example because I remember very well when we did this test, When you administer the test, you're next to your user. You're not there watching their face, watching the screen. You can't. You administer the scenarios, you accompany them during their test. But afterward, you'll rewatch and recontextualize with each of the scenarios. And you'll see their face. Because I can see she tells me, yes, yes, it's going well, everything is going well. But her face tells me, no, no, it's not going well. And in fact, it's difficult for someone to criticize your interface, to criticize your product, to say, no, it's useless, I don't understand. Because if you say, I don't understand, we'll think we're going to judge you. I don't understand because maybe I'm not smart enough, I missed something. It has absolutely nothing to do with that. But as a result, people don't dare. That's why we always try to combine the two. There's a debriefing time where I'll rewatch all the videos, try to interpret the expressions they might have combined with what they say, because that information is important, really.
As I told you about the user test part, It happens in three stages. There's a preparation phase. The preparation is when we work on our test protocol. We work on a test protocol because we estimate we'll do it often, we hope to do several user tests on a product because we work iteratively. The goal is really to develop a product, release it, evaluate it, and then iterate and start over each time. So the preparation is my protocol. My protocol tells me how many users I'm going to recruit, what their profile is, and what equipment I'm going to use, whether I film them, what scenario I'm going to test, what hypotheses I've made about my scenarios, what I'm going to evaluate, and for what improvement. So it's something quite comprehensive, but it will be useful to you; you'll be able to reuse it for your other tests, to have a common base and track the evolution, in fact. Then, there's the administration phase. We estimate that a user test should last one hour. Beyond that, it's long, it's very long. Already, one hour is good. Because generally, I think there's a quarter of an hour for introductions, a small interview, half an hour of scenarios. And afterward, we still have time, we have a quarter of an hour left for everything related to the interview and conducting the test.
And the debriefing phase, because once you've conducted the tests, you'll have to rewatch the videos to identify where all the problems encountered by your users are, identify the... major problems and also find solutions. So, that's my problem that was encountered by everyone. Here are the solutions we could implement to address this problem.
So, what types of tests exist? We have guerrilla testing. Guerrilla testing, I don't know if you've ever heard of it, But the idea is to very quickly get information and user feedback. So it's often done in a Starbucks. So why Starbucks? Because people who go to Starbucks often go there with their laptop because they want a Wi-Fi connection, they want to sit down and work. So they have time. Whereas if, for example, you go to a train station, people don't have time at a train station. They're waiting for their train to be displayed and to go catch their train. You really need to be in a place where people have time to give you. Even if it's very quick, even if you say it's only 10 minutes, psychologically, people think it won't last 10 minutes, it will be longer. And in exchange, what we do is offer a coffee. It's always a bit of a reward. You give me 10 minutes of your time, I'll buy you a coffee, and that's how we arrange it. So you also need to—you haven't prepared a test protocol like for eye-tracking tests, for example. However, you still need to know what you want to test and what potential problems you've identified and why you want to test them.
So that's guerrilla testing. Then, we have everything related to A/B testing. A/B testing is funny; when I started in UX, all my clients told me, 'We're going to do A/B testing.' I said, 'But no, why would we do A/B testing?' It depends on the context, it depends on what we want to test. But A/B testing is the most well-known. I don't know why. A/B testing can be done qualitatively or quantitatively. It's about being able to compare two interfaces. Often, we want to compare two interfaces to get the one with a higher conversion rate.
When you do it qualitatively, you can do it in guerrilla testing mode because it's interesting to know why the majority of people chose solution A over B, because it will help you avoid repeating mistakes later. And otherwise, you have the quantitative mode. There are sites, for example, like Usability Hub, that allow you to publish the two interfaces. And people, they go and they vote A, B, A, B. They choose the interface they prefer. The disadvantage here is that you don't know why people voted A or B. There's still some information missing. But it still gives you feedback. For example, Netflix does A/B testing. So, the way they do it is that they publish—well, they develop the images, they put them on their site, and they see the conversion rate. I don’t remember how long they leave them on their site, but it allows them to see... Indeed, how long... Well, not how long, sorry.
It allows them to see which image is clicked the most.
And I’ll finish with eye tracking. So, eye tracking is measuring the user journey through the eyes. So, we can measure where the gaze lands and how long the gaze stays on each spot.
So, it gives you heatmaps, for example. So, it allows you to see, for example, in my example, that the header and the footer are the most viewed areas. However, you need to recontextualize them. Because in my scenario here, it was: you want to subscribe to a newsletter. So, we see that when people want to subscribe to a newsletter, they first go to the top and then go directly to the footer. If you take it out of context, someone could tell me, 'Ah, but when people arrive on a homepage, they go to the header and the footer.' Well, no, actually. You need to recontextualize it to the scenario.
Another representation is an intention point map. Now, it looks a bit messy, but actually, it’s not that messy. There’s a color code for each user. We know the order in which they visited each point on the page. And depending on the size of the point, it means they spent more time in those areas. So, you see that with eye tracking, we’re dealing with much more precise data. For example, guerrilla testing is more about impressions. With eye tracking, I can say exactly that, in fact, my button might need to be moved a little to the right—it’s a bit too centered here—or that I should move it up a bit. Really focusing on finer levels of the interface. Or perhaps realizing there are design issues that need to be reworked.
The advantage of eye tracking is that it can be used not only on interfaces but also in physical spaces. For example, stores use eye tracking a lot to see how people navigate. It’s also used sometimes in museums to see how people orient themselves with information panels, how they navigate a space, or even within a store aisle. How do people see the products? And on a product, what do I look at first? Am I drawn to the color or the name of the product? That’s all that kind of information. So, it’s something that can be adapted.
And finally, as I told you, when you want to measure the user experience, You first need to define your objectives, because you need to know what information you want, Because what impacts you want it to have, what improvements you can make to your product afterward. And above all, use the right tools and the right methods at the right time. Thank you.

Transcript (Translated)