What is the perfect cookie? How do you decide this? Many research groups use FaceReader to investigate emotions during eating and drinking behaviour (see an example here). Generally, there are some correlations between different measures of hedonic liking (or disliking) and facial expressions. Benefits of using facial expression analysis to measure responses towards food products are that you can measure emotions automatically, and during the event (instead of having to ask afterwards). There are also some difficulties; most importantly, food causes an occlusion in front of the face. In the Virtual Emotion Reader project, the goal of VicarVision, Noldus IT and NIZO food research was to create a better measurement of emotion whilst eating. The partners in the project performed an online validation experiment where people ate cookies in front of the camera.

A Yummy Cookie Experiment

Participants were given a package with six different cookies (for example, a cholate chip cookie and an Oreo). Through FaceReader Online each participant received a link that allowed their expressions to be measured from any computer with internet and a camera. Once the link was opened, the the facial expressions of participants were analyzed while they answered a questionnaire and took a bite of the cookie. Then participants rated the cookie on crispiness, sweetness, and tastiness. The faces of 54 people from locations all over the Netherlands were analyzed while they ate and rated the cookies. The goal of the experiment was to investigate emotional expressions towards different types of cookies. It was also a test if this was feasible using an online platform, where people participated from different locations. Ultimately, the goal is also to validate and test the reliability of a newly developed FaceReader functionality that improves facial expression recognition under occlusion.

Results of the Study

People rated the cookies differently in how they tasted. They also scored differently on happy and surprised expressions, indicating that facial expressions can help to differentiate the cookies. An interesting and surprising finding was that liking of the cookie correlated slightly with angry expressions, and negatively with happy expressions. This occurred while people were answering questions; therefore, it might be related to thinking about the taste of the cookie. Just consider the thinking emoji 🤔, thinking may make you look a bit angry and sad, even when you are thinking of the lovely dinner you will cook tonight. In addition, a smile is not necessary a likely response towards food that is liked, as seen in other studies. For starters, because a strong emotion of happiness is expressed with an open mouth, something you do not want to do with your mouth full! Standard emotional labels may not always be the most optimal for tasting food and filling out questionnaires. It may be useful to create new emotion labels that apply to a specific context, something that is possible with action units (classifications of the movements of the face).

The Future of Food Research

This study showed that it is technically feasible to measure facial expressions in a home environment while people consume food or fill-out questionnaires. This approach can bring food research to a new level. The online platform makes this research much easier, because participants can stay at home while they do the tasting. In addition, with the data from this experiment we will further develop optimal techniques to measure eating behaviour. By answering questions like: How many bites do people take? How much do they chew? What emotions do they express? Food researchers try to find the crunchiest and tastiest cookies, chips, or other products. To conclude, this is a very interesting field where a lot of research is still to be done! Are you interested in doing an experiment like this? Our partner, Applegg, already offers this testing technique as a service.