Investigating the Effect of Auditory Cues on Food Perception by Measurement of Electrophysiological, and Emotional Responses

Abstract

Auditory elements such as sounds and music, as well as visual stimuli, can significantly influence emotions and, consequently, consumer behaviour. This research seeks to provide a new perspective on understanding the temporal dynamics of food perception and emotional responses by combining subjective evaluations with objective measures such as electrophysiological responses. The primary aim of this research was to examine the influence of auditory cues on food perception through the evaluation of electrophysiological and emotional responses. The research is grounded in the premise that sensory perception, emotional responses, and electrophysiological responses are interconnected and can be influenced by external stimuli such as auditory and visual cues. The research was divided into four chapters, with the studies focusing on: 1) examining the impact of negative and positive valence combinations of musical and non-musical sounds on participants' affective states and their perceptions of chocolate ice cream; 2) examining the effects of auditory stimuli on the consumption of chocolate milkshake, and liked (eggplant) and less liked (asparagus) vegetable ice creams, representing emotional and non-emotional foods respectively, which expands our understanding of how sound influences our reactions to different types of food; 3) understanding the influence of music and sound, differing in arousal levels, on temporal changes in food perception and the associated emotional reactions, assessed by both subjective and objective (i.e., electrophysiological) methodologies; and 4) examining the influence of video clips that evoke different emotions on the perception of chocolate ice cream and further examine the relationship between self-reported emotions and their electrophysiological correlates using Bayesian Network (BN) modelling. The first working chapter aimed to explore how auditory cues, such as sounds and music varying in valence, impact our perception of food. Specifically, it sought to examine the effects of mixed musical and non-musical sounds with negative and positive valences on participants' emotional states and their perception of chocolate ice cream. The findings revealed that consuming ice cream while listening to liked music (LM) and the combination of liked music and pleasant sound (LMPS) resulted in more positive emotions compared to just listening to pleasant sound (PS). Additionally, LM and LMPS elicited stronger positive emotions than PS. Consuming ice cream under the LM condition resulted in the longest duration of perceived sweetness, lasting from 0 seconds to 27.28 seconds (61% of the standardised time) over an overall duration of 45 seconds. Meanwhile, in the PS condition, cocoa became significantly dominant from 15.75 seconds to 45 seconds, persisting until the end of the duration. Similarly, in the LMPS condition, cocoa emerged as the next dominant attribute from 27.9 seconds to 45 seconds, persisting until the end of the duration. Bitterness and roasted flavours were dominant under the disliked music and unpleasant sound (DMUS) and disliked music (DM) conditions, respectively. Positive emotions of “calm”, “at ease”, “pleasant”, “relaxed”, “satisfied”, “joy”, “interested”, and “active” were strongly associated with the perception of sweetness and cocoa while consuming chocolate ice cream in the positive valence auditory condition. Conversely, negative emotions of “uneasy”, “unhappy”, “anxious”, “tired”, “lonely”, “boredom”, “restless”, “annoyed”, and “unable to concentrate” correlated with bitter and roasted flavours under negatively valenced auditory conditions. In the second working chapter, the study aimed to examine the impact of pleasant and unpleasant sounds on the perception and emotional responses during the consumption of chocolate milkshake (CM) and vegetable ice creams, representing emotional and less-emotional foods, respectively. Fourteen sounds (seven pleasant and seven unpleasant) were selected through a preliminary study. Participants rated the pleasantness of 20 sound samples using an unstructured line scale (0 = most unpleasant, 5 = neutral, 10 = most pleasant). The seven sounds with the highest ratings were categorized as pleasant sounds (e.g., "waterfall and jungle sounds", "bird chirping", "river flowing and birds singing sounds", "waterfall", "rain and thunder", "waves hitting sea rocks", and "burning fire"), while the seven with the lowest ratings were categorized as unpleasant (e.g., "outdoor restaurant dining", "traffic", "indoor marketplace", "inside of airport", "busy traffic", "bar", and "fast food restaurant background"). The temporal changes in perceived flavour were determined using the temporal dominance of sensations (TDS) method. Additionally, ratings of food liking, affective states, and emotional responses were obtained following the consumption of the two foods. Canonical variate analysis (CVA) was employed to further evaluate the flavour of CM, most liked vegetable ice cream (LV), and less liked vegetable ice cream (LLV) consumed under varying auditory conditions. The results showed that consumption of CM and LV evoked positive emotions of “interested”, “active”, “satisfied”, “joy”, “pleasant”, “enthusiastic”, “energetic”, “relaxed”, “at ease”, and “calm” and increased dominance of sweetness and milky attributes, respectively, when listening to pleasant sounds. In contrast, CM and LLV evoked negative emotions of “unable to concentration”, “restless”, “unhappy”, “anxious”, “tired”, “uneasy”, “annoyed”, and “boredom” that were associated with thickness, cocoa, and earthy attributes when listening to unpleasant sounds. In the third working chapter, the study aimed to understand the influence of music and sound, varying in arousal levels, on the perception of chocolate ice cream and the associated emotional responses. The results revealed that exciting sounds were correlated with low valence and high arousal, while calm sounds were correlated with high valence and low arousal. Chocolate ice cream samples consumed under Calm music (CM), calm sound (CS), and a combination of calm music and sound (CMCS) were correlated to perceptions of sweet, creamy, cocoa, and vanilla, affective responses of valence, emotions of “pleasant”, “joy”, “relaxed”, “calm” and “at ease”, as well as physiological measure of respiratory rate amplitude (RSP amp). In contrast, ice cream samples consumed under the excited conditions (excited music (EM), excited sound (ES), and the combination of exciting music and sound (EMES) were correlated with perceptions of bitter, milky, and roasted, arousal, and emotional terms that included “active”, “enthusiastic”, “energetic”, “excited”, “unable to concentrate”, “annoyed”, “restless”, “tired”, “uneasy”, “anxious”, “boredom”, and “lonely”. In the final chapter, the study aimed to examine the influence of 60-second video clips that evoke different emotions on the perception of chocolate ice cream and to investigate the relationship between self-reported emotions and their electrophysiological correlates using Bayesian Network (BN) modelling. The results revealed that videos evoking positive emotions of “quiet”, “at ease”, “relaxed”, and “uninspired” were associated with an increased perception of sweetness and creaminess, whereas video clips eliciting negative emotions of “happy” and “enthusiastic” were linked to a increased perception of bitterness. Additionally, changes in electrophysiological measures aligned with the different emotional states evoked by the video clips and consumption of chocolate ice cream. Videos that are high in arousal resulted in increased heart rate and skin conductance, while videos that induced calmness had the opposite effect. The Bayesian Network modelling also indicated connections between emotional states and electrophysiological responses in relation to flavour perception. The model suggested that heart rate (HR) and skin conductance (SC) were positively correlated with positive emotions, enhancing sweet and milky attributes, respectively. Blood volume pulse (BVP) amplitude had a negative correlation with arousal and chocolate flavour perception. Furthermore, emotions that included “calm” and “enthusiastic” were positively associated with creaminess, while “tense” was correlated with the roasted attribute. The research findings from the four working chapters provide comprehensive insights into the impact of auditory and visual stimuli on the perception of food and the corresponding emotional and physiological responses. The studies explored the influence of music, sound, and video clips on flavour perception, demonstrating that different stimuli can evoke varying emotional states and affect the perceived sensory attributes of foods. The results highlighted the intricate relationships between sensory stimuli, emotional responses, and electrophysiological measures, showing how valence and arousal levels of auditory cues can modulate electrophysiological measures. Understanding these complex relationships can lead to the development of food experiences that enhance consumer enjoyment.