• KLF15
• University Of Kentucky
• Thomas Chaillou

Objective: The purpose of this study was to investigate the effects of the duration of expressions on the recognition of microexpressions, which are closely related to deception. Methods: In two experiments, participants were briefly (from 20 to 300 ms) shown one of six basic expressions and then were asked to identify the expression. Results: The results showed that the participants’ performance in recognition of microexpressions increased with the duration of the expressions, reaching a turning point at 200 ms before levelling off.

The results also indicated that practice could improve the participants’ performance. Conclusions: The results of this study suggest that the proper upper limit of the duration of microexpressions might be around 1/5 of a second and confirmed that the ability to recognize microexpressions can be enhanced with practice.

1. Introduction During interpersonal communication, it is important to be aware of others’ emotions (Matsumoto et al.,; Ekman, ). As a Chinese saying goes, “Look at the weather when you step out; look at men’s faces when you step in.” However, not everyone shows their feelings on their faces. Some individuals may suppress true emotion and express a false facial expression. There are also some situations in which social norms force people to conceal, mask, or inhibit true feelings in ways depending on politeness, context, culture or their status (Ekman, ). Fortunately, the suppressed expressions can be expressed subconsciously in the form of microexpressions and therefore can be detected by a skilled observer (Ekman and Friesen,; Ekman, ).

A microexpression is a brief and subtle facial movement which usually lasts for from 1/25 to 1/5 of a second revealing an emotion a person is trying to conceal (Ekman and Friesen,; Ekman,; ). It expresses one of the six universal emotions: disgust, anger, fear, sadness, happiness, and surprise. This kind of facial expression usually occurs in high stake situations, where people have something valuable to gain or lose (Ekman et al., ). According to Ekman et al.

, a microexpression is considered to reflect a human’s real intent, especially one of a hostile nature. A microexpression, therefore, can be an essential behavioral clue for lie detection and can be employed as a means of detecting a dangerous demeanor (Metzinger,; Schubert,; Weinberger, ). Some extreme actions like terrorist attacks around the world necessitate the use of various technologies to detect dangerous individuals and prevent such actions. Analysis of microexpressions was found to be a suitable approach (Weinberger, ). Haggard and Isaacs first discovered microexpressions which they termed “micromomentary” expressions, and Paul EKMAN then formally named them microexpressions (Ekman and Friesen,; Ekman, ). Today, there are two main research groups in this field: one led by Paul EKMAN and including Mark G.

FRANK and Maureen O′SULLIVAN, and another led by Porter (Porter and ten Brinke, ). Although few peer-reviewed studies about microexpressions have been published (Porter and ten Brinke,; Vrij et al.,; Matsumoto and Hwang, ), applications of microexpressions seem to be booming in the USA (Hoffman,; Mervis, ). A microexpression and a macroexpression differ only in their duration (Ekman and Friesen,; Ekman, ).

However, there have been at least six estimates of the duration of a microexpression: (1) “A micro expression flashes on and off the face in less than one-quarter of a second” (Ekman, ); (2) “Microexpressions (that last 1/3 sec. Materials and procedure A Lenovo computer with a 17-inch cathode-ray tube (CRT) monitor running at a refresh rate of 85 Hz and the software package E-Prime (version 1.1) were used for stimuli presentation and data collection. The target stimuli were pictures of six basic expressions posed by two human models which were selected from the DFAT-504 database (Kanade et al., ) and trimmed to 192 pixels×220 pixels. All stimuli were presented on a uniform silver gray background, which remained silver gray throughout the experiment.

Before the formal experiment, there were two practice runs. Each run contained six pictures of facial expressions including happiness, disgust, anger, fear, sadness, and surprise.

Each picture was presented on the screen for 2 s. Participants were asked to identify each expression by selecting from among the six emotional labels. After giving their answers, participants were given feedback to inform them of the correct answer or to confirm a correct answer. At the end of each practice run, the program reported the total accuracy rate. The program would not continue until the accuracy rate of recognition of microexpressions reached 100%. Errors made by the participants had two sources: first, that the participants did not recognize the expression shown in the picture, or second, that they did not fully process the pictures during the brief exposure time. The practice runs eliminated the first possibility.

After recognition practice, two experimental conditions (the BART and METT paradigm conditions) were tested. In the BART condition, a “+” as the fixation appeared at the center of the computer monitor screen for 1 s; then one of the six basic expressions was presented in the center of the screen for 40, 120, 200, or 300 ms. Participants were asked to select one of the six emotional labels using a mouse. Under the condition of the METT paradigm, a neutral expression was first presented in the center of the screen for 2 s. Then one of the six basic expressions flashed briefly for 40, 120, 200, or 300 ms. Exposures of from 40 to 200 ms are deemed to be microexpressions (Ekman,; ). Finally, the neutral expression was presented for another 2 s.

At the end of the presentation the participants had to identify the fleeting expression by selecting one of the six emotional labels. After the practice runs involving recognizing the two models’ expressions, twelve trials were run without feedback on accuracy to familiarize the participants with the formal experimental procedures. Then, the formal experiments were run with the two conditions (BART and METT, blocked). No feedback was given during each formal experiment. Each of the two formal experimental blocks consisted of 384 trials (4 durations×2 models×6 basic emotions×8 replicates).

Results A 2 conditions (BART and METT paradigm)×4 durations (40, 120, 200, and 300 ms)×6 expressions (disgust, anger, fear, sadness, happiness, and surprise) repeated measures analysis of variance (ANOVA) was conducted with the participants’ accuracy score (%) entered as the dependent variable. The mean accuracies of the two conditions are shown in Fig. The main effect of condition was not significant F(1, 10)=0.962, P=0.350, η p 2=0.088. There was a significant main effect for duration F(3, 30)=109.027, P0.684). Accuracy of recognition of each expression under two conditions (BART and METT paradigm) in Experiment 1 In Table, the accuracy of recognition of expressions of each participant was computed by calculating the ratio of the cases in which one facial expression was classified correctly as one of the six expressions and the total number of classifications for that facial expression. Then, the accuracy rates of all participants were averaged to gain the mean accuracy for each expression.

For instance, in this experiment the number of times that one participant labeled the facial expression ‘happiness’ correctly (122) was divided by the total number of classifications for that facial expression (128). So, the recognition accuracy for happiness for this participant was 95.3%, and so on. Finally, the accuracy rates of all participants were averaged to give the mean accuracy of 86.8% for ‘happiness’ in Experiment 1. The percentages in the diagonal line are the mean accuracies of recognition for each expression across the two conditions.

Discussion This experiment investigated the effects of the duration of expressions on the recognition of microexpressions. In contrast to general “macroexpressions”, the participants’ performance was impaired by the short duration of the microexpressions. As their performance showed no significant difference only after durations of 200 and 300 ms, the duration of 200 ms appeared to be a critical turning point in defining microexpressions.

The participants showed confusion between anger and disgust, fear and surprise, and sadness and disgust (Table ). Happiness was rarely mistaken for other expressions and therefore was the easiest expression to recognize, while anger and fear were distinguished relatively poorly (Tables and ). 3. Experiment 2 Ekman reported that if people had as little as 40 min of microexpression training, they could, on average, improve their performance from about 30% to 40% from the pretest to the posttest in an METT test. In Experiment 2, to test the effects of practice and to find out whether people would still be able to identify microexpressions when the presentation duration was below the lower limit of the duration of microexpressions as defined by Ekman and Friesen , the participants were required to achieve 100% accuracy in two consecutive runs during a practice session to ensure intensive practice and recognition of expressions presented for 20 ms. Moreover, the experiment aimed to further test the effects of the duration of expressions on the performance of recognition of microexpressions and to find the proper upper limit of microexpressions. Eight levels of duration (20, 40, 80, 120, 160, 200, 240, and 280 ms) were employed in this experiment. Materials and procedure The materials and the procedures were similar to those used in Experiment 1, except for the following changes.

KLF15

Two different models’ (one male and one female) six basic expressions were used and the practice of expression recognition did not cease until participants had correctly identified all six basic expressions in two consecutive runs (once classification accuracy reached 100%, the program did not proceed to the next run immediately but repeated the previous run to eliminate the possibility of a correct guess). In addition, to make the practice more intensive, a block of mixed recognition practice was added, in which the two models’ expressions were mixed. Only when the participants had achieved 100% recognition of the model’s six expressions in two successive runs would the program advance. In both the BART and METT paradigm conditions, the levels of duration of the six basic expressions were changed to eight levels: 20, 40, 80, 120, 160, 200, 240, and 280 ms. Results The mean accuracies of the two conditions are shown in Fig. A 2 conditions×8 durations×6 expressions repeated measures ANOVA was used to analyze the data. Each of the factors and the combination of factors had significant effects.

There were significant main effects for condition, duration, and expression type F(1, 11)=51.932, P0.455). Furthermore, the simple effect analysis for the effect of duration in the BART condition showed that there were only two significant effects for pairs of 20 and 240 ms ( P=0.024), and 40 and 240 ms ( P=0.003). An analysis of simple effects of duration in the condition of the METT paradigm revealed no significant differences in pairs of duration after 120 ms (all P0.147).

A simple effect analysis was also employed to investigate the effect of condition within each level of duration. There were significant simple effects before 160 ms (all P0.139). The two curves (BART and METT) were almost superimposed at 200, 240, and 280 ms (Fig. Accuracy of recognition of each expression under two conditions (BART and METT paradigm) in Experiment 2 To explore the effects of practice, we compared the results from the two conditions in Experiments 1 and 2 (Fig. Under the condition of the METT paradigm, the accuracy rates for the levels of 20 ms (Experiment 2) and 40 ms (Experiments 1 and 2) were compared with the accuracy expected by chance, which was 1/6 (if the participants did not recognize anything, they would by chance obtain the correct answer one in six times simply by guessing one of the six expressions). An independent sample t-test was employed to calculate the values of t for the difference between the means for each of the three accuracy levels and the accuracy expected by chance.

The results showed all the comparisons were significant t(11)=8.349, P. Discussion Again, the duration of expressions had significant effects on the recognition of microexpressions.

Furthermore, the participants’ performance improved with practice (Fig. However, the increasing trends did not differ between the conditions of the two experiments. The results also showed that, with considerable practice, participants could identify microexpressions even when the duration of expressions was below the lower limit of microexpressions as defined by Ekman (40 ms). The results suggest that the perception of human facial expression information is very fast and can be tuned by experience. The pattern of recognition accuracy found in this experiment was similar to that found by Matsumoto et al.

However, the accuracy of participants in our study was much higher, perhaps because we employed fewer models (two). In general, the expression of surprise was the easiest to recognize, followed by that of happiness. Fear was the most difficult expression to recognize. These results were similar to those of McAndrew. It could be argued that our experiments employed pictures of small size which resulted in poor performance in recognizing the expressions. However, there is evidence to suggest that image size does not affect the recognition of expressions (Ekman et al., ). 4. General discussion We explored mainly the upper limit of duration of microexpressions.

University Of Kentucky

In both experiments, the duration of the expressions affected the recognition of the microexpressions. Training in recognizing expressions also played an important role in the identification of the microexpressions. The results indicated that there was a turning point at 200 ms in Experiment 1 and at 160 ms in Experiment 2, suggesting that the critical time point differentiating microexpressions from macroexpressions was about 200 ms and most likely less than 200 ms. The results showed that there was a large difference in the accuracy of recognition of microexpressions between the conditions of BART and METT, particularly at short durations (40 ms in Experiment 1 and less than 160 ms in Experiment 2).

The reason may lie in the neutral image under the condition of METT, which played the role of a mask, preventing the fleeting expression from remaining on the retina. Thus, for a short duration, the processing of microexpressions could be more impaired by a mask (the neutral images) under the condition of METT paradigm, compared to the condition of BART without a mask. With an increase in duration, the effects of afterimage of a brief expression may decrease (Breitmeyer and Ogmen, ) and the information of the microexpression may already be transferred to more durable memory storage. Thus, the mask may then have little effect on the processing of microexpressions under the condition of the METT paradigm. Consequently, the performance under the two conditions showed no significant difference when the duration of the microexpressions was longer.

The simple effect analysis showed that the two conditions were different only after a short exposure (40 ms) when there had been no extensive practising (in Experiment 1). When the extent of practice was increased (in Experiment 2), the difference between the two conditions persisted until 160 ms, suggesting that adding two neutral images, one before and one after the microexpression, was necessary when the duration was short.

Thus, for the purpose of testing the ability to recognize microexpressions with a duration of less than 200 ms, the METT paradigm was appropriate because it was more sensitive to the change of duration which was the only difference between microexpressions and macroexpressions (it also had a wider range of accuracy). The ability to identify microexpressions is related to the detection of deception. The main aim of microexpression studies is to find indicators of deception via microexpressions. Unfortunately, evidence about the link between microexpressions and deception is far from conclusive. However, microexpressions may be the most promising approach to detecting deception (Ekman, ). According to Weinberger , the Transportation Security Administration in the USA has already employed a technique called screening passengers by observation techniques (SPOT) which is largely based on the findings from microexpression research.

Department of Defense has funded David MASUMOTO, a longtime collaborator with Paul EKMAN, in his work on microexpressions, for what he calls “behavior-detection techniques” (Mervis, ). Now there are at least 3 000 behavior detection officers deployed at 161 airports in USA (Lord, ).

Thomas Chaillou

pointed out that there is a physiological basis for microexpressions: a pyramidal motor system drives voluntary facial actions and an extrapyramidal motor system drives the more involuntary and emotional facial actions which can also be viewed as the physiological basis of the Darwin’s inhibition hypothesis (Porter and ten Brinke, ). Thus, it is reasonable to regard the analysis of microexpressions as a useful and promising tool for detecting deception. There has been some debate and criticism about microexpression analysis and its applications (Committee on Science, Space and Technology, ). First, many scientists have argued that microexpressions and their applications have not been subjected to controlled scientific tests (Lord,; Vrij et al.,; Weinberger, ). Empirical studies are needed to establish the validation for linking microexpressions to deception. Also, the application of microexpressions to airport security is not very effective as less than 1% of suspects have subsequently been arrested (Weinberger, ).

An appropriate attitude to microexpressions may be similar to that proposed by Porter and ten Brinke , who concluded that nonverbal cues can only assist investigators to detect deception. Many aspects of microexpressions need to be elucidated in the future including the lower limit, individual differences in expression and recognition of microexpressions (Bond and DePaulo,; O′Sullivan et al.,; Wang and Fu,; Warren et al., ), effective methods of training and the retention of training effects (Hurley,; Matsumoto and Hwang, ), and automatic brief facial expression analysis systems (Polikovsky et al.,; Wu et al., ). As for individual differences, in groups closely related to deception detection, such as crime interrogators, national security personnel, visa interviewers, sales personnel, negotiators, and mental health professionals, the expression and recognition of microexpressions might not be the same as for other people. While microexpressions have some valuable applications in the field of security, they may also have applications in healthcare and forensic contexts (Porter et al., ). To use microexpressions widely, it is important to design and develop an automatic brief facial expression analysis system in the field of computer vision to aid trained or untrained people to detect microexpressions and improve hit rates.

The questions and debates about microexpressions can be solved by conducting much more empirical research which will fill the gap between knowledge (production features, dynamic features, recognition of microexpressions filmed in real life situations, etc.) and applications (e.g. Aviation security), and establish a more stable connection between microexpressions and deception (the main purpose of studies on microexpressions). In conclusion, this study explored the upper limit of duration of microexpressions and had two major findings. Firstly, to some degree, a short training program can improve lay individuals’ ability to recognize microexpressions. Secondly, the accuracy of recognition of microexpressions is a function of the duration of the expressions and reaches a turning point at 200 ms (perhaps less than 200 ms) and then levels off, suggesting that the proper upper limit of duration of microexpressions may be around 1/5 of a second.

Much more research is needed to move towards a more complete understanding of microexpressions.

Responding Effectively Training Tool: Law Enforcement For the benefit of officers and citizens alike. Ekman gives his insight on interrogation techniques. Success Stories “Dr. Ekman’s training tools have helped me read emotions, predict reactions, and interact with others in a more proficient and professional way.” -Joe S., Residential Manager, boarding school for underprivileged kids “I have been a great admirer of Dr. Ekman’s work for many years. As a private investigator, I use the knowledge gained from the tools every day, and am impressed with their usefulness.” -Matt M., Private Investigator “I’m very sincere when I say that Dr. Ekman’s research has been so helpful for emotional identification and regulations with the autism population, and we’ve only scratched the surface of this work.” -Emily W., Director of Education.