Multimedia ResearchISSN:2582-547X

An Ecological Approach to Measuring User Experience (UX) from Facial Expressions

Abstract

A system for assessing UX issues automatically is proposed in this paper. The facial behavior of an individual performing a specific activity is tracked in real-time with software that tracks facial motion features. Evaluated with the conventional studies, this approach has several advantages: ease of deployment in the user's natural setting; avoidance of invasive devices; and severe cost minimization. An evaluation of the user experience of the system was conducted using 144 videos that showed 12 users executing three tasks on four commercial media players. To predict the presence/absence of UX issues based on the tracker's features, we used different machine learning algorithms. We show promising outcomes that open up opportunities for automated real-time UX estimation in an environmental context.

References

  • Hassenzahl, M., & Tractinsky, N. (2006). User experience-a researchagenda. Behaviour & information technology, 25(2), 91-97.
  • Benedek, J. and Miner, T.: Measuring Desirability: New methodsfor evaluating desirability in a usability lab setting. In: Proceedingsof Usability Professionals Association, pp. 8-12 (2002)
  • Desmet, P.M.A.: Measuring emotion; development and applicationof an instrument to measure emotional responses to products.In: M.A. Blythe, A.F. Monk, K. Overbeeke, & P.C. Wright (Eds.),Funology: from usability to enjoyment. pp. 111-123. Dordrecht:Kluwer Academic Publishers (2003)
  • Katherine Isbister, Kia H ̈o ̈ok, JarmoLaaksolahti, and MichaelSharp.: The sensual evaluation instrument: Developing a transcultural self-report measure of affect. Int. J. Hum.-Comput. Stud.65(4), 315-328 (2007)
  • Derbaix, C.M.: The impact of affective reactions on attitudes towardthe advertisement and the brand: A step toward ecological validity.Journal of Marketing Research. 32(4), 470-479 (1995)
  • Den Uyl, MJ and Van Kuilenburg, H.: The FaceReader: Online facialexpression recognition. In: Proc. Measuring Behaviour, pp. 589-590.(2005).
  • Mehrabian, A.: Pleasure-arousal-dominance: A general frameworkfor describing and measuring individual differences in temperament. Current Psychology. 14(4), 261-292 (1996)
  • Chorianopoulos, K. and Spinellis, D.: User interface evaluation ofinteractive TV: a media studies perspective. Universal Access in theInformation Society. 5(2), 209-218 (2006)
  • Swindells, C. and MacLean, K.E. and Booth, K.S. and Meitner, M.:A case-study of affect measurement tools for physical user interfacedesign. In: Proceedings of Graphics Interface 2006, pp. 243-250.Canadian Information Processing Society (2006)
  • Bradley, M.M. and Lang, P.J.: Measuring emotion: the self-assessment manikin and the semantic differential. Journal of behavior therapy and experimental psychiatry. 25(1), 49-59, Elsevier(1994).
  • Millard, N. and Britton, R.: Calling time: an effective and affectiveevaluation of two versions of the MIT Beer Game. In: Proceedingsof the 21st British HCI Group Annual Conference on HCI 2008:People and Computers XXI: HCI... but not as we know it-Volume2, pp. 75-77. British Computer Society (2007)
  • Scherer, K. R.: What are emotions? And how can they be measured? Social Science Information, 44(4), 693-727 (2005)
  • Dawes, R.M.: Fundamentals of Attitude Measurement, Wiley, NewYork, NY (1972)
  • Branco, P., Firth, P., Encarnao, L.M., Bonato, P.: Faces of emotionin human-computer interaction. In: CHI’05 extended abstracts onHuman factors in computing systems, pp. 1236-1239. ACM (2005)
  • Hazlett, R.L. and Benedek, J.: Measuring emotional valence tounderstand the user’s experience of software. International Journalof Human-Computer Studies. 65(4), 306-314 (2007)
  • Ravaja, N. and Saari, T. and Laarni, J. and Kallinen, K. and Salminen, M. and Holopainen, J. and Jarvinen, A.: The Psychophysiologyof Video Gaming: Phasic Emotional Responses to Game Events.DIGRA Conf. (2005)
  • Ravaja, N., Turpeinen, M., Saari, T., Puttonen, S., Keltikangas, J.:arvinen, L.: The psychophysiology of James Bond: Phasic emotionalresponses to violent video game events. Emotion 8( 1), 114–120(2008)
  • Zaman, B. and Shrimpton-Smith, T.: The FaceReader: Measuringinstant fun of use. In: Proceedings of the 4th Nordic Conference onHuman-computer interaction: changing roles, pp. 457-460. ACM(2006)
  • Westerman, S., Sutherland, E., Robinson, L., Powell, H., Tuck, G.: AMulti- method Approach to the Assessment of Web Page Designs.In: Proceedings of the 2nd international conference on AffectiveComputing and Intelligent Interaction, pp. 302-313. Springer-Verlag(2007)
  • Mahlke, S. and Minge, M., Thuring, M.: Measuring multiplecomponents of emotions in interactive contexts. In: CHI’06 extendedabstracts on Human factors in computing systems, pp. 1061-1066.ACM(2006)
  • Ekman, P., & Friesen, W.: Facial Action Coding System: A tech-nique for the measurement of facial movement. Palo Alto, CA:Consulting Psychologists Press (1978)
  • Ryan, A., Cohn, J. F., Lucey, S., Saragih, J., Lucey, P., la Torre, F. D.,and Ross, A.: Automated facial expression recognition system. In:IEEE International Carnahan Conference on Security Technology(2009)
  • Kuilenburg, H. V., Wiering, M., and Uyl, M. D.: A model basedmethod for automatic facial expression recognition. In: Proceedingsof the European Conference on Machine Learning, pp. 194–205(2005)
  • Tao, H., and Huang, T.: A piecewise bezier volume deformationmodel and its applications in facial motion capture. Advances inImage Processing and Understanding. 39–56 (2002)
  • Joho, H., Staiano, J., Sebe, N., and Jose, J.: Looking at the viewer:analysing facial activity to detect personal highlights of multimediacontents. Multimedia Tools and Applications. 51, 505–523 (2011)
  • Lavie, T. and Tractinsky, N.: Assessing Dimensions of PerceivedVisual Aesthetics of Web Sites. International Journal of Human-Computer Studies. 60(3), pp. 269-298 (2004)
  • Tractinsky, N. and Zmiri, D.: Exploring attributes of skins aspotential antecedents of emotion in HCI. Aesthetic computing. 405-422 (2006)
  • Lund, A.: Measuring usability with the USE questionnaire. Usability and User Experience. 8( 2)(2001)
  • De Angeli, A. Sutcliffe, A. Hartmann, J. (2006).: Interaction, usability and aesthetics: what influences users’ preferences? In: DISConference Proceedings, pp. 271-280. ACM (2006)
  • McDaniel, B., D’Mello, S., King, B., Chipman, P., Tapp, K.Graesser, A.: Facial Features for Affective State Detection in Learning Environments. In: CogSci07, pp. 467-472, (2007)
    •

Access options

  • Price $
  • Offer $
Buy article PDF

DOI : https://doi.org/10.46253/j.mr.v5i3.a4

Author information

Affiliations

Shanto-Mariam University of Creative Technology, Bangladesh

Publisher Information

  • Received

    26 Mar

  • Revised

    10 June

  • Accepted

    7 July

Keywords
User; Automated; Facial; Media; Tasks

Publisher : Resbee Info Technologies Pvt Ltd