在游戏中评估游戏用户体验的方法

Methodologies for Evaluating Player Experience inGame Play

在游戏中评估游戏用户体验的方法

Kimberly Chu, Chui Yin Wong, and Chee Weng Khong

Universal Usability and Interaction Design (UUID) SIG, Interface Design Department, Faculty of Creative Multimedia, Multimedia University, 63100 Cyberjaya, Selangor, Malaysia

{kimberly.chu,cywong,cwkhong}mmu.edu.my

Abstract. Player experience constitutes one of the most significant factors indetermining the success rate of games. Games which do not provide enormoususer experience usually will not gain intense interest from players. The conceptof player experience is normally interchanged with concepts such as fun, flow,fulfillment, enjoyment, engagement, satisfaction, pleasure and playability. Inthis paper,we reviewed, analyzed and discussed the different attributes andmethodologies used to evaluate player experience for game play. We concluded the finding in a playability matrix based on an analysis of methodologies forevaluating player experience in game play. The matrix was constructed fromliterature analysis, which prised of attributes consisting of qualitative andquantitative, verbal and non-verbal, empirical and non-empirical methods.

摘要：在决定游戏是否能成功占有市场的问题上，游戏玩家的用户体验是最为关键性的因素之一。对于那些没有提供极大量的用户体验效劳的游戏通常都不会引起玩家们十分密切的关注。一般来说玩家的用户体验的概念是很多概念交织的集合，比方说游戏所带给玩家的欢乐感受，使玩家心情舒畅，有成就感，有娱乐感，有参与感，有满足感，无尽的乐趣以及游戏本身的可玩性。在这篇论文当中，我们将一道回忆，分析并且讨论能够用来评估在游戏中的游戏玩家用户体验的各种不同的方法和其属性特征。基于对评估在游戏中玩家的用户体验的众多方法的分析，我们得出可玩性模型的结论。这样的模型采用文学分析的形式构建起来，这其中包括数量与质量的分析方法，言语和非言语的分析方法，经历研究和非实证研究的分析方法。

Keywords: Player experience, measurement, game play, playability. 关键词：玩家的用户体验，测量，游戏体验，可玩性。

1 Introduction 1 简介

In the past decade, game industries have been flourishing on various platforms fromconsoles, to personal puters (PC) to mobile devices. The trend is also coupledwith enhanced graphic processing power, advanced user interfaces and increasedplexity in game play. The number of players grows exponentially due to theincreased enjoyability of game play. Therefore, in Human-puter Interaction(HCI) discipline, there are a number of research contributions, which are related to themethodologies for evaluating player experience for game play.

在过去的十年的当中，游戏产业在各种各样的平台上蓬勃开展起来，从小型游戏机到个人计算机〔PC〕再到移动电子设备。这种开展趋势也随着数字图像处理的能力的不断提高，也在成倍的加快步伐，不断地更新与推进游戏中用户使用界面的图形质量，增加游戏过程中内容的复杂与多样性。游戏玩家在数量上的

-

-可修编-

. -

增涨，在极大的程度上取决于这些游戏的娱乐性的加强。因此，在人与计算机交互〔HCI〕的这个领域,已经有数量极为庞大的研究报告为人机交互的开展做出了大量奉献，这些研究报告都是关乎于评估游戏中玩家的用户体验的方法的。

The concept of player experience is often interchangeable with other concepts,such as fun, flow, fulfillment, engagement, satisfaction, pleasure and playability.Player experience does not depend on a particular mode of emotion, but enpass awide variety of emotions that contribute to game player experience. For example,while playing game, players experience fear, excitement, happiness, alert, anger,relief, pleasure, hope, discouragement, proud, joy, and distress.

游戏玩家的用户体验的概念一般来说是包含许许多多概念交织的集合，比方游戏所带给玩家的欢乐感，让玩家的心情舒畅，给玩家带来的成就感，和愉悦感，参与感，以及满足感，还有丰富的乐趣以及游戏本身的可玩性。玩家的用户体验并不仅仅是只依靠一种特定的人类的情感上的模式，而是包含了那些种类繁多的人们的情感类型，这些情感类型共同组成了游戏中玩家们的用户体验的内容。比方说，在玩游戏的过程当中，玩家们能体验那些品种繁多的人们的情感如恐惧感，兴奋感，快乐感，警觉感，愤怒感，抚慰感，愉悦感，期待感，挫败感，成就感，快乐感和压力感。

These different modesof emotions eventually form the elements of gaming experience. In addition, Clanton[3] and Federoff [6] mentioned three aspects of gaming, i.e. interface, mechanics andgame play, which highly affected the level of player’s experience. Nevertheless, theconcept of player experience is fairly difficult to distinguish as they seem to be asintangible as they are appealing, which requires hands-on skills and grasping theexperience of enjoyment [15, 18]. So far, the concept itself seems fragmented, withvarious viewpoints, and it has not defined a cohesive integrated framework [11].Sweetser and Wyeth [16] also highlighted that there was no integrated model on howplayers evaluate their enjoyment level in games. Moreover, Bernhaupt [1] addressedthere was no general framework on what methods shall be used to assess interactionconcept on games. As a result, this paper aims to review and analyze the existingmethodologies for evaluating player experience, particularly on game play.

这些各式各样的人类的情感的模式最终便会共同构成玩家们的游戏体验。另外，Clanton【3】和Federoff【6】曾提到过有关游戏用户体验的三个方面，这三个方面说的内容包括游戏的用户界面，游戏的运行机制和玩家的游戏过程，这三个方面在极大的程度上影响着游戏玩家的用户体验的水平。然而，要想区分游戏玩家的用户体验这个概念是十分困难的，由于玩家的用户体验这个概念看上起让人十分难以捉摸以至于它们是如此的吸引人去研究，要想弄清这个概念就需要我们拥有亲自动参与游戏体验的技能并能够把握好玩家们在游戏娱乐中的用户体验【15，18】。至现在为止，从许许多多不同的角度来看，玩家们的用户体验这个概念本身看上去好似零零散散，并且它任然没有给予一个连贯整合框架式的定义【11】。Sweetser和Wyeth【16】也作出这样的强调，他们表示目前来说仍然没有一个可以用来正确评价玩家们在游戏体验过程中所得到的愉快感的水平的完整的模式。除此之外，Bernhaupt还强调，对于如何评价游戏中互动的概念也没有一个总体的框架能为之提供固定的方法。总而言之，这篇论文的主要目的是回忆和分析当前已经存在的用户体验评估方法，尤其是在游戏当中。

2 A Review of Methodologies for Evaluating Player Experience 2 用户体验评估方法回忆

Numerous research studies from different domains have been conducted in academiaand industries to evaluate player experience. Csikzentmihalyi [4] introduced the studyof “flow〞 where it

- -可修编-

. -

defines the optimalexperience of enjoyment regardless of age,gender or social class. On one hand, researchers from the usability domain[5, 6, 12]adapted traditional usability methods such as heuristics to evaluate playability ingames. For instance, Malone [11] constructed a list of heuristic guidelines foreducational games, but it is based on designing enjoyable interfaces rather than playerexperience itself. Federoff [6] piled a list of heuristics by reviewing ingames industry guidelines and did a case study at a game development pany. Inaddition, Desurvire et. al. [5] created Heuristics Evaluation for Playability(HEP)from games designliterature, HEP results were then cross validated the findings fromuser studies. User studies act as a benchmark for game evaluation tools in HEP, whilstFederoff [6] and Malone [11] did not involve any users in their heuristics studies.Desurvire stressed that although certain player issues were determined through HEP;nonetheless, some of the problems could only be found by direct player observation.Sweetser and Wyeth [16] mentioned although there were many heuristics studies inliterature, however, there is a need to integrate these heuristics into a validated modelto assess player enjoyment for games. As a result, they created GameFlow andvalidated through expert evaluators’ point of view. There was still no direct userinvolvement in their method.

为了来评估用户体验的作用，大量的来自不同领域的研究学习工程被我们运用到学术界和工业界之中去。Csikzentmihalyi【4】引入了对“流动玩家〞的研究，也就是抛开年龄，性别或等级的束缚来定义的最正确的游戏乐趣体验。在一方面，可用性研究领域【5，6，12】里的研究人员将如启发式的试探这样的传统的评估可用性的方法用于在游戏中评估游戏的可玩性。例如，Mslone【11】曾经列出了一份对一些有教育意义的游戏的启发性指导方针的列表，但是它是基于有趣的用户界面的设计而不是游戏玩家的用户体验本身。Federoff【6】通过反复学习研究游戏产业中具有指导方针意义的文献编写出了一份新的启发性指导列表，并且在一个游戏开发公司做了一个个案研究。除此之外，Desurvire et. al.【5】通过结合许多游戏的设计文献创造了可玩性的启发式评价〔HEP〕的方法，可玩性的启发式评价方法的产生随后通过对用户的研究发现得到了全面的证明。在可玩性的启发式评价中用户的研究表现为游戏评估工具中的一项基准，然而，Federoff【6】和Malone【11】在他们那些启发式研究中并没有涉及到任何用户。Desurvire强调说，虽然通过实用可玩性的启发式评价方法可以确定某些玩家的问题所在，但是，一些问题却只能通过直接观察玩家得以发现。Sweetser和Wyeth【16】提到过尽管在文献中有很多启发性的研究，但是，研究人员仍然需要将这些启发性的研究整合到已经经过验证的模型之中去，才能够去评价游戏当中玩家的享受程度。

On the other hand, researchers in the physiological design domain of Humanputer Interaction (HCI) employ physiological metrics for user evaluation andoptimizing relationship between human and technological systems. Mandryk[13]conducted a study of continuous emotion in games based on physiological responsessuch as Galvanic Skin Response (GSR), electrocardiography (EKG), andelectromyography of face (EMG smiling and EMG frowning). Heart rate (HR) wasputed from the EKG signal. The author pared the modeled emotion fromusers and reported subjective evaluation on a 5-point likert scale. Affect Grid wasadapted as part of their modeled emotion. However, other perspectives have seenlimited success in adopting physiology to indentify emotional states [2]. In addition,Zaman and Smith[19] presented studies to measure fun through FaceReader, where itdistinguished six emotional states.They pared the results of FaceReader withother sources such as user questionnairesand researcher’s loggings. He found thatuser questionnaire did not provide precisionas it reflected more on the content of theapplication or the oute of the task (successfulor not) rather than fun orenjoyment. Sykes and Brown [17] investigated a player’s state of arousal, relating tothe pressure to press buttons ona gamepad. The results indicated that it was possibleto determine game player’s arousal by the pressure they used when controlling thegamepad. However, there was no measure of emotional valence for game players inthis study.Hazlett [7] used facial electromyography (EMG) as a measure for positiveand

- -可修编-

. -

negative emotional valence. On one hand, facial EMG is limited to only positiveand negative result as the method was not able to recognize discrete emotions. Usersare interviewed on which factors that were positive and negative to them based on theevents of the game.However, by using various physiological methods, they need toensure participants had no qualms being attached with wires and connectors, or else itwould appear to intrude participants emotionally, thus affecting the final result.Recent developments on questionnaire have included dimensions such as tension,frustration or negative affect. Ijsesslsteijn et. al. [9] suggested using GamesExperience Questionnaire (GEQ) as a qualitative questionnaire to understand playerexperience in games after the game play session.

在另一方面，从事人与计算机互动〔HCI〕心理设计领域的研究人员通过运用心理度量指标来作用户方面的评估以及优化人类和技术系统之间的关系。Mandryk【13】根据基于生理反响的研究，如皮肤电反响〔GSR〕，心电图扫描法〔EKG〕，和面部肌电描记法〔面部肌电描记法中的笑容表情和面部肌电描计法中的皱眉表情〕对正在游戏中的玩家持续的情感做了一系列实验研究。肌电描记法发出的信号会将测试者的心率〔HR〕输入到电脑中去。随后，研究者比照了不同的用户和情感模型，并利用5点式利开特量测量表作出主观评价的报告。这份情感表将会作为情感模型的一局部。然而，就其他方面来看，采用心理学方法来确定感情状态【2】的几乎没有成功过的。另外，Zaman和Smith【19】发表了许多关于通过解读面部表情来测量愉快感程度的研究，从而能够区别出六种情感状态。他们将解读面部表情的结果同其他的资源进展比拟，比方许许多多的用户问卷调查和许多研究人员的记录。他发现用户调查问卷并没有提供很准确的结果，这是因为它反响的更多的是这些应用程序在内容上的问题或者仅仅只反响了这个任务的结果〔是成功了还是没有成功〕，而不是反响情感上的愉悦程度和享受程度。Sykes和Brown【17】从玩家受到的应激的状态方面着手调查研究，这种状态和玩家按下游戏操纵器上的按钮时感到的压力有关系。这些研究结果说明，在玩家控制游戏操纵设备时受到的压力可能会决定他在游戏中受到的应激的效果。然而，在这个研究中并没有可以测量游戏玩家们情感的度量标准。Hazlett【7】使用了面部用的肌电描记法〔EMG〕作为测量玩家积极和消极的情感指数的方法。在一方面，面部用的肌电描记法仅仅只限于测量积极和消极的情感，这是由于这种方法还不能识别许多没有相互联系的情感。研究人员会基于许多的游戏事件来询问用户们哪些情况是积极的哪些是消极的。即使如此，为了实用各种各样的心理测试方法，他们必须要确保所有参与者都已经消除了对于将被连接电线和连接器的顾虑和不安，否那么，这样的一些情绪会参与者的心理产生干扰，将会对实验的最终结果造成影响。最近的用户问卷调查方面的进展包括多维情感，如紧X感，挫败感或消极的情感。Ijsesslsteijn et.al.【9】暗示过通过使用用户的游戏体验调查问卷【GEQ】来作一份有质量的调查问卷，通过这样的调查问卷在游戏时间之后来理解在游戏中的玩家的用户体验。

3 Analysis of Methodologies 3 方法分析

Based on literature reviews, we categorize the methodologies for player experienceinto three categories, general research methods (qualitative versus quantitative),instruments for measuring emotions (verbal versus non-verbal), and measurements forpleasurable design products (empirical versus non-empirical). Table 1 shows the threemain evaluation categories for evaluating player experience in game play.

根据对文献的反复学习，我们将分析用户体验的方法分成三种，即概括型研究方法〔质量相对于数量〕，测量情感的工具的方法〔言语相对于非言语〕，以及所设计产品带来的愉悦程度的测量〔经历研究和非实证研究〕。图表1说明了游戏中用来评估玩家的用户体验的三种主要的评估类型。

3.1 General Research Methods: Qualitative vs. Quantitative 3.1 概括性研究方法：质量的测量方法与数量的测量方法

- -可修编-

. -

Qualitative research concentrates on exploring and understanding user’s perceptionsand interactions. Generally this research approach generates non-numerical datawhere it documents the experience of players. On the other hand, quantitativeresearch produces numerical data or data that can be converted into numbers[14].The qualitative manner allows the evaluator to write down their experiences duringgame play, and quantitative method provides the statistics to show how interestedthere are. As for verbal and non verbal diagnostics, not every player shows theiremotions or talk freely during game play. During game play, players have difficulty toconcentrate and talk about their experience at the same time. Therefore, both methodsshould be adopted to get a more objective and thorough analysis for evaluation.

质量方面的研究主要集中于探索和了解用户的许多感知方面及许多互动方面的行为上。从总体上来说，这种研究方法将会得出没有数字的数据，这些数据记载着玩家的用户体验。在另一方面，数量上的研究那么会得出数字型的数据，或者产生的数据可以被转化为数字【14】。这种质量方面的研究方法允许评估员在游戏过程中记下他们的体验感受，而数量方面的研究方法那么会展现出数字上的数据，通过这些数据可以证明玩家的兴趣的大小。而对于言语和非言语类的测量方法，并不是每一位游戏玩家都会在游戏过程中显露出自己的情感或着随意的进展交流。另外，在游戏过程中，玩家们也很难在玩游戏的同时专注地谈论他们此时的感受。因此，这两种方法都应该被使用，从而能够获得一些更加客观与全面的评估分析。

3.2 Instruments for Measuring Emotions: Verbal vs. Non-verbal 3.2 用来测量情感的工具:言语测量方法与非言语测量方法

Hirschman [8] states that researchers prefer to develop their own approaches tomeasure emotion. A distinction is made between non verbal (objective) and verbal(subjective) instruments. Non verbal measures the expressive or the physiologicalelement of experience emotion evaluated by facial or vocal expressions while verbalinstruments assesses the subjective feeling ponent of emotion evaluated throughself report, rating scales and verbal protocols [19].

Hirschman【8】提到研究人员都喜欢创造属于他们自己的方式途径来测量用户的情感。非言语工具〔一种客观的工具〕和言语工具〔一种主观的工具〕之间存在着一个区别。非言语测量方法通过对面部的或者语言上的评估来测量用户心理上的或者生理上的体验，而与此同时，言语测量方法通过自我体验报告，等级定量表和口头报告来评估情感上的的主观感情方面因素【19】。

3.3 Measurements for Pleasurable Design Products: Empirical vs. Nonempirical 3.3 设计产品愉悦程度的测量方法：以观察或实验为依据的测量方法与不是以观察或实验为依据的测量方法

In designing pleasurable products, Jordan [10] highlighted two methods to measurethe relation of understanding people, products and evaluating design concepts, that is(i) non-empirical, which requires no participant involvement, and (ii) empirical,which involves user engagement in the design process. Both empirical and non-empiricalmethods are important when it involves players during the game play,especially at the final evaluation stage. However, the non-empirical method is alsocrucial, particularly at the beginning of creating or designing game play, as expertsare able to provide greater deal of feedback for further game play development.

在设计那些用来娱乐的产品的时候，Jordan【10】强调了两种方法来测量理解人，产品以及评估设计理念之间的关系，这两种方法是：(i)不是以观察或实验为依据的测量方法，这种方法不要求参与者的参与，

- -可修编-

. -

和(ii)以观察或实验为依据的测量方法，这种方法包含用户参与到设计进程中去。当涉及到正在游戏过程中的玩家的时候，以观察或实验为依据的测量方法和不以观察或实验为依据的测量方法都很重要，尤其是在使用方法评估的最后阶段。然而，不以观察或实验为依据的测量方法也起着至关重要的作用，尤其是在创造或者设计游戏的最初阶段，因为游戏专家们能够提供更为大量的通过反响而来的主意进而促进游戏的进一步完善和开发。

Table 1. A review of methodologies for evaluating player experience in game play

表一评估游戏中玩家的用户体验方法的复习回忆

玩家的游戏体验的启发法 可玩性的启发式评价〔HEP〕 游戏中的面部表情面部肌电描记法 游戏体验手指按压情感 生理测量方法〔皮肤电反响，面部肌电描记法，心电图扫描发，心率〕 流动玩家 阅读者 调查问卷 所传递的质量的测量方法 数量的测量方法 言语测量方法 非言语测量方法 以观察或实验为依据的测量方法 不是以观察或实验为依据的测量方法 √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ 概括性研究方法：质量的测量方法与数量的测量方法

-

-可修编-

. -

用来测量情感的工具:言语测量方法与非言语测量方法

设计产品愉悦程度的测量方法：以观察或实验为依据的测量方法与不是以观察或实验为依据的测量方法

To summarize, we suggest an adoption of the three methods from qualitative andquantitative, verbal and non verbal, and lastly empirical and non-empirical in order toprovide a prehensive approach to evaluate player experience for game play.

总的来说，我们建议能够同时采用这三种方法，首先是用质量的方法和数量的方法，其次是言语方法和非言语方法，最后是以观察或实验为依据的方法和不以观察或实验为依据的方法，以至于能为评估玩家在游戏中的用户体验提供一套完整的综合的分析方法。

4 Summary 4 总结

With regards to the perception of game play experience, it is different for individualplayers. Hence, both the expert evaluator and game player are crucial in assessing theelements involved in game play and the level of game player experience. This paperexamined and reviewed methodologies that evaluate player’s experience and analyzedthe different attributes through literature studies. Here we remend three maincategories consisting of general research methods, instruments to measure emotionsand pleasurable product measurement. A mixed method approach suggested in thispaper provides a holistic approach in determining player experience for game play.

关于游戏用户体验的感知情况，不同的玩家会有不同的感知结果。因此，评估游戏中涉及到的元素和评估用户体验的水平时，评估专家和游戏玩家都起着至关重要的作用。这篇论文仔细研究和回忆了评价用户体验的方法，并且通过对文献的研究学习分析了不同的特征。这里，我们一共推荐了三种主要的方法，其中包括概括性研究方法，测量情感的工具和娱乐产品测量。多种方法混合研究说明这篇论文综合而全面地研究分析了游戏中的用户体验。

References 参考文献

[1] Bernhaupt, R., Eckschlager, M., Tscheligi, M.: Methods for Evaluating Games – How to Measure Usability and User Experience in Games? In: Proc. ACE 2007, pp. 309–310. ACM Press, New York (2007)

[2] Cacioppo, J.T., Bernston, G.G., Larsen, J.T., Poehlmann, K.M., Ito, T.A.: The Psychophysiology of Emotion. In: Lewis, M., Haviland–Jones, J.M. (eds.) Handbook of Emotions. The Guilford Press, New York (2000)

[3] Clanton, C.: An Interpretation Demonstration of puter Game Design. In: Proc. of CHI 1998’ Summary: Human Factors in puting Systems, CHI 1998, pp. 1–2 (1998) [4] Csikszentmihalyi, M.: Flow: The Psychology of Optimal Experience. Harper Perennial, New York (1990)

[5] Desurvire, H., Caplan, M., Toth, J.A.: Using Heuristics to Evaluate the Playability of Games. In: Extended Abstract, CHI 2004, pp. 1509–1512. ACM Press, New York (2004) [6] Federoff, M.: Heuristics and Usability Guidelines for The Creation and evaluation of Fun in Video Games (February 2021), .melissafederoff./thesis.html

[7] Hazlett, R.: Measuring Emotional Valence during Interactive Experience: Boys at Video

- -可修编-

. -

Game Play. In: Proc. CHI 2006, pp. 1023–1026. ACM Press, New York (2006)

[8] Hirschman, E.C., Holbrook, M.B.: Hedonic Consumption: Emerging concepts, methods and propositions. Journal of Marketing 46, 92–101 (1982)

[9] Ijsselsteijn, W.A., van den Hoogen, W.H.M., Klimmt, C., de Kort, Y.A.W., Lindley, C., Mathiak, K., Poels, K., Ravaja, N., Turpeinen, M., Vorderer, P.: Measuring the Experience of Digital Game Enjoyment. In: Proc. of Measuring Behavior 2021, pp. 88–89 (2021) [10] Jordan, P.: Designing Pleasurable Products: An Introduction to the New Human Factors. Taylor and Francis, London (2000)

[11] Law, E., Roto, V., Vermeeren, A.P.O.S., Kort, J., Hassenzahl, M.: Towards a Shared Definition of User Experience. In: Proc. SIGs, CHI 2021, pp. 2395–2398 (2021) [12] Malone, T.W.: Heuristics for Designing Enjoyable User Interfaces: Lessons from puter Games. In: Thomas, J.C., Schneider, M.L. (eds.) Human Factors in puting System, Norwood. Ablex Publishing Corporation, NJ (1982)

[13] Mandryk, R.L., Atkins, M.S., Inkpen, K.M.: A Continuous and Objective Evaluation of Emotional Experience with Interactive Play Environments. In: Proc. CHI 2006, pp. 1027–1036. ACM Press, New York (2006)

[14] Neill, J.: Qualitative & Quantitative Research (March 2021), wilderdom./research/

QualitativeVersusQuantitativeResearch.html

[15] Overbeeke, C.J., Djajadiningrat, J.P., Hummels, C.C.M., Wensveen, S.A.G.: Beauty in Usability: Forget About the Ease of Use? In: Green, W.S., Jordan, P.W. (eds.) Pleasure with Products: Beyond Usability, pp. 9–18. Taylor & Francis, London (2002)

[16] Sweetser, P., Wyeth, P.: GameFlow: A Model for Evaluating Player Enjoyment in Games. ACM puters in Entertainment 3(3), article 3A (2005)

[17] Sykes, J., Brown, S.: Affective Gaming: Measuring Emotion through Gamepad. In: Proc. of SIGCHI Conference on Human Factors in puting Systems (CHI 2003), pp. 732–733. ACM Press, New York (2003)

[18] Wittgenstein, L.: Philosophical investigations. Blackwell, Oxford (1953)

[19] Zaman, B., Smith, T.S.: The FaceReader: Measuring instant fun of use. In: Proc. of the 4th Nordic Conference on Human-puter Interaction, pp. 457–460 (2006)

- -可修编-