Stealth Assessment in Games - Essay Sample

Introduction

Stealth assessments are performance-based assessments embedded on digital learning or gaming environments to invisibly capture crucial data required by the learner in the development of one's skills, knowledge or personal attributes. Shute (2011) proposed the use of stealth assessment as a valid and reliable tool for evaluating complex competencies amongst students who are unaware of the assessment. The two main features analyzed to determine the effectiveness of stealth assessment include learning support features aimed at selecting relevant information and those that organize and integrate information (Waters & van Oostendrop, 2013).

Is your time best spent reading someone else’s essay? Get a 100% original essay FROM A CERTIFIED WRITER!

Save 25%

Various frameworks including evidence-centered design, assessment engineering, and cognitive design systems are used in the development of stealth assessment. Wang, Shute, and Moore (2015) used an evidence-centered model to design a stealth assessment model aimed at assessing in-depth competencies of students. The designer based the design process on defined competencies and evidence indicators. Stealth assessments have been used to gain insights into various cognitive and non-cognitive skills. For instance, systems thinking through the acquisition of science content knowledge and inquiry skills have been assessed based on the Taiga Park game (Shute, Masduki, & Donmez, 2010). The main limitation of the evidence-centered model is time, where the game designers and learning scientists among other experts take close one or more years to implement a stealth assessment (Almond et al., 2014).

Moore & Shute (2017), designed a stealth assessment to evaluate non-cognitive competencies such as conscientiousness. Stealth assessments evaluate the learner's level of competency on a particular skill which is accurately updated over time. The evaluation is based on games due to their significant contribution to learning environments through positive learning outcome (Wilson et al., 2009). The assessment of conscientiousness adopted the use of evidence-centered designs which comprised three models the competency, evidence, and task models (Shute & Ventura, 2013). Additionally, the Bayesian networks were used for effective implementation of the stealth assessment by embedding competency to the three models. The networks collect and store updated evidence on the learners' beliefs by establishing conditional dependencies between the competency and observation (Almond et al., 2015).

Shute & Kim (2011), used stealth assessments by adopting the evidence-centered designs on the World of Goo to conduct a cognitive analysis of three competencies. These competencies included problem-solving skills, causal reasoning, and knowledge of static equilibrium. The study mainly focused on qualitative physics knowledge (Shute, Ventura, & Kim, 2013). The World of Goo is a puzzle game that borrows physics principles to build suction pipes from goo balls (Davidson, 2009). The popularity of games among young adults make it an effective vehicle to base the stealth assessment research. The games highly engaged the learners; therefore, if utilized appropriately builds on problem-solving skills (Gee 2007; Shute, Ventura, & Ke, 2015). The findings on the study by Shute & Kim (2011), asserted that the level of interaction of the learner and the game determines the success of learning. Therefore, merely playing the game may have no impact on learning.

Ke & Shute (2015), analyzed the design process of stealth assessments and learning support on Portal 2 to assess math competencies. The research found that procedural data mining and study-based analytics are vital tools in the retrieval of complex learning information in a gaming environment. However, complexities arise in the interpretation of log data as the game levels change due to misalignment of stealth assessment's objectives and design adopted. In this regard, Shute and Ventura (2013), propose the identification and integration of the learning context and evidence-centered design into the early stage of the game to address the complexity issues.

References

Almond, G., Kim, J., Velesquez, G., & Shute, J. (2014). How Task Features Impact Evidence from Assessments Embedded in Simulations and Games. Measurement: An Interdisciplinary Perspective, 12, 1-33.

Almond, G., Mislevy, J., Steinberg, S., Yan, D., & Williamson, M. (2015). Bayesian networks in educational assessment. New York, NY: Springer.

Davidson, D. (2009). From experiment gameplay to the wonderful world of goo and how physics is your friend. In D. Davidson (Ed.). Well played 1.0: video games, values and meaning (pp. 334-367). Pittsburgh, PA: ETS Press.

Gee, P. (2007). What video games have to teach use about learning and literacy. New York, NY: Palgrave/Macmillan.

Ke, F., & Shute, V. (2015). Design of game-based stealth assessment and learning support. In C. Loh, Y. Sheng, & D. Ifenthaler (Eds.). Serious games analytics (pp. 301-318). New York: Springer.

Moore, G., & Shute, V. (2017). Improving Learning through Stealth Assessment of Conscientiousness. In Marcus-Quinn, A., & Hourigan, T (Eds.). Handbook on Digital Learning for K-12 Schools (355-368). Cham: Springer International Publishing.

Shute, J. (2011). Stealth assessment in computer-based games to support learning. In S. Tobias & J. D. Fletcher (Eds.), Computer games and instruction (pp. 503-524). Charlotte, NC: Information Age Publishers.

Shute, J., & Kim, J. (2011). Does playing the World of Goo facilitate learning? In D. Y. Dai (Ed.), Design research on learning and thinking in educational setting: Enhancing intellectual growth and functioning (pp. 359-387). New York, NY: Routledge Books.

Shute, J., & Ventura, M. (2013). Measuring and supporting learning in games: Stealth assessment. Cambridge, MA: The MIT Press.

Shute, J., Masduki, I., & Donmez, O. (2010). Conceptual framework for modeling, assessing, and supporting competencies within game environments. Technology, Instruction, Cognition, and Learning, 8(2), 137-161.

Shute, J., Ventura, M., & Ke, F. (2015). The power of play: The effects of Portal 2 and Lumosity on cognitive and non-cognitive skills. Computers & Education, 80, 58-67.

Shute, J., Ventura, M., & Kim, J. (2013). Assessment and learning of qualitative physics in Newton's Playground. The Journal of Education Research, 106, 423-430.

Wang, L., Shute, V., & Moore, G. (2015). Lessons Learned and Best Practices of Stealth Assessment. International Journal of Gaming and Computer-Mediated Simulations, 7(4), 66-87.

Wilson, A., Bedwell, L., Lazzara, H., Salas, E., Burke, S., Estock, L., Conkey, C. (2009). Relationships between game attributes and learning outcomes. Simulation & Gaming, 40(2), 217-266.

Wouters, P., & Van Oostendorp, H. (2013). A meta-analytic review of the role of instructional support in game-based learning. Computers & Education, 60(1), 412-425.