1. SFT
  2. Archiwum
  3. Vol. 64
  4. A Personalised Optimising Level Adapt...

A Personalised Optimising Level Adaptation (OLA) Difficulty Algorithm for Scenario Simulations in Professional VR Simulators

Robert Marcin Wolański, Ph. D. , Karol Jędrasiak, Ph. D.

Download article
Open Access
DOI: 10.12845/sft.64.2.2024.4, pages: 56 – 65

Abstract

Aim: This study introduces the Optimising Level Adaptation (OLA) algorithm, designed to enhance scenario simulations for professional VR training by dynamically adjusting difficulty levels to match user performance, thereby supporting personalised learning and readiness for high-stakes situations such as firefighting and emergency response.

Project and methods: The OLA algorithm divides scenario activities into blocks and adjusts their difficulty based on user performance in comparison to a reference group of AI-controlled agents. The algorithm’s efficacy was tested across three proprietary VR simulators covering diverse professional scenarios: public speaking, hydrogen electrolysis and mechanical technician operations. Each scenario was divided into ten blocks of varying difficulty (easy, medium, difficult), dynamically adjusted based on the user's performance. This structure enables rapid adaptation, making it particularly beneficial for fire and rescue training, where realistic, yet scalable, scenario complexity is critical to preparing for unpredictable conditions in the field.

Results: Testing with 30 participants per simulator revealed an average final score of approximately 75%, closely aligning with the target success rate of 70%. The average number of difficulty level switches (between 0.8 and 1.16 across scenarios) demonstrated the algorithm’s effective adaptation to user performance, thus ensuring optimal engagement. The OLA algorithm’s capacity to tailor training difficulty in real time reflects its potential to enhance skill retention and readiness in emergency response settings, where maintaining user engagement at appropriate challenge levels is essential for preparedness in life-threatening situations.

Conclusions: The OLA algorithm provides significant advancements in personalised VR training, particularly within fire- and rescue-related applications, by maintaining optimal engagement and adaptive challenge levels. The adaptability demonstrated across multiple scenarios indicates its versatility and potential for use in diverse high-risk training applications. Future research could enhance the OLA algorithm’s effectiveness by refining scenario block determination, therefore contributing to improved response times, decision-making and operational efficiency in the emergency services.

Keywords: VR training, personalised learning, adaptive difficulty, fire and rescue training, scenario simulation, professional development, Optimising Level Adaptation (OLA), emergency response preparedness

Type of article: original scientific article

Bibliography:

  1. Cannon-Bowers J., Burns J., Salas E., Pruitt J., Advanced technology in scenario-based training, in: Making Decisions Under Stress, J. Cannon-Bowers, E. Salas (eds), APA, 1998, 365–374, https://doi.org/10.1037/10278-014.
  2. Van den Bosch K., Riemersma J.B.J., Reflections on scenario-based training in tactical command, in: Scaled worlds: Development, validation, and applications, S. Schiflett (ed.), Ashgate 2004, 1–21.
  3. Oser R.L., Cannon-Bowers J.A., Salas E., Dwyer D.J., Enhancing human performance in technology-rich environments: guidelines for scenario-based training, “Human Technology Interaction in Complex Systems” 1999, 9, 175–202.
  4. Riedl M.O., Young R.M., Narrative planning: Balancing plot and character, “Journal of Artificial Intelligence Research”, 2010, 39, 1, 217–268.
  5. Böckle M., Novak J., Bick M., Towards adaptive gamification: a synthesis of current developments, in: Proceedings of the 25th European Conference on Information Systems (ECIS). Guimarães, Portugal.
  6. Gockley, R., Bruce, A., Forlizzi, J., Michalowski, M., Mundell, A., Rosenthal, S., Designing robots for long-term social interaction, in: IEEE/RSJ international conference on intelligent robots and systems, IROS 2005, 1338–1343.
  7. Kanda T., Hirano T., Eaton D., Ishiguro H., Interactive robots as social partners and peer tutors for children: A field trial, “Human-Computer Interaction” 2004, 19(1), 61–84.
  8. Leite I., Martinho C., Paiva A., Social robots for long-term interaction: A survey, “International Journal of Social Robotics” 2013, 5(2), 291–308, https://doi.org/10.1007/s12369-013-0178-y.
  9. Yee N., Gaming motivations align with personality traits, 2016, https://quanticfoundry.com/2016/01/05/personality-correlates/ [accessed: 04.2020].
  10. Rodrigues L., Brancher J.D., Improving players’ profiles clustering from game data through feature extraction, in: Proc. SBGames Comput. Track, 2018, 177–186.
  11. Orji R., Vassileva J., Mandryk R.L., Modeling the efficacy of persuasive strategies for different gamer types in serious games for health, “User Model. User-Adapted Interact.” 2014, 24, 5, 453–498.
  12. Knutas A., Van Roy R., Hynninen T., Granato M., Kasurinen J., Ikonen J., A process for designing algorithm-based personalized gamification, ”Multimedia Tools Appl.”, 78, 10, 13593––13612, 2019.
  13. Rodrigues L., Toda A., Oliveira W., Palomino P., Isotani S., Just beat it: Exploring the influences of competition and task-related factors in gamified learning environments, in: Proc. Braz. Symp. Comput. Educ., 2020, 461–470.
  14. Csikszentmihalyi M., Flow: The psychology of the optimal experience, Harper Collins Publishers, New York 1990.
  15. Deci E.L., Ryan R.M., Intrinsic motivation and self-determination in human behaviour Plenum, New York 1985.
  16. Vygotsky L.S., Mind in society: The development of higher psychological processes, Harvard University Press, Cambridge 1978, https://doi.org/10.2307/j.ctvjf9vz4.
  17. Janssen J.B., Van der Wal C.C., Neerincx M.A., Looije R., Motivating children to learn arithmetic with an adaptive robot game, in: Proceedings of the third international conference on Social Robotics, 2011, 153–162, https://doi.org/10.1007/978-3-642-25504-5_16.
  18. Leyzberg D., Spaulding S., Scassellati B., Personalizing robot tutors to individuals’ learning differences, in: Proceedings of the 2014 ACM/IEEE international conference on human-robot interaction, 2014, 423–430.
  19. Gordon G., Breazeal C., Bayesian active learning-based robot tutor for children’s word-reading skills, in: Proceedings of the AAAI Conference on Artificial Intelligence, 29(1), https://doi.org/10.1609/aaai.v29i1.9376.
  20. VanLehn K., The relative effectiveness of human tutoring,intelligent tutoring systems, and other tutoring systems, “Educational Psychologist” 2011, 46(4), 197–221, https://doi.org/10.1080/00461520.2011.611369.
  21. Polson M.C., Richardson J.J., Foundations of intelligent tutoring systems, Psychology Press, New York 2013.
  22. Lord F.M., Novick M.R., Birnbaum A., Statistical theories of mental test scores, Addison-Wesley 1968.
  23. Chrysafiadi K., Virvou M., Student modeling approaches: A literature review for the last decade, “Expert Systems with Applications” 2013, 40(11), 4715–4729, https://doi.org/10.1016/j.eswa.2013.02.007.
  24. Desmarais M.C., de Baker R.S., A review of recent advances in learner and skill modeling in intelligent learning environments, “User Modeling and User-Adapted Interaction” 2012, 22(1-2), 9–38, https://doi.org/10.1007/s11257-011-9106-8.
  25. Corbett A.T., Anderson J.R., Knowledge tracing: Modeling the acquisition of procedural knowledge, “User Modeling and User Adapted Interaction” 1994, 4(4), 253–278.
  26. Pelanek R., Applications of the ELO rating system in adaptive educational systems, “Computers & Education” 2016, 98, 169–179, https://doi.org/10.1016/j.compedu.2016.03.017.
  27. Hvattum L.M., Arntzen H., Using ELO ratings for match result prediction in association football, “International Journal of forecasting” 2010, 26(3), 460–470, https://doi.org/10.1016/j.ijforecast.2009.10.002.
  28. Klinkenberg S., Straatemeier M., Van der Maas H.L.J., Computer adaptive practice of maths ability using a new item response model for on the fly ability and difficulty estimation, “Computers & Education” 2011, 57(2), 1813–1824, https://doi.org/10.1016/j.compedu.2011.02.003.
  29. Glickman M.E., Parameter estimation in large dynamic paired comparison experiments. Journal of the Royal Statistical Society, “Series C-Applied Statistics” 1999, 48, 377–394.
  30. Elo A.E., The Rating of Chess Players Past and Present, Arco, New York1978.
  31. Rasch G., Probabilistic models for some intelligence and attainment tests, The Danish Institute of Educational Research, Copenhagen 1960.
  32. Eggen T.J.H.M., Verschoor A.J., Optimal testing with easy or difficult items in computerized adaptive testing, “Applied Psychological Measurement” 2006, 30(5), 379–393, https://doi.org/10.1177/0146621606288890.
  33. Toda A.M., Valle P.H.D., Isotani S., The dark side of gamification: An overview of negative effects of gamification in education, in: Higher Education for All. From Challenges to Novel Technology Enhanced Solutions, A.I. Cristea, I.I. Bittencourt, F. Lima (eds.), Springer, Berlin 2018, 143–156.
  34. Morschheuser B., Hassan L., Werder K., Hamari J., How to design gamification? A method for engineering gamified software, “Inf. Softw. Technol.” 2018, 95, 219–237.
  35. Liu D., Santhanam R., Webster J., Toward meaningful engagement: A framework for design and research of gamified information systems, “MIS Quart.” 2017, 41, 4, 1011–1034.
  36. Liapis A., Yannakakis G. N., Togelius J., Adapting models of visual aesthetics for personalized content creation, “IEEE Transactions on Computational Intelligence and AI in Games”, 2012, 4(3), 213–228, https://doi.org/10.1109/TCIAIG.2012.2192438.
  37. Snodgrass S., Mohaddesi O., Harteveld C., Towards a generalized player model through the PEAS framework, in: FDG '19: Proceedings of the 14th International Conference on the Foundations of Digital Games, https://doi.org/10.1145/3337722.3341856.
  38. Gignac G.E., Szodorai E.T., Effect size guidelines for individual differences researchers, “Personality and individual differences” 2016, 102, 74–78, https://doi.org/10.1016/j.paid.2016.06.069.
  39. Enzmann D., Notes on effect size measures for the difference of means from two independent groups: The case of Cohen’sd and Hedges’g, 2015, 12, https://doi.org/10.13140/2.1.1578.2725.