Military Police Host Nation Policing Simulation: A Case Study into Designing Experiential Games to Meet Instructional Objectives
•Descargar como PPTX, PDF•
1 recomendación•678 vistas
Case Study presentation given at LEEF 2011 by Amy Morrison, Joy Pachuki and Ron Punako for Concurrent Technologies Corporation (CTC)
Recomendados
Más contenido relacionado
La actualidad más candente
La actualidad más candente (11)
Similar a Military Police Host Nation Policing Simulation: A Case Study into Designing Experiential Games to Meet Instructional Objectives
Similar a Military Police Host Nation Policing Simulation: A Case Study into Designing Experiential Games to Meet Instructional Objectives (20)
Último
Último (20)
Military Police Host Nation Policing Simulation: A Case Study into Designing Experiential Games to Meet Instructional Objectives
- 1. Military Police Host Nation Policing Simulation: A Case Study into Designing Experiential Games to Meet Instructional Objectives Amy Morrison Director, Software Development Joy PachuckiInstructional Designer Ron PunakoSoftware EngineerJune 16, 2011
- 2. CTC Overview 501(c)(3) nonprofit established in 1987 Staff of 1,400+ professionals More than 50 locations 900,000 sq. ft., including labs & demonstration space Top 100 Government Contractor Quality/EH&S Management System comprised of industry-best models: ISO 9001 (Quality) and 14001 (Environmental), AS9100 (Aerospace), and CMMI-SE/SW (Systems/Software Engineering) Nationally recognized security capabilities with 300,000+ sq. ft. of Top Secret/Sensitive Compartmented Information Facility Space, JWICS, SIPRNet, and NIPRNet access Award-winning education and training solutions team
- 3. Learning and Human Performance Solutions
- 5. BackgroundProgram goals Counter Insurgency (COIN) & Stabilization, Security, Transition, and Reconstruction Operations (SSTRO) Overall goal: Develop a training exercise that enables learners to evaluate the security posture of a host-nation police station and build rapport with the host-nation police force.
- 6. BackgroundStakeholders Army Research Laboratory (ARL) Leonard Wood Institute (LWI) US Army Military Police School (USAMPS)
- 8. BackgroundDevelopment challenges Easily operable / no training time Train students who have finished Police Transition Team and Police Mentorship Team Training Continue with Virtools or move to Unity? Small time frame and budget
- 9. BackgroundKey development facts 2009-2010 prototype Development time 6 months Rebuilt Virtools REAL engine and SAT in Unity Unity received Certificate of Networthiness from Army & Air Force in May Designed and implemented scenario Small team
- 11. Sound Engineer
- 13. Video Engineer
- 14. Voice Actors
- 16. Learners experience realistic environments, characters and cultural contexts
- 17. Learners explore a virtual host nation police station
- 18. A supporting cast of characters influence the learner
- 19. Learners investigate and make decisionsregarding station security while building rapport
- 20. Learners make challenging decisions
- 21. Learners progress through a scenario event sequence
- 22. Feedback and after action reviews enable reflection on performance
- 23. Learners receive just-in-time performance feedback
- 24. Learner performance is scored Quantitative Objective 1: 50%Objective 2: 100%… Final: 75% 1. Choice A Choice B Choice C Learner makes a choice Score is averaged by objective and recoded Final score is averaged across objective scores + Qualitative Good Job! You chose choice A. Choice A was best because… Objective 1: Good Job! … 1. Choice A Choice B Choice C Feedback is provided to learner Learner makes a choice Feedback is recorded by objective.
- 25. Systems Development Life Cycle
- 26. Systems Development Life CycleAnalysis Project Planning Initial Client Meetings Instructional Intent Assess Training Needs Determine Training Solution Client Requirements Technical Specifications Identified and Analyzed
- 27. EngineeringRequirements development and management Use Case “The player makes decisions through dialogue with Non-Player Characters (NPC).” Client Requirement Derived Requirement TRACEABILITY & PROCESS VISIBILITY “The simulation shall enable the learner to progress through making decisions.” “The simulation shall provide the capability to respond to character dialogue.” Test Case “PASS: The simulation enables the learner to progress through making decisions.”
- 28. Systems Development Life CycleDesign Concept Paper Development Simulation Course Outline Screenplay Development Initial Product Asset Development Technical System Design
- 29. Systems Development Life CycleDesign Continued
- 30. EngineeringDesign Art Plan Technical Solution Information Architecture
- 31. Systems Development Life CycleDevelopment Text Simulation Development Asset Development Product Development
- 32. EngineeringDevelopment Scenario Authoring Tool Game Engine Event Sequencing Subsystem Sound Subsystem UI/Menu Subsystem Character AI Subsystem REAL Notification Subsystem Animation Subsystem ScoringSubsystem Lipsync Subsystem
- 33. EngineeringDevelopment Asset Development Scenario Integration with Authoring Tool Sound Sound Video Video Scenario Event Sequence Event Sequence Models & Animation Models & Animation Scoring Scoring
- 34. Systems Development Life CycleImplementation Verification for adherence to instructional specifications Final product installation Validation testing and discrepancies resolved Product training conducted Final review and acceptance
- 35. EngineeringVerification testing ID: TST1 Prerequisites: 1. The system must be started 2. The simulation subsystem must be started Actions: 1. NPC engages the learner’s character in dialogue 2. Learner attempts to select a dialogue response Outcome: 1. Pass: The learner is able to select a response to character dialogue 2. Fail: The learner is not able to select a response to character dialogue Derived Requirement Trace to “The simulation shall provide the capability to respond to character dialogue.”
- 36. Systems Development Life CycleEvaluation Evaluation tasks directly planned in support of individual efforts, if applicable, would apply to each part of the process
- 37. Direct outcomes from challenges Rebuilt REAL system and developed exercise in 6 months REAL enabled rapid development of the MPHNP exercise Employed information architecture for easily operable training Employed sound development processes
- 38. Other results Visual representation of structured training for rapid development Teams build small simulations in a few weeks Savings vs. traditional development
- 40. Contact Information Amy Morrison(814) 269-2879 morrisoa@ctc.com Joy Pachucki (814) 269-2833 pachuckj@ctc.com Ron Punako (814) 269-6538 punakor@ctc.com
- 41. Conclusion and Hands-on Demo REAL Video (http://www.youtube.com/user/ ConcurrentTechCorp#p/u/0/27QMKTeWk8c)
- 42. Putting ideas into action.SM 1-800-CTC-4392www.ctc.com
Notas del editor
- Talking Points: This is the customary summary that clients sometimes expect of an organization. These are the features; it is important that you translate those features into client benefits. Emphasize "What this means for you and your organization.” Slides follow that correspond to each of the above line items, allowing you to elaborate on these benefits. Notes:Feature: 501(c)(3) nonprofit established in 1987. Benefit: As a nonprofit, CTC is an independent, tax-exempt organization able to deliver total solutions without bias for proprietary technologies. Reinforce the message that CTC is trusted to function as an unbiased resource to provide technology transition for the good of the country. Feature: Staff of 1,400+ professionals and more than 50 locationsBenefit: So what if we have a lot of people working here? Why should clients care? Because we can show that we strive to place the right people in locations of client convenience. CTC employees even work at client sites to best meet client needs.Feature: Approximately 900,000 sq. ft. (Related benefit slide: We Demonstrate. Evaluate. Educate. To test and transfer new technologies to the DoD and private industry, CTC operates a variety of state-of-the-art laboratories and demonstration facilities.) Benefit: The benefit of CTC's 900,000 sq. ft. of workspace is that we have more than adequate space to demonstrate, evaluate, educate, and meet client needs. The slide about CTC's friction stir welding system gives an example of just how good those labs and demonstration facilities are. That machine is big enough to handle an entire combat vehicle. When purchased, it was the world’s largest.Feature: Top 100 Government ContractorBenefit: CTC was listed by Washington Technology Magazine as a top 100 defense contractor, which punctuates our ability to serve clients well as a federally compliant contractor who knows what it takes to meet government regulations. Feature: First nonprofit research and development organization to simultaneously certify to both the ISO 9001 and ISO 14001 international standards. AS9100 certification (for aerospace related activities) received April 12, 2005. Assessed at CMMI-SE/SW® Maturity Level 3 within one operating unit in 2003. Benefit: CTC is a trusted, respected business partner with a national reputation for doing things right. We not only attained the standards, we live up to the standards and have an intense quality focus that benefits each client.Feature: Security Benefit: CTC earned the Defense Security Service’s James S. Cogswell Industrial Security Achievement Award, one of only ten awards given in 2006 to defense contractors who consistently demonstrate excellence in all aspects of their security. We have the experience and infrastructure to support nearly any classified requirement. CTC has secure facilities approved by the Defense Security Service since 1/96. JWICS = Joint Worldwide Intelligence Communication System. SIPRNet = Secret Internet Protocol Router Network. NIPRNet = Unclassified but Sensitive Internet Protocol Router Network.
- The call for immersive training has led to the development of Concurrent Technologies Corporation’s (CTC’s) REAL system. REAL is an immersive training simulation system and builder that enables learners to experience challenging learning scenarios containing realistic environments, people, cultures and their operational contexts before they experience them in real-life. REAL combines the best of 3D virtual worlds, gaming and experiential learning methodology into immersive learning experiences that engage today’s learners. Focused on analytic and decision making skills, learners use their knowledge, skills and abilities to synthesize information and execute available courses of action (COAs) to solve complex problems. Selected COAs provide dynamic scenario branching opportunities that simulate the 2nd and 3rd order effects of decision making in the real-world. REAL allows rapid development of simulations by training designers and developers making training more cost effective and easy to develop and deploy. Overall benefits:Designed for teams of developers, integrators, artists, designers and instructional designers.Leverages a visual, drag and drop building feature to quickly analyze, build and review structured training development while also quickly make changes to alter training content and logic.Easily deploys to Web, PC, Mac and iOS devicesCreates quick, realistic experiences for a variety of environments, training objectives and audiences. Features for learners and training designers: Portrays realistic environments, avatars and cultural contexts.Designed for training analytic and decision making skills.Supports simulation of 2nd and 3rd order decision making.Designed to support structured and unstructured scenario play.Tracks quantitative learner performance against learning objectives.Provides immediate reinforcement and remedial feedback to learners.Manage difficulty and learner flow experiences through scenario branching.Provides replay of scenario acts and discreet learning activities for reflection.Provides mini-games to add additional challenge and change scenario pacing.Provides After Action Reviews to learners. TECHQuest Pennsylvania Technology Product of the Yearhttps://ext.sharepoint.CTC.com/CTCComContent/StageMedia/Press%20Releases/09may07_pr.pdfREAL Videohttp://www.youtube.com/user/ConcurrentTechCorp#p/u/0/27QMKTeWk8cREAL system for rapid developmentFaster and cheaperRapid development - Instructionally sound and development processesInformation architecture for gamingQuicker visual representation for rapid developmentSmall simulations in the matter of a few weeksCut the time in 1/3???Savings you can see by taking this approach
- Military doctrine changesIn relatively recent years there has been considerable change in military doctrine and its application. In the past, in spite of doctrinal changes to the contrary, US forces continued to apply conventional warfare tactics while the enemy had changed from regular warfare to a guerilla warfare. During the second Iraq war, the US started earnestly embracing counter-insurgency (COIN) activities, however it was not easy for the military as it suffered military and political losses early on. In order to support counter insurgency activities Stabilization, Security, Transition, and Reconstruction Operations (SSTRO) were implemented. SSTRO deals helping to create a safe and secure environment while multinational agencies establish a new social order in countries where the government is weak, corrupt, incompetent or non-existent. Many SSTROs deal with restoration of civic functions and utilities like power, water and police forces.Overall goalOur program goals were to support COIN and SSTRO by developing a training exercise that enables learners to evaluate the security posture of a host-nation police station and build rapport with the host-nation police force after formal warfare has ceased. The need for policing in war torn countries is great. If the US does not assist and support the police then insurgents may instead fill the power vacuum and social conditions will continue to deteriorate.
- Our stakeholdersArmy Research Laboratory (ARL)Leonard Wood Institute (LWI)US Army Military Police School (USAMPS) at Fort Leonard Wood, MOImage source: http://en.wikipedia.org/wiki/File:ARL_logo.pngImage source: http://www.leonardwoodinstitute.org/Image source: http://www.wood.army.mil/usamps/ClipArt/Regimental_Crest.html
- DemographicsNon-Commissioned Officers (NCOs) and Junior OfficersAge = 21 years-32 years Experience = 1 year – 20 years Unique challenge go from conventional war fare to rapport building immediately.Imagine going from conventional warfare with all of its inherent violence one day and the next being put in charge of reconstruction of critical civic functions and rapport building the next. This is what our audience is asked to do.Little real-world civic trainingOur audience has little or no practical training in the area of civil restoration they are asked to address. The ability to think on their feet and to improvise is critical.Little lessons learned availableThere is little information to be passed down from one squad to the next as squads rotate off of assignments. For incoming squads the expert information and experience gained by the previous squad is lost.Image source: http://www.ng.mil/news/archives/2009/02/020609-nco.aspx
- Easily operable / no training timeOur client made it clear that there would be no time for formal training of learners to use the system. Therefore, the system would need to be highly usable with help and feedback automated to the extent possible. Prototypes are not typically refined to this level of usability.Train students who have finished Police Transition Team and Police Mentorship Team TrainingThe simulation would need to be capable of training learners fresh from their PTT and PMT trainings. Therefore, the simulation product would likely help provide these learners with their initial look at the kinds of environments that they would be deployed in.Continue with Virtools or move to Unity?As we all know technology changes quickly. After conducting a gap analysis of available technology vs. the current Virtools technology used to develop REAL it was determined that the Unity Game Development Tool would provide more capability, deployment to more platforms and be a better organizational fit. Unity was quickly gaining steam in the game development community and with potential clients. A decision needed to be made to continue with Virtools or move to Unity.
- 2009-2010 prototypeThe MPHNP prototype was developed between 2009-2010. Our mission was to provide a proof-of-concept to help client decide if this is the kind of training that they would like to formally develop in the future.6 months development timeThese are all standard industry metrics regarding serious game development. Serious game efforts like MPHNP are typically short and low cost efforts, relatively speaking, when compared to commercial game development efforts.Rebuilt Virtools REAL in UnityWe decided to rebuild the REAL technology in Unity. This proved to be a valuable and timely decision. MPHNP and other REAL based simulations may now be deployed to PC, Web, Mac and iPad using a single source and push button deployment. Then, in May Unity and the Unity Web Player were added to the list of products cleared for use by the US Army and Air force under a certificate of networthiness. That made the Flash and Unity players the two key Web based players capable of delivering immersive simulation to the military. This event coupled with the rebuild of the REAL player has well positioned our organization for continued work through government contracts.Designed and implemented scenarioWe designed and implemented learning scenario.
- Similar team for making moviesDeveloping a simulation like MPHNP is similar to producing a small movie with an outcome that is determined by the learner.That being said, the development effort requires a range of talents from writing to animating to programming. The slide shows a typical list of staff required on these efforts. Being that serious game development efforts are typically smaller efforts in terms of time and budget, it is advantageous to find staff that are cross-trained in these disciplines for integration and backup purposes. For example, Artists may also serve as Modelers/Animators and Video Engineers may also serve as Sound Engineers. It is also advantageous to have Program and Project Managers that understand the unique needs and capabilities of this kind of team as there are many different kinds of perspectives to take into account.
- Role Playing GameMPHNP was designed as a first-person perspective Role Playing Game (RPG). MPHNP enables the learner to experience training scenarios as a character within a 3D virtual world. There are a number of inherent learning principles at work in well-designed RPGs that are independent of the content that is being taught. Some of these principles include identity and co-design.Principle of IdentityIdentity is when the learner takes on the role of the person they are training to be or their own real-life role. Doing so provides the learner a environment in which they experience, first-hand, the values, attitudes and actions associated with a role that they may not receive otherwise through memorization of terms and definitions. (J, Gee., 2008)The player is Squad Leader Josh HardinStudents experience the simulation as Squad Leader Josh Hardin. The Hardin character is a composite of the target demographic for the students using the simulation. Principle of Co-designCo-design enables the learner to become active contributors to the training environment instead of passive observers. Unlike a movie in which an actor is observed performing his role throughout a finite scenario, the learner becomes an active agent within a scenario so that they may change the scenario. Being able to change the scenario enables the learner to experience training from different angles over time which is a key enabler to reflective learning and horizontal learning before they move to vertical learning. (J, Gee., 2008)
- Learner as Active AgentThe simulation is designed to make the student feel that they are an active agent in the training instead of a passive observer. We use games to engage learners in the same way that good movies do by placing them at the center of the action with the addition that the learner CONTROLS the outcomes. We depend upon immersion(recruitment of sight and sound modalities through rich multimedia) and flow (intense and single minded focus/ordering of the learner’s consciousness) to engage the user.Physiological reaction to movies/gamesFor example, consider movie goers watching a scary movie. They may become so immersed in the movie that it elicits feelings of fear physiologically indistinguishable from real-life feelings of fear (Palmer, 2008). They grab their seats and recoil in fear. Similarly, movies about tragedy may elicit intense feelings of sadness causing the watcher to cry.These are powerful experiences that reach humans on a very visceral level that are essential to human experience. Importantly to training, they are memorable. That’s why you can clearly recall facts from that movie from all those years ago, but can’t remember what was on your last standardized test.
- Police Station as CompositeThe host nation police station depicted in the simulation are based upon a composite of stations found in outlying Iraq and Afghanistan areas. These stations contrast greatly with western police stations. The are many times dusty, non-descript outposts in the desert or mountains. Lets look at some of the challenges of these stations.Challenges of Host Nation Police Stations1. The host nation police force may or may not be dressed in formally recognizable uniforms and may be mistaken for civilians.2. They may not possesses expected weapons and formal training making their performance and unpredictable.3. They may have porous security4. Host nation police may also be insurgents... To be sure insurgents are watching what US forces are doing in the area.
- Hardin is assisted by Platoon Leader Robertson, Sergeant Rooney and Interpreter HabibHardin is not alone in the simulation. There is a host of computer controlled characters who populate the simulated station. Directly in Hardin’s squad is Platoon Leader Robertson, Sergeant Rooney and Interpreter Habib. Other soldiers are deployed are deployed around the perimeter of the station for security prior to the squad exiting their Humvee, but they are not shown. Platoon Leader RobertsonPlatoon Leader Robertson is your ranking superior and serves as your personal mentor for the mission. She lets you take the lead. In this way Robertson provides a realistic mechanism for delivery of just-in-time feedback and remediation of decisions made by the learner. Robertson provides a way for the learner to receive feedback from a peer without fear of judgment.Sergeant RooneySergeant Rooney is there to provide support by offering alternative opinions and insight into issues encountered during their visit.Interpreter HabibInterpreter Habib provides translation services for the squad. Interpreters in squads typically are not US forces and are often hired from the indigenous population after passing a screening process. As a Squad Leader Hardin must make sure that the interpreter keep the squad in the loop in terms of conversation with host nation forces and ensure that the interpreter follow his lead.
- Objectives and decision makingAt the beginning of the simulation learners are presented with a list of scenario objectives that they are tasked to fulfill.These objectives involve learners making decisions to improve the security posture of a host nation police station. They must also do so while building rapport with the host nation police force and prepare to discuss station security with the host nation Police Chief. The objective to improve the host nation police station can be considered an “observe and report” task. However, the task of simultaneously building rapport greatly increases the challenge. The squad must be careful to prepare to discuss their requests with the Police Chief without insulting the Chief and or trampling cultural observances along the way. Building relationships in this way are key to SSTRO.Decision points, event sequences and interactive objectsTo implement decision making MPHNP uses decision points, event sequences and interactive objects
- Our simulation is designed to train just-in-time analysis and decision making skills.Just-in-time Analysis and decision makingAnalysis is conducted based upon synthesis of information from environment, characters, communication and intelligence gathering. Analyzed information is then used to make decisions that immediately and significantly effect the outcome of the simulation story. Why is this important? When we analyzed the audience we understood that failure to quickly make correct decisions may mean the loss of life of squad members, host nation forces and civilians. These learners must be able to think on their feet with a small window of opportunity and with incomplete or imperfect information. Decision point user interfaceChoices take place primarily through dialogue with squad members and host nation police forces. In the imageabove the learner is given the choice of 2 to 3 weighted dialog options to choose from in a visual list. Why Hardin’s voice is not heardWhile engaged in conversation, Hardin’s voice is not heard for the choices he makes. Why is this important? Again, this goes back to the idea that the student is placed in the center of the action as if he were there. By playing an actor’s voice we would be undermining the learner’s “inner voice.”Interactive ObjectsLearners also interact with simple or complex interactive objects that enable decision making through the object to affect event sequencing and outcomes. For example, the user has an opportunity to investigate a hole in the perimeter wall in MPHNP that is an interactive object.
- Overview of event sequencesAs in real life, the choices we make today help to shape many possible future outcomes tomorrow. The simulation employs an event sequence capable of simulating outcomes that mirror real-life events and outcomes. Our event sequencing engine has been designed to support linear, branching, recursive and any-order sequences. These are similar in concept to SCORM sequencing, but are not directly compatible at this point. The MPHNP simulation in particular takes advantage of the Linear and Any Order models. These are modeled by Exercise Designers in the REAL Scenario Authoring Tool. I will briefly discuss these.LinearLinear sequencing is the sequencing of events from start to end with no opportunity for deviation. Engineers and Program Managers may recognize this as the “waterfall” model. This is typically reserved for simple sequence complexity or low budget simulation.BranchingBranching sequencing allows opportunity for deviation from the sequence trunk. These are useful for placing learners within different branches of events based upon the decisions they have made in the simulation giving the learners “control” of outcomes. Developers must be careful with branching as the event sequences get wider they also become more complex, longer and, therefore, expensive.RecursiveRecursive sequencing can be thought of as reverse-branching to previously experienced events. This is useful for when the learner requires multiple repetitions of a task for practice or remediation. Exercise developers must be careful to ensure that there is some exit condition or risk having a never ending loop of the same set of events.Any OrderAny order sequencing allows the user to experience a number of events from a discreet group in any order. After completing the group of events the learner is allowed to move on. In the MPHNP simulation this kind of sequencing is useful for letting learners explore aspects of the host nation police station in the order of their preference.FlowAs previously mentioned, Flow helps to organize our consciousness for the task at hand. How do you know when you are experiencing flow? It varies, but many cite the rapid passing of time, forgetting of one’s self and the ability to intensely focus at the task at hand (Csikszentmihalyi, 1990). Athletes call it “The Zone.” It is important to engagement and retention.As game designers, we want to keep the leaner in an “optimal flow state”. Ideally, that state would run directly up the middle of the “Flow Channel” in the diagram perfectly balanced between anxiety and boredom to move quickly to mastery. However, in real-life and in games learners typically come up against a challenge the first time with little skill and so the challenge can cause high anxiety at first. As we gain skill, anxiety is reduced as we are now able to overcome challenge. However, linger to long at the same challenge with the same skills and soon the learner will become bored and so new challenges must be introduced. Therefore, in reality the path through the flow channel looks more like a snake representing the rise and fall of challenge and skills (Schell, 2008). This keeps learners on the engaged and “on the edge of competency”.(Gee, 2008) Our event sequences allow us to support flow by throttling difficulty and acquisition of skill. Students quickly moving through branches that are easy for them may be branched to more difficult branches and vice-vice versa.
- Overview of feedback and reviewIn order to quantify the effectiveness of training retention, the results of the test are compiled and shared with the learner to reflect on their performance relative to simulation objectives. Just-in-time feedback, scoring and after action reviewTo implement feedback and review MPHNPemploysjust-in-time feedback, scoring and after action review.
- Just-in-time performance feedbackSimilar to just-in-time analysis and decision making. The student receives just-in-time feedback in MPHNP. Feedback is designed to be multi-modal to increase retention. The learner makes decisions and is IMMEDIATELY given feedback about the decision. The learner SEES the physical reaction of characters, HEARS their spoken reaction and SEES relevant reinforcing and remediating textual feedback. All performance data is stored for later analysis and review.
- Overview of scoringHow does the scoring work? Typically, each decision in the simulation is weighted or given a color code representing a good, fair or bad choice, but can be any number of colors and weights. These choices are dictated by the educational requirements of the simulation.Three types of scoring are used including quantitative, qualitative and combined. MPHNP uses qualitative scoring.QuantitativeUsing quantitative scoring the learners make choices that are individually scored and then averaged across objectives. Then, all of the objective scores are compiled into a final score for an after action review. This methodenables a bridge to standardized test assessment methods and for storage into an LMS. (Not yet implemented in latest rewrite)QualitativeUsing quantitative scoring the learners make choices and associated qualitative feedback is stored across objectives. Then, the qualitative information is compiled into an after action review. This method is useful for those who simply want practice and feedback and or for exercise developers who do not require or possess standardized test assessment thresholds.CombinationBoth quantitative and qualitative scoring occurs for a comprehensive assessment.After Action ReviewThe review offers an opportunity for deep reflection of performance through a detailed walkthrough. The learner SEES the decision they made with a referential in-game image for activation they SEE remediating and reinforcing feedback on their performance and they SEE how it all relates to underlying objectives.
- The Systems Development Life Cycle adheres to instructional design and systems development processes and methodologies with constant consideration for Quality Assurance (QA). While key milestones are defined separately in specific project plans and schedules, they will reflect and step through these stages. It is important to note that this process is not linear and that many of the stages overlap. When developing any system, including games, it is advantageous to follow standards and processes. Every company’s approach to choosing and institutionalizing them is different. To give a bit of background on our own processes, we follow the ISO 9001 quality standard, Carnegie Mellon’s Capability Model Maturity Integration (CMMI) for Systems Development and proprietary Systems Development Process (SDP). We are accredited in both the ISO 9001 standard and at the CMMI Dev Level 3, which means that our processes and procedures are standardized throughout our organization. Each of these build upon the other in an increasing level of operational detail. The reason we go through the trouble of following these is to ensure quality of processes and products throughout our process, the lifecycle.
- Initial client meetings are critical. This is the time in which initial expectations about the game are set. Questions that need to be answered may include: What is the instructional intent? Leveraging instructional design expertise, advanced technologies, and past performance to develop performance-enhancing instruction. Whether that be a game, a simulation, or a combination of both (which is most common). It is good to be knowledgeable of the principles of learning in games and not rely solely upon well worn explanations (It’s engaging, it’s what young adults do) to answer this. During this phase, the instructional design team is working very closely with the client to assess training needs. This is often done through a variety of means – most commonly the organization’s training support plans, strategic plans, deficiency reports and meetings (workshops, summits) with key decision makers and subject matter experts (SMEs). Once the training needs are assessed and gathered, the instructional design team works with those key decision makers (and SMEs, if applicable) to prioritize training needs. Skill deficiencies, new skill development and return on investment are core topics of these discussions. Together a training solution is determined, an outline developed and resources identified and obtained. Note that for serious games this stage of the process is no different than it would be for facilitated instruction or web-based training. Just like all training solutions, analysis is a key process area in the systems development effort. What are the client requirements? Questions that need to be answered may include: What are the system specs in the classrooms where the system will be deployed and/or will the learners be distributed, and where? Does the system have to comply with the American’s with Disabilities Act Section 508 standards? These are just some of the questions that help determine the software and hardware required for integration, and for the engineers to begin preparing client requirements for the system. What are the technical specifications: Should the system provide the capability to author scenarios without technical knowledge? Has the software passed any certification and accreditation process, or will it need to? (Especially important if running on a government or military network.) If the system is a prototype on a short timeline this may be very difficult. This runs hand-and-hand with another important question, is there a requirement to teach classified subject matter in the system? (Classified systems have much different requirements, accreditations and processes overall than unclassified systems.)These questions are not what commercial game development houses typically have to worry about. These questions and their answers distinguish a serious game. At the end of this phase, the key decision makers work closely with the project managers, instructional design team and software engineers to come to a decision and determine the requirements.
- CLIENT REQUIREMENTSClient requirements that are written in “Natural” or “Conversational” language. That is, these requirements are written as compound statements of functionality that the client clearly understands. Before further planning may occur, the client must approve these requirements as they are the basis for more detailed planning and design required to create the game.For example, a client requirement may read “The simulation shall enable the learner to progress through making decisions”USE CASESUse cases help Software Engineers elaborate how goals or tasks are achieved in the game. Several use cases may be created per client requirement or from multiple client requirements to help determine different scenarios in which goals and tasks could be achieved. In short, it helps Software Engineers think deeply about the problem that must be solved by using the system.For example, a use case that supports our client requirement may read “The player makes decisions through dialogue with Non-Player Characters (NPC).”DERIVED REQUIREMENTSDerived requirements are created based upon functionality selected from the use cases. One or more requirements may be generated for a single use case. In contrast to client requirements, derived requirements are written in “Requirements Language”. That is that these requirements are atomic, unambiguous, complete and are testable. These are the requirements that Software Engineers will use to create the game. These requirements can cover a variety of requirement types such as functionality, design, documentation, interfaces, supportability and others.In order to partially satisfy the use case we createdwe will create a new derived requirement “The simulation shall provide the capability for the learnerto respond to character dialogue”.TEST CASESOnce a derived requirement has been completed it may be traced to one or more design elements or test cases.TRACEABILITYIn order to be able to track what has been requested by the client, what has been designed, developed and tested we use requirements traceability. Consider the back of your entertainment center. Do you have a rats nest of wires there between your various components or do you have your wires labeled? The rat’s nest represents poor traceability and the labeled wires represent good traceability. It is not difficult to plug the wires in any which way when you are hooking the components together initially. The problem is when you have to trace the wires back to their source to verify connections or disconnect things. This is the problem created during monitoring and control when we need to change to something in the game project, but don’t know where the authority or initiating requirement for the thing being changed came from.
- As we noted in the previous slide, the instructional intent of the training solution has been determined. This allows for the Systems Development Life Cycle to really be spun into action. All team members are beginning to layout some initial ground work. The engineers are determining the design and build of the game, the artists are beginning initial product and asset design and development. And the instructional design team is working closely with the client to develop a concept paper and outline of the game. This concept paper, as we term it, details the terminal and enabling objectives of the game, along with a character list and descriptions, and a thorough scenario synopsis and outline of the games branching. The level (Beginner, Intermediate, Advanced, etc.) of the game is determined (again this stems back to the solution determination in Analysis) and from that the technical and instructional design expanded. Level of the game refers to the type of interactivity and amount of guidance provided through remedial feedback, resources, etc. Many organizations have their own definitions and requirements for these levels, such as the DoD Handbook for the Development of Interactive Multimedia Instruction (IMI).Once the concept paper is approved by the client, we move forward to expand the scenario, its branches, key decision points or knowledge reinforcement activities, and remedial feedback fully to support the client’s current curriculum and/or training needs.
- During this stage the instructional design team is working closely with the engineers to develop a rubric for the game, the example shown here shows a game where learners will be required to apply knowledge and skills within a simulated environment by making the correct choices at decision points. These decisions may (and/or could) require the learner to execute defined procedures, complete required tasks, or develop specific work products (such as briefings, reports, quad charts, etc.). Authentic and traditional assessment methods will be applied to determine gains in knowledge and skills. Methods such as self-assessments, objective tests, observations, and work products will assess knowledge and application of concepts, procedures, and principles. These assessments are then mapped back to the instructional objectives of the game, through a cognitive map. Bloom’s Taxonomy is most often used as the basis for categorizing behaviors and assessment methods through the cognitive map.
- ART PLANThe art plan is used to detail the vignettes, graphics, models and animations to be used in the game as specified in the game scripts. The art plan is critical to communicating the overall visual theme and style of the game and ensuring that the visual design “hangs together” well. The plan works in an increasing order of fidelity from initial concept art to the final art as created in our production graphic and modeling tools. The inputs for the plan come primarily from the scripts developed by the writer. Each character, animation and environment are given an matching ID within the Scripts and the Art Plan to establish traceability throughout design. Props and less significant details are not assigned IDs for practicality. Inputs to the plan may also come from design requirements specified by the customer. For example, the client may want to co-design a character’s physical appearance based upon specific demographics. However, in most cases we find that the client typically provides reference and instructional material from which our artists are able by be creative in design.The Art Plan is also the where all visual quality standards and character, environment and animation development processes are established for artistic components.TECHNICAL SOLUTION PLANThe Technical Solution Plan is used to do several things such as create the system architecture, create the system data model and evaluate system design alternatives. In order to do so, derived requirements are allocated to design. For example, returning to our character dialogue requirement to provide the capability for the user to respond to character dialogue,we may determine that we will require a character dialogue subsystem that is a part of a greater decision subsystem that controls the sequencing for events in the game. We could then develop a data model that details the classes and methods required to support the character dialogue subsystem. We may evaluate two or more alternative approaches to how the data model is organized based upon a set of selection criteria and log the results to substantiate the choice. INFORMATION ARCHITECTUREThe Information Architecture Plan is used to develop the information model for MPHNP. Our information architecture requires four different layers that when laid one on another like pieces of acetate, create a comprehensive information model including the structure, content model, skeleton and visual layers. The structure layer consists of the hierarchical structure of the game screens. Create a structure that is too deep and risk the learner not being able to find what is needed. Create one too broad and risk information overload. The second layer consists of a content model. The content model builds upon each screen by specifying an “inventory” of what goes on each screen such as text, images and controls. Note that this layer lists the inventory items by name and is not a visual inventory. The next layer, the skeleton layer, provides wire frame diagrams that visually build upon the content model inventory items. Now we essentially have blueprints detailing each screen and their contents. Finally, the visual layer is where graphics and prototype screen shots are developed to realize the wire frame blue prints.The Information Architecture provides several advantages that may not be immediately noticeable. First, the plan provides a single document that enables a decomposition of the system from conception to visual design. Therefore, the document is great to share with the client for understanding on how design decisions were made. Second, the plan provides a single document that Information Architects, Software Engineers, Graphic Designers and Managers can all contribute to and can use to coordinate situational awareness of the product being developed on many levels. Last, but not least, the Information Architecture helps us to think deeply in terms of game usability. We can apply usability heuristics to the wireframe designs and provide paper prototypes of the wireframes to test users before we have written a single line of code, thereby reducing rework due to usability issues later in the process.
- Now that the hard work of analysis and design have been completed, we are ready to develop and integrate the products. At this point the development team will execute the Technical Solution Plan, Information Architecture Plan and Art Plan to develop game authoring tools, the game engine and art assets. Each component of the game will be reviewed for adherence to the learning objectives and tested for functionality. Three points to cover for this slide:[Early Phase] The text simulation (full script) provides the team the ability to step through the flow and decision points in a simulation prior to an investment in multimedia elements.[Mid Phase] Product assets reinforce the elements of the learning scenario by incorporating realistic, engaging multimedia as well as incorporating guidance tools and supporting materials. Product assets may include characters, props, scenes, reference material, and remedial feedback.[Late Phase] The alpha product represents the opportunity to verify and validate that all components of the product are fully integrated, have been met; and the product contains all technical and functional requirements as designed. The alpha product is carefully reviewed prior to final product development.
- SCENARIOAUTHORING TOOLThe scenario authoring enables teams of developers, multimedia specialists and exercise designers to create scenarios. Game development companies depend upon this capability to reduce future development time and cost associated with building games from scratch. It provides the capability for integrators to bring together all game assets and the game script into the production game quickly and easily as opposed to having to integrate them with programming code.GAME ENGINE DEVELOPMENTOur approach to game engine development is to buy a license to them so that we may append to them and abstract their functionality for our own purposes. For example, we have purchased an engine that provides, among other capabilities, the built-in capability to easily integrate 3D character movement and walk animations. On top of that capability we appended a waypoint artificial intelligence subsystem that enables the characters to walk from point A to point B while navigating around objects. Then, we may abstract, or hide, the complexity of the waypoint system by only requiring that points A and B be entered into the system for the characters to navigate. This approach helps provide the functionality that the game integrator will need to author the game using the scenario authoring tool.
- ASSET DEVELOPMENTAsset development goes on in parallel with authoring tool and game engine development. Assets include the graphics, characters, models, animations, videos, sound and user interfaces required to develop the game. The assets are developed using the design specified in the Art Plan which is based upon the scripts. It is important that artists and modelers have access to game development authoring tools in order to see how their model assets look and behave within the game world. This is where the virtual dressing room tool comes into play. The dressing room provides just enough capability to view charactersand their animations from different angles as they would be viewed in the finished game before the game is finished.INTEGRATIONIntegration is the culmination point in which the assets are brought together with the scenario authoring tool by the integrator to build the game as it will be experienced by the learner. It is also the point at which all of the scripts, design and development are put to the test. Often, as the assets are brought together in practice we are able to see for the first time how well things really work together. At this point we move out of formal design and development into bug discovery and fix mode using short tight discovery, design, build and test spirals. It is very helpful at this point to have documented how the scenario authoring tool works and the entire asset inventory so that integration can move forward smoothly. It is also imperative that an asset pipeline be planned and executed at this point. The asset pipeline is the process by which assets move from the artists’ and modelers’ hands into the integrators’ hands and into the game as efficiently as possible. Each pipeline is different based upon the roles, development tools and assets involved in the process. Some developers will develop their own automation tools to help drive the pipeline. In our current project we provide a staging or, sandboxed, version of the game where our modeler can import environment models and character models to build the game levels and verify that the models look and animate correctly. This step helps provide assurance to the integrator that the assets are ready for the next step in the pipeline which is to import the staging version models into the production version of the game using the scenario authoring tool. The Integrator then uses the functionality provided by the authoring tool give life to the models, sequence game events and score learner decisions.
- The most effective instructional tool is the instructor, whether that be in the classroom or through an online collaborative environment. It is imperative that faculty and others responsible for training and/or developing systems are equipped with the necessary skills and tools to enhance learning. Thus, the implementation phase is vital to a game’s success, not only for technical testing but also product training.During this phase we provide electronic copies of all training material and conduct a train-the-trainer session workshop. These workshops allow for organizations to develop a plan for the integration of gaming and simulation technologies into classroom instruction. This plan will promote collaboration and knowledge development amongst instructors and support staff, as well as ensure additional expertise and support resources during solution implementation.
- Verification testingVerification testing asks “Has the system been developed according to requirements?” System verification testing allows Software Engineers to test the discrete functionality of the system using test cases. One or more test cases are developed to test each requirement. Each test case contains prerequisites, actions and outcomes necessary to verify its associated requirement. Requirements traceability is critical to verification testing as we need to be able to trace test cases back to requirements being tested. It is advantageous to stub-out, or create the test case shells during requirements formation as a reminder and as a way to get a jump start on traceability and testing.
- We understand the importance of evaluating the effectiveness and impact of the games we develop; therefore we work closely with our clients to conduct both formative and summative evaluation, as required. Formative evaluation, conducted during the development of the product, will aid in ensuring that the goals of the instruction are being achieved. Summative evaluation efforts are often supported per request and focus on evaluating the learning materials and learning process. After Action Reviews (AARs) that drill down and discuss core feedback, and surveys, on the usability and instructional integrity of the game, are vital to constant verification and validation, and success, of the games we develop.
- With our remaining time I am going to run a brief video demo of the simulation and then we will take any questions that you might have.