In this presentation, we discuss the integration pf OpenSyllabus, a scenario creation tool, to EMAEVAL, a competency management and gradebook tool. OpenSyllabus proposes a model-based approach to create simple scenarios to describe activities, actors and resources (competencies, files, evaluation rubrics, comments, etc.). EMAEVAL offers extensive functionalities to describe competencies frameworks, coordinate the actors’ tasks including those of the tutors, validate students’ competencies, produce summary reports and emit proficiency certificates according to simple or elaborate rules. This integration offers a very nice flexibility for learning scenarios yet preserving a very robust back-end to monitor students’ accomplishments.

1. Assessment Portfolios: An Integrated Model-Based Approach Supporting The Needs And Scenarios Across Users Jacques Raynauld, HEC Montréal and MATI Montréal Christian Martel, Université de Savoie and Pentila

2. Key Words Flexible Comprehensive Epic 2011 Conference – London, UK– July 11-13 2

3. Overview Our portfolio vision Twofundamental portfolio challenges Our solution : integration of twocomplementarytools OpenSyllabus EmaEvaL Demo of proof of concept Future work and Road-map Epic 2011 Conference – London, UK– July 11-13 3

4. Context MATI Montréal is a research center for HEC Montréal, École polytechnique and University of Montréal Started a strategic initiative on guided- assessment portfolios in 2008 – manyschoolsinvolved 4 Epic 2011 Conference – London, UK– July 11-13

5. Context Université de Savoie– group of researchers worked on the project since 2007 Pentila : spin-off on an assessment solution EvalComp Consortium: community of French universities using the system since 2009 5 Epic 2011 Conference – London, UK– July 11-13

6. Our portfolio vision The assessment or evaluation process is the key driving force for individuals to create artefacts that will populate their portfolios Epic 2011 Conference – London, UK– July 11-13 6

7. Two fundamental challenges Many actors or activities involved in the creation of artefacts Course related activities (instructors) Internships (outside supervisors) Co-curricular activities (peer) Others Epic 2011 Conference – London, UK– July 11-13 7

8. Two fundamental challenges Need to keep track of all the results in light of skills or competencies framework Different scales Different rules Over time Need for a very flexible tool or set of tools Epic 2011 Conference – London, UK– July 11-13 8

9. Our solution – integration of two complementary tools A flexible and semantic authoring environment to create and sequence learning activities Open Syllabus : used since 2003 at HEC Montréal – now a Sakai 2.8 tool (10 000 students) A comprehensive tool to manage any evaluation process EmaEval : used in numerous French universities since 2009 (6 000 students) Epic 2011 Conference – London, UK– July 11-13 9

10. Open Syllabus An open source model-based approach to organize the content of any pedagogical structure (skills framework, program, course, portfolio) 1. Resources + 2. structure + 3. role + 4. workflow Epic 2011 Conference – London, UK– July 11-13 10

11. Open Syllabus 1. Resources Text Files (Power Point, PDF, Doc, videos, etc.) URL Comments 11 Epic 2011 Conference – London, UK– July 11-13

12. Open Syllabus Learning outcomes/competencies Civic knowledge & engagement (local & global) Creativethinking Criticalthinking Ethicalreasoning & action Foundations & skills for lifelonglearning Information literacy Inquiry & analysis Integrativelearning Interculturalknowledge & competence Oral communication Problemsolving ……. Association of American Colleges and Universities http://www.aacu.org/ 12 Epic 2011 Conference – London, UK– July 11-13

13. 1. Resources Rubrics (forms) Association of American Colleges and Universities http://www.aacu.org/ 13 Epic 2011 Conference – London, UK– July 11-13

14. Open Syllabus Organizing the resources according to a model approach Structure Unit Context Resources Root Structure Unit Context Resource 14 Epic 2011 Conference – London, UK– July 11-13

15. Open Syllabus Root Structure Competency 1 Porfolio Unit Context Instructions Text file (instructor) Year1 Resource Competency 1 Competency 2 Submit form Form to submit the artefact (student) Year 2 Competency 1 Competency 2 Evaluation Grading rubric form (tutor) 4. Workflow 3. Role 15 Epic 2011 Conference – London, UK– July 11-13

16. EmaEval A fully integrated management tool Integration of all the functionalities related to the evaluation, validation and certification of knowledge and competencies for the education sector 16 Epic 2011 Conference – London, UK– July 11-13

17. EmaEval Functionalities Definition of training programs (role distribution, institutional rules, etc.) Definition of the competency framework Choice of the evaluation scenario (quiz, session paper, validation of non-formal and informal learning, auto-evaluation, peer evaluation, …) Choice of evaluation methods (criteria, scales, ..) Follow-up of individual of collective progress Committeeand certificates emission Link to administrative systems (results, etc.) Epic 2011 Conference – London, UK– July 11-13 17

18. EmaEval 18 Epic 2011 Conference – London, UK– July 11-13

19. EmaEval 19 Epic 2011 Conference – London, UK– July 11-13

20. Open Syllabus EmaEval Google WebToolKit BonitaWorkflow Jack-Rabbit Google WebToolKitjavascriptwidgets Sling REST API Jack-Rabbit Use of external open source tools (ORBEON, EMaEval) 20 Epic 2011 Conference – London, UK– July 11-13

21. Demo of proof of concept OpenSyllabus Set the competencyframework Create a set of activities Specify the roles Identify the evaluationactivities , theirsequences and the criteriaused EmaEval Synchronizemanually the comp. framework Synchronize the roles Synchronize the activities and the criteriaused Set of Web services thatlink the differenttools to play the assessementactivities

22. July 2009 All these entries are activities to beexecutedsequentially by the actors 10th Sakai Conference - Boston, MA, U.S.A. 22

23. 23 Epic 2011 Conference – London, UK– July 11-13

24. Integration and demo Additional scenarios Course work : different scales by different instructors Workshop : auto-evaluation Internship : same skills are graded by numerous evaluators Co-curricular activities : the students choose the skills + peer evaluation Epic 2011 Conference – London, UK– July 11-13 24

25. Road-map Proof of concept isworking fine – weunderstand the wholechain – very flexible Small Pilot in Septemberwith École polytechnique de Montréal Automation of the links between the twoplatforms More stable and robust application nextyear Epic 2011 Conference – London, UK– July 11-13 25

26. Road-map Vimoscreencast Paper on the conference site Jacques.raynauld@hec.ca Christian.Martel@univ-savoie.fr Questions? Epic 2011 Conference – London, UK– July 11-13 26