1. The Advantages of Monitoring
Sakai Usage
By
Jose L. Silva-Lugo
Carl J. Perry
John R. Boekenoogen
e-Learning Support Services
Office of Academic Technology
University of Florida

2. Introduction
2
 The University of Florida (UF)’s contract with
Blackboard WebCT was going to end in October 2010.
 In Spring of 2009, UF came up with four options: Sakai,
Blackboard, Moodle, and Angel.
 UF ran open meetings with each vender.
 UF set up test sites for faculty, staff, and students.
 UF selected Sakai
6/10/2012

3. Introduction: Planning, Testing, & Implementation
3
2009 2010 2011 2012
Process Dec Spr Sum Fall Spr Sum Fall Spr
Planning
Pilot Study
Migration
Introd. to UF
Roll Out
Workshops
Open Labs
Webinars
6/10/2012

4. Introduction: Training & Assistant Offer to UF
4
• Getting Started
• Moving On
Workshops • Evaluating knowledge with online assessment
• Begin a new semester
• End a semester
• 1 h consultation per week
Open labs • There is no need of appointment
• Every Tuesdays through the semester
Webinar • 30 minutes per week
• Every Wednesday through the semester
6/10/2012

5. Introduction: The Need of Monitoring Sakai Usage
5
 Even though the effort of promoting Sakai carried out
from summer 2010 to 2011, we did not have the
expected attendance.
 We decided to look our records to:
 Find out the percentage of courses, sections, and instructors
using Sakai per college/year
 Identify the colleges with percentages less than 50% used of
Sakai
 Plot the percentage of courses, sections, and instructors per
semester
 Target colleges with a marketing program
6/10/2012

6. Methodology: Databases
6
 Data was retrieved from the network:
 Registrar Office: IBM DB2 database
 Middleware: MySQL database
 Language used to query the database:
 A PHP script queried both database (< 5 seconds).
 The same script merged and compared the results (5-
20 seconds).
6/10/2012

7. Methods: Points of Interest about Querying
7
 You need to understand the database.
 You have to find how to best compare the results:
 The Registrar DB2 had zero to many instructors associated with a
section
 The Registrar DB2 used postfix characters in the course name, such
as for labs
 The results produced from each database were accurate and
error free.
 Manipulating large data sets and verifying the results were
accurate.
6/10/2012

8. Methodology: Calculating Percentages
8
Registrar (Total) Middleware (Sakai)
• Courses • Courses
• Sections • Sections
• Instructors • Instructors
6/10/2012

9. Methodology: Statistics
9
 Statistical Analysis:
 Proportion comparison:
 Between 2010 and 2011 for courses, sections, and
instructors per college
 Between semesters for courses, sections, and instructors
 Alpha level = 0.05
 Two-tailed test
6/10/2012

10. No. of Courses from the Registrar Office and Middleware in 2010 & 2011
2010 2011
College Registrar 10
Middleware Percentage Registrar Middleware Percentage
DESIGN, CONSTRUCTION AND PLANNING 307 91 29.64 348 256 73.56
HEALTH AND HUMAN PERFORMANCE 383 91 23.76 423 282 66.67
ENGINEERING 1,100 218 19.82 1,127 445 39.49
JOURNALISM AND COMMUNICATIONS 311 57 18.33 289 191 66.09
MEDICINE 534 92 17.23 519 116 22.35
PHARMACY 244 42 17.21 296 150 50.68
LIBERAL ARTS AND SCIENCES 3,138 455 14.50 2,872 1,237 43.07
PUBLIC HEALTH & HEALTH PROFESSIONS 457 65 14.22 458 245 53.49
BUSINESS ADMINISTRATION 645 88 13.64 653 406 62.17
AGRICULTURAL AND LIFE SCIENCES 1,602 213 13.30 1,647 573 34.79
CROSS-COLLEGE 36 4 11.11 47 0 0.00
FINE ARTS 1,022 56 5.48 1,037 179 17.26
VETERINARY MEDICINE 259 13 5.02 264 51 19.32
EDUCATION 630 18 2.86 662 54 8.16
DENTISTRY 244 1 0.41 279 5 1.79
ACCOUNTING 64 0 0.00 65 0 0.00
BUILDING CONSTRUCTION 211 0 0.00 218 0 0.00
CONTINUING EDUCATION 209 0 0.00 171 0 0.00
LEVIN COLLEGE OF LAW 190 0 0.00 550 11 2.00
MILITARY 62 0 0.00 55 0 0.00
NURSING 133 0 0.00 130 88 67.69
Total 11,781 1,504 12.77 12,110 4,289 35.42
6/10/2012

11. 11
6/10/2012

12. Courses: Proportion Comparison between 2010 and 2011 per College
12
Semester π1 π2 Π σ π1- π2 |Z| Zα P-value Result
AGRICULTURAL 0.13 0.35 0.2419 0.0150 14.3034 1.645 < 0.0001 P < 0.05
BUSINESS ADMINISTRATION 0.14 0.62 0.3806 0.0270 18.0054 1.645 < 0.0001 P < 0.05
DENTISTRY 0.00 0.02 0.0115 0.0093 1.4809 1.645 0.0694 P > 0.05
DESIGN, CONSTRUCTION 0.30 0.74 0.5298 0.0391 11.2387 1.645 < 0.0001 P < 0.05
EDUCATION 0.03 0.08 0.0557 0.0128 4.1510 1.645 < 0.0001 P < 0.05
ENGINEERING 0.20 0.39 0.2977 0.0194 10.1481 1.645 < 0.0001 P < 0.05
FINE ARTS 0.05 0.17 0.1141 0.0140 8.4064 1.645 < 0.0001 P < 0.05
HEALTH & HUMAN PERFORMANCE 0.24 0.67 0.4628 0.0352 12.2002 1.645 < 0.0001 P < 0.05
JOURNALISM & COMMUNICATIONS 0.18 0.66 0.4133 0.0402 11.8711 1.645 < 0.0001 P < 0.05
LEVIN COLLEGE OF LAW 0.00 0.02 0.0149 0.0102 1.9640 1.645 0.0250 P < 0.05
LIBERAL ARTS & SCIENCES 0.14 0.43 0.2787 0.0115 25.0227 1.645 < 0.0001 P < 0.05
MEDICINE 0.17 0.22 0.1975 0.0245 2.0872 1.645 0.0188 P < 0.05
NURSING 0.00 0.68 0.3346 0.0582 11.6320 1.645 < 0.0001 P < 0.05
PHARMACY 0.17 0.51 0.3556 0.0414 8.0846 1.645 < 0.0001 P < 0.05
PUBLIC HEALTH 0.14 0.53 0.3388 0.0313 12.5489 1.645 < 0.0001 P < 0.05
VETERINARY MEDICINE 0.05 0.19 0.1224 0.0287 4.9889 1.645 < 0.0001 P < 0.05
2010 0.13 0.50 0.3142 0.0060 61.6931 1.645 < 0.0001 P < 0.05
2011 0.35 0.50 0.4248 0.0064 22.2413 1.645 < 0.0001 P < 0.05
6/10/2012

13. 13
6/10/2012

14. 14
6/10/2012

15. Courses: Proportion Comparison between Semesters
15
Semester π1 π2 Π σ π1- π2 |Z| Zα P-value Result
Spring 2010 - Summer 2010 0.00 0.04 0.0166 0.0031 13.4318 1.645 < 0.0001 P < 0.05
Summer 2010 - Summer 2011 0.04 0.29 0.1772 0.0102 24.5078 1.645 < 0.0001 P < 0.05
Fall 2010 - Spring 2011 0.30 0.37 0.3344 0.0099 6.4253 1.645 < 0.0001 P < 0.05
Fall 2010 - Summer 2011 0.30 0.29 0.2985 0.0107 1.0054 1.645 0.1562 P > 0.05
Fall 2010 - Fall 2011 0.30 0.38 0.3432 0.0099 8.1293 1.645 < 0.0001 P < 0.05
Spring 2011 - Summer 2011 0.37 0.29 0.3363 0.0111 6.7244 1.645 < 0.0001 P < 0.05
Spring 2011 - Fall 2011 0.37 0.38 0.3752 0.0102 1.6609 1.645 0.0485 P < 0.05
Summer 2011 - Fall 2011 0.29 0.38 0.3469 0.0111 8.2315 1.645 < 0.0001 P < 0.05
6/10/2012

16. 16
6/10/2012

17. No. of Sections from the Registrar Office and Middleware in 2010 & 2011
2010 2011
College Registrar 17
Middleware Percentage Registrar Middleware Percentage
HEALTH AND HUMAN PERFORMANCE 628 169 26.91 894 563 62.98
MEDICINE 604 134 22.19 682 165 24.19
DESIGN, CONSTRUCTION AND PLANNING 577 125 21.66 582 341 58.59
ENGINEERING 2467 500 20.27 2384 784 32.89
PHARMACY 718 131 18.25 735 426 57.96
LIBERAL ARTS AND SCIENCES 7653 1308 17.09 7632 3649 47.81
CROSS-COLLEGE 66 11 16.67 71 0 0.00
JOURNALISM AND COMMUNICATIONS 588 98 16.67 638 327 51.25
BUSINESS ADMINISTRATION 1264 185 14.64 1306 915 70.06
AGRICULTURAL AND LIFE SCIENCES 3220 457 14.19 3870 1144 29.56
PUBLIC HEALTH & HEALTH PROFESSIONS 811 90 11.10 919 347 37.76
FINE ARTS 1843 87 4.72 1975 273 13.82
VETERINARY MEDICINE 325 14 4.31 370 63 17.03
EDUCATION 1299 22 1.69 1389 78 5.62
DENTISTRY 356 1 0.28 372 4 1.08
ACCOUNTING 136 0 0.00 116 0 0.00
BUILDING CONSTRUCTION 269 0 0.00 277 0 0.00
CONTINUING EDUCATION 228 0 0.00 177 0 0.00
LEVIN COLLEGE OF LAW 441 0 0.00 503 11 2.19
MILITARY 90 0 0.00 79 0 0.00
NURSING 279 0 0.00 279 208 74.55
Total 23,862 3,332 13.96 25,250 9,298 36.82
6/10/2012

18. 18
6/10/2012

19. Sections: Proportion Comparison between 2010 and 2011 per College
19
Semester π1 π2 Π σ π1- π2 |Z| Zα P-value Result
AGRICULTURAL 0.14 0.30 0.2258 0.0100 15.4094 1.645 < 0.0001 P < 0.05
BUSINESS ADMINISTRATION 0.15 0.70 0.4280 0.0195 28.3899 1.645 < 0.0001 P < 0.05
DENTISTRY 0.00 0.01 0.0069 0.0061 1.2973 1.645 0.0968 P > 0.05
DESIGN, CONSTRUCTION & PLANNING 0.22 0.59 0.4021 0.0288 12.8197 1.645 < 0.0001 P < 0.05
EDUCATION 0.02 0.06 0.0372 0.0073 5.3689 1.645 < 0.0001 P < 0.05
ENGINEERING 0.20 0.33 0.2647 0.0127 9.9592 1.645 < 0.0001 P < 0.05
FINE ARTS 0.05 0.14 0.0943 0.0095 9.6172 1.645 < 0.0001 P < 0.05
HEALTH & HUMAN PERFORMANCE 0.27 0.63 0.4809 0.0260 13.8633 1.645 < 0.0001 P < 0.05
JOURNALISM & COMMUNICATIONS 0.17 0.51 0.3467 0.0272 12.7130 1.645 < 0.0001 P < 0.05
LEVIN COLLEGE OF LAW 0.00 0.02 0.0117 0.0070 3.1238 1.645 < 0.001 P < 0.05
LIBERAL ARTS & SCIENCES 0.17 0.48 0.3243 0.0076 40.5675 1.645 < 0.0001 P < 0.05
MEDICINE 0.22 0.24 0.2325 0.0236 0.8508 1.645 0.1917 P > 0.05
NURSING 0.00 0.75 0.3728 0.0409 18.2102 1.645 < 0.0001 P < 0.05
PHARMACY 0.18 0.58 0.3833 0.0255 15.5669 1.645 < 0.0001 P < 0.05
PUBLIC HEALTH & HEALH PROFESSIONS 0.11 0.38 0.2526 0.0209 12.7359 1.645 < 0.0001 P < 0.05
VETERINARY MEDICINE 0.04 0.17 0.1108 0.0239 5.3304 1.645 < 0.0001 P < 0.05
2010 0.14 0.50 0.3213 0.0043 84.9707 1.645 < 0.0001 P < 0.05
2011 0.37 0.50 0.4316 0.0044 28.7675 1.645 < 0.0001 P < 0.05
6/10/2012

20. 20
6/10/2012

21. 21
6/10/2012

22. Sections: Proportion Comparison between Semesters
22
Semester π1 π2 Π σ π1- π2 |Z| Zα P-value Result
Spring 2010 - Summer 2010 0.00 0.05 0.0163 0.0023 21.5659 1.645 < 0.0001 P < 0.05
Summer 2010 - Summer 2011 0.05 0.24 0.1576 0.0074 25.3576 1.645 < 0.0001 P < 0.05
Fall 2010 - Spring 2011 0.31 0.40 0.3528 0.0068 12.3286 1.645 < 0.0001 P < 0.05
Fall 2010 - Summer 2011 0.31 0.24 0.2850 0.0075 9.7743 1.645 < 0.0001 P < 0.05
Fall 2010 - Fall 2011 0.31 0.42 0.3636 0.0068 15.2238 1.645 < 0.0001 P < 0.05
Spring 2011 - Summer 2011 0.40 0.24 0.3371 0.0079 19.8939 1.645 < 0.0001 P < 0.05
Spring 2011 - Fall 2011 0.40 0.42 0.4058 0.0070 2.7055 1.645 0.0034 P < 0.05
Summer 2011 - Fall 2011 0.24 0.42 0.3514 0.0079 22.3055 1.645 < 0.0001 P < 0.05
6/10/2012

23. No. of Instructors from the Registrar Office and Middleware in 2010 & 2011
2010 2011
College Registrar 23
Middleware Percentage Registrar Middleware Percentage
CROSS-COLLEGE 36 10 27.78 37 0 0.00
HEALTH AND HUMAN PERFORMANCE 220 56 25.45 267 175 65.54
ENGINEERING 697 176 25.25 758 417 55.01
DESIGN, CONSTRUCTION AND PLANNING 218 46 21.10 236 129 54.66
BUSINESS ADMINISTRATION 405 80 19.75 414 124 29.95
JOURNALISM AND COMMUNICATIONS 286 56 19.58 318 180 56.60
LIBERAL ARTS AND SCIENCES 3,364 519 15.43 3,843 1,289 33.54
PUBLIC HEALTH & HEALTH PROFESSIONS 403 61 15.14 444 162 36.49
AGRICULTURAL AND LIFE SCIENCES 1,003 147 14.66 1,050 388 36.95
FINE ARTS 621 50 8.05 659 141 21.40
VETERINARY MEDICINE 256 14 5.47 489 32 6.54
MEDICINE 324 16 4.94 541 49 9.06
EDUCATION 583 21 3.60 581 67 11.53
PHARMACY 164 3 1.83 236 8 3.39
DENTISTRY 132 1 0.76 571 4 0.70
ACCOUNTING 40 0 0.00 45 0 0.00
BUILDING CONSTRUCTION 67 0 0.00 82 0 0.00
CONTINUING EDUCATION 12 0 0.00 5 0 0.00
LEVIN COLLEGE OF LAW 256 0 0.00 297 9 3.03
MILITARY 28 0 0.00 31 0 0.00
NURSING 174 0 0.00 205 6 2.93
Total 9,289 1,256 13.52 11,109 3,180 28.63
6/10/2012

24. 24
6/10/2012

25. Instructors: Proportion Comparison between 2010 and 2011 per College
25
Semester π1 π2 Π σ π1- π2 |Z| Zα P-value Result
AGRICULTURAL 0.15 0.37 0.2606 0.0194 11.5043 1.645 < 0.0001 P < 0.05
BUSINESS ADMINISTRATION 0.20 0.30 0.2491 0.0302 3.3741 1.645 < 0.001 P < 0.05
DENTISTRY 0.01 0.01 0.0071 0.0081 0.0703 1.645 0.4721 P > 0.05
DESIGN, CONSTRUCTION 0.21 0.55 0.3855 0.0457 7.3403 1.645 < 0.0001 P < 0.05
EDUCATION 0.04 0.12 0.0756 0.0155 5.1170 1.645 < 0.0001 P < 0.05
ENGINEERING 0.25 0.55 0.4076 0.0258 11.5416 1.645 < 0.0001 P < 0.05
FINE ARTS 0.08 0.21 0.1492 0.0199 6.6968 1.645 < 0.0001 P < 0.05
HEALTH & HUMAN PERFORMANCE 0.25 0.66 0.4743 0.0455 8.8171 1.645 < 0.0001 P < 0.05
JOURNALISM & COMMUNICATIONS 0.20 0.57 0.3907 0.0398 9.3113 1.645 < 0.0001 P < 0.05
LEVIN COLLEGE OF LAW 0.00 0.03 0.0163 0.0108 2.8082 1.645 0.0025 P < 0.05
LIBERAL ARTS & SCIENCES 0.15 0.34 0.2509 0.0102 17.6964 1.645 < 0.0001 P < 0.05
MEDICINE 0.05 0.09 0.0751 0.0185 2.2242 1.645 0.0132 P < 0.05
NURSING 0.00 0.03 0.0158 0.0129 2.2748 1.645 0.0116 P < 0.05
PHARMACY 0.02 0.03 0.0275 0.0166 0.9387 1.645 0.1736 P > 0.05
PUBLIC HEALTH 0.15 0.36 0.2633 0.0303 7.0459 1.645 < 0.0001 P < 0.05
VETERINARY MEDICINE 0.05 0.07 0.0617 0.0186 0.5791 1.645 0.2810 P > 0.05
2010 0.14 0.50 0.3152 0.0068 52.8065 1.645 < 0.0001 P < 0.05
2011 0.50 0.29 0.3907 0.0065 31.9009 1.645 < 0.0001 P < 0.05
6/10/2012

26. 26
6/10/2012

27. 27
6/10/2012

28. Instructors: Proportion Comparison between Semesters
28
Semester π1 π2 Π σ π1- π2 |Z| Zα P-value Result
Spring 2010 - Summer 2010 0.00 0.07 0.0235 0.0044 14.5203 1.645 < 0.0001 P < 0.05
Summer 2010 - Summer 2011 0.07 0.23 0.1582 0.0113 14.3211 1.645 < 0.0001 P < 0.05
Fall 2010 - Spring 2011 0.29 0.32 0.3065 0.0103 3.5117 1.645 < 0.001 P < 0.05
Fall 2010 - Summer 2011 0.29 0.23 0.2645 0.0114 5.3643 1.645 < 0.0001 P < 0.05
Fall 2010 - Fall 2011 0.29 0.28 0.2856 0.0099 0.4479 1.645 0.3264 P > 0.05
Spring 2011 - Summer 2011 0.32 0.23 0.2881 0.0115 8.4099 1.645 < 0.0001 P < 0.05
Spring 2011 - Fall 2011 0.32 0.28 0.3030 0.0099 4.0899 1.645 < 0.0001 P < 0.05
Summer 2011 - Fall 2011 0.23 0.28 0.2636 0.0111 5.1051 1.645 < 0.0001 P < 0.05
6/10/2012

29. 29
6/10/2012

30. Discussion
30
 In general, course, sections, and instructors
using Sakai are not monitored.
 This simple procedure has the following
advantages:
 Provide information about the proportion of
instructors, courses, and sections using Sakai.
 It helps to monitor the implementation process.
 It creates the database for marketing programs.
 It might be used to evaluate success at the college and
at university level.
6/10/2012

31. Conclusions
31
There was significant increase in their proportions
of courses, sections, and instructors from 2010 to
2011 in the majority of the colleges.
There was significant increase in their proportions
of courses, sections, and instructors per semester.
We have identified the colleges that need more our
attention, and we are targeting them with a
marketing program.
Monitoring Sakai usage is important during the
implementation and evaluation of the overall
success at the college and university level.
6/10/2012