Slides for my presentation at EuroRV3 2016 Paper: https://www.researchgate.net/publication/303080505_An_Introduction_to_Evaluation_in_Medical_Visualization Abstract: Medical visualization papers often deal with data that is interpreted by medical domain experts in a research or clinical context. Since visualizations are by definition designed to be interpreted by a human observer, often an evaluation is performed to confirm the utility of a presented method. The exact type of evaluation required is not always clear, especially to new researchers. With this paper, we hope to clarify the different types of evaluation methods that exist and provide practical guidelines to choose the most suitable evaluation method to increase the value of the work.

1. An Introduction to Evaluation in
Medical Visualization
Noeska Smit1 and Kai Lawonn2
1TU Delft, The Netherlands
2University of Koblenz · Landau, Germany

2. 2

3. 3

4. 4
https://commons.wikimedia.org/w/index.php?curid=10570695

5. And he liked it very much…
5

6. Evaluation
• Munzner’s nested model for visualization design and validation
6
Munzner: "A nested model for visualization design and
validation."Visualization and Computer Graphics, IEEE
Transactions on 15.6 (2009): 921-928.

7. Evaluation
• Isenberg et al.: 10 years of VIS paper evaluations
• Trend: more evaluations involving participants
7
Isenberg et al.: "A systematic review on the practice of
evaluating visualization.“IEEE Transactions on
Visualization and Computer Graphics, 19.12 (2013): 2818-
2827.

8. Evaluation Types
• Performance evaluation
• Case study
• User study
8

9. Performance Evaluation
• Measurable quantities:
• Timings: rendering, preprocessing, framerates
• Compared to similar methods
• On the same data (OSIRIX, AIM@SHAPE)
9

10. Case Study
• Apply method to clinical data
• What is shown/new insights
• Should be done by domain experts, otherwise usage scenarios
10

11. User Study
• Observation
• Task comparison
• Interview
• Evaluation form
11

12. Which Users?
• Domain experts, students in the field
• People in the lab, students
• Depending on task: random people on the street
12

13. Practical Examples
• Medical education application
• Visualization of multimodal PET/CT data
13

14. The Online Anatomical Human
14
Smit et al.: “The Online Anatomical Human: Web-based Anatomy
Education” Proceedings of Eurographics Education Papers 2016

15. The Online Anatomical Human
15https://vimeo.com/156236130

16. MOOC Deployment
• Massive Open Online Course (MOOC) on human anatomy via
Coursera:
• Over 14000 active learners worldwide
• https://www.coursera.org/learn/anatomy
16

17. Online Evaluation
• Large scale user study
• Survey:
• Statements with Likert scale responses
• Open questions
• Linked to Coursera user profiles
17
User experience
User performance

18. Illustrative Multi-volume Rendering for
PET/CT Scans
18
Lawonn et al.: "Illustrative multi-volume rendering for PET/CT scans.“
Proceedings of the Eurographics Workshop on Visual Computing for
Biology and Medicine 2015

19. 19

20. 20

21. 21

22. 22

23. 23

24. Discussion
• Standardized evaluation procedures
• Standardized datasets
• Medvis challenges?
24

25. Conclusion
• Evaluation methods for medical visualization:
• Performance
• Case study
• User study
• Medvis Challenge?
25

26. Thank you
26
• http://anatomy.tudelft.nl
• Live OAHction
• http://medvis.org
• The blog on all things medical visualization