ParaView is an open-source graphical user interface for VTK with additional functionality including the capability to perform rendering in parallel and a client-server architecture enabling visualization and analysis to be performed on a server while being viewed and driven from a client. ParaView, like VTK, is open-sourced under a BSD license and its development is overseen by the commercial entity, Kitware, Inc. ParaView is multi-platform, extensible via its plugin architecture, and natively supports many common data analysis tasks and data formats. As it builds upon VTK, any VTK functionality can in principle be invoked. In practice not all VTK functionality is exposed by default but can easily be exposed or extended via the plugin architecture previously mentioned and discussed in more detail below. Exposing VTK functionality is as easy as writing a short XML file. In this talk I present the process of plugging into ParaView to do visualization and analysis of terabytes of data in real time.

1. PARAVIEW AND ASTROVIZ
ParallelVisualizationTools for the Masses
Christine Corbett Moran
Institute forTheoretical Physics

2. GOALS OFTHISTALK
• Review the state-of-the-art in parallel analysis and visualization
• Introduce the ParaView plugin AstroViz
• Focus on generalized techniques of extending ParaView for
custom use cases
• Provide performance numbers and use cases

3. •Visualization
•VTK and ParaView
• AstroViz
• Performance
• Usage
• Next steps
OUTLINE

4. VISUALIZATION GOALS
• Tool for insight, communication, comparison, appreciation

5. ABSTRACT PROCESS

6. ABSTRACT PROCESS

7. ABSTRACT PROCESS

8. ABSTRACT PROCESS

9. DETAILED PROCESS
• Filtering
• Representation
• Perception

10. VISUALIZATION AND
ANALYSIS IN ASTROPHYSICS
• Parallel capabilities
• Wide variety of data
formats
• Extensible
• Robust
• Short learning curve
SimAm AstroMD Salsa Tipsy
Flash
View
ParaView python VisIt
SPLASH IDL MayaVi Splotch
yt gnuplot IFrIT OpenDX

11. •Visualization
•VTK and ParaView
• AstroViz
• Performance
• Usage
• Next steps
OUTLINE

12. VTK
• Open source, multi-platform visualization application
• Supports distributed computation models
• Extensible modular architecture

13. VTK AND PARAVIEW
• Open-source, multi-platform visualization applications
• ParaView is UI on top of theVTK framework
• Easily extensible via plugin development model

14. • ParaView represents
data as collections of
points and cells
• Calculations of filters
on these points and
cells largely
independent
• ParaView uses data
parallelism
PARALLEL COMPUTING IN
PARAVIEW

15. PARALLEL CAPABILITIES:
READERS
• Structured data: each
process reads a specific
coordinate extent
• Unstructured data: each
process reads a specific
fraction of the file

16. PARALLEL CAPABILITIES: DATA
DISTRIBUTION
• ParaView distributes data
spatially and evenly
• Structured data: done
automatically
• Unstructured data: done
manually with D3 filter

17. PARALLEL CAPABILITIES:
FILTERS
• Each process performs
same operation on its
piece of data
• Where necessary ghost
cells are utilized

18. PARALLEL CAPABILITIES:
RENDERING
• ParaView uses IceT
parallel rendering library
• Each process creates an
image based on its
partition of geometry
• Processes collectively
composite images

19. D3 FILTER
Before After

20. GHOST CELLS:
EXTERNAL FACES

21. PARALLEL RENDERING
WITH ICET
IceT Users’ Guide and Reference, Kenneth Moreland 2009

22. PARAVIEW PLUGINS
• Easier to create and deploy than directly modifying sourcecode
• Interface for developers to write modular software projects
and compile them against ParaView to produce libraries
• User can load any number of these libraries at runtime

23. PLUGIN DEVELOPMENT
REQUIREMENTS
• ParaView build and source used to compile it
• C++ source code to implement plugin’s features
• XML files to allow ParaView to use plugin
• CMakeLists.txt to set up plugin’s build environment

24. PLUGINTYPES
• Server-side
• Readers and writers
• Filters
• Client-side
• Readers and writers
• Object panels
• Toolbars
• Custom views

25. •Visualization
•VTK and ParaView
• AstroViz
• Performance
• Usage
• Next steps
OUTLINE

26. GUI
ParaView AstroViz
• Manipulate view in 3D
• Split view
• Link views
• Variety of display choices
• Additional menu, submenus
and buttons for ease of use
of AstroViz features
• Each AstroViz feature has an
option panel with user
customizable options

27. GUI INTERACTION

28. REALTIME INTERACTION

29. FILE FORMATS
ParaView AstroViz
• VTK
• Comma separated values
• XDMF
• Saved ParaView state
• Tipsy binary
• Marked particle files
• Additional ASCII attributes

30. FILE FORMATS:
C++ IMPLEMENTATION

31. FILE FORMATS:
GUI IMPLEMENTATION

32. PARALLEL READING
• Each process reads from to
if and to otherwise

33. DATA REDUCTION
ParaView AstroViz
• Reduce data to particles
selected in GUI
• Export selected particles and
their associated data
• Threshold based on input
attributes or simple functions
of them
• Option to load in only certain
attributes of particles
• Read in particles based on
“marked” file
• Threshold based on quantities
computed with AstroViz’s additional
analysis capabilities
• Halo finding

34. DATA REDUCTION

35. HALO FINDING
• A halo is a group of gravitationally bound objects
• Friends-of-Friends (FOF) halo finding algorithm
• A FOF halo is set of objects for which every object in the
set is within a linking length scale from at least one other
object in the set
• Purely geometric
• One free parameter, the linking length

36. ANALYSIS
ParaView AstroViz
• Simple functions of input data
• Plot data
• Statistics
• Python scripting
• Smooth quantities
• Compute the virial radius
• Center of mass
• Cumulative quantities within a radius
• Radial, tangential, and circular
velocity
• Angular momentum
• Principle moments of inertia

37. ANALYSIS:
FULL SCREENSHOT

38. KD-TREES
• Offers efficient way to
• Locate a point in space
• Identify N-nearest neighbors
• Locate points within a given spatial
area
Figure reproduced from Dr. BenjaminTyner, Department of
Statistics, Purdue University, 2009.
• Build Kd-Tree data structure

39. KD-TREE

40. VIRIAL RADIUS
• Formally the virial radius the radius at which virial equilibrium
holds
• Is often approximated as
(tensor) (scalar with )

41. CENTER OF MASS
• Summation operation is
commutative
• Processes calculate summation
independently
• Results combined and final
division performed

42. PRINCIPLE MOMENTS OF
INERTIA

43. PROFILE
• Calculate various physical
quantities as a function of radius
• Proceeds by grouping particles
into equal radius bins

44. PROFILED QUANTITIES

45. PARALLEL CAPABILITIES
ParaView AstroViz
• Read certain native formats in
parallel
• Parallel rendering
• Data distribution
• Certain analysis and data
reduction operations
automatically done in parallel
• All AstroViz features capable
of running in parallel

46. •Visualization
•VTK and ParaView
• AstroViz
• Performance
• Usage
• Next steps
OUTLINE

47. • Quantify how much faster an algorithm will run on parallel
computer with N processes
• Where is the time it takes to execute the algorithm on
processes
PARALLEL COMPUTING

48. • Every non-trivial algorithm has some serial component
• As processes are added to the computation of a parallel
algorithm, a decreasing return is seen on algorithmic speed
• If is the fraction of the algorithm that is inherently serial
then the algorithm will instead have the following speedup,
upper bounded by , known as Amdahl’s law:
PARALLEL COMPUTING

49. PERFORMANCE BENCHMARKS
• GHALO
• B1, B2, B3 resolutions with 11 million, 141 million, and 3
billion particles respectively
• Machines
• ZBox3 CPU cluster
• Horus GPU cluster

50. MACHINES
• ZBox3 CPU Cluster
• 144 nodes with quad core 2.4 GHZ Intel CPUS, 8GB main
memory per node
• Dolphin SCI highspeed interconnects
• Horus GPU/CPU Cluster
• 16 nodes 2.4 GHZ AMD Opteron Processor 250, 192 GB
total memory
• two NVIDIA Quadro FX 4500 GPUS per node

51. READING

52. ANALYSIS

53. INTERACTIVEVISUALIZATION

54. INTERACTIVEVISUALIZATION

55. LARGE PARTICLE NUMBER

56. LARGE PARTICLE NUMBER

57. •Visualization
•VTK and ParaView
• AstroViz
• Performance
• Usage
• Next steps
OUTLINE

58. DATA REDUCTION

59. DATA REDUCTION

60. DATA REDUCTION

61. DATA REDUCTION

62. DATA REDUCTION

63. DATA REDUCTION

64. DATA REDUCTION

65. REALTIME ANALYSIS

69. REALTIME ANALYSIS

70. REALTIME ANALYSIS

73. REALTIME ANALYSIS

74. REALTIME ANALYSIS

75. SCRIPTED ANALYSIS

76. MOVIES

77. MOVIES

78. •Visualization
•VTK and ParaView
• AstroViz
• Performance
• Usage
• Next steps
OUTLINE

79. ASTROVIZ: NEXT STEPS
• Increased support for hydrodynamical codes (both AMR and
SPH) and their common analysis tasks
• Increased support for observational astrophysics visualization
and analysis tasks including the FITS format and pixel based
operations
• Performance optimizations and additional file formats
• Features suggested by community

80. GOALS OFTHISTALK: REVIEW
• Review the state-of-the art in parallel analysis and visualization
• Introduce the ParaView plugin AstroViz
• Provide performance numbers and use cases

81. RELEASE INFORMATION
• Open source BSD license
• Website
• http://www.itp.uzh.ch/~corbett/astroviz/astroviz.html
• Feedback and feature requests
• http://astroviz.uservoice.com/

82. QUESTIONS?