9. BDTC - Beijing, 2013-12-6
Large Volume Visualization
! Level of Details
! Out of Core
! Parallel Visualization
9
10. BDTC - Beijing, 2013-12-6
10
Top 10 Challenges in Extreme-Scale Data
Visual Analytics
Pak Chung Wong (PNNL)
Han-Wei Shen (OSU)
Chris Johnson (Utah)
Chaomei Chen (Drexel)
Robert Ross (Argonne)
11. BDTC - Beijing, 2013-12-6
Top 10 Challenges in ExtremeScale Data Visual Analytics
11
! In Situ Analysis
! Perform as much analysis as possible while the data are still in
memory
! Interaction and User Interfaces
! Machine-based automated systems vs. Human Cognition
! Large-Data Visualization
! Data projection and dimension Reduction, display technology
! Databases and Storage
! A cloud-based solution might not meet the needs
! Algorithms
! Address both data-size and visual-efficiency issues
12. BDTC - Beijing, 2013-12-6
Top 10 Challenges in ExtremeScale Data Visual Analytics
12
! Data Movement/Transport, & Network Infrastructure
! Efficiently use networking resources and provide convenient
abstractions
! Uncertainty Quantification
! Cope with incomplete data
! Parallelism
! Domain and Development Libraries, Frameworks, and Tools
! Affordable resource libraries, frameworks, and tools
! Social, Community, and Government Engagements
13. BDTC - Beijing, 2013-12-6
Challenges in Big Data
Visualization/Visual Analytics - 1
! Integrating heterogeneous Data from different resources and
scales
13
20. BDTC - Beijing, 2013-12-6
Preprocessing: Map
Matching
Raw taxi
GPS
Data
Raw Road
Network
Cleane
d GPS
Data
Processed
Road
Network
Map Matching
GPS Trajectories
Matched
to the Road Network
20
22. BDTC - Beijing, 2013-12-6
Visual Interface: Single Road
Level
22
! Pixel based visualization
Time of a day: 144 columns (each for a 10min)
Days: 24 rows
(each for one day)
Each cell represents one time bin
Color encode speed
23. BDTC - Beijing, 2013-12-6
Case Study: Road Level
Exploration and Analysis
! Different road congestion patterns
23
24. BDTC - Beijing, 2013-12-6
Case Study: Road Level
Exploration and Analysis
24
25. BDTC - Beijing, 2013-12-6
25
Propagation Graph Analysis
! Spatial Temporal information of one propagation
Large delay
Spatial path
Temporal delay
26. BDTC - Beijing, 2013-12-6
Propagation Pattern
Exploration
! Propagation graphs for one region in the morning of different
days
26
40. BDTC - Beijing, 2013-12-6
Challenges in Big Data
Visualization/Visual Analytics - 2
! Integrating heterogeneous Data from different resources and
scales
! Scalability in Data/Task complexity
! Data inherent properties impose more computational challenges
methods for visualization and visual analysis on big data
40
43. BDTC - Beijing, 2013-12-6
43
Multivariate to Multi-Run
Visual Analysis
QVAPOR
QVAPOR
QCLOUD
Pressure
Speed
Run 1
QCLOUD
QVAPOR
QCLOUD
Pressure
Speed
QVAPOR
Run 2
Pressure
Speed
(Multivariate)
QVAPOR
QCLOUD
Pressure
Speed
(Ensemble Runs)
Run 3
44. BDTC - Beijing, 2013-12-6
Eulerian and Lagriangian
Specifications
! Eulerian:
! Lagriangian:
! Relationships between two specifications (flow map):
44
45. BDTC - Beijing, 2013-12-6
Eulerian-based Attribute
Space Projection
! Samples on data grid !
Samples in attribute space !
Eulerian-based Attribute Space Projection
(EASP)
45
46. BDTC - Beijing, 2013-12-6
Lagrangian-based Attribute
Space Projection
! Pathlines on data grid !
Pathlines in attribute space !
Lagrangian-based Attribute Space Projection (LASP)
! Both multivariate scalar fields and vector field are considered
46
48. BDTC - Beijing, 2013-12-6
48
Couple Ensemble Flow Line Advection
and Analysis (eFLAA)-Concept
! Ensemble data (large)
! Field line data (much larger than ensemble data)
! Variation field (small)
! Filtered lines (even smaller)
[Guo, Yuan, Huang and Zhu TVCG 2013 (SCIVis ‘13)]
53. BDTC - Beijing, 2013-12-6
GEOS-5 Simulation: CO2based Metric
53
! The metric: the differences of locations / CO2
concentration along the pathline
! Findings
! The variation of the wind field is high in the north hemisphere
! However, The CO2 difference is higher in south hemisphere and
some places in the north
! CO2 concentration is not sensitive to wind in above regions
54. BDTC - Beijing, 2013-12-6
Challenges in Big Data
Visualization/Visual Analytics - 3
! Integrating heterogeneous Data from different resources and
scales
! Scalability in Data/Task complexity
! Data inherent properties impose more computational challenges
methods for visualization and visual analysis on big data
! Limited access in Interaction for Large Data
54
57. BDTC - Beijing, 2013-12-6
Real-time Visual Querying of
Big Data
!
imMens
57
58. BDTC - Beijing, 2013-12-6
Real-time Visual Querying of
Big Data
!
!
58
59. BDTC - Beijing, 2013-12-6
Nanocubes for Real-Time Exploration
of Spatiotemporal Datasets
!
59
60. BDTC - Beijing, 2013-12-6
Challenges in Big Data
Visualization/Visual Analytics - 4
! Integrating heterogeneous Data from different resources and
scales
! Scalability in Data/Task complexity
! Data inherent properties impose more computational challenges
methods for visualization and visual analysis on big data
! Limited access in Interaction for Large Data
! Scalability in User
! Collaborative Visualization and Analysis on large data
! Can scientist create novel visualization without programming
60
61. BDTC - Beijing, 2013-12-6
61
Double Gulf
Visualization
Designer
Visualization
User
Representation
Evaluation
Data
Visualization
Conceptual
Model
Execution
Manipulation
62. BDTC - Beijing, 2013-12-6
62
Double Gulf
Visualization
Designer
Visualization
User
Representation
Evaluation
Data
Visualization
Conceptual
Model
Execution
Manipulation
63. BDTC - Beijing, 2013-12-6
63
From Data to User
Visualization
User
Evaluation
Execution
Visualization
Designer
Representation
Manipulation
64. BDTC - Beijing, 2013-12-6
64
Scalability In Users
Visualization
Designer
Visualization
User
Representation
Evaluation
Data
Visualization
Conceptual
Model
Execution
Manipulation
74. BDTC - Beijing, 2013-12-6
Challenges in Big Data
Visualization/Visual Analytics - 5
! Integrating heterogeneous Data from different resources and
scales
! Scalability in Data/Task complexity
! Limited access in Interaction for Large Data
! Scalability in User
! System Development
! Domain and Development Libraries, Frameworks, and Tools
! Social, Community, and Government Engagements
74
75. BDTC - Beijing, 2013-12-6
75
SCIVIS Visualization Systems
! VisIt - LLNL
https://wci.llnl.gov/codes/visit
! ParaView- Kitware/SNL/LANL
http://www.paraview.org
! IceT (Image Composition Engine for Tiles) - Sandia
http://icet.sandia.gov
! Daxtoolkit - Data Analysis at Extreme
http://www.daxtoolkit.org
! PISTON - Portable Data-Parallel Visualization and Analysis Library LANL
http://viz.lanl.gov/projects/PISTON.html
76. BDTC - Beijing, 2013-12-6
VisIt
! Production end-user tool supporting
scientific and engineering
applications.
! Parallel post-processing that scales
from desktops to massive HPC
clusters.
76
77. BDTC - Beijing, 2013-12-6
77
Development of VisIt
! The VisIt project started in 2000 to support LLNL’s large scale ASC
physics codes.
! Supported by multiple organizations: LLNL, LBNL, ORNL, UC Davis,
Univ. of Utah, …
! Over 75 person years effort.
! 1.5+ million lines of code.
Based on SC’11 Tutorial
79. BDTC - Beijing, 2013-12-6
79
VTK
W.J. Schroeder, K. Martin, and W. Lorensen, The
Visualization Toolkit: An Object Oriented Approach to
Computer Graphics, Third Edition, Kitware, Inc.,
ISBN-1-930934-12-2 (2004).
S. E. Rogers, D. Kwak, and U. K. Kaul, A numerical study of
three-dimensional incompressible flow around multiple
post. In Proceedings of AIAA Aerospace Sciences
Conference. AIAA Paper 86-0353. Reno, Nevada, 1986.
80. BDTC - Beijing, 2013-12-6
ParaView
! 2000 Los Alamos National Laboratories and Kitware Inc.
! 2005 Sandia National Laboratories and Kitware Inc.
! Used by academic, government, and commercial institutions
worldwide.
! Downloaded ~100K times per year.
80
83. BDTC - Beijing, 2013-12-6
Starlight Information
Visualization System
83
84. BDTC - Beijing, 2013-12-6
Build a successful vis system
! System Design
! Domain User – Visualization Scientist “Co-design”
! Stable Development Team
! Funding Mechanism
84
85. BDTC - Beijing, 2013-12-6
Build a successful vis system
! System Design
! Domain User – Visualization Scientist “Co-design”
! Stable Development Team
! Funding Mechanism
85
87. BDTC - Beijing, 2013-12-6
Challenges in Big Data
Visualization/Visual Analytics - 6
! Integrating heterogeneous Data from different resources and
scales
! Scalability in Data/Task complexity
! Limited access in Interaction for Large Data
! Scalability in User
! System Development
! Visualization Experts
87
90. BDTC - Beijing, 2013-12-6
90
Social, Community, and
Government Engagements
! Universities
!
!
!
!
!
!
!
!
University of Tennessee in Knoxville
Ohio State University
SCI Institute, University of Utah
University of California, Davis
University of California, San Diego
University of Nebraska-Lincoln
Michigan Technological University
Drexel University
! Supercomputer centers
! San Diego Supercomputer Center (SDSC)
! Texas Advanced Computing Center
(TACC)
! National Center for Supercomputing
Applications at the University of Illinois
(NCSA)
! DoE Labs
! Argonne National Laboratory (ANL)
! Lawrence Berkeley National Laboratory
(LBNL)
! Lawrence Livermore National Laboratory
(LLNL)
! Los Alamos National Laboratory (LANL)
! Pacific Northwest National Laboratory
(PNNL)
! Oak Ridge National Laboratory (ORNL)
! Sandia National Laboratories (SNL)
! National Renewable Energy Laboratory
(NREL)
! Companies
! Kitware
91. BDTC - Beijing, 2013-12-6
91
Good News
! More and more universities started visualization research
program
! Many Companies are aware of the importance of visualization
! Still, lack of national infrastructure