NDA presentation given by Dr. Jon McCarthy, Senior Scientist at Wisconsin Materials Institute

1. Networked Data Storage and Analysis for the
Wisconsin Regional Materials Network
6/3/2015 M&M 2015 Portland 1
Jon J. McCarthy, Paul M. Voyles, Julie A. Last, Davis R. Bittner, Ian W. Sadkovich
Wisconsin Materials Institute, University of Wisconsin, Madison WI, USA
Materials Science and Engineering Department, University of Wisconsin, Madison WI, USA
Materials Science Center, University of Wisconsin, Madison WI, USA
Computer-Aided Engineering Department, University of Wisconsin, Madison WI, USA

2. Our “big” data issues at UW Madison
Instrument data output is growing…. FAST!
Issues with continuing to transfer data the “Same Old Way”
The importance of including “off the local computer “Analysis Capability
A couple examples
Things we have learned so far…
M&M 2015 Portland 26/3/2015

3. Problem Statement:
Large Images/multispectral data sets
• Need to be stored…OFF the Instrument COMPUTER
• Virus avoidance is critical
• Dedicated sub networks
• How Large is large? EM Tomography data
• Now 0.5 Gb per 100 slice stack, soon to be 2 to 16+ GB per stack
• But also Processed ….
• Which leads to even more images…and more storage space
• Often Proprietary commercial software required….
• How many licenses can you afford?
Can your collaborators access data and analysis it output easily?
• Without access to rest of your computer networks and restricted software?
M&M 2015 Portland 36/3/2015
Wi Regional Materials Network

4. EM Tomography Data Storage Requirements:
• Electron Tomography of Cryo Sections :
What is Next?
Happening now in fact!
6/3/2015 M&M 2015 Portland 4
The Output is 0.5 GB
of raw Image data in a
single tilt series.
With no binning its 2
GB of data.
K2 16 Megapixel
Direct Electron Detector
The Output is
potential for 4 to 32
GB for a single tilt
series!
GIF:
binned 2X2

5. AP Tomography Data Storage Requirements:
6/3/2015 M&M 2015 Portland 5
The Output is 2.8 GB of
raw Image “hit” data file
that is processed to
generate this image.

6. Getting data off the instrument: No Portable Drives!
The trouble with Thumb and Portable Drives: Viruses and Limited Capacity
M&M 2015 Portland 66/3/2015
NOT Acceptable, but all to common!

7. Getting data off the instrument: Limited LAN
Internal sub-net to another Lab PC for Storage; Lock rack door to the both Instrument PC’s!
M&M 2015 Portland 76/3/2015
Instrument
Control PC
Support PC
(crossover
Cable)
This was a satisfactory solution for several years
BUT…issues are:
• LAN running out of disk Storage
• 20 users, output is 4 TB per year. Limited
LAN host is maxed out below that level!
• Duplicating hosts and more storage and
Licensed processing software quickly
becomes cost prohibited !
• SO users bring out the portable drives AGAIN
…. OR
• Users must come to the host to process data.
• One user at a time can process data (its
not a server, just a host)
• Access for outside UW Collaborators very
difficult!
We needed a scalable Solution!

8. Networked Data and Analysis Solution
A data storage and processing on a secure, networked platform:
We chose the Citrix XenAPP Platform
M&M 2015 Portland 86/3/2015
Citrix XenAPP provides delivery over the
Internet of data analysis applications with
access to a central data repository.
• Users acquire their data on the instrument.
• Data is copied off the instrument
computers to a centralized data store.
• The data store is connected to an analysis
software server.
• Users can connect to the analysis software
server from any Windows or Mac computer
with Internet connection faster than 1
Mbit/sec.
• Raw data and analysis Output are stored
on the server, or copied back to the client
computer.

9. NDA Advantages
• The Instrument control computers do not see Thumb Drives, USB Storage, or even the
UW Network!
• Secure Login, accesses only client machine and the XenAPP server.
• Hardware easy to expand as storage needs grow; Currently, 24 cores on two
processors and 256 MB of RAM, 4 TB of storage. EASY to SCALE!
• Data can be sent to server directly from instrument Support Computer and downloaded
to client computer as needed.
• The XenApp client runs on the user’s computer, but the application runs on a University
(CAE) managed Windows server.
• XenApp delivery is legal for most end user license agreements (EULAs) for data
analysis applications.
• Outside users cannot access other UW resources or other computers on the CoE
network.
6/3/2015 M&M 2015 Portland 9

10. Example: Launching XenAPP
6/3/2015 M&M 2015 Portland 10
Secure Login
Access to only these applications

11. Client and Server Drives auto-mapped
6/3/2015 M&M 2015 Portland 11
Client drives
XenAPP Server Drives
NO Access to UW computers

12. Example: EM Tomography
6/3/2015 M&M 2015 Portland 12
On my desktop but running on XenAPP Server
Tomogram created with IMOD/etomo on
XenAPP! Made into an AVI with Fiji

13. Frame from a FIJI movie of a rendered Tomogram
6/3/2015 M&M 2015 Portland 13
Movie of FHV
rendered stack
running in XenAPP
• On my laptop (i7,
8GB Dell)
• At home, 11
Mbps download
speed!

14. Collaborator Access Example
6/3/2015 M&M 2015 Portland 14
UW Milwaukee/UW Madison/Thermo Scientific Collaborators
Li Cells, Built, Charged
In Milwaukee.
Anodes extracted in Glove Box, Loaded into
Thermo Vacuum Transfer Vessel in Madison
XPS
Spectral
Imaging
Data
Data on Instrument Computer transferred to NDA Server

15. Collaborator Access Example
6/3/2015 M&M 2015 Portland 15
Spectral Images Processed, Quantified entirely on Thermo Avantage on XenAPP Server
Atomic % for all elements from three anodes determined,
and plotted in Excel, by collaborators in MKE and Madison!

16. TIPS: What we have Learned so far
 Unless you can manage the Server yourself, get a key contact in your IT group to be
the “owner” of XenAPP administration.
 Admin and negotiation of License and EULA
 How do you load and configure plug’s or upgrades?
 Commercial Software seems to be more likely to be easy to configure on a LAN and
XenAPP
o Many packages (with a LAN License) work “out of the box”
o Shareware is often configured for a single user and a desktop/laptop
 Often use code or scripts that select specific files or path names that may be blocked on the Server!
 It took three rounds to get 3DMOD to work, then 3 more to get etomo to work
 Have a Skilled User do testing of any new XenAPP application!
 IT folks are computer savvy, but not experts on your application!
 You don’t want to learn the software and debug XenAPP install issues at the same time!
 Have the tester user write up “how to use IMOD on XenAPP” tutorial document.
6/3/2015 M&M 2015 Portland 16

17. Acknowledgements
6/3/2015 M&M 2015 Portland 17
Co-Authors
Paul M. Voyles
Materials Science and Engineering Department
Julie Last, Alex Kvit and John Jacobs
Materials Science Center
Davis R. Bittner and Ian W. Sadkovich
Computer-Aided Engineering Department
Special Thanks (for tomography work) to:
Ken J. Ertel
Fellow,
Institute For Molecular Virology
UW Madison
Desiree Benefield
Post Doc Research Associate
Morgridge Institute for Research
Weixin Chen, Research Associate
Materials Science and Engineering Department
UW Madison
External Collaborators:
Xingkang Huang
Mechanical Engineering Department
University of Wisconsin-Milwaukee
Bill Sgammato
Thermo Fisher Scientific

18. 6/3/2015 M&M 2015 Portland 18
QUESTIONS?

19. Movie is so Cool I have to show you!
6/3/2015 M&M 2015 Portland 19

20. 6/3/2015 M&M 2015 Portland 20

21. Motivation for moving from Limited LAN to NDA System
6/3/2015 M&M 2015 Portland 21
Data Output and Processing bottlenecks:
• Quickly running out of disk space—if one user produces 16.5 GB in a week, and you have 20
user-> 4 TB per year. Limited LAN host is maxed out below that level. The data output will
quadruple with the direct electron detector!
• Duplicating hosts and more storage and Licensed processing software quickly becomes cost
prohibited !
• Users must come to the host to process data. Only one user at a time can process data (its not
a server, just a host)
• It often takes as long (or longer!) to process tomograms as to collect the data!
• If Host is in the lab (Atom Probe) , it is cold and noisy and crowded. The next user is collecting their
data, space is crowded!
SO what Happens?
• Users get a license for their PC and use the Portable Drives again to move data to their own
machine.
• Multiple licenses for software raise cost for each user group, so still have the one person in the
lab processing at a time!

22. 22
Materials Science Center (MSC)
• Three FESEMs and AURIGA FIB, PHI Auger, 4 XRDs, including Bruker Discover D8 (microfocus
source, automated stage, WAXs detector) JY Horiba Micro-imaging Raman and Micro FT-IR,
UV/VIS/NIR spectroscopy, DI Multimode AFM, Bruker Catalyst Bio AFM, Zeiss META CLSM,
Thermo Imaging XPS, FEI 200KV Titan, FEI TF-30 CryoTEM for electron tomography, Cameca
LEAP 3000si Atom Probe.
Soft Materials Laboratory (SML)
• Rheometer, Dynamic Mechanical Analysis, 2 HPLC, TGA, DSC, contact angle, Ellipsometry,
Zeiss 1550 VPSEM, Thermo Scientific Micro-Imaging Raman
Wisconsin Center for Advanced Microelectronics (WCAM)
• Seven Bay, 10,000 Square foot clean room Facility, over 60 fab tools
• CMOS devices, MEMS, soft lithography, DC/RF sputter deposition tool.
UW College Of Engineering Shared Facilities
Instruments in Blue (most recent!) create Analytical Spectral Images and Image stacks -which
brought us to a tipping point, but Electron Tomography pushed us over the edge!

23. Loading an IMOD stack from client HD
6/3/2015 M&M 2015 Portland 23

24. Getting data off the instrument: Solution #2
Local 1 to 2 TB portable hard
disk
• It stays in the instrument
room - always!
• But does It really??
M&M 2015 Portland 246/3/2015
ISSUES:
• Researchers must come here to
Process
• It often takes as long (or longer!) to
process tomograms as to collect the
data!
• Only one user at a time!
• For the Atom Probe;
• Its cold and noisy in the lab
• The next user is collecting their
data, space is crowded!
 NEED to Process the data
somewhere ELSE!

25. Getting data off the instrument: Limited LAN
A support computer, with crossover Ethernet cable to instrument control computer!
M&M 2015 Portland 256/3/2015
Instrument
Control PC
Support PC
(crossover
Cable)
ISSUES:
• Better than solution 2, but one thumb
drive inserted in the wrong computer
brought down our Titan! Led to a Disk
Rebuild!
• It often takes as long (or longer!) to
process tomograms as to collect the
data!
• Only one user at a time
• Run out of disk space on the support
Computer—if one user generates
16.5 GB, and you have 20 users…
 NEED to process Data somewhere
ELSE
Support computer monitor