The document describes a cloud-based virtual microscopy system called VIRTUM. It allows uploading of various types of microscopy image files which are then processed using a library of 70+ image analysis algorithms running on GPUs. This enables fast visualization, measurement, segmentation and analysis of 2D and 3D microscopy images for applications in medicine, biology and other fields. Key benefits of VIRTUM include flexibility, accelerated processing, and intuitive workflow-based analysis of large image datasets in the cloud.
IAC 2024 - IA Fast Track to Search Focused AI Solutions
PG-4035, Virtual Microscopy in the cloud, by Wojciech Tarnawski
1. VIRTUAL
MICROSCOPY
IN
THE
CLOUD
WOJCIECH
TARNAWSKI
,
CSO
MICROSCOPEIT
LTD.
1
|
PRESENTATION
TITLE
|
NOVEMBER
21,
2013
|
CONFIDENTIAL
2. Virtual
Microscopy
in
the
Cloud
Wojciech
Tarnawski,
PhD,
CSO
MicroscopeIT
Ltd.,
Wroclaw,
Poland
2
|
PRESENTATION
TITLE
|
NOVEMBER
21,
2013
|
CONFIDENTIAL
3. MICROSCOPY
IS
COMPLICATED
! Different formats, different producers.
! Different software for different image processing tasks.
! Image analysis takes time.
! Open Source
vs.
Commercial Software.
! Image types: 2D (fluorescence, phase-contrast), 3D
(confocal), 4D (3D objects in time), different channels targeting
different molecular elements.
3
|
PRESENTATION
TITLE
|
NOVEMBER
21,
2013
|
CONFIDENTIAL
CreaMve
Commons
2.0,
Nicole
Yeary's
photos
via
GeRy
Images
4. WHAT
IS
VIRTUM?
Cloud Computing
Image processing pipeline
integrated accessed in the web
browser.
Acceleration
Time consuming image analysis
ported to GPU.
Robust and fast workflow-based
image analysis
Save time thanks to intelligent
algorithms with „visual” development.
Image
credit:
leverhawk.com,
Why
is
cloud
integraMon
sMll
an
adopMon
barrier,
2012.
4
|
PRESENTATION
TITLE
|
NOVEMBER
21,
2013
|
CONFIDENTIAL
Information Retrieval
Phenotype detection of biologically
relevant information directly from images.
Flexibility
All formats, dimensions and
modality supported
5.
IN
ACTION
Our
system:
32
GPU
cards
(6
donated
by
AMD)
Data
acquisi:on
Database
" Work-‐flow
based
image
processing
and
task
scheduling
5
|
PRESENTATION
TITLE
|
NOVEMBER
21,
2013
|
CONFIDENTIAL
6. FEATURES,
APPLICATIONS
Visualization (Virtual Microscopy)
Medicine and
biology
Clinical trials
Scientific
research
E-learning
Teleconferencing
teleconsultations
Quantitative
data analysis
Biotechnology
High-Content and
High-Throughput
Screening
Data Analysis
6
|
PRESENTATION
TITLE
|
NOVEMBER
21,
2013
|
CONFIDENTIAL
7.
2D
Image
Series
Viewer
Visualiza:on
WSI
VisualizaMon
3D
Image
Series
Movie
ProjecMon
3D
Geometry
Rendering
ReconstrucMon
Input
Data
Types
Not Ordered
WSI
Image z-‐stacks
Time-‐Lapse Time-‐Lapse
Images
( Image Pyramids)
Image Series
Z-‐Stacks
7
|
PRESENTATION
TITLE
|
NOVEMBER
21,
2013
|
CONFIDENTIAL
8. INPUT
DATA
TYPES
:
NOT-‐ORDERED
SETS
AND
TIME-‐LAPSE
IMAGE
SERIES
8
|
PRESENTATION
TITLE
|
NOVEMBER
21,
2013
|
CONFIDENTIAL
9. INPUT
DATA
TYPES
:
Z-‐STACKS
AND
TIME-‐LAPSE
Z-‐STACKS
9
|
PRESENTATION
TITLE
|
NOVEMBER
21,
2013
|
CONFIDENTIAL
10. INPUT
DATA
TYPES
:
IMAGE
PYRAMID
10
|
PRESENTATION
TITLE
|
NOVEMBER
21,
2013
|
CONFIDENTIAL
11.
2D
Image
Series
Viewer
Visualiza:on
WSI
VisualizaMon
3D
Image
Series
Movie
ProjecMon
3D
Geometry
Rendering
ReconstrucMon
Input
Data
Types
Not Ordered
WSI
Image z-‐stacks
Time-‐Lapse Time-‐Lapse
Images
( Image Pyramids)
Image Series
Z-‐Stacks
Image Processing and Analysis Library
2-‐3D Mesurements Image Preprocessing
2-‐3D Object SegmentaDon 2-‐3D Object Analysis StaDsDcs
Data
Analysis
2D Image Processing 2-‐3D Image ReconstrucDon Time-‐Dependent Analysis Post-‐Processing
and Analysis
11
|
PRESENTATION
TITLE
|
NOVEMBER
21,
2013
|
CONFIDENTIAL
12. CLOUD
COMPONENTS
(BACK-‐END)
1/3
Image Processing and Analysis Library : about 70 methods tailored
for microscopy imaga data implemented on CPU and GPU
2-‐3D Mesurements
Image Preprocessing : noise removal, contrast improvement, inhomogeneous
lighDng removal, opDcal deconvoluDon, 2-‐3D Image SDtching, Histogram-‐based
processing, MulD-‐channel Image Composing, Image ArithmeDc, Edge DetecDon, …
etc.
2-‐3D Object SegmentaDon : automaDc or machine-‐learning methods for
segmentaDon of 2-‐3D objects e.g. 2-‐3D Cell Tracking Advanced SegmentaDon in
mulD-‐dimensional space composed with texture and color features, AcDve Contour
and AcDve Mesh, Threshold -‐ and Morphology – based SegmentaDon, Mean-‐Shi[,
…
2-‐3D Object Analysis : Split into 2-‐3D Ellipsoids e.g. for highly clustered cells ,
Morphology Operatos , Weighted Distance Transform, Voronoi TriangulaDon, Object
RecogniDon module for Cell Phase ClassificaDon by Markov chains
12
|
PRESENTATION
TITLE
|
NOVEMBER
21,
2013
|
CONFIDENTIAL
StaDsDcs Module – PCA, Basic StaDsDcs, Cluster Analysis,
13. 3D
IMAGE
SEGMENTATION
:
ACTIVE
MESH
13
|
PRESENTATION
TITLE
|
NOVEMBER
21,
2013
|
CONFIDENTIAL
14. CLOUD
COMPONENTS
(BACK-‐END)
1/3
Image Processing and Analysis Library : about 70 methods tailored
for microscopy imaga data implemented on CPU and GPU
Workflow-‐based
image
processing
*
A
Robust
Algorithm
for
Segmen:ng
and
Tracking
Clustered
Cells
in
Time-‐Lapse
Fluorescent
Microscopy
Tarnawski,
W.
;
Kurtcuoglu,
V.
;
Lorek,
P.
;
Bodych,
M.
;RoRer,
J.
;
Muszkieta,
M.
;
Piwowar,
L.
;
Poulikakos,
D.
;Majkowski,
M.
;
Ferrari,
A.
Biomedical
and
Health
InformaMcs,
IEEE
Journal
of
Volume:
17
,
Issue:
4
PublicaMon
Year:
2013
,
Page(s):
862
-‐
869
14
|
PRESENTATION
TITLE
|
NOVEMBER
21,
2013
|
CONFIDENTIAL
15. WORKFLOW
–
BASED
IMAGE
PROCESSING
AND
ANALYSIS
15
|
PRESENTATION
TITLE
|
NOVEMBER
21,
2013
|
CONFIDENTIAL
16. USE
CASES
! Detection of nuclei and cytoplasm in 80 000 images (512x512 pixels) takes about 2 hours on multicore CPU (AMD
Athlon(tm)
II
X4
640
Processor).
GPU provided up to 4x acceleration
! Optical deconvolution : about 25x acceleration for 512x512 image
! 3D-dimensional diffuse filter on image-stack (z-stack with 1920x1080) : about 10x acceleration
16
|
PRESENTATION
TITLE
|
NOVEMBER
21,
2013
|
CONFIDENTIAL
17. CLOUD
COMPONENTS
(BACK-‐END)
2/3
Image Processing and Analysis Library : about 70 methods tailored
for microscopy imaga data implemented on CPU and GPU
Task Scheduler to provide image analysis results for many users.
Scheduling approach :
Scheduler –> Executor –> Worker –> Task
-‐
Schedules
image
processing
tasks
on
the
CPU
&
GPU
cluster.
-‐
Monitors
CPU,
GPU,
memory,
storage
usage.
-‐
OpMmizes
scalability.
17
|
PRESENTATION
TITLE
|
NOVEMBER
21,
2013
|
CONFIDENTIAL
18. CLOUD
COMPONENTS
(BACK-‐END)
3/3
Image Processing and Analysis Library : about 70 methods tailored for
microscopy imaga data implemented on CPU and GPU
Task Scheduler to provide image analysis results for many users.
Database Module -‐ to store the microscopic image data
Database Module provides upload data module that supports:
• about 100 microscopic image data formats (i.e. lsm, nd2, oly, mulD-‐
channel , 16-‐bit Dff, basic graphic formats, …)
• compressed images series (zip)
• filename parser to upload image series ordered by channel, z-‐stack
layers, Dme-‐points, …
• users data are fully organized
• users can be assigned to many projects
18
|
PRESENTATION
TITLE
|
NOVEMBER
21,
2013
|
CONFIDENTIAL
20. CLIENT
(GUI)
COMPONENTS
Graphical
User
Interface
(GUI)
installed
as
a
plugin
in
the
web
browser:
! Designed
for
touch-‐based
devices.
! Designed
to
tag
microscopic
image
series
with
metadata.
! Includes
different
viewers
to
visualize
mulM-‐dimensional
images.
! Provides
„visual”
interface
to
design
the
workflow
for
image
processing
and
analysis.
! Provides
tools
to
select
the
image
regions
for
futher
iamge
analysis.
20
|
PRESENTATION
TITLE
|
NOVEMBER
21,
2013
|
CONFIDENTIAL