SlideShare una empresa de Scribd logo
1 de 20
Overview of
Analysis Result Manager
         (ARM)
      Jeremy Magland

       27 October, 2011
  brainomics group @ Penn
Motivations for ARM
O Motivation #1:
Visualization
I wanted to be able to
SEE my fMRI data,
intermediate files, and
processing results,
and to share these
with my collaborators.
Motivations for ARM
O Motivation #2:                            Bone-Water Study
                            Real-time Study Subject AP-1
Data Management              REAL_23            Localizer
                       Sample Session 1         Baseline, TR=300
Organize fMRI data     text     Raw fMRI series Follow-up 1, TR=300
                       About   Anatomic localizer
on a central server,   study Session 2    Bone-Water Study
                       Stuff REAL_36       Subject AP-1
and access it from     To fill Session 1     Localizer
                       sapace Session 2      Baseline, TR=300
anywhere (cloud                              Follow-up 1, TR=300

style)
Motivations for ARM
O Motivation #3:
Computing Power
Utilize the processing
power of server
computers (CPU
power, RAM,
automatic parallel
processing)
Motivations for ARM
O Motivation #4:
Provenance Tracking
Track the processing
steps required to
reproduce each
intermediate object and
processing result.
Advantages:
* reproducibility
* auto-non-redundant
processing
Challenge: Richly Interactive,
Web-based Data Visualization
O Browser-based
                           O RemoteWidgets:
  applications are very
  convenient but often       O Framework for
  have limited                developing
  functionality.
O Desktop applications        desktop
  can have rich               applications in Qt4
  functionality. However      with option to
  there are some cons:
   O software
                              deploy on browser.
     downloads/updates
   O OS incompatibility
   O Security risks
   O Etc.
RemoteWidgets: Overview
                           Seegull

O Write a GUI for the
  web using only one                 ARM
  programming
  language: Qt4/C++
O Rapid development       Pickles

  of highly interactive
  web applications.
O Direct access to                         Math Puppy

  server-side
  processes.
RemoteWidgets:
     One App, Two Access Options
                       Seegull in Web Browser

O Web Browser
  Interface (jQuery)
O Desktop Interface
  (thin client
  download)
                       Seegull on Desktop
RemoteWidgets: Hello, World!
RemoteWidgets:
      Server-Side Event Handling
O Handling GUI events is as easy as
 connecting Qt signals and slots on the
 server… no javascript coding needed.
RemoteWidgets: Idea
O Interactive, desktop-style visualization tools
  available from the convenience of a web
  browser.
Analysis Result Manager
             (ARM)
O Cloud-based
  approach to fMRI
  data management,
  visualization, and
  analysis.
O ARM is a
  RemoteWidgets
  application, and can
  therefore be
  accessed remotely
  via a web browser.
Tradeoff Between
    Scripting and GUI for fMRI Processing




O Scripting is way more powerful for research purposes.
  And, if you have a good script interface, you could always
  throw a GUI on top of it.
O But… some problems with Matlab-style scripting:
      O Need to manually shuffle data around in
           files/folders
      O Not easy to remember exactly which parameters
           were used >2 days ago
      O Run a 10-hour script, and discover the last line
           had a bug.
ARM: Script-Generated Workflows
With Robust Tracking of Provenance Data




   The multi-threaded processing     The script immediately sets up
   module then executes each         a workflow, creating all
   processors as soon as all input   intermediate and result objects
   objects have been defined.        ahead of the processing,
                                     storing detailed information
                                     about how objects are related
                                     via processes.
Processors are Generic Plugins
C++ Wrappers (for FSL, SPM, etc)




 Views are also plugins… (custom views of custom datatypes)
Live ARM Demonstration
O   Upload Scan Data                   O   View processing results and
O   Online view, change view               intermediate files.
    settings                           O   Show provenance (process
O   Super-efficient browsing of huge       history) of the output objects.
    data files.                        O   Explain that changing scan id’s
O   Browser access to same app             and even names of the
O   View example script                    processors causes no
                                           problems. Key assumption:
O   Running script does nothing            objects, once created, never
    because processing has already         change.
    completed
                                       O   Clean up by deleting results. (In
O   Change design function and re-         future, intermediate files will be
    run script – show how                  auto-removed after a period if
    processes are queued and               they are no longer linked to
    executed in proper order (No           results)
    redundant processing).
                                       O   In development – share a
                                           processing result by a button
                                           click.
Live Seegull Demonstration
O It is often desirable to SHARE results with others.
  But you don’t always want to give them access to a
  huge system. Furthermore, since ARM is highly
  dynamic, you want to be sure that a result, once
  shared, is available for ever after (snapshot of a
  result)
O Hence, Seegull
Summary and Future
O Share interactive views of data
O Taking full advantage of cloud technology
    to manage, process, visualize, compare
    fMRI data analyses
O   Script-generated workflows
O   Automatic provenance tracking
O   Convenient access via web browser
O   Import publicly available fMRI studies into
    system (powerful meta-analyses)
Especially Thank You to…
O Anna Rose Childress
O Felix Wehrli
O Charlie Epstein


And…
O   Benny Lam        O   John Detre     O Chris Tjoa
O   Oscar Bartra     O   Jesse Lupardus O Brian Avants
O   Kimberly Young   O   Ning Zhang
O   Dan Willard      O   Cheng Li
O   Rob Fabianski    O   Ze Wang

Más contenido relacionado

La actualidad más candente

Dark Silicon, Mobile Devices, and Possible Open-Source Solutions
Dark Silicon, Mobile Devices, and Possible Open-Source SolutionsDark Silicon, Mobile Devices, and Possible Open-Source Solutions
Dark Silicon, Mobile Devices, and Possible Open-Source Solutions
Koan-Sin Tan
 

La actualidad más candente (7)

Async and parallel patterns and application design - TechDays2013 NL
Async and parallel patterns and application design - TechDays2013 NLAsync and parallel patterns and application design - TechDays2013 NL
Async and parallel patterns and application design - TechDays2013 NL
 
GPU Computing for Data Science
GPU Computing for Data Science GPU Computing for Data Science
GPU Computing for Data Science
 
Tensorflow on Android
Tensorflow on AndroidTensorflow on Android
Tensorflow on Android
 
TMPA-2017: Unity Application Testing Automation with Appium and Image Recogni...
TMPA-2017: Unity Application Testing Automation with Appium and Image Recogni...TMPA-2017: Unity Application Testing Automation with Appium and Image Recogni...
TMPA-2017: Unity Application Testing Automation with Appium and Image Recogni...
 
Dark Silicon, Mobile Devices, and Possible Open-Source Solutions
Dark Silicon, Mobile Devices, and Possible Open-Source SolutionsDark Silicon, Mobile Devices, and Possible Open-Source Solutions
Dark Silicon, Mobile Devices, and Possible Open-Source Solutions
 
STKO - A revolutionary toolkit for OpenSees
STKO - A revolutionary toolkit for OpenSeesSTKO - A revolutionary toolkit for OpenSees
STKO - A revolutionary toolkit for OpenSees
 
IoT Supercharged: Complex event processing for MQTT with Eclipse technologies
IoT Supercharged: Complex event processing for MQTT with Eclipse technologiesIoT Supercharged: Complex event processing for MQTT with Eclipse technologies
IoT Supercharged: Complex event processing for MQTT with Eclipse technologies
 

Similar a Analysis Result Manager (ARM)

OORPT Dynamic Analysis
OORPT Dynamic AnalysisOORPT Dynamic Analysis
OORPT Dynamic Analysis
lienhard
 
Running Accurate, Scalable, and Reproducible Simulations of Distributed Syste...
Running Accurate, Scalable, and Reproducible Simulations of Distributed Syste...Running Accurate, Scalable, and Reproducible Simulations of Distributed Syste...
Running Accurate, Scalable, and Reproducible Simulations of Distributed Syste...
Rafael Ferreira da Silva
 
Presentation
PresentationPresentation
Presentation
butest
 
aleph - Malware analysis pipelining for the masses
aleph - Malware analysis pipelining for the massesaleph - Malware analysis pipelining for the masses
aleph - Malware analysis pipelining for the masses
Jan Seidl
 

Similar a Analysis Result Manager (ARM) (20)

Remote Web Desk
Remote Web DeskRemote Web Desk
Remote Web Desk
 
Android overview
Android overviewAndroid overview
Android overview
 
Presentation1.2.pptx
Presentation1.2.pptxPresentation1.2.pptx
Presentation1.2.pptx
 
OORPT Dynamic Analysis
OORPT Dynamic AnalysisOORPT Dynamic Analysis
OORPT Dynamic Analysis
 
Running Accurate, Scalable, and Reproducible Simulations of Distributed Syste...
Running Accurate, Scalable, and Reproducible Simulations of Distributed Syste...Running Accurate, Scalable, and Reproducible Simulations of Distributed Syste...
Running Accurate, Scalable, and Reproducible Simulations of Distributed Syste...
 
OS for AI: Elastic Microservices & the Next Gen of ML
OS for AI: Elastic Microservices & the Next Gen of MLOS for AI: Elastic Microservices & the Next Gen of ML
OS for AI: Elastic Microservices & the Next Gen of ML
 
Real time-collaborative-editor-presentation
Real time-collaborative-editor-presentationReal time-collaborative-editor-presentation
Real time-collaborative-editor-presentation
 
Java Performance & Profiling
Java Performance & ProfilingJava Performance & Profiling
Java Performance & Profiling
 
Presentation
PresentationPresentation
Presentation
 
Running High-Speed Serverless with nuclio
Running High-Speed Serverless with nuclioRunning High-Speed Serverless with nuclio
Running High-Speed Serverless with nuclio
 
Building and deploying LLM applications with Apache Airflow
Building and deploying LLM applications with Apache AirflowBuilding and deploying LLM applications with Apache Airflow
Building and deploying LLM applications with Apache Airflow
 
Realizing the Promise of Portable Data Processing with Apache Beam
Realizing the Promise of Portable Data Processing with Apache BeamRealizing the Promise of Portable Data Processing with Apache Beam
Realizing the Promise of Portable Data Processing with Apache Beam
 
IT TRENDS AND PERSPECTIVES 2016
IT TRENDS AND PERSPECTIVES 2016IT TRENDS AND PERSPECTIVES 2016
IT TRENDS AND PERSPECTIVES 2016
 
OBJECT ORIENTED ROGRAMMING With Question And Answer Full
OBJECT ORIENTED ROGRAMMING With Question And Answer  FullOBJECT ORIENTED ROGRAMMING With Question And Answer  Full
OBJECT ORIENTED ROGRAMMING With Question And Answer Full
 
Java Performance and Profiling
Java Performance and ProfilingJava Performance and Profiling
Java Performance and Profiling
 
Overview of Modern Graph Analysis Tools
Overview of Modern Graph Analysis ToolsOverview of Modern Graph Analysis Tools
Overview of Modern Graph Analysis Tools
 
Software and the Concurrency Revolution : Notes
Software and the Concurrency Revolution : NotesSoftware and the Concurrency Revolution : Notes
Software and the Concurrency Revolution : Notes
 
Internship msc cs
Internship msc csInternship msc cs
Internship msc cs
 
NodeJS guide for beginners
NodeJS guide for beginnersNodeJS guide for beginners
NodeJS guide for beginners
 
aleph - Malware analysis pipelining for the masses
aleph - Malware analysis pipelining for the massesaleph - Malware analysis pipelining for the masses
aleph - Malware analysis pipelining for the masses
 

Último

Architecting Cloud Native Applications
Architecting Cloud Native ApplicationsArchitecting Cloud Native Applications
Architecting Cloud Native Applications
WSO2
 

Último (20)

Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
 
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot TakeoffStrategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
 
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
 
Ransomware_Q4_2023. The report. [EN].pdf
Ransomware_Q4_2023. The report. [EN].pdfRansomware_Q4_2023. The report. [EN].pdf
Ransomware_Q4_2023. The report. [EN].pdf
 
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
 
Corporate and higher education May webinar.pptx
Corporate and higher education May webinar.pptxCorporate and higher education May webinar.pptx
Corporate and higher education May webinar.pptx
 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024
 
Artificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : UncertaintyArtificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : Uncertainty
 
A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?
 
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps ScriptAutomating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Script
 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdf
 
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost SavingRepurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
 
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
 
MS Copilot expands with MS Graph connectors
MS Copilot expands with MS Graph connectorsMS Copilot expands with MS Graph connectors
MS Copilot expands with MS Graph connectors
 
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data DiscoveryTrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
 
DBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor PresentationDBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor Presentation
 
AWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of TerraformAWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of Terraform
 
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, AdobeApidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
 
Architecting Cloud Native Applications
Architecting Cloud Native ApplicationsArchitecting Cloud Native Applications
Architecting Cloud Native Applications
 
2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...
 

Analysis Result Manager (ARM)

  • 1. Overview of Analysis Result Manager (ARM) Jeremy Magland 27 October, 2011 brainomics group @ Penn
  • 2.
  • 3. Motivations for ARM O Motivation #1: Visualization I wanted to be able to SEE my fMRI data, intermediate files, and processing results, and to share these with my collaborators.
  • 4. Motivations for ARM O Motivation #2: Bone-Water Study Real-time Study Subject AP-1 Data Management REAL_23 Localizer Sample Session 1 Baseline, TR=300 Organize fMRI data text Raw fMRI series Follow-up 1, TR=300 About Anatomic localizer on a central server, study Session 2 Bone-Water Study Stuff REAL_36 Subject AP-1 and access it from To fill Session 1 Localizer sapace Session 2 Baseline, TR=300 anywhere (cloud Follow-up 1, TR=300 style)
  • 5. Motivations for ARM O Motivation #3: Computing Power Utilize the processing power of server computers (CPU power, RAM, automatic parallel processing)
  • 6. Motivations for ARM O Motivation #4: Provenance Tracking Track the processing steps required to reproduce each intermediate object and processing result. Advantages: * reproducibility * auto-non-redundant processing
  • 7. Challenge: Richly Interactive, Web-based Data Visualization O Browser-based O RemoteWidgets: applications are very convenient but often O Framework for have limited developing functionality. O Desktop applications desktop can have rich applications in Qt4 functionality. However with option to there are some cons: O software deploy on browser. downloads/updates O OS incompatibility O Security risks O Etc.
  • 8. RemoteWidgets: Overview Seegull O Write a GUI for the web using only one ARM programming language: Qt4/C++ O Rapid development Pickles of highly interactive web applications. O Direct access to Math Puppy server-side processes.
  • 9. RemoteWidgets: One App, Two Access Options Seegull in Web Browser O Web Browser Interface (jQuery) O Desktop Interface (thin client download) Seegull on Desktop
  • 11. RemoteWidgets: Server-Side Event Handling O Handling GUI events is as easy as connecting Qt signals and slots on the server… no javascript coding needed.
  • 12. RemoteWidgets: Idea O Interactive, desktop-style visualization tools available from the convenience of a web browser.
  • 13. Analysis Result Manager (ARM) O Cloud-based approach to fMRI data management, visualization, and analysis. O ARM is a RemoteWidgets application, and can therefore be accessed remotely via a web browser.
  • 14. Tradeoff Between Scripting and GUI for fMRI Processing O Scripting is way more powerful for research purposes. And, if you have a good script interface, you could always throw a GUI on top of it. O But… some problems with Matlab-style scripting: O Need to manually shuffle data around in files/folders O Not easy to remember exactly which parameters were used >2 days ago O Run a 10-hour script, and discover the last line had a bug.
  • 15. ARM: Script-Generated Workflows With Robust Tracking of Provenance Data The multi-threaded processing The script immediately sets up module then executes each a workflow, creating all processors as soon as all input intermediate and result objects objects have been defined. ahead of the processing, storing detailed information about how objects are related via processes.
  • 16. Processors are Generic Plugins C++ Wrappers (for FSL, SPM, etc) Views are also plugins… (custom views of custom datatypes)
  • 17. Live ARM Demonstration O Upload Scan Data O View processing results and O Online view, change view intermediate files. settings O Show provenance (process O Super-efficient browsing of huge history) of the output objects. data files. O Explain that changing scan id’s O Browser access to same app and even names of the O View example script processors causes no problems. Key assumption: O Running script does nothing objects, once created, never because processing has already change. completed O Clean up by deleting results. (In O Change design function and re- future, intermediate files will be run script – show how auto-removed after a period if processes are queued and they are no longer linked to executed in proper order (No results) redundant processing). O In development – share a processing result by a button click.
  • 18. Live Seegull Demonstration O It is often desirable to SHARE results with others. But you don’t always want to give them access to a huge system. Furthermore, since ARM is highly dynamic, you want to be sure that a result, once shared, is available for ever after (snapshot of a result) O Hence, Seegull
  • 19. Summary and Future O Share interactive views of data O Taking full advantage of cloud technology to manage, process, visualize, compare fMRI data analyses O Script-generated workflows O Automatic provenance tracking O Convenient access via web browser O Import publicly available fMRI studies into system (powerful meta-analyses)
  • 20. Especially Thank You to… O Anna Rose Childress O Felix Wehrli O Charlie Epstein And… O Benny Lam O John Detre O Chris Tjoa O Oscar Bartra O Jesse Lupardus O Brian Avants O Kimberly Young O Ning Zhang O Dan Willard O Cheng Li O Rob Fabianski O Ze Wang