SlideShare a Scribd company logo
1 of 37
Download to read offline
globus online
Globus Online for
Research Data Management
Rachana Ananthakrishnan
Great Plains Network Annual Meeting 2013
We started with technology
proven in many large-scale grids
GridFTP
GRAM
MyProxy
GSI-OpenSSH
…
1.2 PB of climate data
delivered to 23,000 users
Typical of large, well funded
research projects using GT
1.2 PB of climate data
delivered to 23,000 users
GT provides robust
infrastructure for the 1%
What about the 99%?
GT provides robust
infrastructure for the 1%
What about the 99%?
BIG SCIENCE. Small labs
GT provides robust
infrastructure for the 1%
globus online
Managing data should be easy …
Registry	
  
Staging	
  
Store	
  
Ingest	
  
Store	
  
Analysis	
  
Store	
  
Community	
  
Store	
  
Archive	
   Mirror	
  
Ingest	
  
Store	
  
Analysis	
  
Store	
  
Community	
  
Store	
  
Archive	
   Mirror	
  
Registry	
  
… but it’s hard and frustrating!
Registry	
  
Staging	
  
Store	
  
Ingest	
  
Store	
  
Analysis	
  
Store	
  
Community	
  
Store	
  
Archive	
   Mirror	
  
Ingest	
  
Store	
  
Analysis	
  
Store	
  
Community	
  
Store	
  
Archive	
   Mirror	
  
Registry	
  
Quota
exceeded
!
Expired
credentials
!
Network
failed. Retry.
!
Permission
denied
!
What is Globus Online?
Transfer and sharing of large
data sets…
…with dropbox-like
characteristics…
…directly from your own
storage systems
We adopted SaaS approaches to
transform the user experience
… for both researchers and
resource owners/system
administrators
We started with reliable, secure,
high-performance file transfer …
Data
Source
Data
Destination
User initiates
transfer request
1
Globus Online
moves and
syncs files
2
Globus Online
notifies user
3
… and then made it simple to share
big data off existing storage systems
Data
Source
User A selects file
(s) to share,
selects user or
group, and sets
permissions
1
Globus Online tracks
shared files; no need
to move files to cloud
storage!
2
User B logs in to
Globus Online
and accesses
shared file
3
Log into Globus Online
Alternate Logins
Login using InCommon
InCommon Login
Source endpoint
Destination endpoint
Activation
Transfer data
Share data
Set permissions
Manage Groups
Interactive login to command line interface:
Running commands remotely:
Using CLI with gsissh:
Globus Online CLI
$ ssh tuecke@cli.globusonline.org
$ ssh tuecke@cli.globusonline.org <command>
$ gsissh tuecke@cli.globusonline.org <command>
$ ssh tuecke@cli.globusonline.org scp –r –s 3 -D 
nersc#dtn:~/myfile* mylaptop:~/projects/p1
Task ID: 4a3c471e-edef-11df-aa30-1231350018b1
$ _
Usage is accelerating
Early Adopters
•  What is GCMU?
–  Globus Connect version for easily creating (sharable) endpoints
on multi-user storage servers
–  Packages a GridFTP server and MyProxy CA authentication
server, pre-configured for use with Globus Online
•  Why GCMU?
–  Create transfer endpoints in minutes
–  Avoid complex GridFTP install
•  To download: www.globusonline.org/gcmu
Globus Connect Multiuser (GCMU)
29
“We	
  used	
  GCMU	
  to	
  form	
  a	
  campus-­‐
wide	
  GSI	
  authenAcaAon	
  service	
  
spanning	
  mulAple	
  servers.	
  Now	
  my	
  
users	
  have	
  a	
  fast,	
  easy	
  way	
  to	
  get	
  their	
  
data	
  wherever	
  it	
  needs	
  to	
  go,	
  and	
  the	
  
setup	
  process	
  was	
  trivial."	
  	
  
-­‐-­‐University	
  of	
  Michigan	
  
“As	
  a	
  resource	
  admin,	
  I've	
  found	
  
GCMU	
  an	
  exceedingly	
  useful	
  
tool....	
  With	
  GCMU,	
  seHng	
  up	
  a	
  
GridFTP	
  server	
  and	
  handling	
  
authenAcaAon	
  for	
  mulAple	
  users	
  
is	
  easy."	
  	
  
-­‐-­‐Oak	
  Ridge	
  Na8onal	
  Lab	
  
We are a non-profit service
provider to the non-profit
research community
Our challenge:
Sustainability
We are a non-profit service
provider to the non-profit
research community
Globus Online Provider Plans
Support ongoing operations
Offer value-added capabilities
Engage more closely with users
•  Endpoint operations management
•  Branded web sites
•  Alternate identity provider
•  Usage reporting
•  MSS optimizations
•  Multiple GridFTP servers per endpoint
Provider Plans offer…
Starting at $20k per year
End User Plans
•  Basic: Free
– File transfer and synchronization to/from
servers
– Personal endpoints with Globus Connect
– Access to shared endpoints created by others
•  Plus: $7/month (or $70/year)
– Create and manage shared endpoints
– Peer-to-peer transfer and sharing
Globus Platform-as-a-Service
Globus Nexus
(Identity, Group, Profile)
…
Sharing Service
Transfer Service
Dataset Services
Globus Toolkit
GlobusOnlineAPIs
GlobusConnect
Our research is supported by:
U. S. DE PART M E N T O F
ENERGY
Questions?
Contact: support@globusonline.org
Providers: globusonline.org/provider-plans
www.globusonline.org

More Related Content

What's hot

MongoDB SoCal 2020: MongoDB Atlas Jump Start
 MongoDB SoCal 2020: MongoDB Atlas Jump Start MongoDB SoCal 2020: MongoDB Atlas Jump Start
MongoDB SoCal 2020: MongoDB Atlas Jump StartMongoDB
 
What does Netflix, NTT and Rubicon Project have in common? Apache Druid.
What does Netflix, NTT and Rubicon Project have in common? Apache Druid.What does Netflix, NTT and Rubicon Project have in common? Apache Druid.
What does Netflix, NTT and Rubicon Project have in common? Apache Druid.Rommel Garcia
 
Introduction to the Globus SaaS (GlobusWorld Tour - STFC)
Introduction to the Globus SaaS (GlobusWorld Tour - STFC)Introduction to the Globus SaaS (GlobusWorld Tour - STFC)
Introduction to the Globus SaaS (GlobusWorld Tour - STFC)Globus
 
Building Streaming Data Pipelines with Google Cloud Dataflow and Confluent Cl...
Building Streaming Data Pipelines with Google Cloud Dataflow and Confluent Cl...Building Streaming Data Pipelines with Google Cloud Dataflow and Confluent Cl...
Building Streaming Data Pipelines with Google Cloud Dataflow and Confluent Cl...HostedbyConfluent
 
Creating a Kafka Topic. Super easy? | Andrew Stevenson and Marios Andreopoulo...
Creating a Kafka Topic. Super easy? | Andrew Stevenson and Marios Andreopoulo...Creating a Kafka Topic. Super easy? | Andrew Stevenson and Marios Andreopoulo...
Creating a Kafka Topic. Super easy? | Andrew Stevenson and Marios Andreopoulo...HostedbyConfluent
 
Towards a self automated CERN Cloud
Towards a self automated CERN CloudTowards a self automated CERN Cloud
Towards a self automated CERN CloudJose Castro Leon
 
[WSO2Con USA 2018] Deploying Applications in K8S and Docker
[WSO2Con USA 2018] Deploying Applications in K8S and Docker[WSO2Con USA 2018] Deploying Applications in K8S and Docker
[WSO2Con USA 2018] Deploying Applications in K8S and DockerWSO2
 
MongoDB SoCal 2020: Using MongoDB Services in Kubernetes: Any Platform, Devel...
MongoDB SoCal 2020: Using MongoDB Services in Kubernetes: Any Platform, Devel...MongoDB SoCal 2020: Using MongoDB Services in Kubernetes: Any Platform, Devel...
MongoDB SoCal 2020: Using MongoDB Services in Kubernetes: Any Platform, Devel...MongoDB
 
Ingesting and Processing IoT Data - using MQTT, Kafka Connect and KSQL
Ingesting and Processing IoT Data - using MQTT, Kafka Connect and KSQLIngesting and Processing IoT Data - using MQTT, Kafka Connect and KSQL
Ingesting and Processing IoT Data - using MQTT, Kafka Connect and KSQLGuido Schmutz
 
MongoDB .local Bengaluru 2019: MongoDB Atlas Data Lake Technical Deep Dive
MongoDB .local Bengaluru 2019: MongoDB Atlas Data Lake Technical Deep DiveMongoDB .local Bengaluru 2019: MongoDB Atlas Data Lake Technical Deep Dive
MongoDB .local Bengaluru 2019: MongoDB Atlas Data Lake Technical Deep DiveMongoDB
 
MongoDB Europe 2016 - Warehousing MongoDB Data using Apache Beam and BigQuery
MongoDB Europe 2016 - Warehousing MongoDB Data using Apache Beam and BigQueryMongoDB Europe 2016 - Warehousing MongoDB Data using Apache Beam and BigQuery
MongoDB Europe 2016 - Warehousing MongoDB Data using Apache Beam and BigQueryMongoDB
 
A Microservices approach with Cassandra and Quarkus | DevNation Tech Talk
A Microservices approach with Cassandra and Quarkus | DevNation Tech TalkA Microservices approach with Cassandra and Quarkus | DevNation Tech Talk
A Microservices approach with Cassandra and Quarkus | DevNation Tech TalkRed Hat Developers
 
Live Event Debugging With ksqlDB at Reddit | Hannah Hagen and Paul Kiernan, R...
Live Event Debugging With ksqlDB at Reddit | Hannah Hagen and Paul Kiernan, R...Live Event Debugging With ksqlDB at Reddit | Hannah Hagen and Paul Kiernan, R...
Live Event Debugging With ksqlDB at Reddit | Hannah Hagen and Paul Kiernan, R...HostedbyConfluent
 
MongoDB Ops Manager + Kubernetes
MongoDB Ops Manager + KubernetesMongoDB Ops Manager + Kubernetes
MongoDB Ops Manager + KubernetesMongoDB
 
Journey to Containerized Application / Google Container Engine
Journey to Containerized Application / Google Container EngineJourney to Containerized Application / Google Container Engine
Journey to Containerized Application / Google Container EngineGoogle Cloud Platform - Japan
 

What's hot (20)

MongoDB SoCal 2020: MongoDB Atlas Jump Start
 MongoDB SoCal 2020: MongoDB Atlas Jump Start MongoDB SoCal 2020: MongoDB Atlas Jump Start
MongoDB SoCal 2020: MongoDB Atlas Jump Start
 
What does Netflix, NTT and Rubicon Project have in common? Apache Druid.
What does Netflix, NTT and Rubicon Project have in common? Apache Druid.What does Netflix, NTT and Rubicon Project have in common? Apache Druid.
What does Netflix, NTT and Rubicon Project have in common? Apache Druid.
 
Google Cloud DNS
Google Cloud DNSGoogle Cloud DNS
Google Cloud DNS
 
Introduction to the Globus SaaS (GlobusWorld Tour - STFC)
Introduction to the Globus SaaS (GlobusWorld Tour - STFC)Introduction to the Globus SaaS (GlobusWorld Tour - STFC)
Introduction to the Globus SaaS (GlobusWorld Tour - STFC)
 
Building Streaming Data Pipelines with Google Cloud Dataflow and Confluent Cl...
Building Streaming Data Pipelines with Google Cloud Dataflow and Confluent Cl...Building Streaming Data Pipelines with Google Cloud Dataflow and Confluent Cl...
Building Streaming Data Pipelines with Google Cloud Dataflow and Confluent Cl...
 
Citrix Netscaler
Citrix NetscalerCitrix Netscaler
Citrix Netscaler
 
Creating a Kafka Topic. Super easy? | Andrew Stevenson and Marios Andreopoulo...
Creating a Kafka Topic. Super easy? | Andrew Stevenson and Marios Andreopoulo...Creating a Kafka Topic. Super easy? | Andrew Stevenson and Marios Andreopoulo...
Creating a Kafka Topic. Super easy? | Andrew Stevenson and Marios Andreopoulo...
 
Kafka Streams
Kafka StreamsKafka Streams
Kafka Streams
 
Towards a self automated CERN Cloud
Towards a self automated CERN CloudTowards a self automated CERN Cloud
Towards a self automated CERN Cloud
 
[WSO2Con USA 2018] Deploying Applications in K8S and Docker
[WSO2Con USA 2018] Deploying Applications in K8S and Docker[WSO2Con USA 2018] Deploying Applications in K8S and Docker
[WSO2Con USA 2018] Deploying Applications in K8S and Docker
 
MongoDB SoCal 2020: Using MongoDB Services in Kubernetes: Any Platform, Devel...
MongoDB SoCal 2020: Using MongoDB Services in Kubernetes: Any Platform, Devel...MongoDB SoCal 2020: Using MongoDB Services in Kubernetes: Any Platform, Devel...
MongoDB SoCal 2020: Using MongoDB Services in Kubernetes: Any Platform, Devel...
 
Ingesting and Processing IoT Data - using MQTT, Kafka Connect and KSQL
Ingesting and Processing IoT Data - using MQTT, Kafka Connect and KSQLIngesting and Processing IoT Data - using MQTT, Kafka Connect and KSQL
Ingesting and Processing IoT Data - using MQTT, Kafka Connect and KSQL
 
MongoDB .local Bengaluru 2019: MongoDB Atlas Data Lake Technical Deep Dive
MongoDB .local Bengaluru 2019: MongoDB Atlas Data Lake Technical Deep DiveMongoDB .local Bengaluru 2019: MongoDB Atlas Data Lake Technical Deep Dive
MongoDB .local Bengaluru 2019: MongoDB Atlas Data Lake Technical Deep Dive
 
MongoDB Europe 2016 - Warehousing MongoDB Data using Apache Beam and BigQuery
MongoDB Europe 2016 - Warehousing MongoDB Data using Apache Beam and BigQueryMongoDB Europe 2016 - Warehousing MongoDB Data using Apache Beam and BigQuery
MongoDB Europe 2016 - Warehousing MongoDB Data using Apache Beam and BigQuery
 
A Microservices approach with Cassandra and Quarkus | DevNation Tech Talk
A Microservices approach with Cassandra and Quarkus | DevNation Tech TalkA Microservices approach with Cassandra and Quarkus | DevNation Tech Talk
A Microservices approach with Cassandra and Quarkus | DevNation Tech Talk
 
Live Event Debugging With ksqlDB at Reddit | Hannah Hagen and Paul Kiernan, R...
Live Event Debugging With ksqlDB at Reddit | Hannah Hagen and Paul Kiernan, R...Live Event Debugging With ksqlDB at Reddit | Hannah Hagen and Paul Kiernan, R...
Live Event Debugging With ksqlDB at Reddit | Hannah Hagen and Paul Kiernan, R...
 
Purser
PurserPurser
Purser
 
Big Data Applications
Big Data ApplicationsBig Data Applications
Big Data Applications
 
MongoDB Ops Manager + Kubernetes
MongoDB Ops Manager + KubernetesMongoDB Ops Manager + Kubernetes
MongoDB Ops Manager + Kubernetes
 
Journey to Containerized Application / Google Container Engine
Journey to Containerized Application / Google Container EngineJourney to Containerized Application / Google Container Engine
Journey to Containerized Application / Google Container Engine
 

Similar to Globus Online for Research Data Management

Campus Bridging with Globus Services
Campus Bridging with Globus ServicesCampus Bridging with Globus Services
Campus Bridging with Globus ServicesIan Foster
 
Architecting Analytic Pipelines on GCP - Chicago Cloud Conference 2020
Architecting Analytic Pipelines on GCP - Chicago Cloud Conference 2020Architecting Analytic Pipelines on GCP - Chicago Cloud Conference 2020
Architecting Analytic Pipelines on GCP - Chicago Cloud Conference 2020Mariano Gonzalez
 
Introduction to Data Transfer and Sharing for Researchers
Introduction to Data Transfer and Sharing for ResearchersIntroduction to Data Transfer and Sharing for Researchers
Introduction to Data Transfer and Sharing for ResearchersGlobus
 
Introduction to Globus for New Users
Introduction to Globus for New UsersIntroduction to Globus for New Users
Introduction to Globus for New UsersGlobus
 
Getting more into GCP.pdf
Getting more into GCP.pdfGetting more into GCP.pdf
Getting more into GCP.pdfKnoldus Inc.
 
What's New in Globus - Internet2 TechEXtra
What's New in Globus - Internet2 TechEXtraWhat's New in Globus - Internet2 TechEXtra
What's New in Globus - Internet2 TechEXtraGlobus
 
Introduction to Google Cloud & GCCP Campaign
Introduction to Google Cloud & GCCP CampaignIntroduction to Google Cloud & GCCP Campaign
Introduction to Google Cloud & GCCP CampaignGDSCVJTI
 
Harnessing the Power of Google Cloud Platform: Strategies and Applications
Harnessing the Power of Google Cloud Platform: Strategies and ApplicationsHarnessing the Power of Google Cloud Platform: Strategies and Applications
Harnessing the Power of Google Cloud Platform: Strategies and ApplicationsHitesh Mohapatra
 
What is expected from Chief Cloud Officers?
What is expected from Chief Cloud Officers?What is expected from Chief Cloud Officers?
What is expected from Chief Cloud Officers?Bernard Paques
 
Comparative Analysis, Security Aspects & Optimization of Workload in Gfs Base...
Comparative Analysis, Security Aspects & Optimization of Workload in Gfs Base...Comparative Analysis, Security Aspects & Optimization of Workload in Gfs Base...
Comparative Analysis, Security Aspects & Optimization of Workload in Gfs Base...IOSR Journals
 
Extending Twitter's Data Platform to Google Cloud
Extending Twitter's Data Platform to Google CloudExtending Twitter's Data Platform to Google Cloud
Extending Twitter's Data Platform to Google CloudDataWorks Summit
 
Grid Computing
Grid ComputingGrid Computing
Grid Computingabhiritva
 
Alluxio Use Cases and Future Directions
Alluxio Use Cases and Future DirectionsAlluxio Use Cases and Future Directions
Alluxio Use Cases and Future DirectionsAlluxio, Inc.
 
Globus: A Data Management Platform for Collaborative Research (CHPC 2019 - So...
Globus: A Data Management Platform for Collaborative Research (CHPC 2019 - So...Globus: A Data Management Platform for Collaborative Research (CHPC 2019 - So...
Globus: A Data Management Platform for Collaborative Research (CHPC 2019 - So...Globus
 
Big Data on Cloud Native Platform
Big Data on Cloud Native PlatformBig Data on Cloud Native Platform
Big Data on Cloud Native PlatformSunil Govindan
 
Big Data on Cloud Native Platform
Big Data on Cloud Native PlatformBig Data on Cloud Native Platform
Big Data on Cloud Native PlatformSunil Govindan
 
Big data in action
Big data in actionBig data in action
Big data in actionTu Pham
 

Similar to Globus Online for Research Data Management (20)

Campus Bridging with Globus Services
Campus Bridging with Globus ServicesCampus Bridging with Globus Services
Campus Bridging with Globus Services
 
OGCE SC10
OGCE SC10OGCE SC10
OGCE SC10
 
Architecting Analytic Pipelines on GCP - Chicago Cloud Conference 2020
Architecting Analytic Pipelines on GCP - Chicago Cloud Conference 2020Architecting Analytic Pipelines on GCP - Chicago Cloud Conference 2020
Architecting Analytic Pipelines on GCP - Chicago Cloud Conference 2020
 
Introduction to Data Transfer and Sharing for Researchers
Introduction to Data Transfer and Sharing for ResearchersIntroduction to Data Transfer and Sharing for Researchers
Introduction to Data Transfer and Sharing for Researchers
 
Introduction to Globus for New Users
Introduction to Globus for New UsersIntroduction to Globus for New Users
Introduction to Globus for New Users
 
Getting more into GCP.pdf
Getting more into GCP.pdfGetting more into GCP.pdf
Getting more into GCP.pdf
 
What's New in Globus - Internet2 TechEXtra
What's New in Globus - Internet2 TechEXtraWhat's New in Globus - Internet2 TechEXtra
What's New in Globus - Internet2 TechEXtra
 
Session19 Globus
Session19 GlobusSession19 Globus
Session19 Globus
 
Introduction to Google Cloud & GCCP Campaign
Introduction to Google Cloud & GCCP CampaignIntroduction to Google Cloud & GCCP Campaign
Introduction to Google Cloud & GCCP Campaign
 
Harnessing the Power of Google Cloud Platform: Strategies and Applications
Harnessing the Power of Google Cloud Platform: Strategies and ApplicationsHarnessing the Power of Google Cloud Platform: Strategies and Applications
Harnessing the Power of Google Cloud Platform: Strategies and Applications
 
What is expected from Chief Cloud Officers?
What is expected from Chief Cloud Officers?What is expected from Chief Cloud Officers?
What is expected from Chief Cloud Officers?
 
H017144148
H017144148H017144148
H017144148
 
Comparative Analysis, Security Aspects & Optimization of Workload in Gfs Base...
Comparative Analysis, Security Aspects & Optimization of Workload in Gfs Base...Comparative Analysis, Security Aspects & Optimization of Workload in Gfs Base...
Comparative Analysis, Security Aspects & Optimization of Workload in Gfs Base...
 
Extending Twitter's Data Platform to Google Cloud
Extending Twitter's Data Platform to Google CloudExtending Twitter's Data Platform to Google Cloud
Extending Twitter's Data Platform to Google Cloud
 
Grid Computing
Grid ComputingGrid Computing
Grid Computing
 
Alluxio Use Cases and Future Directions
Alluxio Use Cases and Future DirectionsAlluxio Use Cases and Future Directions
Alluxio Use Cases and Future Directions
 
Globus: A Data Management Platform for Collaborative Research (CHPC 2019 - So...
Globus: A Data Management Platform for Collaborative Research (CHPC 2019 - So...Globus: A Data Management Platform for Collaborative Research (CHPC 2019 - So...
Globus: A Data Management Platform for Collaborative Research (CHPC 2019 - So...
 
Big Data on Cloud Native Platform
Big Data on Cloud Native PlatformBig Data on Cloud Native Platform
Big Data on Cloud Native Platform
 
Big Data on Cloud Native Platform
Big Data on Cloud Native PlatformBig Data on Cloud Native Platform
Big Data on Cloud Native Platform
 
Big data in action
Big data in actionBig data in action
Big data in action
 

More from Globus

Advanced Globus System Administration Topics
Advanced Globus System Administration TopicsAdvanced Globus System Administration Topics
Advanced Globus System Administration TopicsGlobus
 
Instrument Data Automation: The Life of a Flow
Instrument Data Automation: The Life of a FlowInstrument Data Automation: The Life of a Flow
Instrument Data Automation: The Life of a FlowGlobus
 
Building Research Applications with Globus PaaS
Building Research Applications with Globus PaaSBuilding Research Applications with Globus PaaS
Building Research Applications with Globus PaaSGlobus
 
Reliable, Remote Computation at All Scales
Reliable, Remote Computation at All ScalesReliable, Remote Computation at All Scales
Reliable, Remote Computation at All ScalesGlobus
 
Best Practices for Data Sharing Using Globus
Best Practices for Data Sharing Using GlobusBest Practices for Data Sharing Using Globus
Best Practices for Data Sharing Using GlobusGlobus
 
An Introduction to Globus for Researchers
An Introduction to Globus for ResearchersAn Introduction to Globus for Researchers
An Introduction to Globus for ResearchersGlobus
 
Introduction to Research Automation with Globus
Introduction to Research Automation with GlobusIntroduction to Research Automation with Globus
Introduction to Research Automation with GlobusGlobus
 
Globus for System Administrators
Globus for System AdministratorsGlobus for System Administrators
Globus for System AdministratorsGlobus
 
Introduction to Globus for System Administrators
Introduction to Globus for System AdministratorsIntroduction to Globus for System Administrators
Introduction to Globus for System AdministratorsGlobus
 
Introduction to the Globus Platform for Developers
Introduction to the Globus Platform for DevelopersIntroduction to the Globus Platform for Developers
Introduction to the Globus Platform for DevelopersGlobus
 
Introduction to the Command Line Interface (CLI)
Introduction to the Command Line Interface (CLI)Introduction to the Command Line Interface (CLI)
Introduction to the Command Line Interface (CLI)Globus
 
Automating Research Data with Globus Flows and Compute
Automating Research Data with Globus Flows and ComputeAutomating Research Data with Globus Flows and Compute
Automating Research Data with Globus Flows and ComputeGlobus
 
Automating Research Data Flows and Introduction to the Globus Platform
Automating Research Data Flows and Introduction to the Globus PlatformAutomating Research Data Flows and Introduction to the Globus Platform
Automating Research Data Flows and Introduction to the Globus PlatformGlobus
 
Advanced Globus System Administration
Advanced Globus System AdministrationAdvanced Globus System Administration
Advanced Globus System AdministrationGlobus
 
Introduction to Globus for System Administrators
Introduction to Globus for System AdministratorsIntroduction to Globus for System Administrators
Introduction to Globus for System AdministratorsGlobus
 
Introduction to Globus for New Users
Introduction to Globus for New UsersIntroduction to Globus for New Users
Introduction to Globus for New UsersGlobus
 
Working with Globus Platform Services and Portals
Working with Globus Platform Services and PortalsWorking with Globus Platform Services and Portals
Working with Globus Platform Services and PortalsGlobus
 
Globus Automation
Globus AutomationGlobus Automation
Globus AutomationGlobus
 
Advanced Globus System Administration
Advanced Globus System AdministrationAdvanced Globus System Administration
Advanced Globus System AdministrationGlobus
 
Introduction to Globus
Introduction to GlobusIntroduction to Globus
Introduction to GlobusGlobus
 

More from Globus (20)

Advanced Globus System Administration Topics
Advanced Globus System Administration TopicsAdvanced Globus System Administration Topics
Advanced Globus System Administration Topics
 
Instrument Data Automation: The Life of a Flow
Instrument Data Automation: The Life of a FlowInstrument Data Automation: The Life of a Flow
Instrument Data Automation: The Life of a Flow
 
Building Research Applications with Globus PaaS
Building Research Applications with Globus PaaSBuilding Research Applications with Globus PaaS
Building Research Applications with Globus PaaS
 
Reliable, Remote Computation at All Scales
Reliable, Remote Computation at All ScalesReliable, Remote Computation at All Scales
Reliable, Remote Computation at All Scales
 
Best Practices for Data Sharing Using Globus
Best Practices for Data Sharing Using GlobusBest Practices for Data Sharing Using Globus
Best Practices for Data Sharing Using Globus
 
An Introduction to Globus for Researchers
An Introduction to Globus for ResearchersAn Introduction to Globus for Researchers
An Introduction to Globus for Researchers
 
Introduction to Research Automation with Globus
Introduction to Research Automation with GlobusIntroduction to Research Automation with Globus
Introduction to Research Automation with Globus
 
Globus for System Administrators
Globus for System AdministratorsGlobus for System Administrators
Globus for System Administrators
 
Introduction to Globus for System Administrators
Introduction to Globus for System AdministratorsIntroduction to Globus for System Administrators
Introduction to Globus for System Administrators
 
Introduction to the Globus Platform for Developers
Introduction to the Globus Platform for DevelopersIntroduction to the Globus Platform for Developers
Introduction to the Globus Platform for Developers
 
Introduction to the Command Line Interface (CLI)
Introduction to the Command Line Interface (CLI)Introduction to the Command Line Interface (CLI)
Introduction to the Command Line Interface (CLI)
 
Automating Research Data with Globus Flows and Compute
Automating Research Data with Globus Flows and ComputeAutomating Research Data with Globus Flows and Compute
Automating Research Data with Globus Flows and Compute
 
Automating Research Data Flows and Introduction to the Globus Platform
Automating Research Data Flows and Introduction to the Globus PlatformAutomating Research Data Flows and Introduction to the Globus Platform
Automating Research Data Flows and Introduction to the Globus Platform
 
Advanced Globus System Administration
Advanced Globus System AdministrationAdvanced Globus System Administration
Advanced Globus System Administration
 
Introduction to Globus for System Administrators
Introduction to Globus for System AdministratorsIntroduction to Globus for System Administrators
Introduction to Globus for System Administrators
 
Introduction to Globus for New Users
Introduction to Globus for New UsersIntroduction to Globus for New Users
Introduction to Globus for New Users
 
Working with Globus Platform Services and Portals
Working with Globus Platform Services and PortalsWorking with Globus Platform Services and Portals
Working with Globus Platform Services and Portals
 
Globus Automation
Globus AutomationGlobus Automation
Globus Automation
 
Advanced Globus System Administration
Advanced Globus System AdministrationAdvanced Globus System Administration
Advanced Globus System Administration
 
Introduction to Globus
Introduction to GlobusIntroduction to Globus
Introduction to Globus
 

Globus Online for Research Data Management

  • 1. globus online Globus Online for Research Data Management Rachana Ananthakrishnan Great Plains Network Annual Meeting 2013
  • 2. We started with technology proven in many large-scale grids GridFTP GRAM MyProxy GSI-OpenSSH …
  • 3. 1.2 PB of climate data delivered to 23,000 users
  • 4. Typical of large, well funded research projects using GT 1.2 PB of climate data delivered to 23,000 users
  • 6. What about the 99%? GT provides robust infrastructure for the 1%
  • 7. What about the 99%? BIG SCIENCE. Small labs GT provides robust infrastructure for the 1%
  • 9. Managing data should be easy … Registry   Staging   Store   Ingest   Store   Analysis   Store   Community   Store   Archive   Mirror   Ingest   Store   Analysis   Store   Community   Store   Archive   Mirror   Registry  
  • 10. … but it’s hard and frustrating! Registry   Staging   Store   Ingest   Store   Analysis   Store   Community   Store   Archive   Mirror   Ingest   Store   Analysis   Store   Community   Store   Archive   Mirror   Registry   Quota exceeded ! Expired credentials ! Network failed. Retry. ! Permission denied !
  • 11. What is Globus Online? Transfer and sharing of large data sets… …with dropbox-like characteristics… …directly from your own storage systems
  • 12. We adopted SaaS approaches to transform the user experience … for both researchers and resource owners/system administrators
  • 13. We started with reliable, secure, high-performance file transfer … Data Source Data Destination User initiates transfer request 1 Globus Online moves and syncs files 2 Globus Online notifies user 3
  • 14. … and then made it simple to share big data off existing storage systems Data Source User A selects file (s) to share, selects user or group, and sets permissions 1 Globus Online tracks shared files; no need to move files to cloud storage! 2 User B logs in to Globus Online and accesses shared file 3
  • 15. Log into Globus Online
  • 26. Interactive login to command line interface: Running commands remotely: Using CLI with gsissh: Globus Online CLI $ ssh tuecke@cli.globusonline.org $ ssh tuecke@cli.globusonline.org <command> $ gsissh tuecke@cli.globusonline.org <command> $ ssh tuecke@cli.globusonline.org scp –r –s 3 -D nersc#dtn:~/myfile* mylaptop:~/projects/p1 Task ID: 4a3c471e-edef-11df-aa30-1231350018b1 $ _
  • 29. •  What is GCMU? –  Globus Connect version for easily creating (sharable) endpoints on multi-user storage servers –  Packages a GridFTP server and MyProxy CA authentication server, pre-configured for use with Globus Online •  Why GCMU? –  Create transfer endpoints in minutes –  Avoid complex GridFTP install •  To download: www.globusonline.org/gcmu Globus Connect Multiuser (GCMU) 29 “We  used  GCMU  to  form  a  campus-­‐ wide  GSI  authenAcaAon  service   spanning  mulAple  servers.  Now  my   users  have  a  fast,  easy  way  to  get  their   data  wherever  it  needs  to  go,  and  the   setup  process  was  trivial."     -­‐-­‐University  of  Michigan   “As  a  resource  admin,  I've  found   GCMU  an  exceedingly  useful   tool....  With  GCMU,  seHng  up  a   GridFTP  server  and  handling   authenAcaAon  for  mulAple  users   is  easy."     -­‐-­‐Oak  Ridge  Na8onal  Lab  
  • 30. We are a non-profit service provider to the non-profit research community
  • 31. Our challenge: Sustainability We are a non-profit service provider to the non-profit research community
  • 32. Globus Online Provider Plans Support ongoing operations Offer value-added capabilities Engage more closely with users
  • 33. •  Endpoint operations management •  Branded web sites •  Alternate identity provider •  Usage reporting •  MSS optimizations •  Multiple GridFTP servers per endpoint Provider Plans offer… Starting at $20k per year
  • 34. End User Plans •  Basic: Free – File transfer and synchronization to/from servers – Personal endpoints with Globus Connect – Access to shared endpoints created by others •  Plus: $7/month (or $70/year) – Create and manage shared endpoints – Peer-to-peer transfer and sharing
  • 35. Globus Platform-as-a-Service Globus Nexus (Identity, Group, Profile) … Sharing Service Transfer Service Dataset Services Globus Toolkit GlobusOnlineAPIs GlobusConnect
  • 36. Our research is supported by: U. S. DE PART M E N T O F ENERGY