SlideShare una empresa de Scribd logo
1 de 25
Descargar para leer sin conexión
Porting DMTCP to Mac OS X
                   Liang Yu                                       Xiang Gong
  College of Computer and Information Science   College of Electrical and Computer Engineering
             Northeastern University                        Northeastern University
               Boston MA 02215                                Boston MA 02215

          Email: liangyu@ccs.neu.edu                     Email: gong.xi@ece.neu.edu
What is the DMTCP?
•  DMTCP	
  (Distributed	
  Mul2Threaded	
  Checkpoin2ng)	
  	
  

•  A	
  tool	
  to	
  transparently	
  checkpoint	
  the	
  state	
  of	
  
   mul2ple	
  simultaneous	
  applica2ons,	
  including	
  mul2-­‐
   threaded	
  and	
  distributed	
  applica2ons.	
  

•  hCp://dmtcp.sourceforge.net/	
  
Why porting to Mac
•  Mac	
  is	
  used	
  extensively	
  




•  No	
  DMTCP	
  in	
  Mac	
  recently	
  
How to do that?
What	
  informa2on	
  of	
  a	
  process	
  we	
  	
  
should	
  pay	
  aCen2on	
  to:	
  
	
  
•      libraries	
  
•      data	
  
•      stack	
  
•      heap	
  
•      threads	
  
•      open	
  file	
  descriptors	
  
•      signal	
  status	
  
•      mutexes	
  
•      process	
  envrionment	
  
How implemented in Linux
     and some Unix
But when it comes
 to Mac OS X ...
In Mac
Distinction in Some Commands
In Mac
Distinction in System Construction
WHAT SHOULD WE DO?
Option One :
How to Construct a Linux-like
Procfs in Mac OS X:
•  Is	
  MacFUSE	
  the	
  perfect	
  solu2on	
  for	
  us?	
  

•  MacFUSE	
  is	
  a	
  Mac	
  OS	
  X	
  implementa2on	
  	
  
	
  	
  	
  	
  	
  of	
  the	
  FUSE	
  (File	
  System	
  in	
  User	
  Space)	
  	
  
	
  	
  	
  	
  	
  mechanism.
We feel less hope,
then we try another method.
Then, we roll back to the previous
problem, thinking an alternative.
Option Two:
The VMA of a Process
in Details
A Demo
•  Find the Memory Range for the stack
Different mechanisms when it
comes the memory boundary
•  Signal	
  Func2on	
  

•  Sigac2on	
  Func2on	
  
	
  	
  	
  	
  -­‐-­‐	
  If	
  you	
  use	
  a	
  global	
  variable	
  to	
  flag	
  a	
  signal	
  from	
  	
  
	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  a	
  signal-­‐handler	
  func2on,	
  it	
  should	
  be	
  of	
  the	
  special	
  	
  
	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  type	
  sig_atomic_t.	
  


•  System	
  Call	
  Abuse	
  
	
  	
  	
  	
  -­‐-­‐	
  e.g.	
  access()	
  system	
  call.	
  
Quick Video Demo – Find Heap
Probing the Libraries
•  Considerably	
  Complicated	
  

•  Right	
  now,	
  we	
  choose	
  “libc”	
  as	
  a	
  beginning,	
  
	
  	
  	
  	
  and	
  we	
  make	
  it.	
  
What discovered during the work



        VS	
  
Enhanced Access() Usage
Result of Enhanced Access()
FUTURE WORK …
•  Using	
  access()	
  call	
  to	
  replace	
  sigac2on()	
  call	
  successes	
  
                 in	
  Linux,	
  but	
  crashes	
  on	
  Mac,	
  need	
  
	
  	
  	
  	
  to	
  figure	
  out.	
  

•  Enhanced	
  Access()	
  can	
  only	
  get	
  all	
  the	
  valid	
  
	
  	
  	
  	
  addresses	
  in	
  a	
  vague	
  way,	
  without	
  showing	
  
	
  	
  	
  	
  which	
  does	
  a	
  segment	
  of	
  memory	
  address	
  	
  	
  	
  
	
  	
  	
  	
  should	
  belongs	
  to.	
  

•  We	
  can	
  only	
  find	
  the	
  memory	
  range	
  of	
  libc,	
  
	
  	
  	
  	
  con2nued	
  work	
  is	
  needed	
  to	
  find	
  other	
  libs,	
  
	
  	
  	
  	
  if	
  existed.	
  	
  
	
  
•  The	
  access	
  right	
  of	
  each	
  memory	
  segment	
  is	
  
	
  	
  	
  	
  	
  is	
  also	
  unknown.	
  
Acknowledgments
•  We	
  would	
  like	
  to	
  thank	
  Prof.	
  Gene	
  Cooperman	
  
	
  	
  	
  	
  for	
  every	
  instruc2ons	
  he	
  gave	
  to	
  us	
  and	
  for	
  the	
  	
  
	
  	
  	
  	
  research	
  spirit	
  he	
  conveyed	
  to	
  us.	
  

•  We	
  would	
  like	
  to	
  acknowledge	
  Kapil	
  and	
  Vlad,	
  for	
  
	
  	
  	
  	
  the	
  system	
  knowledge	
  they	
  taught	
  us.	
  

•  Finally,	
  I’d	
  like	
  to	
  thank	
  Mr.	
  Rick,	
  a	
  classmate	
  
	
  	
  	
  	
  in	
  my	
  another	
  course,	
  for	
  introducing	
  me	
  such	
  
	
  	
  	
  	
  a	
  wonderful	
  tool	
  to	
  depict	
  the	
  sketch.	
  

Más contenido relacionado

La actualidad más candente

Spring sim 2010-riley
Spring sim 2010-rileySpring sim 2010-riley
Spring sim 2010-riley
Sopna Sumāto
 
Inter Process Communication Presentation[1]
Inter Process Communication Presentation[1]Inter Process Communication Presentation[1]
Inter Process Communication Presentation[1]
Ravindra Raju Kolahalam
 
Interprocess communication
Interprocess communicationInterprocess communication
Interprocess communication
Sushil Singh
 
Distributed file systems (from Google)
Distributed file systems (from Google)Distributed file systems (from Google)
Distributed file systems (from Google)
Sri Prasanna
 

La actualidad más candente (20)

An Overview of Distributed Debugging
An Overview of Distributed DebuggingAn Overview of Distributed Debugging
An Overview of Distributed Debugging
 
Ns3
Ns3Ns3
Ns3
 
Spring sim 2010-riley
Spring sim 2010-rileySpring sim 2010-riley
Spring sim 2010-riley
 
IPC
IPCIPC
IPC
 
ns-3 Tutorial
ns-3 Tutorialns-3 Tutorial
ns-3 Tutorial
 
Lec1 Intro
Lec1 IntroLec1 Intro
Lec1 Intro
 
Ns2
Ns2Ns2
Ns2
 
Capturando pacotes de rede no kernelspace
Capturando pacotes de rede no kernelspaceCapturando pacotes de rede no kernelspace
Capturando pacotes de rede no kernelspace
 
Network Simulation
Network SimulationNetwork Simulation
Network Simulation
 
Beyond Text - Methods as Objects
Beyond Text - Methods as ObjectsBeyond Text - Methods as Objects
Beyond Text - Methods as Objects
 
Migration To Multi Core - Parallel Programming Models
Migration To Multi Core - Parallel Programming ModelsMigration To Multi Core - Parallel Programming Models
Migration To Multi Core - Parallel Programming Models
 
Inter-Process Communication (IPC) techniques on Mac OS X
Inter-Process Communication (IPC) techniques on Mac OS XInter-Process Communication (IPC) techniques on Mac OS X
Inter-Process Communication (IPC) techniques on Mac OS X
 
Inter Process Communication Presentation[1]
Inter Process Communication Presentation[1]Inter Process Communication Presentation[1]
Inter Process Communication Presentation[1]
 
Interprocess communication
Interprocess communicationInterprocess communication
Interprocess communication
 
Distributed file systems (from Google)
Distributed file systems (from Google)Distributed file systems (from Google)
Distributed file systems (from Google)
 
Google Spanner : our understanding of concepts and implications
Google Spanner : our understanding of concepts and implicationsGoogle Spanner : our understanding of concepts and implications
Google Spanner : our understanding of concepts and implications
 
Threads and multi threading
Threads and multi threadingThreads and multi threading
Threads and multi threading
 
IPC
IPCIPC
IPC
 
ITFT_Inter process communication
ITFT_Inter process communicationITFT_Inter process communication
ITFT_Inter process communication
 
Ground to ns3 - Basic wireless topology implementation
Ground to ns3 - Basic wireless topology implementationGround to ns3 - Basic wireless topology implementation
Ground to ns3 - Basic wireless topology implementation
 

Similar a Porting dmtcp mac_slides

Scalability20140226
Scalability20140226Scalability20140226
Scalability20140226
Nick Kypreos
 
3 f6 9_distributed_systems
3 f6 9_distributed_systems3 f6 9_distributed_systems
3 f6 9_distributed_systems
op205
 
Processes and Threads in Windows Vista
Processes and Threads in Windows VistaProcesses and Threads in Windows Vista
Processes and Threads in Windows Vista
Trinh Phuc Tho
 
Fixing Twitter Improving The Performance And Scalability Of The Worlds Most ...
Fixing Twitter  Improving The Performance And Scalability Of The Worlds Most ...Fixing Twitter  Improving The Performance And Scalability Of The Worlds Most ...
Fixing Twitter Improving The Performance And Scalability Of The Worlds Most ...
xlight
 

Similar a Porting dmtcp mac_slides (20)

Introduction to Big Data
Introduction to Big DataIntroduction to Big Data
Introduction to Big Data
 
The Future of Networking, and the Past of Protocols
The Future of Networking, and the Past of ProtocolsThe Future of Networking, and the Past of Protocols
The Future of Networking, and the Past of Protocols
 
Scalability20140226
Scalability20140226Scalability20140226
Scalability20140226
 
Oldhabits
OldhabitsOldhabits
Oldhabits
 
John adams talk cloudy
John adams   talk cloudyJohn adams   talk cloudy
John adams talk cloudy
 
Network architecure (3).pptx
Network architecure (3).pptxNetwork architecure (3).pptx
Network architecure (3).pptx
 
Deep_Learning_Frameworks_CNTK_PyTorch
Deep_Learning_Frameworks_CNTK_PyTorchDeep_Learning_Frameworks_CNTK_PyTorch
Deep_Learning_Frameworks_CNTK_PyTorch
 
Outsourced database
Outsourced databaseOutsourced database
Outsourced database
 
Kafka 0.8.0 Presentation to Atlanta Java User's Group March 2013
Kafka 0.8.0 Presentation to Atlanta Java User's Group March 2013Kafka 0.8.0 Presentation to Atlanta Java User's Group March 2013
Kafka 0.8.0 Presentation to Atlanta Java User's Group March 2013
 
Outsourced database
Outsourced databaseOutsourced database
Outsourced database
 
Dc lec-06 & 07 (osi model)
Dc lec-06 & 07 (osi model)Dc lec-06 & 07 (osi model)
Dc lec-06 & 07 (osi model)
 
3 f6 9_distributed_systems
3 f6 9_distributed_systems3 f6 9_distributed_systems
3 f6 9_distributed_systems
 
The Power of Determinism in Database Systems
The Power of Determinism in Database SystemsThe Power of Determinism in Database Systems
The Power of Determinism in Database Systems
 
Processes and Threads in Windows Vista
Processes and Threads in Windows VistaProcesses and Threads in Windows Vista
Processes and Threads in Windows Vista
 
Fixing twitter
Fixing twitterFixing twitter
Fixing twitter
 
Fixing_Twitter
Fixing_TwitterFixing_Twitter
Fixing_Twitter
 
Fixing Twitter Improving The Performance And Scalability Of The Worlds Most ...
Fixing Twitter  Improving The Performance And Scalability Of The Worlds Most ...Fixing Twitter  Improving The Performance And Scalability Of The Worlds Most ...
Fixing Twitter Improving The Performance And Scalability Of The Worlds Most ...
 
Fixing Twitter Improving The Performance And Scalability Of The Worlds Most ...
Fixing Twitter  Improving The Performance And Scalability Of The Worlds Most ...Fixing Twitter  Improving The Performance And Scalability Of The Worlds Most ...
Fixing Twitter Improving The Performance And Scalability Of The Worlds Most ...
 
What's inside the black box? Using ML to tune and manage Kafka. (Matthew Stum...
What's inside the black box? Using ML to tune and manage Kafka. (Matthew Stum...What's inside the black box? Using ML to tune and manage Kafka. (Matthew Stum...
What's inside the black box? Using ML to tune and manage Kafka. (Matthew Stum...
 
Thread
ThreadThread
Thread
 

Último

Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Safe Software
 

Último (20)

presentation ICT roal in 21st century education
presentation ICT roal in 21st century educationpresentation ICT roal in 21st century education
presentation ICT roal in 21st century education
 
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot TakeoffStrategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
 
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
 
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemkeProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
 
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
 
ICT role in 21st century education and its challenges
ICT role in 21st century education and its challengesICT role in 21st century education and its challenges
ICT role in 21st century education and its challenges
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
 
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, AdobeApidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
 
Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...
 
A Beginners Guide to Building a RAG App Using Open Source Milvus
A Beginners Guide to Building a RAG App Using Open Source MilvusA Beginners Guide to Building a RAG App Using Open Source Milvus
A Beginners Guide to Building a RAG App Using Open Source Milvus
 
Artificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : UncertaintyArtificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : Uncertainty
 
AXA XL - Insurer Innovation Award Americas 2024
AXA XL - Insurer Innovation Award Americas 2024AXA XL - Insurer Innovation Award Americas 2024
AXA XL - Insurer Innovation Award Americas 2024
 
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin WoodPolkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
 
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost SavingRepurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdf
 
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
 
FWD Group - Insurer Innovation Award 2024
FWD Group - Insurer Innovation Award 2024FWD Group - Insurer Innovation Award 2024
FWD Group - Insurer Innovation Award 2024
 
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
 
Strategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherStrategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a Fresher
 

Porting dmtcp mac_slides

  • 1. Porting DMTCP to Mac OS X Liang Yu Xiang Gong College of Computer and Information Science College of Electrical and Computer Engineering Northeastern University Northeastern University Boston MA 02215 Boston MA 02215 Email: liangyu@ccs.neu.edu Email: gong.xi@ece.neu.edu
  • 2. What is the DMTCP? •  DMTCP  (Distributed  Mul2Threaded  Checkpoin2ng)     •  A  tool  to  transparently  checkpoint  the  state  of   mul2ple  simultaneous  applica2ons,  including  mul2-­‐ threaded  and  distributed  applica2ons.   •  hCp://dmtcp.sourceforge.net/  
  • 3. Why porting to Mac •  Mac  is  used  extensively   •  No  DMTCP  in  Mac  recently  
  • 4. How to do that? What  informa2on  of  a  process  we     should  pay  aCen2on  to:     •  libraries   •  data   •  stack   •  heap   •  threads   •  open  file  descriptors   •  signal  status   •  mutexes   •  process  envrionment  
  • 5. How implemented in Linux and some Unix
  • 6. But when it comes to Mac OS X ...
  • 7. In Mac Distinction in Some Commands
  • 8. In Mac Distinction in System Construction
  • 11. How to Construct a Linux-like Procfs in Mac OS X: •  Is  MacFUSE  the  perfect  solu2on  for  us?   •  MacFUSE  is  a  Mac  OS  X  implementa2on              of  the  FUSE  (File  System  in  User  Space)              mechanism.
  • 12. We feel less hope, then we try another method.
  • 13. Then, we roll back to the previous problem, thinking an alternative.
  • 15. The VMA of a Process in Details
  • 16. A Demo •  Find the Memory Range for the stack
  • 17. Different mechanisms when it comes the memory boundary •  Signal  Func2on   •  Sigac2on  Func2on          -­‐-­‐  If  you  use  a  global  variable  to  flag  a  signal  from                            a  signal-­‐handler  func2on,  it  should  be  of  the  special                            type  sig_atomic_t.   •  System  Call  Abuse          -­‐-­‐  e.g.  access()  system  call.  
  • 18. Quick Video Demo – Find Heap
  • 19. Probing the Libraries •  Considerably  Complicated   •  Right  now,  we  choose  “libc”  as  a  beginning,          and  we  make  it.  
  • 20. What discovered during the work VS  
  • 22. Result of Enhanced Access()
  • 24. •  Using  access()  call  to  replace  sigac2on()  call  successes   in  Linux,  but  crashes  on  Mac,  need          to  figure  out.   •  Enhanced  Access()  can  only  get  all  the  valid          addresses  in  a  vague  way,  without  showing          which  does  a  segment  of  memory  address                should  belongs  to.   •  We  can  only  find  the  memory  range  of  libc,          con2nued  work  is  needed  to  find  other  libs,          if  existed.       •  The  access  right  of  each  memory  segment  is            is  also  unknown.  
  • 25. Acknowledgments •  We  would  like  to  thank  Prof.  Gene  Cooperman          for  every  instruc2ons  he  gave  to  us  and  for  the            research  spirit  he  conveyed  to  us.   •  We  would  like  to  acknowledge  Kapil  and  Vlad,  for          the  system  knowledge  they  taught  us.   •  Finally,  I’d  like  to  thank  Mr.  Rick,  a  classmate          in  my  another  course,  for  introducing  me  such          a  wonderful  tool  to  depict  the  sketch.