Cloud Storage - Azure Blob and Amazon S3.
Please note that some slides are not converted by slideshare. Click Save link to download the original presentation.
Boston Code Camp - Cloud Storage (Azure Blob and Amazon S3)
1. Cloud Storage
(Azure Blob and Amazon S3)
Udaiappa Ramachandran (Udai)
Chief Software Architect @ Ektron, Inc.,
New Hampshire – Nashua Cloud/.NET User Group Organizer
LinkedIn: http://www.linkedin.com/in/udair
Blog: http://cloudycode.wordpress.com
2. Cloud Storages - Agenda
• Overview of Cloud Storage
• Azure Blob Storage
• Amazon S3
• Comparing Azure Blob and Amazon S3
• Benchmark comparisons for performance, stability, and
scalability
• Best Practice
• Resource
• QA
3. Overview of Cloud Storage
• Evaluation
• Network Storage
• Hosted Storage
• Cloud Based Storage
• Benefits
• Highly Available with Strong Consistency
• Provides access to data in face of failures/partitioning
• Durability
• Replicate data several times within and across data centers
• Scalability
• Need to scale to Exabyte’s and beyond
• Provide a global namespace to access data around the world
• Automatically load balance data to meet peak traffic demands
• Easy setups and low cost
• Pay as you go
• API Accessible, Emulator, Web Portal
5. Azure Blob Storage Abstractions
• Blobs
• File with a metadata
• Blob Types
• Block Blob
• Targeted for streaming workloads
• Each blob consists of a sequence of blocks, Each block is identified by a Block ID targeted for streaming workloads
• Size limit 200GB per blob
• Optimistic concurrency via ETags
• Page Blob
• Targeted for random read/write workloads
• Each blob consists of an array of pages. Each page is identified by its offset from the start of the blob
• Size limit 1TB per blob
• Optimistic or Pessimistic (locking) concurrency via leases
• Drives
• Durable NTFS volumes for Windows Azure applications to use
• Based on Blobs
• Drive access across multiple instance(s) possible thru SMB
6. Azure Blob Storage Details
• Storage End Point
• http://<account>.blob.core.windows.net/<container>/<blobname>
• [Account][Container][Blobs][Pages/Blocks]
• Mock Storage Emulator
• Can CDN Enable Account
• Blobs delivered via 24 global CDN nodes
• Can co-locate storage account with compute account
• Explicitly or using affinity groups
• Accounts have two independent 512 bit shared secret keys
• 100 TBs per account
• Geo-Replication
• Storage Analytics
• Logs: Provide trace of executed requests for your storage accounts
• Metrics: Provide summary of key capacity and request statistics for Blobs, Tables, and Queues
• HTTP headers for Blobs
• RESTful and Client API support
7. Azure Blob Storage Details Continued…
• Associate Metadata With blob
• Standard HTTP metadata/headers
(Cache-Control, Content-Encoding, Content-Type, etc)
• Metadata is <name, value> pairs, up to 8KB per blob
• Either as part of PutBlob or independently
• Blob always accessed by name
• Can include ‘/‘ or other delimiters in name
e.g. /<container>/myblobs/blob.jpg
• Basic Blob Operations
• GetBlob
• DeleteBlob
• CopyBlob
• SnapshotBlob
• LeaseBlob
• ListBLobs
8. Azure Blob Storage -Drives
• Durable NTFS volume for Windows Azure Instances
• Use existing NTFS APIs to access a network attached durable drive
• Use System.IO from .NET
• Benefits
• Move existing apps using NTFS more easily to the cloud
• Durability and survival of data on instance recycle
• A Windows Azure Drive is an NTFS VHD Page Blob
• Mounts Page Blob over the network as an NTFS drive
• Local cache on instance for read operations
• All flushed and un buffered writes to drive are made durable to the Page Blob
• A Windows Azure Drive is a Page Blob formatted
as a NTFS single volume Virtual Hard Drive (VHD)
• Drives can be up to 1TB
9. Azure Blob Storage -Drives Continued…
• A Page Blob can be mounted:
• On one instance at a time for read/write
• Using read-only snapshots to multiple instances at once
• An instance can dynamically mount up
to 16 drives
• Remote Access via standard Blob UI
• Can’t remotely mount drive
• Can upload the VHD to a Page Blob using the blob interface, and then mount it as a
Drive
• Can download the VHD to a local file and mount locally
• Operations performed via Drive API not REST Calls
• Operations on Drives
• Create Drive
• Mount / UnMount Drive
• Get Mounted Drives
• Snapshot Drive
11. Simple Storage Service (S3)
• Storage Endpoints
• Virtual Hosted Style: http[s]://<bucketname>.s3-website-[us-east-1].amazonaws.com/object
• Path-Style: http[s]://s3.amazonaws.com/<bucketname>/object
• Associate Metadata With blob
• Standard HTTP metadata/headers
(Cache-Control, Content-Encoding, Content-Type, etc)
• Metadata is <name, value> pairs, up to 8KB per blob
• Either as part of PutBlob or independently
• Blob always accessed by key
• Can include ‘/‘ or other delimeter in name but folder must end with ‘/’
e.g. /<bucketname>/myblobs/blob.jpg
• 1-5 TB per blob
• Unlimited Size per accounts
• RESTful/API support
• Storage Analytics
• Ability to host static template from blob
• Bittorrent protocol support
12. S3- Basic Blob Operations
• Put
• Get
• Delete
• Copy
• List
13. Comparing Azure Blob and Amazon S3
Please note that next 3 slides may not be converted by slideshare.
Click Save link to download the original presentation.
17. Best Practices
• Avoid lock-in to specific provider
• Choose the location closest to your customer
• Use only lower cases for container/key name
• Always remove public access from container/bucket. Instead
grant public access to keys if required.
• Avoid unnecessary request (such a call to check if blob exists
instead use the response)
• Use compression for large file if possible
• Enable CDN