4. Azure Storage
Object Storage
Data Transport
File Storage
Hybrid Storage
Lift and shift of legacy
applications that require file
shares to the cloud
Secure, Scalable storage
for unstructured data
Move or migrate data into
Azure
Secure, intelligent data tiering
between on-premises and
cloud storage
Azure Blobs Azure Files
Azure NetApp Files
Azure Import/Export
Azure DataBox
Azure StorSimple
Azure File Sync
Avere
Disk Storage
Reliable, persistent, high
performing storage for
Virtual Machines
Premium
Standard
5. • Self-service data migration
• Simple: order, fill, and return to
upload data to Azure
• Fast: Move ~80 TB per Data Box
• Secure: AES 256-bit encryption
• Status: Preview
Azure Data Box
• Self-service data migration
• Customer supplied disk(s)
• Send disks for upload into Azure
• Secure: AES 128-bit encryption
• Status: Generally Available
Import/Export
• Partner specialists provide HW,
facilities, & expertise
• Migrate tapes, optical drives,
hard disks, even film to Azure
• Contact partners
• Offline Media Import Website
Offline Media
Import
Self-serve and partner-led data transfer services to Azure.
6.
7. Blob Storage
Blob Storage - Azure’s Object
Storage platform
Store and serve unstructured
data
App and Web scale data
Backups and Archive
Big Data from IoT, Genomics,
etc.
Broad integration for Blobs
across Azure services
Enables many scenarios
Blob
Storage
8. Blob Storage Pillars
Durable &
Available
Secure &
Compliant
Manageable &
Cost Efficient
Scalable &
Performant
Open &
Interoperable
9. Blob Storage Pillars
Durable &
Available
Secure &
Compliant
Manageable &
Cost Efficient
Scalable &
Performant
Open &
Interoperable
10. Azure Storage Replication Options
LRS
3 replicas, 1 region
Protect against disk,
node, rack failures
Write is ack’d when all
replicas are committed
Superior to dual-parity
RAID
GRS
6 replicas, 2 regions (3/region)
Protects against major
regional disasters
Asynchronous to secondary
RA-GRS
GRS + Read access to
secondary
Separate secondary
endpoint
RPO delay to secondary can
be queried
ZRS
3 replicas across 3 Zones
Protect against disk, node, rack and
zone failures
Synchronous writes to all 3 zones
Generally Available now in 6 regions
Zone 1 Zone 2 Zone 3
11. Blob Storage Pillars
Durable &
Available
Secure &
Compliant
Manageable &
Cost Efficient
Scalable &
Performant
Open &
Interoperable
14. Security &
Compliance
Encryption at Rest
Always on with MS keys
Customer supplied keys – GA
since 03/18
Firewalls and Virtual Networks
Restrict access from public
internet – GA since 02/18
15. Challenges
Storage Account Keys are all-
powerful
Exposed storage account keys in code
Must build a token service to
issue appropriate SAS Tokens
Map user identities to appropriate permissions
No user identity in storage
diagnostic logs
Requirements
Use enterprise identities
Leverage advanced
authentication technologies
Multi-Factor Auth
Conditional Access Policies
Integrate with of Azure Role-
Based Access Control
16. Security &
Compliance
Encryption at Rest
Always on with MS keys
Customer supplied keys – GA
since 03/18
Firewalls and Virtual Networks
Restrict access from public
internet – GA since 02/18
Azure AD and RBAC
AAD creds for Auth instead of
Shared key/SAS
Full RBAC down to container level
Preview June; GA – H2 CY18
21. Azure Host
Azure VM
Azure Service
(e.g. ARM, Azure Storage)
Your code
IMDS Identity
Credentials
1
2
3
http://169.254.169.254/metadata/identity/oauth2/token
Azure Active Directory
Azure (inject and roll credentials)
22.
23. Security &
Compliance
Encryption at Rest
Always on with MS keys
Customer supplied keys – GA
since 03/18
Firewalls and Virtual Networks
Restrict access from public
internet – GA since 02/18
AAD, OAuth and RBAC
AAD creds for Auth instead of
Shared key/SAS
Full RBAC at container level
Preview June; GA – H2 CY18
Immutable Blob Storage
SEC 17a-4(f) compliant
Across all Storage tiers
Preview – May, GA – Q3 CY18
24.
25. Blob Storage Pillars
Durable &
Available
Secure &
Compliant
Manageable &
Cost Efficient
Scalable &
Performant
Open &
Interoperable
29. Blob Level
Tiering
Blob-Level Tiering
Individual blobs can move between
tiers
All tiers co-exist in the same storage
account
New API to set blob tier:
SetBlobTier
Acknowledged immediately from
service
Get APIs (GetBlobProperties and
ListBlobs) return current tier and
archive status
New headers "x-ms-access-tier” and
“x-ms-archive-status”
Blob REST API
Hot Tier
Lower Transaction cost
Cool Tier
Lower Capacity cost
Archive Tier
Lowest Capacity cost
34. 5.3 Million
Files were Archived in 6
months
Project Payback < 6 Months
and more savings to come
Improved retention
policy compliance
Unified Enterprise Archiving Offer
Available to the whole company
BP Attic Cloud
Archive
35.
36. Blob Storage Pillars
Durable &
Available
Secure &
Compliant
Manageable &
Cost Efficient
Scalable &
Performant
Open &
Interoperable
37. Stateless, front door for request handling (auth, request metering/throttling, validation)
Partition Layer
Serves data based on partitions, enables batch transactions and strong consistency
Stream Layer
Stores multiple replicas of the data, deals with failures, bit rot, etc.
FE 2
Partition 3
(F-J)
Stream 2
38.
39. Per Account
Limits
Through 08/17 Current CY18
Capacity 500 TB 5PB 20 PB
TPS 20K tps 50K tps 100K tps
Throughput 20/30 Gbps 50 Gbps 100 Gbps
Single 4GB blob read from 500 containers on a single account:
100 Gbps!
Blob Storage
Scale & Performance
Per Blob
Single Block Blob max size
increased 25x to 5TB
Read and write throughput for a
single object can scale to as high
as 12.5 GB/s (100 Gb/s)
50% improvement in read latency
for small blobs
Per Storage Account
FE 2
Partition 3
(F-J)
Stream 2
42. Blob Storage Pillars
Durable &
Available
Secure &
Compliant
Manageable &
Cost Efficient
Scalable &
Performant
Open &
Interoperable
43. Open &
Interoperable
Developers & Tools
Open Source client libraries on
GitHub:
Retry/resume, Parallel
upload/downloads, etc.
Mobile platform support
Recent – Python/PHP/Ruby GA, Go
preview
OpenAPI/Swagger based client
libraries
Data movement library
High performance data transfers
Cross platform compliant –
CoreCLR based
Tools
CLI – Python based, Linux friendly
AZCopy – Robocopy for Azure
Storage
Storage Explorer – Cross platform
UX
44. New
Blob Service – 2017-07-29 Blob SDK v11
OpenAPI spec AutoRest Hand-writtenProtocol API Convenience SDK
github.com/azure/azure-storage-java
45. Open &
Interoperable
Linux FUSE Adapter
Mount a container and use regular
filesystem commands
Serves HPC and big data workloads
on Linux
Mount container for reads on many
nodes (no write sync)
Parallel uploads/downloads for fast
access to large blobs
Local read cache to improve
subsequent access times
Partner support for S3
Variety of data, deployment and cost
models
Blob data is non-opaque
Use objects with Blob API including
other Azure services
Azure Blob
48. Blobs CY18
Roadmap
Durable &
Available
Secure &
Compliant
Manageable &
Cost Efficient
Scalable &
Performant
Open &
Interoperable
ZRS V2 Custom Enc. Keys
Firewalls and
Virtual Networks
AAD, OAuth &
RBAC
Immutable Blobs
Soft Delete
Versioning
Monitor
integration
Data Lifecycle
policies
Archive features
Per Blob perf
Per Account
scale
Storage Events
FUSE driver
Static Websites
Re-architected
SDKs
49.
50. Where should I put my files?
Azure Blobs Azure FilesAzure Compute +
Disks
On-premises
File Servers
SharePoint Online
51. What is Azure Files?
Cloud file shares that are…
Secure
Data encrypted at rest and in transit
Easily managed
No servers required
Smart
Make the most of limited networks
with intelligent caching
Cross-platform
Mount on Windows, Linux, and macOS
Built for a hybrid world
Access data where you want to,
how you want to
Harmonious
Migrate applications to the cloud
without the headaches
52. Use Case 1: General Purpose Files as-a-service
HQ Cluster
Branch
Branch
Centralize General Purpose File Services
Multi-site sync for distributed access
Cache for fast, local performance
Utilize cloud-based backup and fast DR
Azure Backup Vault
53. Azure File Sync
HQ
Seattle
File Share
West US
Cloud Tiering
Multi-site sync
Rapid file server DR
Direct cloud access
Integrated cloud backup
PaaS
Azure Backup Vault
3rd party data mngmt
Branch Office
New York
Work Folders
SMB
NFS
Users
Applications
55. Cache shares on-premises with Azure File Sync
Install the Azure File Sync agent (not shown)
Register the server
56. Cache shares on-premises with Azure File Sync
Install the Azure File Sync agent (not shown)
Register the server
Create a file share (if one does not exist)
57. Cache shares on-premises with Azure File Sync
Install the Azure File Sync agent (not shown)
Register the server
Create a file share (if one does not exist)
Create a sync group (with a cloud endpoint)
58. Cache shares on-premises with Azure File Sync
Install the Azure File Sync agent (not shown)
Register the server
Create a file share (if one does not exist)
Create a sync group (with a cloud endpoint)
Create a server endpoint
59. Cache shares on-premises with Azure File Sync
Install the Azure File Sync agent (not shown)
Register the server
Create a file share (if one does not exist)
Create a sync group (with a cloud endpoint)
Create a server endpoint
Validate sync works
60.
61.
62. Background
Checkpoint
State
Persistent
Temporary
Sync DB
SMB & REST
SAS
MAB
Sync
Client
USN
Journal
Recall
Engine
Tiering
Engine
Mgmt
Storage Sync Service (PaaS)
Front
Ends
Front
Ends
Front
Ends
Sync
Worker
Sync
Worker
Sync
Worker
Background
Worker
Background
Worker
Background
Worker MngmtMngmtMgmt
State
Server Agent
Sync
Engine
Filter Driver
Local StorageLocal StorageSync Root
File REST
File REST
SMB
Sync
State
ARM portal, Powershell
Queue
Azure File
Share
Azure File
Share
Azure File
Share
63. Use Case 2: Migrate applications to the cloud
Migrate on-premises file or share based apps often with no code changes
Test migration in stages – first move data, then move application
Interact with app data from on-prem securely
Reuse ISVs existing file based data access layers in your cloud apps
AAD authentication and ACL support coming this year
Azure US-EastOn-premises
SMB3.0
64. Azure US-East
SMBCloud Services
Worker Roles
REST
REST APIs enable integration with modern cloud applications
Integrate modern cloud applications with legacy applications
Develop new file system based applications using REST APIs
REST APIs support
Shared access signature (SAS) token support for delegated authorization
Cross origin resource sharing (CORS) for direct access from browser clients
Storage Client Library Support in .NET, Java, Node and Python
On Premise
Web Sites
Azure US-West
Web Sites
Cloud Services Worker Roles
65. Use Case 4: Simplify hosting HA workload data
SMB2.1 well suited for client / server scenarios
Server applications (e.g. SQL, IIS) require greater reliability during network failures
Using SMB 3.0 persistent handles supports continuous availability
Applications include
Hosting IIS/FTP sites using shared storage
SQL Server Failover Cluster Instance
IIS IIS IIS
IIS Hosting
66. Example 1: FTP highly available
FTP server using Disks
Single VM instance is prone to
availability loss
VM
Azure Disks
FTP server using Files
Increased availability through multiple
VM instances
VM 1 VM2
Azure FilesAzure Files
Azure load
balancer
Azure load
balancer
75. Fully managed file-share service
Native Azure integration (Portal/REST/CLI, Billing, Monitoring, Security)
Sold and supported as a Microsoft product
The power of data with ONTAP
HA, Data Protection, Data Management (Snapshots, Clones), Performance
NFS v3 and v4
Hybrid
Data migration and replication capabilities (SnapMirror)
Secure
Data at Rest Encryption, RBAC
Limited Preview coming in June
Azure NetApp Files
76.
77.
78. Azure Disks
Performance Tiers Management Options
Premium
Disks
SSD based,
provisioned
performance
Standard
Disks
HDD based,
cost effective
Resource Group
Managed Disks
Highly available
and manageable
< 1ms latency for
cached
operations
Blob Cache technology
Up to 160,000 IOPS
Industry leading
ZERO % Annual
Failure Rate
Enterprise grade
durability with 3 replicas
Best in class
High IOPS/BW
80,000 IOPS & 2,000 MB/s
Disk throughput per VM
Recent Feature releases:
Azure Backup and ASR Support
for larger Managed Disks
Managed Disks support for
Azure Availability Zone
Improved Managed Disk metrics
79. Disks
Roadmap
Features for H2 CY2018
Move Managed Disks across
Subscriptions/Resource Groups
Portal support for conversion to
Managed Disks
Managed Disks incremental
snapshots and Diff APIs
Improve data import experience
for Managed Disks
Scale/Perf
Improvements
Standard Managed Disks Premium Managed Disks
Disk Latency Double digit milliseconds
Uncached Read: ~4ms
Uncached Write: ~2ms ~1ms
Max Disk Size 4TB 32TB 4TB 32TB
