Virtualization and Containers

Virtualization- To Infinity and Beyond
For Developers and DBAs
Kellyn Pot’vin-Gorman | Technical Intelligence Manager for the Office of CTO

2© 2017 Delphix Corporation
Kellyn Pot’Vin-Gorman
Technical Intelligence Manager for the Office
of CTO, Delphix
• 2 decades experience as multi-platform DBA,
(Oracle, MSSQL, MySQL, Sybase,
Postgres…..)
• Oracle ACE Director, (Alumni)
• Oak Table Network
• APEX Women in Technology Award, CTA 2014
• STEM education with Raspberry Pi and Python
• Liaison for Denver SQL Server User Group
• President of Rocky Mountain Oracle User
Group, (RMOUG)
• Author, blogger, (http://dbakevlar.com)

Working with Relational Databases Isn’t
Easy
• Projects using RDBMS wait on average of
85% for database creation and refreshes
• Version Control is for code, but data can be
impacted.
• Complex multi-tier systems
• Often requires advance skills to manage and
maintain.
• Rarely if ever just a database tier release.
• Then we have the cloud… 

Virtualization Adoption Trend
.
Source: Gartner Forecasts
0% 20% 40% 60% 80% 100%
2013
2014
2015
2016
2017
2018
Percentage of Virtualized
Worklaods

Cloud Trends
 85% of Enterprises have a multi-cloud strategy
 77% are hybrid cloud, (up over 20% from 2016 cloud survey)
 Workloads are being run in the cloud-
- 41% in public clouds
- 38% in private clouds
 Enterprise Companies are choosing cloud
- 65% want public cloud
- 63% want private cloud solutions
- 93% will be hybrid
State of the Cloud Survey, RightScale

This trend will only increase in the next five years as cloud
continues to overtake the industry

Cloud Adoption and How It’s Changing
0%
20%
40%
60%
80%
100%
Public Cloud Private Cloud Hybrid Cloud Any Cloud
2016
2017
2018
http://www.rightscale.com/blog/cloud-industry-insights/cloud-computing-trends-2017-state-cloud-survey

Cloud by Provider
Cloud Endure 2017
Amazon
Microsoft
Google
IBM
Oracle
Other
Trivia: Which Cloud
Vendor benefitted most
from the Amazon S3
outage?

Timeline in a Project
Waterfall- 5 Releases
January February March April
Agile with Virtualization- 12 Releases
January February March April
Database Refresh
Development Production Release
Testing Cycle
Refresh and
Retest
Skip Test and
Release

How We Really Deploy…
DB
App
Other
DB
App
Other

Make it Easier…
DB
App
Other
DB
App
Other

Release Script-
SQL> delete from kinder_tbl
where c2 like '%40%';
143256 rows deleted.
SQL> commit;
Oh-oh!

Or…
You need more space…
>rm –rf /u02/data/*.c*
“Uhm, that was supposed to say *.cfg*.”
Whoops!
“Why doesn’t the app work? Who released this piece? This
isn’t ready yet!”

Developer or Tester…
Find SCN or Timestamp and….
>FLASHBACK DATABASE TO SCN 198251;
Oh, you don’t have privileges. OK, go find a DBA
You want the files back on the host- You need someone to recover them
or transfer them over- Need a storage admin or a configuration manager.
What if you could do this as part of our job?

Virtualization On-prem, with containers, (templates)
Source DB server
Oracle 10g, 11g, 12c
8TB storage
Delphix Virtualization Engine
8 TB storage
Target DB server
NFS
RMAN
Instance with background
processes, CPU usage and
redo logs
Application Application
Unstructured
Data
Unstructured
Data
Templates/
Container

DBA- Data Version Control

Developer/Tester- Do it Yourself!

After Rewind

Containers=Power
• As we often deploy as a package, (application tier, database tier
and middleware/server, etc.) containerize your dev and test to
deliver as packages, make it simple to refresh and simple to
recover if catastrophic changes occur.
• Significantly speeds up the development cycle, the deployment
delivery cycle to those that need to investigate issues and
refreshes.

What Else Would You Do?
• Flashback or Recovery database?= Time
• Backout scripts
• Refresh environment
Virtualization Engines for Databases Can Be…
• software appliances, (VM) on any hardware.
• tracks continual changes on regular intervals
• Using native technology
• Uses a source, (Dsource) to track changes
• Restores, refreshes and creates Virtual Databases,
recovers files and applications in seconds.
• Uses little to no storage during life of VDB/vfiles.
• IO on Demand

Data Virtualization from the DB Tier
Source DB server
Oracle 9i, 10g, 11g, 12c
8TB storage
8TB storage
RMAN
Note- Storage is
same for All of
Development and
Test as Single
Production

Data Virtualization and Provisioning
8 TB storage
Target DB server
NFS

Data Virtualization and Creating Targets
Target DB server
8 TB storage
NFS
Target DB server
NFS

Data Virtualization and Unique Blocks
Target DB server
8 TB storage
NFS
Target DB server
NFS
Target DB server
NFS

Data Virtualization and Resources for VDBs
Target DB server
8 TB storage
NFS
Target DB server
NFS
Target DB server
NFS
Source DB server
8TB storage
RMAN

Robust and Full Read and Write VDBs
• As many as development, test, reporting
etc. require.
• Full background processes, (each have
their own buffer cache, mmon, log writer,
redo logs, etc.)
• Delphix engine tracks the changes by
SCN and timestamp to each VDB.
--Unstructured data is just pointers to the
gold copy, same with applications.

Why Move to the Cloud?
Savings!!
• Cost
• Resource
• Storage
• Time
Well, that depends on….
• Price per Gb
• Complexity
• SLAs and the
biggest savings….

First Things, First
The Cloud? IaaS/PaaS is not THAT much
different than how you’re a DBA or a Developer.
Virtualized environments? That’s not that
different, either.
Don’t fret, it’s all a big dramatic show…

Managing the Cloud
Set Inbound rules for
connections for your IP
Address, (and reset if your IP
Address changes…) and
you’re set.
• Putty/terminal
• SQL Developer, SQL Tools
• Monitoring Tools connect

Well, That Looks Familiar…
SSH Into Hosts
My Commands
Work!
Env
variables
and SQL is
the Same in
PaaS!

OnPrem to the Cloud
Database server
Cloud Storage
Application Server
File Server
Application Server
Database server
File Server

Standard Cloud Migration Limitations
Even if only development or test are moved to the cloud..
 Databases, apps and some unstructured data may be moved, but
this doesn’t count for ongoing data loads, remaining data and some
applications.
 Refreshes are time consuming
 Often use archaic methods to refresh or replication is required.
 Rarely are difference in cost structures taken into consideration in
cloud migration projects from on-prem configurations.

Traditional copy data management techniques
 Developed without concerns about infrastructure chargeback
 This corresponds to higher cost.
IaaS vendors monitor storage and data transfers
 Help meet SLAs, garner profits
 It’s not just the data that exists in the end, so transformations can equal
big money for cloud vendors.
Different cost structures

On-demand EC2 instances
 Range from $0.0065/hr to $8.184/hr
 Reserved instances provide discounts from 29% to 75%
over on-demand
Storage in Elastic Block Storage
 EBS General Purpose SSD (gp2) volumes
 $0.10 per GB-month of provisioned storage
 EBS Provisioned IOPS SSD (io1) volumes
 $0.065 per provisioned IOPS-month
 EBS Throughput Optimized HDD (st1) volumes
Data transfer in EC2
 DATA IN from “internet” is free, but internally within EC2 can be
$0.01/GB
Cost Estimates for Cloud Vendors
 Source: https://aws.amazon.com/ec2/pricing/ Sep
2016

Much Improved Option-
Data virtualization
 Easy, secure, revolutionary
 Optimized for faster agile
development and testing
 Optimized for new costing
structures in IaaS
 Self Service to make the
developer and tester in your
life ecstatic!
Migrating from on-premise to IaaS

For Non-Production Systems…
- Change the way you’ve always performed tasks.
 Performing common tasks the same way as previously
might end up costing more.
- Secure Data
 All IaaS alternatives promote encryption for data in-flight
and for data at-rest, but encryption may not be the right
answer…
Secondary Considerations

Optimized for the cloud in the first place…
1. Different cost structures
 Much smaller storage footprint,
much less data-transfer
How does data virtualization enhance the Cloud
experience?

Data virtualization to the Cloud as a Container
Source DB server
10TB storage
8 TB storage
Target DB server
No database storage
Application Server
Application Server
File Server
File Server

All IaaS solutions provide encryption in-flight
and encryption at-rest
 But encryption doesn’t protect data as
much as it needs to be .
Europe already requires data masking, not
just data encryption for any confindential
data:
http://ec.europa.eu/justice/data-
protection/article-
29/documentation/opinion-
recommendation/files/2014/wp216_en.pd
f
Confidential data

ob·fus·ca·tion
ˌäbfəˈskāSH(ə)n/Submit
noun
the action of making something obscure, unclear, or unintelligible.
”The hackers attempted to break into the development and test
databases, but left frustrated after they found all the critical data had
significant obfuscation."

Encryption is reversible data obfuscation, which is very different from
masking data.
 Data masking is non-reversible.
 It solves the issue at the data level.
Is authentication and authorization in non-production in compliance with
security goals?
 All organizations will soon need to review if critical data in non-
production environments be accessible to developers, testers
and users.
Confidential data

 Masking personally-identifiable, (PII, HIPPA, PCI, etc.) information
renders it useless from a security standpoint
 Resolves both the technical and personal responsibility issue.
 The data can be masked before it moves to non-production,
removing unnecessary risk.
Why Masking is Important-

Confidential data
Exposure
Production
Non-production

Encryption is reversible data obfuscation, which is very different from
masking data.
 Data masking is non-reversible.
 It solves the issue at the data level.
Is authentication and authorization in non-production in compliance with
security goals?
 All organizations will soon need to review if critical data in non-
production environments be accessible to developers, testers
and users.
Confidential data

 Masking personally-identifiable,
(PII, HIPPA, PCI, etc.) information
renders it useless from a security
standpoint
 Resolves both the technical and
personal responsibility issue.
 The data can be masked before it
moves to non-production,
removing unnecessary risk.
Why Masking is Important-

Agile Masking and File Formats
https://docs.delphix.com/docs/delphix-masking/masking-engine-user-guide/understand-and-manage-the-masking-engine/file-masking/flat-file-masking
• Multi-record
• CSV
• XML
• Word
• Excel
• PowerPoint
• Unstructured
• EDI
• Installation: Agile masking installed with
valid license (file masking option)
• Characteristics: Description and type of
files

Confidential data
Exposure
Production
Non-production
Encryption
Mask
Solution

Data Virtualization, On-Prem with Masking
Linux Source
8 TB database
Delphix Masking Engine
8 TB storage
Linux Target
Application Server
File Server

Data virtualization to the Cloud
8TB storage
Delphix Masking Engine
8 TB storage
Target DB server
Linux Source
8 TB database
Linux Target
Application Server
File Server

Data Virtualization ONLY replicates when
needed…
 For data protection, for scalability, and for
migration
 Virtualized data is already compressed at-
rest, further compressed and encrypted in-
flight, incrementally replicated to another
engine
Virtual Databases and Files Have Selective Replication

And From the Command Line
Full and robust CLI allows for mass
provisioning and scripting options that aren’t
feasible with a GUI.
 Manage environment
 Simple and Complex Provisioning
 User Management
 Storage Management

Set Environment:
snapshot list database=VEmp_826
timeflow "dexample" timeflowRanges;
Commit
Delphix Command Line Examples
Provision a new VDB and use the defaults:
delphix> database provision
After each command, note that the CLI relates the hierarchy:
delphix database provision> defaults
Using the defaults, an example is used for the deployment:

delphix database provision *> set sourceConfig.type=OracleSIConfig
delphix database provision *> set sourceConfig.databaseName=VEM_833
delphix database provision *> set sourceConfig.uniqueName=VEM_833
VDB Configuration

Set the appropriate information just as
you would to create a database
deployment from the GUI:
delphix database provision
defaults *> set
location=DEFAULT_SNAPSHOT
delphix database provision *> set
container.group=“Dev Copies"
Setting Snapshots and Group

Once data has been virtualized on-premises…
 It’s transformed cross-platform on-premises...
 It’s transformed by securely masking on-premises...
Then...
 securely virtualized data can be safely replicated to IaaS
Data virtualization with Transformations

Data virtualization For Releases, Production On-Prem
Target DB server
8 TB storage
8 TB storage
Production Source
Oracle 9i, 10g, 11g, 12c
4TB database
Application Server
Application Server
File Server
File Server
Application
Source
DB Source
File Source

 Watch for Chargebacks
 Data virtualization minimizes storage and data
transfer and transformation costs.
 Confidential Data
 Data masking integrated with data virtualization
truly secures PCI, PII, HIPPA and other
confidential data.
 Focus on Creating Containers to ease
development, deployments and releases.
Summary: Virtualization- Infinity and Beyond

Virtualization and Containers

Recomendados

Recomendados

Más contenido relacionado

La actualidad más candente

La actualidad más candente (20)

Similar a Virtualization and Containers

Similar a Virtualization and Containers (20)

Más de Kellyn Pot'Vin-Gorman

Más de Kellyn Pot'Vin-Gorman (20)

Último

Último (20)

Virtualization and Containers

Notas del editor