Más contenido relacionado La actualidad más candente (20) Similar a Put your Java apps to sleep? Find out how - John Matthew Holt (Waratek) (20) Más de jaxLondonConference (18) Put your Java apps to sleep? Find out how - John Matthew Holt (Waratek)1. Put
your
Java
Apps
to
Sleep?
Find
out
How…
John
Ma(hew
Holt
|
JAX
London
2013
!
© Copyright Waratek 2013
2. John Matthew Holt!
• CTO
and
co-‐Founder
of
Waratek
• Been
doing
JVM
engineering
for
10
years
• Invented
“Java
VirtualizaCon”
to
transform
how
JVMs
operate
in
virtualized
and
cloud-‐
compuCng
environments
• Lead
the
design
of
the
first
Type-‐1
VMM
for
a
JVM
© Copyright Waratek 2013
3. What will we cover?!
• First
some
background…
– Why
virtualize
INSIDE
the
JVM?
– Think
“VMware
for
Java”
• PuSng
you
Java
Apps
to
Sleep…
– Why
are
Java
apps
Cred?
(or
restated:
why
should
they
be
slept?)
– Let
your
apps
nap:
“wake-‐on-‐LAN
for
JVCs”
© Copyright Waratek 2013
5. Because…!
• Because
JVMs
make
bad
bedfellows
with
HW
hypervisors
(HV)
• A
JVM
thwarts
HW
hypervisor
features
like:
– The
GC
in
a
JVM
thwarts
memory-‐overcommit
in
a
HW
HV
– The
GC’s
heap
memory,
and
the
JIT’s
permgen/
codecache,
thwarts
‘transparent
page
sharing’
(memory
de-‐dup)
in
a
HW
HV
– Dedicated
GC
and
JIT
for
EVERY
Java
applicaCon
wastes
lots
and
lots
of
CPU
Cme
and
memory
capacity
doing
the
same
thing
over,
and
over,
and
over,
and…
© Copyright Waratek 2013
6. Today’s JVMs are OLD!
• Were
designed
~20
years
ago,
and
haven’t
changed
design
since
then
• JVM
design
PREDATES
the
major
2
trends
of
last
decade:
virtualizaCon
and
cloud-‐compuCng
• Only
2
innovaCons
in
JVMs
in
20
years:
– Beder
JIT
compilers
– Concurrent
GCs
– ….all
blindly
focused
on
single-‐tenant
performance
at
the
expense
of
mulC-‐tenant
efficiency
© Copyright Waratek 2013
7. Server Java has problems!
• JVMs
are
grossly
inefficient
users
of
CPU
and
memory
– A
JVM
assumes
it
can
use
all
available
CPU
and
memory
capacity
for
itself
and
its
single
applicaCon,
eg:
– 1
JBoss
can
consume
up
to
500MB
RSS
with
no
apps
– 1
Tomcat
can
consume
up
to
250MB
RSS
with
no
webapps
– Running
mulCple
JVMs
on
a
server
rapidly
exhausts
physically
memory
while
CPU
uClisaCon
averages
5%
• >100
different
JVM
versions
makes
compliance/support
a
NIGHTMARE
• Security
vulnerabiliCes
and
compromises
are
EVERYWHERE
inside
Java
© Copyright Waratek 2013
8. VMWare does not virtualize
Java well!
• Realworld
stats
using
VMware:
– >80%
of
Server
JVMs
have
<=2GB
of
heap
memory
• Yet
actual
applicaCon
heap
uClisaCon
averages
<20%
– A
2GB
JVM
heap
needs
~3GB
of
OS
RSS
memory,
which
in
turn
needs
~4GB
of
VMWare
tenant
memory
• …reducing
actual
applicaCon
memory
uClisaCon
to
<10%
– CPU
uClisaCon
running
Java
on
Vmware
averages
<7%
• Yet
physical
memory
is
exhausted
because
of
running
mulCple
JVMs
© Copyright Waratek 2013
9. Java is a compliance nightmare!
• Realworld
stats:
– 93%
of
organisaCons
are
running
versions
of
Java
at
least
5
years
old
– The
average
organisaCon
has
51
disCnct
versions
of
Java
installed
– <1%
of
installed
Java
is
upgraded
to
the
latest
version
• Recurring
enterprise
conversaCon:
– QuesCon:
“what
version
of
Java
do
you
run?”
– Answer:
“every
version
except
the
latest
version!”
© Copyright Waratek 2013
10. Java is a security nightmare!
• Realworld
stats:
– In
last
12
months
alone,
>250
CVE
alerts
issued
by
US-‐
CERT/NIST
for
Java
– The
most
prevalent
version
of
Java
has
96
“perfect
10”
vulnerabiliCes
– “There
are
virtually
no
modern
versions
of
Java
without
any
known
severe
vulnerabiliCes”
(source:
Bit9)
• CriCcal
Security
Patch
Updates
are
released
so
oven
that
developers
and
admins
can’t
keep
up
© Copyright Waratek 2013
11. Virtualization is the answer…!
• …but
HIGHER
up
the
Server
Stack
• VirtualizaCon
is
creeping
up
the
Server
Stack
already
– x86
HW
got
virtualized
in
1999
(VMware
et
al)
– x86
OS
got
virtualized
in
2006
(OpenVZ,
LXC,
et
al)
– …now
it’s
Java’s
turn
in
2013
• Virtualizing
Java
means
virtualizing
at
the
TOP
of
the
Server
Stack,
whereas
VMware
virtualizes
at
the
BOTTOM
© Copyright Waratek 2013
12. Cost-per-tentant to zero !
• The
higher
up
virtualizaCon
goes
up
the
Server
Stack,
the
cheaper
a
“tenant”
becomes
• Virtualizing
INSIDE
the
JVM
gets
close
to
“zero-‐
cost
tenants”:
– Helloworld
in
<1MB
of
tenant
memory
– Examples.war
(Tomcat
demo
webapps)
in
<5MB
of
tenant
memory
– …Java
becomes
CHEAPER
than
python/ruby/perl
when
the
JVM
itself
gets
virtualized!
© Copyright Waratek 2013
13. Virtualization == Security!
• Virtualizing
the
JVM
makes
Server
Java
secure
and
compliant
– Virtualized
Java
Apps
run
as
“tenants”
in
a
shared
JVM
– Tenants
run
in
Java
Virtual
Container
(JVC)
“jails”
– Tenants
are
isolated
and
“locked-‐down”
– Tenants
can
run
any
Java
version
• Doesn’t
mader
what
“Java
version”
a
tenant
uses,
it
gets
the
latest
JVM
• Means
a
“Java
version”
becomes
just
a
tenant
API
for
the
convenience
of
developers
• Admins
control
the
REAL
version
at
the
JVM
level
so
that
tenants
(developers)
can
stop
caring
about
“versions”
altogether
© Copyright Waratek 2013
15. A hypervisor INSIDE a JVM!
• Waratek
has
wriden
a
type-‐1
VMM
that
runs
INSIDE
the
Oracle
HotSpot
and
OpenJDK
JVMs
– >300,000
lines
of
Java
code,
<5K
lines
of
C-‐code
– Can
easily
run
on
any
OS/CPU
pair
(only
Linux/x86
supported
so
far)
– Does
EVERYTHING
VMware
does,
but
INSIDE
the
JVM
•
•
•
•
•
•
Virtual
FS
Virtual
networks/IPs/NICs
CPU
pinning
and
quotas
Memory
overcommit,
elasCc
memory,
memory
quotas
I/O
QoS,
rate-‐limiCng
Etc…
– …all
with
ZERO
code
change
to
Java
applicaCons
© Copyright Waratek 2013
16. A real VMM inside the JVM!
© Copyright Waratek 2013
17. VMWare for Java!
Java Code
JVC
Java App
java.* APIs
SecurityManager
Waratek Java
Hypervisor
Process
C/C++ Code
JNI
Libs
• Waratek
virtualizes
Java
exactly
like
VMware
virtualizes
HW
– Waratek
inserts
a
“Java
Hypervisor”
between
the
JVM
and
the
Java
App
+
java.*
APIs
+
SecurityManager
– The
Java
app
PLUS
java.*
APIs
and
SecurityManager
run
inside
a
Java
Virtual
Container
(JVC)
“jail”
– For
a
Java
App
to
go
outside
of
its
JVC
“jail”
it
has
to
“trap”
to
the
Java
Hypervisor
Waratek
JVM
© Copyright Waratek 2013
18. VMWare for Java!
Java Code
JVC
Java App
java.* APIs
SecurityManager
Waratek Java
Hypervisor
Process
C/C++ Code
JNI
Libs
Waratek
JVM
• Every
incoming
and
outgoing
access
by
a
JVC
is
intercepted
and
checked
by
the
Java
Hypervisor,
e.g:
–
–
–
–
–
–
–
–
–
–
–
–
–
–
Class
loading
intercepted
and
checked
JNI
invocaCon
intercepted
Parameter
and
return
value
checks
File
access
(delete/copy/move)
checks
Network
operaCon
checks
IO
rate-‐limiCng
and
flow-‐control
CPU
Cme
control
Memory
consumpCon
control
Buffer
overflow
checks
Null
pointer
checks
Type
safety
checks
OS
access
checks
JNI
Libraries
isolated
…
© Copyright Waratek 2013
19. Slots into existing VM tools!
• You
can
manage
the
Java
hypervisor
exactly
like
you
manage
exisCng
HW
hypervisors:
–
–
–
–
SSH
CLI
RedHat
LibVirt/KVM
integraCon
(beta)
VMWare
vCAC
integraCon
shortly
OpenStack
integraCon
shortly
• Plus
other
management
interfaces
–
–
–
–
–
HTTP
REST
API
Local
and
remote
JMX
APIs
/proc/PID/*
pseudo-‐filesystem
for
JVM
and
each
JVC
Vmware
CloudFoundry
integraCon
(beta)
RedHat
OpenShiv
integraCon
soon
© Copyright Waratek 2013
21. JVCs make Java secure!
Java Code
JVC
Java App
java.* APIs
SecurityManager
Waratek Java
Hypervisor
Process
C/C++ Code
JNI
Libs
Waratek
JVM
• If
a
Java
App
inside
a
JVC
compromises
either/both
of
java.*
APIs
or
SecurityManager,
then
only
that
JVC
is
effected
and
NOT
the
Java
Hypervisor
or
JVM
or
other
JVCs
• JNI
Libraries
are
isolatable
in
their
own
OS
process
• Thus
a
compromised
JVC
cannot:
–
–
–
–
–
–
–
Insert
malicious
code
outside
its
JVC
Delete/move/copy
files
outside
its
VROOT
Trigger
buffer
overflows
Do
null-‐pointer
segmentaCon
faults
Perform
unsafe
heap
memory
access
Fork
OS
processes
etc…
© Copyright Waratek 2013
22. “Come in, we’re open”!
• The
Java
Hypervisor
has
an
open
API
for
unlimited
customizaCon
and
extensibility
• Anyone
can
write
their
own
PURE-‐JAVA
plugins
to
the
Java
hypervisor
to
customize
ANYTHING
about
JVCs,
eg:
– Make
a
DropBox.com
account
the
virtual
filesystem
for
a
JVC
– Create
a
virtual
LAN
between
distributed
JVCs
anywhere
in
the
world
– Clone
live
network
I/O
of
an
acCve
JVC
to
a
test
JVC
for
side-‐by-‐side
applicaCon
tesCng
© Copyright Waratek 2013
23. Example hypervisor plugin!
• Waratek
wrote
ElasCcat
as
one
example
of
a
Java
Hypervisor
plugin
• ElasCcat
is
open
source,
is
<1000
lines
of
Java
code,
and
loads
into
the
Java
hypervisor
at
startup
• ElasCcat
virtualizes
Apache
Tomcat
so
it
becomes
the
world’s
first
mulC-‐tenant
JavaEE
container:
– Tomcat
runs
in
one
JVC
and
each
webapp
runs
in
it’s
own
dedicated
JVC
– If
one
webapp
crashes
it
won’t
crash
any
other
JVC
– …all
this
achieved
with
ZERO
code
change
to
Tomcat
or
webapps,
and
without
them
knowing
they’ve
been
virtualized!
© Copyright Waratek 2013
25. Putting Java to jSleep…!
• Why
are
Java
apps
Cred?
(or:
why
should
they
be
slept?)
© Copyright Waratek 2013
26. Java Apps are tired!!
• Realworld
stats:
– >50%
of
PaaS
apps
are
“dead
bone
idle”
at
any
given
Cme
– The
average
Dev/Test
environment
runs
at
<10%
sustained
uClizaCon
– Lots
and
lots
of
enterprise
Java
apps
are
only
used
periodically,
e.g.
end-‐of-‐week,
end-‐of-‐month…
• But
these
apps
have
SLAs
and
can’t
be
shut-‐off!
© Copyright Waratek 2013
27. Fact from the field:!
• So
frustrated
was
one
Fortune500
admin
that
he
shut-‐off
a
bunch
of
Java
apps
that
had
been
idle
for
2
weeks
(no
TCP
traffic)
and
waited
to
see
what
would
happen…
the
results
were:
– …a
small
number
of
app-‐owners
noCced
immediately
(1-‐2
days)
– …the
largest
number
of
app-‐owners
noCced
within
1-‐2
weeks
– …and
a
small
number
of
app-‐owners
NEVER
noCced
at
all!
© Copyright Waratek 2013
28. Wouldn’t it be great if…!
• …you
could
“compress”
an
idle
app
and
re-‐
awaken
it
instantly
WITHOUT
THE
APP
KNOWING?
• …and
without
the
END-‐POINTS
knowing?
• …and
without
ever
RESTARTING
the
app
later?
© Copyright Waratek 2013
29. Actually it exists…almost!!
• The
technology
for
this
nearly
already
exists:
– It’s
called
“wake-‐on-‐LAN”
(WoL)
• …but
there’s
a
problem:
WoL
only
works
at
the
HARDWARE
level,
so
is
of
NO
USE
in
mulCtenant
virtualized
cloud-‐compuCng
environments
• Think
about
it:
– As
long
as
ONE
tenant
on
a
server
is
acCve
then
the
server
HAS
to
stay
alive
and
WoL
CANNOT
be
used!
© Copyright Waratek 2013
30. Putting Java to jSleep…!
• Let
your
apps
nap:
wake-‐on-‐LAN
for
JVCs
© Copyright Waratek 2013
31. “Wake-on-LAN for JVCs”!
• What’s
needed
is
a
NEW
kind
of
WoL
that
“naps”
single
idle
tenants
on
a
mulCtenant
server…
• …and
reclaims
the
memory/CPU/IO
quotas
that
were
provisioned
for
the
now-‐napped
app,
to
be
used
to
run
other
new
apps
• This
is
what
Waratek
jSleep
does
for
Java
apps!
© Copyright Waratek 2013
32. jSleep in a Slide!
• Waratek
jSleep
works
inside
the
Java
Hypervisor
to
do
four
things:
– 1)
monitor
the
acCvity
of
individual
apps
(JVCs)
– 2)
put
idle
JVCs
to
sleep
aver
defined
inacCvity
thresholds
and
reclaim
their
CPU/mem/IO
quotas
– 3)
monitor
the
I/O
connecCons
(TCP/UDP
Sockets,
ServerSockets,
Files,
etc)
of
a
sleeping
JVC
– 4)
and
awaken
a
sleeping
JVC
when
I/O
acCvity
is
detected,
reinstaCng
reclaimed
quotas
• …it’s
wake-‐on-‐LAN
for
JVCs!
© Copyright Waratek 2013
33. jSleep is Very Powerful!
• Lots
of
“knobs
and
dials”
make
it
possible
to
define
sophisCcated
sleep/awake
profiles
that
respond
to:
– CPU
acCvity
– Network/IO
acCvity
– IO
Cmeouts
– Wall-‐clock
Cme
– …or
just
about
anything
else
• THERE’S
ALSO
a
built-‐in
network
profile
that
allows
for
immediate
configuraCon
via
the
JIRSH
CLI
© Copyright Waratek 2013
34. Simple jSleep Example!
• Set
a
jSleep
policy
based
on
network
traffic
of
defined
sockets/ports,
eg:
– jirsh # sleepinfo 1 –-set
network=120000,include_local=:8080,10.1.0.8:4444
– Above
command:
• Suspends
JVC
1
aver
the
network
interface
is
idle
120s
• The
include_local param
is
a
series
of
host:port
combinaCons
that
specify
what
traffic
of
remote
hosts
and/or
local
ports
is
allowed
to
wake
the
JVC,
so
in
above:
– Traffic
from
any
remote
host
to
local
port
8080
will
wake
the
JVC
– Traffic
from
remote
host
10.1.0.8
to
local
port
4444
will
wake
the
JVC
© Copyright Waratek 2013
35. jSleep API in 60 Seconds!
• 2
packages,
8
classes:
– cloudvm.jsleep.event.*
•
•
•
•
BytesReceivedEvent
CpuEvent
NetworkIdleEvent
TimeoutEvent
– cloudvm.jsleep.profile.*
•
•
•
•
JSleepProfile
JSleepProfile.MemoryType
JSleepProfileState
NetworkProfile
© Copyright Waratek 2013
36. jSleep API in 60 Seconds!
• Main
class:
cloudvm.jsleep.profile.JSleepProfile
– Every
JVC
can
have
one
of
these
– Defines
the
two
CORE
methods:
• checkSuspend( VirtualContainerMXBean )
• checkResume( VirtualContainerMXBean )
– Is
an
abstract
class
so
anyone
can
write
their
own
custom
sleep/resume
profile
– Loaded
and
configured
using
JMX
(local
or
remote)
© Copyright Waratek 2013
37. Custom jSleep API Example!
• Here’s
a
custom
jSleep
profile…
public
boolean
checkSuspend(VirtualContainerMXBean
paramVirtualContainerMXBean)
{
Date
currentDate
=
new
Date();
if(currentDate.aver(sleepDate)
&&
currentDate.before(resumeDate))
{
return
true;
}
else
{
return
false;
}
}
public
boolean
checkResume(VirtualContainerMXBean
paramVirtualContainerMXBean)
{
Date
currentDate
=
new
Date();
if(currentDate.before(sleepDate)
||
currentDate.aver(resumeDate))
{
return
true;
}
else
{
return
false;
}
}
© Copyright Waratek 2013
38. What happens when…!
• …a
JVC
sleeps?:
– The
threads
of
the
JVC
are
suspended
on
the
OS
kernel
(but
NOT
destroyed)
– The
JVC’s
dead
objects
are
asynchronously
reclaimed
by
the
GC
– If
the
JSleepProfile
was
configured
with
MemoryType.VARIABLE,
the
JVCs
unused
memory
quota
will
be
reclaimed
for
use
by
other
JVCs
– All
open
file
descriptors
will
be
kept
open
– …all
this
in
<1
second
© Copyright Waratek 2013
39. What happens when…!
• …a
JVC
wakes
up?:
– The
threads
of
the
JVC
are
resumed
on
the
OS
kernel
– If
the
JSleepProfile
was
configured
with
MemoryType.VARIABLE,
the
JVM
will
reinstate
the
prior
memory
quota
of
the
JVC
– Any
file
descriptors
with
received
but
unread
data
will
now
be
“readable”
by
the
applicaCon
– …all
this
in
<1
second
© Copyright Waratek 2013
40. Why you should care…!
• Dev/Test,
PaaS,
and
middleware
environments
are
characterized
by
LOTS
of
“applicaCon
idleness”
• Idle
apps
today
are
treated
NO
DIFFERENTLY
than
acCve,
busy
apps
• Years
ago
H/W
wake-‐on-‐LAN
was
an
effecCve
technology
for
powering
down
idle
servers…
but
it
no
longer
works
in
mulCtenant
cloud
infrastructure
• jSleep
is
“wake-‐on-‐LAN
for
JVCs”
and
can
increase
the
number
of
Java
apps-‐per-‐server
by
up
to
10x
– …this
means
a
reducCon
in
server
sprawl
and
OpEx
up
to
90%
© Copyright Waratek 2013
41. Summary!
• Recap
and
links…
© Copyright Waratek 2013
42. Biggest change to Java in yrs!
• A
hypervisor
inside
the
JVM
transforms
Java
with:
– Ultra-‐liteweight
JVCs
for
near
zero-‐cost
tenants
– JVC
“jails”
for
Java
security
and
isolaCon
– Mixed-‐version
JVCs
for
version
consolidaCon
and
legacy
compliance
– jSleep
(“Wake-‐On-‐LAN
for
JVCs”)
for
near-‐zero
footprint
of
idle
tenants
© Copyright Waratek 2013
43. Links!
• General
informaCon
about
the
Waratek
JVM
– hdp://www.waratek.com
• jSleep,
JVI
(Java
VirtualizaCon
Interface),
and
JMX
APIs
– hdp://javadoc.waratek.com
• ElasCcat
source
code
– hdps://github.com/waratek/elasCcat-‐driver
© Copyright Waratek 2013