Más contenido relacionado La actualidad más candente (20) Similar a Bytecode Verification, the Hero That Java Needs [JavaOne 2016 CON1500] (20) Bytecode Verification, the Hero That Java Needs [JavaOne 2016 CON1500]3. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Bytecode Verification
The Hero That Java Needs
David Buck
Principal Member of Technical Staff
Java SE Sustaining Engineering
September, 2016
4. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Safe Harbor Statement
The following is intended to outline our general product direction. It is intended for
information purposes only, and may not be incorporated into any contract. It is not a
commitment to deliver any material, code, or functionality, and should not be relied upon
in making purchasing decisions. The development, release, and timing of any features or
functionality described for Oracle’s products remains at the sole discretion of Oracle.
4
5. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
About Me
David Buck
• Java SE Sustaining Engineering
• Mostly JRockit fixes
• OpenJDK 8 Updates
Project Maintainer
• Hobbies: Programming
5
6. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Program Agenda
Introduction
Dangers
Demo
Implementation
Importance
Usage
Conclusions
1
2
3
4
5
6
6
7
7. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Introduction
7
8. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
What Is It?
• Analysis of bytecode
• Syntax check
• Symantec check
• Ensures stability / security of runtime
8
9. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
When does it happen?
• Analysis done during class loading
• Sometimes delayed until right before method execution
• But only done at most once per loaded method
9
10. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Traditional Interpreted Language
Source Code Interpreter
10
11. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Traditional Interpreted Language
Source Code Interpreter
11
12. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Traditional Compiled Language
Source Code ExecutableCompile
12
13. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Traditional Compiled Language
Source Code ExecutableCompile
13
14. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Java
Source Code BytecodeCompile
14
JVM
15. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Java
Source Code BytecodeCompile
15
JVM
16. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Java
Source Code BytecodeCompile
16
JVM
17. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Java
Source Code BytecodeCompile
17
JVM
18. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
What Does It Do?
18
19. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
What Does It Do?
• Protects runtime from bad people
"Why the verifier is so important…. write once and crack anywhere“
-Keith McGuigan
19
20. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
What Does It Do?
• Protects runtime from bad people
"Why the verifier is so important…. write once and crack anywhere“
-Keith McGuigan
• Protects runtime from you
20
21. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Why learn about it?
• The best technologies are invisible…
• Victim of its own success
21
22. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Dangers
22
23. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Class Metadata
23
24. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Class Metadata
• Has a direct superclass
24
25. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Class Metadata
• Has a direct superclass
• Superclass is not marked final
25
26. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Class Metadata
• Has a direct superclass
• Superclass is not marked final
• No final methods are overridden
26
27. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Class Metadata
• Has a direct superclass
• Superclass is not marked final
• No final methods are overridden
27
28. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Operand Stack Overflow
stack=2, locals=1, args_size=1
0: iconst_0
1: iconst_1
2: iconst_2
28
29. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Operand Stack Overflow
stack=2, locals=1, args_size=1
0: iconst_0
1: iconst_1
2: iconst_2
29
30. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Operand Stack Overflow
stack=2, locals=1, args_size=1
0: iconst_0
1: iconst_1
2: iconst_2
0
LIMIT
30
31. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Operand Stack Overflow
stack=2, locals=1, args_size=1
0: iconst_0
1: iconst_1
2: iconst_2
0
LIMIT
1
31
32. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Operand Stack Overflow
stack=2, locals=1, args_size=1
0: iconst_0
1: iconst_1
2: iconst_2
0
LIMIT
1
2
32
33. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Operand Stack Overflow
stack=2, locals=1, args_size=1
0: iconst_0
1: iconst_1
2: iconst_2
0
LIMIT
1
2
33
34. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Operand Stack Underflow
stack=3, locals=1, args_size=1
0: iadd
1: iadd
2: iadd
0
LIMIT
1
2
START
34
35. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Operand Stack Underflow
stack=3, locals=1, args_size=1
0: iadd
1: iadd
2: iadd
0
LIMIT
3
START
35
36. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Operand Stack Underflow
stack=3, locals=1, args_size=1
0: iadd
1: iadd
2: iadd
3
LIMIT
START
36
37. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Operand Stack Underflow
stack=3, locals=1, args_size=1
0: iadd
1: iadd
2: iadd
?
LIMIT
START
37
38. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Operand Stack Underflow
stack=3, locals=1, args_size=1
0: iadd
1: iadd
2: iadd
?
LIMIT
START
38
39. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Type Checking
• Each operation is checked
– Correct types on the stack
– Correct types in local variable “slots”
• Specification uses Prolog to define requirements
39
40. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Prolog?!
• Predicate logic of type system are described by Prolog well
• Java is probably the first of this kind of use by a mainstream programming
language
40
41. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Prolog?!
Facts:
cat(tom).
41
42. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Prolog?!
Facts:
parent_child(sally, bob).
42
43. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Prolog?!
Rules:
Head :- Body.
43
44. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Prolog?!
Rules:
sibling(X, Y) :- parent_child(Z, X),
parent_child(Z, Y).
44
45. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
The Specification
45
46. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Java Bytecode
Expressive Power
Java Language
46
48. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
ClassA
public class ClassA {
public int doSomething(int i1, int i2, int i3)
{
return i1+i2+i3;
}
}
48
49. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
ClassB
public class ClassB {
public Integer doSomethingElse(int i1, int i2, int i3)
{
return new Integer(i1+i2+i3);
}
}
49
50. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
ClassC
public class ClassC extends ClassA {}
50
51. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Demo
public class Demo {
public static void main(String[] args) {
ClassA obj = new ClassC();
System.out.println(obj.doSomething(1,2,3));
}
}
51
52. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Object
ClassA ClassB
ClassC
Demo
52
53. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
It works…
$ java Demo
6
$
53
54. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Lets do something bad…
public class ClassC extends ClassB {}
54
55. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Lets do something bad…
public class ClassC extends ClassB {}
55
56. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Object
ClassA ClassB
ClassC
Demo
56
57. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Object
ClassA ClassB
ClassC
Demo
57
58. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Object
ClassA ClassB
ClassC
Demo
58
59. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Demo
public class Demo {
public static void main(String[] args) {
ClassA obj = new ClassC();
System.out.println(obj.doSomething(1,2,3));
}
}
59
60. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
$ java Demo
Error: A JNI error has occurred, please check your installation and try again
Exception in thread "main" java.lang.VerifyError: Bad type on operand stack
Exception Details:
Location:
Demo.main([Ljava/lang/String;)V @15: invokevirtual
Reason:
Type 'ClassC' (current frame, stack[1]) is not assignable to 'ClassA'
Current Frame:
bci: @15
flags: { }
locals: { '[Ljava/lang/String;', 'ClassC' }
stack: { 'java/io/PrintStream', 'ClassC', integer, integer, integer }
Bytecode:
0x0000000: bb00 0259 b700 034c b200 042b 0405 06b6
0x0000010: 0005 b600 06b1
60
61. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
As expected, the verifier protects us from ourselves.
61
62. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
As expected, the verifier protects us from ourselves.
What if we disable it…
62
63. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
We reap what we sow…
$ java -Xverify:none Demo
#
# A fatal error has been detected by the Java Runtime Environment:
#
# SIGSEGV (0xb) at pc=0x00007fa93be7991c, pid=22925, tid=140364857087744
#
# JRE version: OpenJDK Runtime Environment (8.0_91-b14) (build 1.8.0_91-b14)
# Java VM: OpenJDK 64-Bit Server VM (25.91-b14 mixed mode linux-amd64 compressed
oops)
# Problematic frame:
# V [libjvm.so+0x46391c]
63
64. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Demo Takeaways
• No obvious evidence that bad bytecode was root cause of crash
• A class is only valid in the context of previously loaded classes
• No malicious intent / 3rd party tools used
64
65. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Optional 2nd Demo
$ java -Xverify:none Crack
=============== DEBUG MESSAGE: illegal bytecode sequence -
method not verified ================
Exception in thread "Thread-0"
java.lang.NullPointerException
at Pointer.deref(Pointer.jasm)
at Crack.breakLock(Crack.java:13)
at Crack$1.run(Crack.java:29)
Thread Thread[main,5,main] leaving monitor
$
65
66. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Implementation
66
67. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Type Inference Verifier
– AKA the Old Verifier
Type Checking Verifier
– AKA Split Verifier
– AKA The New Hotness
67
A Tale of Two Verifiers…
68. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Type Inference Verifier
– AKA the Old Verifier
Type Checking Verifier
– AKA Split Verifier
– AKA The New Hotness
68
A Tale of Two Verifiers…
69. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Type Inference Verifier
• Class files <= 49 (JDK 5)
• Requires CFG construction
• Worst case scenario can require
many passes
Diagram by JMP EAX - Own work, CC BY-SA 4.0, https://commons.wikimedia.org/w/index.php?curid=34222288
69
70. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
When We Do Syntactic / Semantic checking
Source Code BytecodeCompile
70
JVM
71. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Type Checking Verifier (AKA Split Verifier)
• Class files >= 50 (JDK 6)
• Depends on StackMapTable Attribute
• Transfers much of the responsibility to javac
Source Code BytecodeCompile JVM
StackMap
Tables
71
72. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
StackMapTable
• Identifies the type of each stack position / local variable
• One needed for every instruction that is the target of a jump
– Methods without branches will not have them
• Are stored as deltas to save space
• Allow single pass verification
72
73. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Importance
73
74. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
3rd Party Tools
• Non-Java languages
• Bytecode obfuscators
• Bytecode optimizers
• 3rd party Java compilers
• Bytecode assemblers
– Oolong
– Jasmin
– JASM
74
75. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Runtime Mischief
• Runtime Code Generation
• Runtime Code Modification
• Usual suspects:
– BCEL
– ASM
– AOP
– Instrumentation tools / agents
75
76. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Compatibility Issues
• A serious limitation for bytecode manipulation
• Tools like instrumentation agents may not know the rules of more recent
classfile versions
76
77. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
r/programminghorror
try {
new OraclePKIProvider();
} catch (Throwable t) { ; }
77
78. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
r/programminghorror
• Verification enabled
– VerifyError silently eaten by catch clause
– Application runs fine
• Verification disabled
– Broken bytecode loaded, environment breaks
78
80. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
options
• -Xverify:
– none
• disables all verification. Only use for debugging!
– remote
• default. Verifies all classes not loaded by boot class path.
– all
• Verifies everything.
• -noverify
• Same as –Xverify:none
80
81. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Xverify:remote
• Has nothing to do with remote / local
• Horribly named
• Our own documentation was wrong for well over a decade
81
82. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Cost of Verification
• Classloading could be CPU-bound in the 90s
• Skipping verification could speed up class loading, giving a faster startup
82
83. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Cost of Verification
83
84. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Cost of Verification
• On modern hardware, class loading is no longer CPU-bound, it is IO-bound
– Even on SSD hardware
• Verification is more or less free
84
85. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Development Usage
• Verification is just as important in Development as in Production (if not
more!)
• Some products explicitly disable verification by default in “Developer”
configurations!
• Previously unseen verify errors thrown when code is moved into
production
85
86. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Verification Support by Class File Version
• <= class file version 49 (JDK 5)
– Only type inference supported
• class file version 50 (JDK 6)
– Type checking w/ fallback to type inference
• >= class file version 51 (JDK 7)
– only type checking supported
– (JDK 7 only) force use type inference w/ -XX:-UseSplitVerifier
86
87. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Conclusions
87
88. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Summary
• Always use verification
– Even in development
– Even with trusted code
– Even when startup time is important
• Verification depends on already loaded classes
• Split Verifier is here to stay
88
89. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
References
[ Cracking the Hotspot JVM ]
https://blogs.oracle.com/kamg/entry/cracking_the_hotspot_jvm
[ 4.10. Verification of class Files ]
https://docs.oracle.com/javase/specs/jvms/se8/html/jvms-4.html#jvms-4.10
89
90. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Safe Harbor Statement
The preceding is intended to outline our general product direction. It is intended for
information purposes only, and may not be incorporated into any contract. It is not a
commitment to deliver any material, code, or functionality, and should not be relied upon
in making purchasing decisions. The development, release, and timing of any features or
functionality described for Oracle’s products remains at the sole discretion of Oracle.
90