Axa Assurance Maroc - Insurer Innovation Award 2024
I Know Kung Fu - Juggling Java Bytecode
1. I Know Kung Fu – Writing Java
Bytecode
Alexander Shopov <ash@kambanaria.org>
2. [ash@edge ~]$ whoami
By day: Software Engineer at Cisco
By night: OSS contributor
Coordinator of Bulgarian Gnome TP
Contacts:
E-mail: ash@kambanaria.org
Jabber: al_shopov@jabber.minus273.org
LinkedIn: http://www.linkedin.com/in/alshopov
Web: Just search “al_shopov”
5. Plan
● What is bytecode and how to juggle it
● Why do it, why not do it
● How to do it
– Class file structure
– Manipulation with ASM
● How to consume your own dog food
– Classloaders
– Proxies
– Agents
6. What it is
● Bytecodes are the assembler of JVM. By writing new
bytecode sequences or changing existing ones we can
program the JVM directly – outside of the domain of the
Java Programming Language;
● Lower level than writing Java - JVM assembler;
● Much easier than writing a whole (optimizing) compiler;
● Harder than using a compiler;
● Similar to the Jedi Dark Side – pathway to many abilities
some consider to be unnatural.
7. Why do it?
● Makes impossible things possible &
● Makes hard things attainable &
● Makes long tasks shorter &
● Makes slow things faster &
● Makes repetition go away &
● Makes your head blow away!!!
8. Common use cases
● Program analysis – find bugs, unused code
paths, reverse engineering, metrics, etc.
● Program generation – compilers, stub/skeleton
compilers, fast layers without reflection.
● Transformation – optimize,
deoptimize/obfuscate, aspect weaving,
performance monitoring.
9. Why not do it?
● Source stops being close to executable &
● Executable stops being close to any source &
● There can be better or easier ways of doing things &
● Fewer people knowing what happens means no one
can help you &
● Problems will be your fault! Unit test ∀ ! &
● You take the red pill and find out the rabbit hole goes
deep, deep, deep… EOT
10. Common misuse cases
● I write it, you maintain it.
● Do it because it is cool or others are doing it.
● Over engineering / over generalization.
● Solution in a search of a problem.
● Preferring NIH to a popular solution.
11. Bytecode crash course in 1-2-3
F4
Thread D
Thread C
Thread B
F3 F3
Thread A
F2 F2 F2
F1 F1 F1 F1
F0 F0 F0 F0
12. Bytecode crash course in 1-2-3
JVM (heap)
0 1 2 3 4 5 6 … Class
PC
Local variables Method code
F0
Class
Pool of
constants
Stack
13. Bytecode crash course in 1-2-3
● Bytecodes retrieve things from class bytestream,
constant pool, local variables array
● Perform computations only on stack
● Store temporary results in local variables array
● Enter frame by method call
● Return from frame with a possible result (top
thing on stack), throw top of stack
● JVM can also throw and unroll frames
14. If you need more information
● Check my other presentation:
Lifting the Veil – Reading Java Bytecode
15. How to do it
● Many ways with different pluses and minuses
● Easiest way to write byte code – use the Java
compiler.
– javac – you all know that
– java compiler api – since 1.6
● But more of that – wait for the end
● Until then…
17. Soot
jDec CGlib
JCF Editor cojen
Jiapi One of the older frameworks,
Retroweaver
JavaAssist BCEL
Tea Trove
jReloader
jclasslib
Roo
Serp
AspectJ
gnu.bytecode
18. Bytecode manipulation
● Not constrained by Java
● Not constrained by compiler
● Not constrained by source availability
● Still constrained by JVM
● Generate or change bytecode to the limits of
JVM
19. Classfile Structure, 0xcafebabe
Modifiers, name, super class, interfaces
Constant pool: numeric, string and type constants
Source file name (optional)
Enclosing class reference
Annotation*
Attribute*
Inner class* Name
Field* Modifiers, name, type
Annotation*
Attribute*
Method* Modifiers, name, return and parameter types
Annotation*
Attribute*
Compiled code
20. What is ASM
● Extremely well designed bytecode
manipulation library
● Modular, small and fast – pick all
● Events based – visitor (+SAX), additionally
has tree API (DOM like)
● Provides a small bytecode generation variant
● Minimal – just transformation, can stack
transformations, no classloading
21. Timeline
● 1.5 – 30 Aug 2004 – up to JDK 1.6 incl.
● 2.0 – 17 May 2005
● 3.0 – 01 Nov 2006
● 4.0 – 29 Oct 2011 – framework for
compatibility
22. Who uses ASM?
● Languages and AOP tools: AspectJ,
BeanShell, CGLIB, Clojure, Groovy, Jruby,
Jython, Coroutines
● Tools and frameworks: Fractal, Terracotta,
Javeleon, JRebel
● Persistence: OpenEJB, Oracle BerkleyDB,
EclipseLink
● Monitoring: WebLogic, JiP
● Testing and code analysis: Cobertura, Eclipse
23. Most JDKs Use It!
● Oracle
– Sun – HotSpot – JDK, OpenJDK
– BEA/Appeal VM – JRockit
● IBM – J9
● Azul – Zing
– $JAVA_HOME/jre/lib/rt.jar
– com.sun.xml.internal.ws.org.objectweb.asm
– com.sun.xml.internal.ws.model.WrapperBeanGenerator ⇒
RuntimeModeler ⇒ Web Services implementation
26. I know what I am doing
Class Class Class Class Class Class Class Class
Reader Visitor Visitor Visitor Visitor Visitor Visitor Writer
27. I believe I can fly, I believe I can
touch the sky…
Class Class Class Class Class Class
Reader Visitor Visitor Visitor Visitor Writer
Class Class Class Class Class
Visitor Visitor Visitor Visitor Writer
Class Class Class Class Class Class
Reader Visitor Visitor Visitor Visitor Writer
Class Class Class Class
Visitor Visitor Visitor Writer
Class Class Class Class Class
Reader Visitor Visitor Visitor Visitor
28. Basic Pattern Of Transformation
● Read bytestream of a class
● Generate events from that
● Make changes to the events
● Receive the events and serialize to
bytestream
● Lather, rinse, repeat
29. Basic diagram of class visiting
User ClassVisitor
visit
visitSource
● Order is important
visitOuterClass
visitAnnotation
● Bold return other
*
visitAttribute visitors
visitInnerClass
visitFiled
* ● * marks repeat
vistiMethod
visitEnd
30. Basic diagram of annotation visiting
User AnnotationVisitor
visit *
visitEnum
● Order is important
visitAnnotation
visitArray
● Bold return other
visitEnd visitors
● * marks repeat
31. Basic diagram of field visiting
User FieldVisitor
visitAnnotation *
visitAttribute
● Order is important
visitEnd
● Bold return other
visitors
● * marks repeat
32. Basic diagram of method visiting
User MethodVisitor
visitAnnotationDefault
visitAnnotation *
● Order is important
visitParameterAnnotation
visitAttribute
● Bold return other
visitCode visitors
visitFrame
visitXInsn
* ● * marks repeat
visitLabel
visitTryCatchBlock
visitLocalVariable
visitLineNumber
visitMaxs
visitEnd
33. Our humble beginning
package org.kambanaria.writebytecode.asm;
public class Zombunny { //Зомбайо
public Integer getVersion() {
return Integer.valueOf(1);
}
}
34. Our building blocks
public class StupidClassLoader extends ClassLoader {
private Map<String, byte[]> bytes = new HashMap<String, byte[]>();
public StupidClassLoader() {
super(StupidClassLoader.class.getClassLoader());
}
public void provide(String className, byte[] classBytes) {
bytes.put(className, classBytes);
}
@Override
public Class<?> loadClass(String name) throws ClassNotFoundException {
byte[] classBytes = bytes.get(name);
Class<?> loaded;
if (classBytes != null) {
loaded = defineClass(name, classBytes, 0, classBytes.length);
} else {
ClassLoader parent = getParent();
if (null == parent) {
parent = ClassLoader.getSystemClassLoader();
}
loaded = parent.loadClass(name);
}
return loaded;
}
}
35. Our building blocks
public final class Utilities {
private Utilities() { }
public static final String CLASS_NAME = "org.kambanaria.writebytecode.asm.Zombunny";
public static final String METHOD_NAME = "getVersion";
public static String toClassPathResourceName(String clazz) {
return clazz.replace('.', '/') + ".class";
}
public static byte[] retrieveBytesFromClassPath(String className) throws IOException {
String path = toClassPathResourceName(className);
InputStream is = Utilities.class.getClassLoader().getResourceAsStream(path);
InputStream classBytes = new BufferedInputStream(is);
return is2bytes(classBytes);
}
public static byte[] patch(byte[] bytes, DemoClassAdapter adapter) {
ClassReader cr = new ClassReader(bytes);
cr.accept(adapter, ClassReader.SKIP_FRAMES);
return adapter.getCw().toByteArray();
}
public static byte[] is2bytes(InputStream is) throws IOException {
int max = 1024 * 1024;
byte[] bytes = new byte[max]; // 1MB
int read = is.read(bytes); // Don't do that
byte[] result = new byte[read];
System.arraycopy(bytes, 0, result, 0, result.length);
return result;
}
public static Object call0ArgsMethodOn(Object o, String methodName) //
throws ReflectiveOperationException {
Class<?> c = o.getClass();
Method m = c.getDeclaredMethod(methodName, (Class<?>[]) null);
m.setAccessible(true);
return m.invoke(o, (Object[]) null);
}
}
36. Our building blocks
public class DemoClassAdapter extends ClassVisitor {
public DemoClassAdapter(ClassVisitor cv) {
super(Opcodes.ASM4, cv);
}
public ClassWriter getCw() {
return (ClassWriter) cv;
}
}
37. Rename class
public class Rename extends DemoClassAdapter {
private String suffix;
public Rename(ClassVisitor cv, String suffix) {
super(cv);
this.suffix = suffix;
}
@Override
public void visit(int version, int access, String name, //
String signature, String superName, String[] interfaces) {
cv.visit(version, access, name + suffix, signature, superName, interfaces);
}
}
38. Result is as if
package org.kambanaria.writebytecode.asm;
public class ZombunnySUFFIX { //Зомбайо
public Integer getVersion() {
return Integer.valueOf(1);
}
}
39. Add field
public class AddField extends DemoClassAdapter {
public AddField(ClassVisitor cv) {
super(cv);
}
@Override
public void visitEnd() {
cv.visitField(ACC_PRIVATE, "_version", //
Type.getDescriptor(Integer.class), null, null);
cv.visitEnd();
}
}
40. Add field
public class Zombunny { //Зомбайо
private Integer _version;
public Integer getVersion() {
return Integer.valueOf(1);
}
}
41. Change method
public class ChangeMethod extends DemoClassAdapter {
public ChangeMethod(ClassVisitor cv) {
super(cv);
}
@Override
public MethodVisitor visitMethod(int access, String name, String desc, //
String signature, String[] exceptions) {
MethodVisitor mv = super.visitMethod(access, name, desc, signature, exceptions);
if (Utilities.METHOD_NAME.equals(name) && "()Ljava/lang/Integer;".equals(desc)) {
return new DemoMethodVisitor(Opcodes.ASM4, mv);
} else {
return mv;
}
}
class DemoMethodVisitor extends MethodVisitor {
public DemoMethodVisitor(int version, MethodVisitor mv) {
super(version, mv);
mv.visitCode();
mv.visitVarInsn(ALOAD, 0);
mv.visitFieldInsn(GETFIELD, "org/kambanaria/writebytecode/asm/Zombunny", //
"_version", "Ljava/lang/Integer;");
mv.visitInsn(ARETURN);
mv.visitEnd();
}
}
}
42. And change method
public class Zombunny { //Зомбайо
private Integer _version;
public Integer getVersion() {
return _version;
}
}
48. And add interface
public class Zombunny implements Comparable { //Зомбайо
private Integer _version;
public Zombunny() {
_version = new Integer(2);
}
public Zombunny(Integer version) {
_version = version;
}
public Integer getVersion() {
return _version;
}
public int compareTo(Object o) {
if (o instanceof Zombunny) {
return compareTo((Zombunny) o);
} else {
throw new ClassCastException();
}
}
public int compareTo(Zombunny o) {
return _version.compareTo(o._version);
}
}
49. The class we get method from
package org.kambanaria.writebytecode.asm;
public class Version {
private Integer _version;
public Version(Integer version) {
_version = version;
}
public Integer getVersion() {
return _version;
}
@Override
public String toString() {
return "Version: " + _version;
}
}
50. Chimerization
public class Chimerize extends DemoClassAdapter {
protected ClassNode twig;
protected MethodNode nm;
private static final String TWIG_NAME = "org.kambanaria.writebytecode.asm.Version";
public Chimerize(ClassVisitor cv) throws IOException {
super(cv);
ClassReader rdr = new ClassReader(TWIG_NAME);
twig = new ClassNode();
rdr.accept(twig, 0);
for (Object o : twig.methods) {
MethodNode method = (MethodNode) o;
if (method.name.equals("toString")) {
// Guess what is missing?
nm = method;
}
}
}
@Override
public void visitEnd() {
MethodVisitor chimeric =cv.visitMethod(nm.access, nm.name, nm.desc,
nm.signature, null);
nm.instructions.resetLabels();
Remapper remapper = new Remapper() {
@Override
public String map(String name) {
return name.replace("Version", "Zombunny");
}
};
nm.accept(new RemappingMethodAdapter(nm.access, nm.desc, chimeric, remapper));
cv.visitEnd();
}
}
51. Finally
public class Zombunny implements Comparable { //Зомбайо
private Integer _version;
public Zombunny() {
_version = new Integer(2);
}
public Zombunny(Integer version) {
_version = version;
}
public Integer getVersion() {
return _version;
}
public int compareTo(Object o) {
if (o instanceof Zombunny) {
return compareTo((Zombunny) o);
} else {
throw new ClassCastException();
}
}
public int compareTo(Zombunny o) {
return _version.compareTo(o._version);
}
@Override
public String toString(){
return "Version: " + _version;
}
}
52. Peek at test construction chain
public class ChimerizeTest {
Comparable sut;
@Before
public void setUp() throws IOException, ReflectiveOperationException {
ClassWriter cw = new ClassWriter(ClassWriter.COMPUTE_FRAMES + //
ClassWriter.COMPUTE_MAXS);
ClassVisitor cv = new AddField(new ManipulateConstructors( //
new AddMethod(new AddInterface(new Chimerize(cw)))));
ClassReader rdr = new ClassReader(Utilities.CLASS_NAME);
rdr.accept(cv, 0);
byte[] newClassBytes = cw.toByteArray();
StupidClassLoader ldr = new StupidClassLoader();
ldr.provide(Utilities.CLASS_NAME, newClassBytes);
Class<?> newClass = ldr.loadClass(Utilities.CLASS_NAME);
Constructor<?> constructor = newClass.getDeclaredConstructor(Integer.class);
sut = (Comparable) constructor.newInstance(new Integer(42));
}
}
53. Utilities
● Type
● Generate the visitors with: Asmfier
– java -classpath asm.jar:asm-util.jar:OUR_CP
org.objectweb.asm.util.ASMifier CLASS
● TraceClassVisitor
● CheckClassAdapter
54. Additional APIs
● org.objectweb.asm.commons – commonly
needed adaptors
● org.objectweb.asm.xml – bridge to SAX 2.0,
manipulate via XSLT, XQuery
● org.objectweb.asm.tree – deserialize to tree
55. Classloaders
● Dynamically load software components for
Java platform
● Lazy – loaded on demand, as late as possible
● Type-safe linkage – must not violate type
safety, no runtime checks
● User-defined extensibility – normal, user
controlled objects
● Multiple communicating namespaces – types
determined by class name and classloader
59. Class Loader API
public abstract class ClassLoader {
protected ClassLoader(ClassLoader parent);
protected ClassLoader();
protected Class<?> loadClass(String name, boolean resolve);
protected Class<?> findClass(String name);
protected final Class<?> defineClass(String name, byte[] b,
int off, int len);
protected final void resolveClass(Class<?> c);
public URL getResource(String name);
public Enumeration<URL> getResources(String name);
public final MyClassLoader getParent();
public void setDefaultAssertionStatus(boolean);
public void setPackageAssertionStatus(String packageName,boolean enabled);
public void setClassAssertionStatus(String className, boolean enabled);
public void clearAssertionStatus();
}
60. When are class loaded?
● Statically:
– Instance creation: new Integer(42);
– Reference to static field or method: System.out;
● Dynamically:
– Class.forName("java.lang.HashMap");
– Class.forName("java.lang.HashMap",
boolean initialize,
ClassLoader loader);
64. Type Compatibility
● A classloader can see and use (with exact
type) instances of classes loaded by the
ancestral chain and the classloader itself
● Instances of classes loaded by sibling or
descendant classloaders are invisible, they
are just java.lang.Object
Object a;
"SomeClass".equals(a.getClass().getName());
a instanceof Object;
(SomeClass)a -> ClassCastException
● Use reflection
65. Proxies
● Dynamic proxy acts as a pass through/router
to the real object
– runtime implementations of interfaces
– public, final and not abstract
– extend java.lang.reflect.Proxy
● Proxy’s behaviour is determined by an
implementation of
java.lang.reflect.InvocationHandler
66. Square peg in a non square hole
Some object
Some interface
67. Fit it with an InvocationHandler
Proxy
Object
public interface InvocationHandler {
public Object invoke(Object proxy, Method method, Object[] args)
throws Throwable;
}
68. public class MyInvocationHandler implements InvocationHandler {
private Object delegate;
public MyInvocationHandler(Object... params) {
delegate = makeDelegate();
}
@Override
public Object invoke(Object proxy, Method method, Object[] args) throws Throwable {
String methodName = method.getName();
Class<?>[] types = method.getParameterTypes();
Method m = reachMethod(methodName, types);
Object result;
try {
result = m.invoke(delegate, args);
} catch (InvocationTargetException e) {
throw e.getCause();
}
return result;
}
private static Method reachMethod(String name, Class<?>... parTypes) {
Method m = null;
/* Logic to determine method */
return m;
}
private static Object makeDelegate(Object... params) {
/* Produce real object instead */
return new Object();
}
}
69. Java agents
● Package java.lang.instrument - allow Java
programming language agents to instrument
programs running on the JVM.
● java ... -javaagent:jarpath[=options]
● Manifest attributes
● byte[] -> byte[]
80. Finds the culprit
SimpleDateFormat frmt =
new SimpleDateFormat("E MM/dd/yyyy");
SimpleDateFormat
publicSimpleDateFormat(String pattern)
Constructs a SimpleDateFormat using the
given pattern and the default date format
symbols for the default locale.
Note: This constructor may not support all
Locales. For full coverage, use the factory
methods in the DateFormat class.
Parameters:
pattern - the pattern describing the date and
time format
Throws:
NullPointerException - if the given pattern
is null
IllegalArgumentException - if the given pattern
is invalid
81. Could this have been avoided?
Hindi
(300×106, 4th)
Shukravār
LANG=de_DE.UTF-8 java -jar SimpleDateFormat.jar Freitag 11/16/2012
LANG=en_US.UTF-8 java -jar SimpleDateFormat.jar Friday 11/16/2012
LANG=hi_IN.UTF-8 java -jar SimpleDateFormat.jar शुकवार ११/१६/२०१२
LANG=bn_IN.UTF-8 java -jar SimpleDateFormat.jar Friday 11/16/2012
LANG=bg_BG.UTF-8 java -jar SimpleDateFormat.jar Петък 11/16/2012
Bengali
(200×106, 7th)
শুক্রার
্রব
Shukrobar
93. import java.util.Random;
public class I {
public boolean singOutOfTune() {
return new Random().nextBoolean();
}
}
public class You {
public static String DID = "StandUp&WalkOutOnMe";
public static String DIDNOT = "LendMeAnEar";
public String wouldDo(boolean iF) {
return iF ? "StandUp&WalkOutOnMe" : "LendMeAnEar";
}
}
94. import java.util.Random;
public class I {
public boolean singOutOfTune() {
return new Random().nextBoolean();
}
}
public class You {
public static String DID = "StandUp&WalkOutOnMe";
public static String DIDNOT = "LendMeAnEar";
public String wouldDo(boolean iF) {
return iF ? "StandUp&WalkOutOnMe" : "LendMeAnEar";
}
}
public class ExistingClassesTest {
@Test
public void test1000times() {
int times = 1000;
do {
boolean didI;
I i = new I();
You you = new You();
String what = you.wouldDo(didI = i.singOutOfTune());
assertEquals(what, didI ? You.DID : You.DIDNOT);
} while (--times > 0);
}
}
95. import java.util.Random;
public class I {
public boolean singOutOfTune() {
return new Random().nextBoolean();
}
}
public class You {
public static String DID = "StandUp&WalkOutOnMe";
public static String DIDNOT = "LendMeAnEar";
public String wouldDo(boolean iF) {
return iF ? "StandUp&WalkOutOnMe" : "LendMeAnEar";
}
}
public class ExistingClassesTest {
@Test
public void test1000times() {
int times = 1000;
do {
boolean didI;
I i = new I();
You you = new You();
String what = you.wouldDo(didI = i.singOutOfTune());
assertEquals(what, didI ? You.DID : You.DIDNOT);
} while (--times > 0);
// What would you do if I sang out of tune ? The Beatles
}
}
96. public class SourceStrings {
private SourceStrings() }
public final static String I = " "
+ "import java.util.Random; "
+ "public class I { "
+ " public boolean singOutOfTune() { "
+ " return new Random().nextBoolean(); "
+ " } "
+ "} ";
public final static String YOU = " "
+ "public class You { "
+ " public static String DID = "StandUp&WalkOutOnMe"; "
+ " public static String DIDNOT = "LendMeAnEar"; "
+ " public String wouldDo(boolean iF) { "
+ " return iF ? "StandUp&WalkOutOnMe" : "LendMeAnEar";"
+ " } "
+ "} ";
}
class StringSourceCodeObject extends SimpleJavaFileObject {
final String _source;
public StringSourceCodeObject(String fqName, String source) {
super(URI.create("string:///" + fqName.replaceAll(".", "/") //
+ Kind.SOURCE.extension), Kind.SOURCE);
_source = source;
}
@Override
public CharSequence getCharContent(boolean ignoreEncodingErrors) {
return _source;
}
97. public class CompilerAPI {
public static void main(String args[]) throws Exception {
/* Creating dynamic java source code file object */
JavaFileObject iObject = new StringSourceCodeObject("I", SourceStrings.I);
JavaFileObject youObject = new StringSourceCodeObject("You", SourceStrings.YOU);
JavaFileObject jfObjects[] = new JavaFileObject[]{iObject, youObject};
/* Units to compile */
Iterable<JavaFileObject> units = Arrays.asList(jfObjects);
/* Instantiating the java compiler */
JavaCompiler compiler = ToolProvider.getSystemJavaCompiler();
/* Get compiler file manager to show what to read. */
// (DEFAULT LISTENER, Locale.getDefault(), Charset.defaultCharset() )
JavaFileManager manager = compiler.getStandardFileManager(null, null, null);
/* Compilation options - here: place in target directory */
String[] compileOptions = new String[]{"-d", "target/classes"};
Iterable<String> options = Arrays.asList(compileOptions);
/* Diagnostic placeholder */
DiagnosticCollector<JavaFileObject> sink = new DiagnosticCollector<JavaFileObject>();
/* 1st null: where to write (default), 2nd null: no annotations processed */
CompilationTask task = compiler.getTask(null, manager, sink, options, null, units);
/* Go, go, go */
boolean status = task.call();
if (!status) {
for (Diagnostic<? extends JavaFileObject> d : sink.getDiagnostics()) {
System.err.format("Error on line %d in %s", d.getLineNumber(), d);
}
}
manager.close();// TRY to close the file manager
}
}
98. public class CompilerAPITest {
Object i;
Object you;
@BeforeClass
public static void setUpClass() throws Exception {
CompilerAPI.main(null);
}
@Before
public void setUp() throws ReflectiveOperationException {
i = Class.forName("I").newInstance();
you = Class.forName("You").newInstance();
}
@After
public void tearDown() {
i = null;
you = null;
}
@Test
public void testMain() throws Exception {
assertEquals(i.getClass().getName(), "I");
assertEquals(you.getClass().getName(), "You");
Method m1 = i.getClass().getMethod("singOutOfTune", (Class<?>[]) null);
Object didI = m1.invoke(i, (Object[]) null);
for (Method m2 : you.getClass().getMethods()) {
if ("wouldDo".equals(m2.getName())) {
Class<?>[] args = m2.getParameterTypes();
if (1 == args.length && args[0].isAssignableFrom(boolean.class)) {
System.out.println("m2.invoke(you, didI)");
return;
}
}
}
fail("We did not find our method!");
}
99. Some Links
● Code:
https://github.com/alshopov/WriteBytecode
● Presentation:
The presentation is to be improved.