In this talk, I will share my journey of using machine learning in Java to build a visual recognition system that can identify Lego blocks. As a Java developer, I wanted to use Java for this project rather than Python, which is more commonly used for machine learning projects. I will explain the basics of machine learning and give an overview of the current Java libraries for machine learning and transferring pre-trained models. I will demonstrate how to train and modify existing models using transfer learning. The goal of the project is to create a Java solution that can identify the top 1000 most popular Lego bricks. I will explain all of this without using any complex mathematical formulas, making it accessible to those with no prior knowledge of machine learning.

1. LESSONS LEARNED USING
MACHINE LEARNING IN JAVA
By Jago de Vreede

6. PYTHON MACHINE LEARNING FRAMEWORKS
• Keras
• SciPy
• PyTorch
• Tensorflow
• Theano
• Pandas

7. JAVA MACHINE LEARNING FRAMEWORKS*
• deeplearning4j
• Elki
• DeepJavaLibrary (djl)
• Apache MXNet
• PyTorch
• Tensorflow
• ONNX Runtime
* With a release in the last year and free

8. JAVA MACHINE LEARNING FRAMEWORKS*
• deeplearning4j
• Elki
• DeepJavaLibrary (djl)
• Apache MXNet
• PyTorch
• Tensorflow
• ONNX Runtime
* With a release in the last year and free

9. BENCHMARKS
DJL DL4J

12. BENCHMARKS - TRAINING
Java Python
github.com/jagodevreede/ml-banchmark
Ran on AWS: g4dn.xlarge

13. BENCHMARKS - PREDICTION
Java Python
github.com/jagodevreede/ml-banchmark
Ran on laptop

14. BENCHMARKS - RESULTS

15. BENCHMARKS - RESULTS

18. Chihuahua
Muffin

19. Chihuahua
Muffin

20. DATASET
$ pip install simple_image_download
import simple_image_download.simple_image_download as simp
my_downloader = simp.Downloader()
my_downloader.download('Chihuahua', limit=2000)
my_downloader.download('Muffin', limit=2000)

21. DATASET

22. DATASET

23. DATASET - CLEANUP
$ pip install ImageHash

25. $ pip install flickrapi

26. DATASET - FLICKR

27. DATASET - ARGUMENTING

28. DATASET - ARGUMENTING
$ pip install keras
datagen = ImageDataGenerator(
rotation_range=20,
width_shift_range=0.1,
height_shift_range=0.1,
shear_range=0.2,
zoom_range=0.2,
horizontal_flip=True,
fill_mode='nearest'
)

29. TIME TO BUILD

34. The number of samples that are processed at once
during each iteration of training.
BATCH SIZES
The presented results confirm that using small
batch sizes achieves the best training stability and
generalization performance, for a given
computational cost, across a wide range of
experiments. In all cases the best results have
been obtained with batch sizes m = 32 or
smaller, often as small as m = 2 or m = 4.
arxiv.org/abs/1804.07612

35. 1 2 4 8 16 32 64
BATCH SIZES - VALIDATION ACCURACY > 85%

36. 1 2 4 8 16 32 64
BATCH SIZES - SPEED (SECONDS)

48. ACCURACY
The lower the value of the entropy loss, the better the
model's predictions are aligned with the labels, and
take into account the confidence of the predictions
accuracy measures the proportion of
correctly classified instances out of the total
number of instances
VS
ENTROPYLOSS

59. Muffin Chihuahua
Muffin 5 3
Chihuahua 1 7
CONFUSION MATRIX

61. VALIDATION ACCURACY

65. LEARNING RATE

66. LEARNING RATE

67. LEARNING RATE

68. LEARNING RATE

69. LEARNING RATE

70. LEARNING RATE

71. LEARNING RATE - OPTIMIZERS

72. factor 0.9 Adam AdaGrad AdaDelta
OPTIMIZERS

74. Muffin Chihuahua
Muffin 7 1
Chihuahua 0 8
CONFUSION MATRIX

75. TRANSFER LEARNING
Muffin Chihuahua

76. TRANSFER LEARNING
Muffin Chihuahua

79. PYTHON AND JAVA TOGETHER

80. CONVERT THE MODEL
import tensorflow as tf
import tensorflow.keras as keras
loaded_model = keras.models.load_model("flowers.h5")
tf.saved_model.save(loaded_model, "model/1")

85. GENERATE IMAGES

86. GENERATE IMAGES

88. ZOOM OR NOT TO ZOOM

91. Sim
SIM - VS - REAL

92. Sim Real
SIM - VS - REAL

93. 3001 3002 3004
3001 31 0 0
3002 28 0 0
3004 29 0 0
CONFUSION MATRIX EPOCH 17

94. 3001 3002 3004
3001 4 27 0
3002 5 23 0
3004 8 21 0
CONFUSION MATRIX EPOCH 47

95. SIM - VS - REAL

96. FAILURE IS ALWAYS AN (TEMPORARY) OPTION

97. @jagovreede
github.com/jagodevreede/
java-dll-dog-vs-muffin
ml-banchmark
lego-sorter