s1160123 Tomoyuki Soeta

1. Information Retrieved
- Image based search
s1160123 Tomoyuki Soeta
Supervised by Prof. Qiangfu Zhao
System Intelligence Lab
1

2. Outline
 Introduction
 Information Retrieval
 VQ (Vector Quantization)
 Divide into the 8x8 block
 Making of Code book
 K-means algorithm
 Extract each image’s feature vector
 Result
 Image and feature vector
 Distance of feature vector
 Conclusion
 Future work 2

3. Introduction
 I want to aim at the improvement of information retrieval
system to search it even if the input data are documents
or images.
 I have charge of a research on information retrieval
based on a image.
 To search images using a search engine, we may use the
index attached to the image, the file name, etc. as the
key-words. We may also use "the contents of an image
themselves."
 I study a new image search technique based on the code
book information.
3

4. Information Retrieved
Image
Text
Divide into the block
(1 block 8x8)
Morphological
Analysis
Code book
Word Filtering
Code of each block
Feature Vector
Feature Vector
NNTree or SVM
4

5. VQ (Vector Quantization)
 Compression coding of images
 Image compression technology
image  In my study, I use VQ to
Vector translate an image into a bag-
Quantization of-blocks (BOB)
(VQ)
same way as document search
5
feature vector

6. Divide into the 8x8 block (1)
 I used 10 facial images with the size 256x256.
 images are converted to gray scale images.
 Divided into the block (one-block 8x8 size).
 Each image obtains the block of 32×32
pieces severally.
32 blocks
32
b
l 1 block 8x8
o
c
k
s
6

7. Divide into the 8x8 block (2)
 Block’s pixel value is read.
 Pixel read value is stored in the array of
1x64.
 One image can be divided into 1024 blocks,
and an array of 1024 rows can be obtained.
1x64
2 3 4 6 8 3 7 2 2 3 4 6 8 3 7 2 8 2 8 2
8 2 8 2 8 2
1 block 8x8
・
・
・
・
8x8 1024 rows 7

8. 10 images
Image 0 Image 1 Image 2 Image 3 Image 4
Image 5 Image 6 Image 7 Image 8 Image 9
8

9. Making of Code book
 The array of 10240 that can be done
by reading 10 images is made
 The code book is made by using the
k-means method.
 Making Code book (size 256)
9

10. K-means algorithm
Step 1) k initial "means" are randomly selected from
the data set .
Step 2) k clusters are created by associating every
observation with the nearest mean.
Step 3) The centroid of each of the k clusters
becomes the new means.
Step 4) Steps 2 and 3 are repeated until
convergence has been reached.
Step 1 Step 2 Step 3 Step 4 10

11. Extract each image’s feature vector (1)
 The feature vector are extracted by using code book.
 There is arrangement 1024 per one image.
 Arranging an individual distance of the array each one
and code book is measured
 The number of the nearest code is returned.
 Which code how many times came out is preserved as
an array.
5
1x64
4
2 3 4 7 8 9 2 ########## Code 7
3
Code 38
Code 72 2
Code 200 1
Code 7
Code 200
Code 72
Code 38
7 0
・ 1 256
・ 7 38 72 200
・
1024 rows Code book 11

12. Result – image and feature vector(1)
Image 0 Image 1
12

13. Result – image and feature vector(2)
Image 2 Image 3
13

14. Result – image and feature vector(3)
Image 4 Image 5
14

15. Result – image and feature vector(4)
Image 6 Image 7
15

16. Result – image and feature vector(5)
Image 8 Image 9
16

17. Result - Distance of feature vector(1)
 Euclidean distance between feature vectors
is measured, and the accuracy of the code
book is seen.
 P and Q are assumed to be two feature vectors.
 Data : x = (x1, x2, ..., xn) and y = (y1, y2, ..., yn)
 n : size of the feature vector
 The distance of P and Q is below.
17

18. Result - Distance of feature vector(2)
256 feature0 feature1 feature2 feature3 feature4 feature5 feature6 feature7 feature8 feature9
feature0 0 0.279945 0.280761 0.226158 0.291376 0.322875 0.300502 0.2307 0.23509 0.228708
feature1 0.279945 0 0.19849 0.271927 0.318353 0.352126 0.324807 0.272823 0.269333 0.30847
feature2 0.280761 0.19849 0 0.308124 0.352732 0.378846 0.359333 0.310492 0.316141 0.324054
feature3 0.226158 0.271927 0.308124 0 0.221109 0.276269 0.240734 0.09959 0.086469 0.136439
feature4 0.291376 0.318353 0.352732 0.221109 0 0.222279 0.17478 0.202749 0.210865 0.248531
feature5 0.322875 0.352126 0.378846 0.276269 0.222279 0 0.084866 0.282603 0.270858 0.306136
feature6 0.300502 0.324807 0.359333 0.240734 0.17478 0.084866 0 0.245255 0.232873 0.276931
feature7 0.2307 0.272823 0.310492 0.09959 0.202749 0.282603 0.245255 0 0.105974 0.155957
feature8 0.23509 0.269333 0.316141 0.086469 0.210865 0.270858 0.232873 0.105974 0 0.152093
feature9 0.228708 0.30847 0.324054 0.136439 0.248531 0.306136 0.276931 0.155957 0.152093 0
 :minimum distance
 The image5 and image6 is the same
persons, image5 doesn't wear glasses,
and image6 wears glasses.
 Between feature5 and feature6 is
minimum distance.
Image 5 Image 6 18

19. Conclusion
 In my research, I study a new image search
technique based on the code book
information. The code book is obtained using
the VQ method.
 It is thought that an accurate feature vector
was able to be extracted about the accuracy
of the feature vector because the distance
between Feature5 and 6 was short.
Information retrieval based on
"the contents of a image themselves." 19

20. Future work
 The background is nullified.
 The feature vector is extracted in the block
of a different size like the block of not the
block of 8x8 size but 16x16 size etc.
 Multimedia retrieval that uses SVM.
20

21. Thank you for your attention!
21