1) The document describes the Recod team's approach for the MediaEval 2014 task of retrieving diverse social images for tourism contexts. 2) The approach uses geographic and face-based filtering to improve precision, followed by visual and credibility re-ranking. 3) The best results were achieved using geographic filtering, a simple face counting filter, visual re-ranking, and credibility re-ranking, followed by k-medoids clustering and sorting images within and between clusters.

1. Recod @ MediaEval 2014:
Diverse Social Images Retrieval
Rodrigo T. Calumby1,2, Vinícius P. Santana2, Felipe S. Cordeiro2,
Otávio A. B. Penatti3, Lin T. Li1, Giovani Chiachia1, Ricardo da S. Torres1
1RECOD Lab, Institute of Computing, University of Campinas (UNICAMP), Campinas, SP - Brazil
2Department of Exact Sciences, University of Feira de Santana (UEFS), Feira de Santana, BA - Brazil
3SAMSUNG Research Institute Brazil, Campinas, SP - Brazil
[rtcalumby, vpsantana, fscordeiro]@ecomp.uefs.br, o.penatti@samsung.com, [lintzyli, chiachia, rtorres]@ic.unicamp.br
Acknowledgments: UEFS/PROBIC, Samsung Research Institute Brazil, and FAPESP (2013/11359-0).
UEFS
Filtering
Geographic Face detection
Diversification Re-ranking
Visual
input
list
Clustering
output
list Selection
Credibility
TASK
Retrieving Diverse Social Images Task at MediaEval 2014
Improve relevace and diversity of image retrieval results in a tourism context
Location representatives:
(location: Casa Batlló, Barcelona)
1
√n+1 4
Test Image:
RelScore (nth image):
Finalcore: CredScore x RelScore
RESULTS – OFFICIAL MEASURES
Run Filtering Re-ranking Diversification P@20 CR@20 F1@20
1 GeoFilter and NumFacesFilter - kMedoids (BoVWsparse
max + HOG) 0.7130 0.4030 0.5077
2 GeoFilter and NumFacesFilter - kMedoids (Cosine) 0.6976 0.4139 0.5133
3 GeoFilter and NumFacesFilter Visual re-ranking (CM3x3 + HOG + BIC) kMedoids (BoVWsparse
max + HOG + Cosine) 0.7016 0.4177 0.5168
4 GeoFilter and NumFacesFilter Visual re-ranking (CM3x3 + HOG + BIC)
and Credibility re-ranking kMedoids (CN3x3) 0.7598 0.4288 0.5423
5 GeoFilter and FaceClassifierFilter Visual re-ranking (CM3x3 + HOG + BIC)
and Credibility re-ranking kMedoids (CN3x3) 0.7407 0.4076 0.5206
PROPOSED APPROACH
Visual Credibility
Clustering
Selection
Geo Filter
Face Filter
CredScore:
visualScore X faceProportion X tagSpecificity
Original Re-ranked
Original Re-ranked
NumFacesFilter
> 1 face → non-relevant
(location: Christ the Redeemer, Rio de Janeiro)
1-NN Classifier
Features
1) number of faces
2) biggest face size
3) smallest face size
4) average face size
5) total face size
Validation (devset)
leave-one-location-out
Runs (testset)
target: full devset
kNN
10km radius
limit from
reference
lat/long of the
location
(location: Iguazu Falls, Brazil/Argentina)
- Descending cluster size
- Most relevant item from
each cluster
(location: Arc de Triomphe, Paris)
- kMedoids: 50 clusters
- Initial medoids: rank offset positions
Output list
FaceClassifierFilter
CONCLUSIONS
- Geographic and face-based filtering improved precision.
- The simple NumFacesFilter outperformed the FaceClassifierFilter.
- Visual and credibility re-ranking improved precision.
- kMedoids outperfomed MMR on devset and was applied for the runs.
- Inter and inner cluster sorting were effective for improving relevance.