Photo lineups play a significant role in the eyewitness identifica-tion process. This method is used to provide evidence in the prosecution and subsequent conviction of suspects. Unfortu-nately, there are many cases where lineups have led to the con-viction of an innocent suspect. One of the key factors affecting the incorrect identification of a suspect is the lack of lineup fair-ness, i.e. that the suspect differs significantly from all other candidates. Although the process of assembling fair lineup is both highly important and time-consuming, only a handful of tools are available to simplify the task.
In these slides, we describe our work towards using recommend-er systems for the photo lineup assembling task. We propose and evaluate two complementary methods for item-based rec-ommendation: one based on the visual descriptors of the deep neural network, the other based on the content-based attrib-utes of persons.
The initial evaluation made by forensic technicians shows that although results favored visual descriptors over attribute-based similarity, both approaches are functional and highly diverse in terms of recommended objects. Thus, future work should in-volve incorporating both approaches in a single prediction method, preference learning based on the feedback from forensic technicians and recommendation of assembled lineups instead of single candidates.
Towards Recommender Systems for Police Photo Lineup
1. Towards Recommender Systems
for Police Photo Lineup
Ladislav Peškaa and Hana Trojanováb
a Department of Software Engineering,
b Department of Psychology,
Charles University, Prague,
Czech Republic
2. Ladislav Peska & Hana Trojanova: Towards
Recommender Systems for Police Photo Lineup
2
Recommender Systems
Propose relevant items to the right persons at the right time
Successful on many various domains
Multimedia
E-commerce
POIs
Events
Recepies
…
However, some novel (relevant, important) tasks are yet to
discover
DLRS@RecSys2017, Como
3. Ladislav Peska & Hana Trojanova: Towards
Recommender Systems for Police Photo Lineup
3
Police Photo Lineup
Eyewitness identification of the suspect / offender during
criminal proceedings
Most common case: select suspect among other persons (fillers)
Either „in natura“ or based on photographies
DLRS@RecSys2017, Como
- Suspect is positioned at
random
- 3-7 fillers are added, witness
should not know them
- Wittness is instructed that the
offender may or may not be
present
4. Ladislav Peska & Hana Trojanova: Towards
Recommender Systems for Police Photo Lineup
4
Errors in Photo Lineup
DLRS@RecSys2017, Como
1) Observe the event
2) Identify the offender
- among other candidates
5. Ladislav Peska & Hana Trojanova: Towards
Recommender Systems for Police Photo Lineup
5
Errors in Photo Lineup
DLRS@RecSys2017, Como
1) Observe the event.
- numerous sources of error (poor lighting,
large distance, short time, fear, fatigue…)
- affects memory and recognition processes
- decrease witness certainty and reliability
2) Identify the offender
- decreased certainty may lead
to incorrect identification
- and conviction of innocent
persons
6. Ladislav Peska & Hana Trojanova: Towards
Recommender Systems for Police Photo Lineup
6
Assembling Fair Photo Lineups
System should be set to eliminate testimony of uncertain
witnesses
Double-blind administration
Fair (unbiased) assembling of lineups
Out of the dataset of candidates, lineup administrator should select
fillers similar to the suspect
Currently, only feature-based filtering systems are available
Better automatization in content processing is necessary
How to better approximate human-perceived similarity of
persons?
Convolutional Neural Networks?
DLRS@RecSys2017, Como
7. Ladislav Peska & Hana Trojanova: Towards
Recommender Systems for Police Photo Lineup
7
Assembling Fair Photo Lineups
First approach: item-based recommendations
Propose top-k candidates for each suspect
CB-RS, cosine similarity of candidates’ and suspect’s CB features
Nationality, Age, Appearence features
Baseline (mimic feature-based filters)
Visual-RS, cosine similarity of candidates’ and suspect’s VGG-
Face probability layers
Approximate the similarity of a candidate and the suspect through their
similarities with other persons
VGG-Face1 network
VGG network for facial recognition problem
Trained as a multi-classification problem
Dataset of 2,622 celebrities with 1,000 images each
1Parkhi, O. M., Vedaldi, A., & Zisserman, A. (2015). Deep Face Recognition. British Machine Vision Conference.
DLRS@RecSys2017, Como
8. Ladislav Peska & Hana Trojanova: Towards
Recommender Systems for Police Photo Lineup
8
Research Questions
Is candidates recommendation (based on VGG-Face’s
features) suitable for the lineup assembling?
How well is the human-perceived similarity approximated within top-k
candidates?
What is the role of diversity in lineup assembling?
Is there a space for personalized recommendations (i.e. personalized per
lineup administrator)?
Can there be „too similar“ candidates?
DLRS@RecSys2017, Como
9. Ladislav Peska & Hana Trojanova: Towards
Recommender Systems for Police Photo Lineup
9
Evaluation – Protocol
Dataset of missing and wanted persons in Czech Republic
4423 males
User study, 30 different lineups, 7 forensic technicians
For each lineup, both CB-RS and Visual-RS proposed top-20 candidates
Merged into one list, forensic technician should select relevant fillers for
particular suspect
Follow-up questionaire and discussion
DLRS@RecSys2017, Como
10. Ladislav Peska & Hana Trojanova: Towards
Recommender Systems for Police Photo Lineup
10
Results
In total 202 completed lineups, 800 selected candidates
Recommendations were seemingly relevant in most cases
Visual-RS considerably outperformed CB-RS
The performance difference was much smaller for suspects from outside
of central Europe
Effect of different ethnicity? (Known in the forensic psychology literature).
DLRS@RecSys2017, Como
Total lineups Selected candidates Level of agreement
All lineups
Visual-RS
202
466 (58%) 0.178
CB-RS 298 (37%) 0.138
Lineups with suspects from countries outside Central Europe
Visual-RS
53
105 (51%) 0.128
CB-RS 82 (40%) 0.205
11. Ladislav Peska & Hana Trojanova: Towards
Recommender Systems for Police Photo Lineup
11
Results II.
Very low intersection (1.8%) of top-20 CB-RS and Visual-RS candidates
Potential to apply some combination in the future work
Average position of selected candidate
CB-RS: 8.9 (+/- 5.2); Visual-RS: 8.2 (+/- 5.7)
Ordering seems relevant even in top-20, however can be improved
Low level of agreement among participants on the selected candidates (0.178, 0.138)
Limits of the global ordering?
Potential for personalized recommendation?
Less diversity is better
Participants agreed on the need for providing homogeneous lineups
Dynamical recommendation based on the selected candidates?
No seemingly too similar candidates
However, the dataset contained some duplicates
Should be addressed in the future work
DLRS@RecSys2017, Como
12. Conclusions
Recommending candidates based on CNN’s features seems
plausible in police photo lineup
Item-based recommendations seems like a good first approach
There is a potential for:
Session-based personalized recommendations (need for uniformity of the final list)
Long-term preference learning (due to the level of disaggreeement among lineup
administrators)
Approaches combining CNN’s features with content-based attributes
Future work
Move from recommended candidates to recommended lineups
Evaluation through mock witnesses
Deployable demo
DLRS@RecSys2017, Como Ladislav Peska & Hana Trojanova: Towards
Recommender Systems for Police Photo Lineup
12
13. DLRS@RecSys2017, Como Ladislav Peska & Hana Trojanova: Towards
Recommender Systems for Police Photo Lineup
13
Thank you!
Questions, comments?
Supplementary materials: http://github.com/lpeska/lineups, https://tinyurl.com/yat3rtr2
Slides: https://www.slideshare.net/LadislavPeska/towards-recommender-systems-for-police-photo-lineup