VIP Model Call Girls Kothrud ( Pune ) Call ON 8005736733 Starting From 5K to ...
Surpassing Humans and Computers with JELLYBEAN: Crowd-Vision Hybrid Counting Algorithms
1. Surpassing Humans and Computers with
JELLYBEAN: Crowd-Vision Hybrid Counting
Algorithms
Akash Das Sarma (Stanford University)
Ayush Jain (University of Illinois)
Arnab Nandi (The Ohio State University)
Aditya Parameswaran (University of Illinois)
Jennifer Widom (Stanford University)
1
3. What is counting, and why is it important?
• Counting number of items of a particular type in an image
2
4. What is counting, and why is it important?
• Counting number of items of a particular type in an image
• Fundamental primitive for computer vision
2
5. • Many applications - two examples from our paper:
• Biologists: number of cell colonies over time (currently
manually counted by researchers)
What is counting, and why is it important?
• Counting number of items of a particular type in an image
• Fundamental primitive for computer vision
2
6. • Many applications - two examples from our paper:
• Biologists: number of cell colonies over time (currently
manually counted by researchers)
What is counting, and why is it important?
• Counting number of items of a particular type in an image
• Fundamental primitive for computer vision
(SIMCEP)
2
7. • Surveillance: number of people at gatherings (for
example, politicians wanting to know attendance at
rallies)
• Many applications - two examples from our paper:
• Biologists: number of cell colonies over time (currently
manually counted by researchers)
What is counting, and why is it important?
• Counting number of items of a particular type in an image
• Fundamental primitive for computer vision
(SIMCEP)
2
8. • Surveillance: number of people at gatherings (for
example, politicians wanting to know attendance at
rallies)
• Many applications - two examples from our paper:
• Biologists: number of cell colonies over time (currently
manually counted by researchers)
What is counting, and why is it important?
• Counting number of items of a particular type in an image
• Fundamental primitive for computer vision
(SIMCEP)
(Flickr)
2
11. True count = 59
Counting is hard for computers
Given image
3
12. True count = 59
Counting is hard for computers
Pre-trained head detector
(Zhu and Ramanan, CVPR 2012)
Given image
3
13. True count = 59
Counting is hard for computers
Detected count = 35
Pre-trained head detector
(Zhu and Ramanan, CVPR 2012)
Given image
3
14. True count = 59
Counting is hard for computers
Detected count = 35
Pre-trained head detector
(Zhu and Ramanan, CVPR 2012)
Given image
3
Not generalizable
22. 6
Our Goal
•When using humans for counting, minimize cost and error
(images of people from Flickr)
23. 6
Our Goal
•When using humans for counting, minimize cost and error
(images of people from Flickr)
•Devise computer-human hybrid solutions when both options are
available (biological images using SIMCEP)
24. Worker error model and basis for approach
7
Worker Error
0
2
4
6
8
10
12
14
16
0 10 20 30 40 50 60 70 80
AverageError
Number of Objects
25. Worker error model and basis for approach
7
Worker Error
0
2
4
6
8
10
12
14
16
0 10 20 30 40 50 60 70 80
AverageError
Number of Objects
26. Threshold Count
Worker error model and basis for approach
7
Worker Error
0
2
4
6
8
10
12
14
16
0 10 20 30 40 50 60 70 80
AverageError
Number of Objects
27. Threshold Count
Count <= 20: Correct
Worker error model and basis for approach
7
Worker Error
0
2
4
6
8
10
12
14
16
0 10 20 30 40 50 60 70 80
AverageError
Number of Objects
28. Threshold Count
Count <= 20: Correct
Count > 20: Incorrect
Worker error model and basis for approach
7
Worker Error
0
2
4
6
8
10
12
14
16
0 10 20 30 40 50 60 70 80
AverageError
Number of Objects
29. Threshold Count
Count <= 20: Correct
Count > 20: Incorrect
Worker error model and basis for approach
Split images until count < 20
7
Worker Error
0
2
4
6
8
10
12
14
16
0 10 20 30 40 50 60 70 80
AverageError
Number of Objects
69. Hybrid approach: Complexity and Algorithm
•Problem: Find minimum number of bins such that (estimated
prior) count of each bin < threshold (20)
11
70. Hybrid approach: Complexity and Algorithm
•Problem: Find minimum number of bins such that (estimated
prior) count of each bin < threshold (20)
• NP-Complete (reduction from partitioning planar bipartite graphs
(Dyer and Frieze 1985))
11
71. Hybrid approach: Complexity and Algorithm
•Problem: Find minimum number of bins such that (estimated
prior) count of each bin < threshold (20)
• NP-Complete (reduction from partitioning planar bipartite graphs
(Dyer and Frieze 1985))
11
72. Hybrid approach: Complexity and Algorithm
•Problem: Find minimum number of bins such that (estimated
prior) count of each bin < threshold (20)
• NP-Complete (reduction from partitioning planar bipartite graphs
(Dyer and Frieze 1985))
•Greedy heuristic:
11
73. Hybrid approach: Complexity and Algorithm
•Problem: Find minimum number of bins such that (estimated
prior) count of each bin < threshold (20)
• NP-Complete (reduction from partitioning planar bipartite graphs
(Dyer and Frieze 1985))
•Greedy heuristic:
• Grow bin by adding adjacent partitions until threshold reached
11
74. Hybrid approach: Complexity and Algorithm
•Problem: Find minimum number of bins such that (estimated
prior) count of each bin < threshold (20)
• NP-Complete (reduction from partitioning planar bipartite graphs
(Dyer and Frieze 1985))
•Greedy heuristic:
• Grow bin by adding adjacent partitions until threshold reached
• Gets stuck in bad local optima
11
75. Hybrid approach: Complexity and Algorithm
•Problem: Find minimum number of bins such that (estimated
prior) count of each bin < threshold (20)
• NP-Complete (reduction from partitioning planar bipartite graphs
(Dyer and Frieze 1985))
•Greedy heuristic:
• Grow bin by adding adjacent partitions until threshold reached
• Gets stuck in bad local optima
11
76. Hybrid approach: Complexity and Algorithm
•Problem: Find minimum number of bins such that (estimated
prior) count of each bin < threshold (20)
• NP-Complete (reduction from partitioning planar bipartite graphs
(Dyer and Frieze 1985))
•Greedy heuristic:
• Grow bin by adding adjacent partitions until threshold reached
• Gets stuck in bad local optima
•ArticulationAvoidance algorithm:
11
77. Hybrid approach: Complexity and Algorithm
•Problem: Find minimum number of bins such that (estimated
prior) count of each bin < threshold (20)
• NP-Complete (reduction from partitioning planar bipartite graphs
(Dyer and Frieze 1985))
•Greedy heuristic:
• Grow bin by adding adjacent partitions until threshold reached
• Gets stuck in bad local optima
•ArticulationAvoidance algorithm:
• Grow bins by adding partitions one at a time up to threshold
11
78. Hybrid approach: Complexity and Algorithm
•Problem: Find minimum number of bins such that (estimated
prior) count of each bin < threshold (20)
• NP-Complete (reduction from partitioning planar bipartite graphs
(Dyer and Frieze 1985))
•Greedy heuristic:
• Grow bin by adding adjacent partitions until threshold reached
• Gets stuck in bad local optima
•ArticulationAvoidance algorithm:
• Grow bins by adding partitions one at a time up to threshold
• Order of partitions decided by priority queue
11
79. Hybrid approach: Complexity and Algorithm
•Problem: Find minimum number of bins such that (estimated
prior) count of each bin < threshold (20)
• NP-Complete (reduction from partitioning planar bipartite graphs
(Dyer and Frieze 1985))
•Greedy heuristic:
• Grow bin by adding adjacent partitions until threshold reached
• Gets stuck in bad local optima
•ArticulationAvoidance algorithm:
• Grow bins by adding partitions one at a time up to threshold
• Order of partitions decided by priority queue
• Articulation points have lower priority
11
80. Hybrid approach: Complexity and Algorithm
•Problem: Find minimum number of bins such that (estimated
prior) count of each bin < threshold (20)
• NP-Complete (reduction from partitioning planar bipartite graphs
(Dyer and Frieze 1985))
•Greedy heuristic:
• Grow bin by adding adjacent partitions until threshold reached
• Gets stuck in bad local optima
•ArticulationAvoidance algorithm:
• Grow bins by adding partitions one at a time up to threshold
• Order of partitions decided by priority queue
• Articulation points have lower priority
• Experimentally observed to achieve near minimum possible number
of bins on our dataset
11
81. 0
5
10
15
20
<=-5 -4 -3 -2 -1 0 1 2 3 4 >=5
Deviation from Actual Count
FS
AA
ML
Experimental Results
(ACCURACY)
Numberofimages FS = Human only
AA = Vision-human hybrid
ML = Computer vision only
12
82. 0
5
10
15
20
<=-5 -4 -3 -2 -1 0 1 2 3 4 >=5
Deviation from Actual Count
FS
AA
ML
Experimental Results
(ACCURACY)
NumberofimagesCost($)
(COST)
FS = Human only
AA = Vision-human hybrid
ML = Computer vision only
12
83. 13
Our Contributions
•Novel formulation of counting as search problem over
segmentation tree
•Provably optimal humans-only solution
•Hybrid computer+humans algorithm when possible for lower
cost and error
•Real data experiments
•Images of people under different settings (Flickr)
•Biological images (SIMCEP)
96. 18
On the crowd dataset, FS has a much higher accuracy of 97.5%
relative to 70.1% for ML on the 5/12 images ML works on; for the
remaining 7/12 images, ML detects no faces at all.
On the biological dataset, FS has an accuracy of 96.4%. In
comparison, AA increases the accuracy to 99.87%, returning exact
counts for 85% of the images (off on the rest by counts of 1 to at
most 3), while Bio-ML gets only 45% correct (off on the rest by
counts of at least 5).
Accuracy results
