My Slide in ICML 2014. Joint work with Naonori Kakimura, Kazuhiro Inaba, and Ken-ichi Kawarabayashi.

1. Optimal Budget Allocation:
Theoretical Guarantee and Efficient Algorithm
Tasuku Soma Naonori Kakimura Kazuhiro Inaba Ken-ichi Kawarabayashi
Univ. of Tokyo Univ. of Tokyo Google NII, JST, ERATO
Kawarabayashi LargeGraphProject
1 / 16

2. Budget Allocation Problem (Alon et al. ’11)
A mathematical model for company’s ads.
source1
source2
source3
Total: 6
cap:3
cap:5
cap:2
2 / 16

3. Budget Allocation Problem (Alon et al. ’11)
A mathematical model for company’s ads.
source1
source2
source3
alloc:2 ! cap:3
alloc:3 ! cap:5
alloc:1 ! cap:2
Total: 6
2 / 16

4. Budget Allocation Problem (Alon et al. ’11)
A mathematical model for company’s ads.
source1
alloc:2 ! cap:3
Influence Probs
1st trial: Pr 0.3
2nd trial: Pr 0.7
3rd trial: Pr 0.2
2 / 16

5. Budget Allocation Problem (Alon et al. ’11)
A mathematical model for company’s ads.
source1
alloc:2 ! cap:3
Influence Probs
1st trial: Pr 0.3
2nd trial: Pr 0.7
3rd trial: Pr 0.2
fail
success
fail
2 / 16

6. Budget Allocation Problem (Alon et al. ’11)
A mathematical model for company’s ads.
source1
alloc:2 ! cap:3
Influence Probs
1st trial: Pr 0.3
2nd trial: Pr 0.7
3rd trial: Pr 0.2
2 / 16

7. Budget Allocation Problem (Alon et al. ’11)
A mathematical model for company’s ads.
source1
alloc:2 ! cap:3
Influence Probs
1st trial: Pr 0.3
2nd trial: Pr 0.7
3rd trial: Pr 0.2
fail
success
success
2 / 16

8. Budget Allocation Problem (formal)
G = (S; T; E): Bipartite graph
For each source s 2 S:
capacity c(s)
Pr of success of the ith trial p(i)
s (i = 1; : : : ; c(s))
B 2 Z+: Total budget
f (b) := the expected # of influenced nodes by budget allocation b 2 ZS,
which satisfies:
x y =) f (x) f (y) (monotonicity)
f (x) + f (y) f (x _ y) + f (x ^ y) (8x; y 2 ZS) (submodularity)
3 / 16

9. Budget Allocation Problem (formal)
G = (S; T; E): Bipartite graph
For each source s 2 S:
capacity c(s)
Pr of success of the ith trial p(i)
s (i = 1; : : : ; c(s))
B 2 Z+: Total budget
f (b) := the expected # of influenced nodes by budget allocation b 2 ZS,
which satisfies:
x y =) f (x) f (y) (monotonicity)
f (x) + f (y) f (x _ y) + f (x ^ y) (8x; y 2 ZS) (submodularity)
Maximize f (b)
subject to 0 b(s) c(s) (s 2 S) X
s2S
b(s) B
3 / 16

10. Previous Work
NP-hard
If P , NP, (1 1=e)-approximation is best possible
Previous algorithm (Alon et al. ’11) is polytime, but impractical
due to the heavy time complexity
What about more complicated real scenarios?
4 / 16

11. Our Results
1 Submodular Function Maximization over Integer Lattice:
A gerenal framework including more complicated scenarios
(1 1=e)-approximation algorithm
2 Faster Algorithm for Nonincreasing Influence Probabilities:
Speeding up Alon et al.’s algorithm under natural assumption
Almost linear time for graph size
Numerical experiments for real big data
5 / 16

12. 1 Submodular Function Maximization over Integer Lattice
2 Faster Algorithm for Nonincreasing Influence Probabilities
6 / 16

13. Submodular Func. Maximization over ZS
+
f : ZS
+ ! R ... monotone submodular function over integer lattice
x y =) f (x) f (y)
f (x) + f (y) f (x _ y) + f (x ^ y)
(8x; y 2 ZS)
(x _ y : elem-wise max, x ^ y : elem-wise min)
7 / 16

14. Submodular Func. Maximization over ZS
+
f : ZS
+ ! R ... monotone submodular function over integer lattice
x y =) f (x) f (y)
f (x) + f (y) f (x _ y) + f (x ^ y)
(8x; y 2 ZS)
(x _ y : elem-wise max, x ^ y : elem-wise min)
cf. submodularity for
set functions:
f (X) + f (Y)
f (X [ Y) + f (X Y)
(8X; Y S)
7 / 16

15. Submodular Func. Maximization over ZS
+
f : ZS
+ ! R ... monotone submodular function over integer lattice
x y =) f (x) f (y)
f (x) + f (y) f (x _ y) + f (x ^ y)
(8x; y 2 ZS)
(x _ y : elem-wise max, x ^ y : elem-wise min)
cf. submodularity for
set functions:
f (X) + f (Y)
f (X [ Y) + f (X Y)
(8X; Y S)
c 2 ZS
+, w 2 ZS
+, B 2 Z0
Maximize f (b)
subject to 0 b c
X
s2S
w(s)b(s) B (knapsack constraint)
b 2 ZS
7 / 16

16. Submodular Func. Maximization over ZS
+
General Framework Including:
Optimal budget allocation with various unit costs
Optimal budget allocation with competitor
Maximum coverage, Sensor placement, Text summarization, ...
8 / 16

17. Submodular Func. Maximization over ZS
+
General Framework Including:
Optimal budget allocation with various unit costs
Optimal budget allocation with competitor
Maximum coverage, Sensor placement, Text summarization, ...
Pseudo Polytime (1 1=e)-Approximation Algorithm:
Theorem
A (1 1=e)-approximate solution can be found in O(B5jSj4) time
(: the running time of oracle for f ) for a monotone submodular function
maximization over the integer lattice subject to knapsack constraint.
Extends the algorithm for optimal budget allocation (Alon et al. ’11)
8 / 16

18. 1 Submodular Function Maximization over Integer Lattice
2 Faster Algorithm for Nonincreasing Influence Probabilities
9 / 16

19. Nonincreasing Influence Probabilities
Under a natural assumption, Alon et al.’s algorithm can be accelerated.
Assumption:
Influence probabilities of each ad source are nonincreasing:
p(1)
s p(2)
s p(c(s))
s (8s 2 S)
(i.e., Effectiveness of each ad is nonincreasing with time)
10 / 16

20. Nonincreasing Influence Probabilities
Under a natural assumption, Alon et al.’s algorithm can be accelerated.
Assumption:
Influence probabilities of each ad source are nonincreasing:
p(1)
s p(2)
s p(c(s))
s (8s 2 S)
(i.e., Effectiveness of each ad is nonincreasing with time)
Alon et al.’s algorithm: O(B6jSj5jTj) time
Our algorithm: O(B(jSj + jTj + jEj)) time (almost linear for graph size)
10 / 16

21. Algorithm
Under the assumption, f satisfies the following diminishing marginal
return property:
f (b + 2es) f (b + es) f (b + es) f (b) (b 2 ZS; s 2 S)
Note: This property is NOT implied by submodularity!
11 / 16

22. Algorithm
Under the assumption, f satisfies the following diminishing marginal
return property:
f (b + 2es) f (b + es) f (b + es) f (b) (b 2 ZS; s 2 S)
Note: This property is NOT implied by submodularity!
Algorithm
1: b := 0
2: while
P
s2S b(s) B do
3: Among s with b(s) c(s), choose one of f (b +es)f (b) maximum.
4: Set b := b + es.
5: end while
6: return b
11 / 16

23. Algorithm
One more trick:
For optimal budget allocation, f (b + es) f (b) can be computed in
O(1) time.
Theorem
A (1 1=e)-approximate solution can be found in O(B(jVj + jEj)) time if
the influence probabilities of each ad source are nonincreasing.
If B = O(1), runs in linear time for graph size.
12 / 16

24. Experiments
Our algorithm for nonincreasing probabilities vs Heuristics.
Graphs:
Real Data (Yahoo! Webscope Dataset)
10,000 nodes 50,000 edges
Random Graph 2M nodes 8M edges
Heuristics:
Degree-prob ... allocate to nodes with higher degree and prob.
Degree ... allocate to nodes with higher degree
Random ... allocate at random (baseline)
Machine: Xeon E5-2690 2.9GHz CPU, 64GB RAM
13 / 16

25. Real Data (10k nodes 50k edges)
9000
8000
7000
Influenced nodes: f (b) Budget: B
6000
5000
4000
3000
2000
1000
0
Greedy (Ours)
Degree-prob
Degree
Random
0 100 200 300 400 500 600 700 800 900 1000
Maximum 15% outperforming
14 / 16

26. Random Graph (2M nodes 8M edges)
1400000
1200000
Influenced nodes: f (b) Budget: B
1000000
800000
600000
400000
200000
0
Greedy (Ours)
Degree-prob
Degree
Random
0 100 200 300 400 500 600 700 800 900 1000
Our algorithm finds an approx solution in a few seconds
15 / 16

27. Our Results
1 Submodular Function Maximization over Integer Lattice:
A gerenal framework including more complicated scenarios
(1 1=e)-approximation algorithm
2 Faster Algorithm for Nonincreasing Influence Probabilities:
Speeding up Alon et al.’s algorithm under natural assumption
Almost linear time for graph size
Numerical experiments for real big data
16 / 16