Why biased matrix factorization works well?

Why Biased Matrix
Factorization Works Well?
2018. 8. 29.
JoonyoungYi
joonyoung.yi@kaist.ac.kr 
Machine Learning & Intelligence Laboratory
School of Computing 
Korea Advanced Institute of Science andTechnology

TABLE OF CONTENTS
1. Netflix Prize and Winner’s Algorithm
2. Funk SVD (2006) and Biased Matrix Factorization (IEEE 2009)
3. Probabilistic Matrix Factorization (NIPS 2008)
4. Why We Use Alternating Minimization?
5. Overcoming Local Minima Problem of Alternating Minimization
6. Why Random Initialization Works?

NETFLIX PRIZE
• In October 2016, Netflix open a $1M Prize called Netflix Prize.
• To improve recommendation accuracy by 10%.
• They splitted the data for prize.
• Training set: 100M
• Test set: 1.4M
• Training set contains 1.2 % of entries.
• 480,000 users and 17,700 movies.
• From a matrix viewpoint, training set contains only 1.2% entries of the matrix.
!4
1 2 3 4 5 6 7 8 …
1 4 2 4
2 3 3 3
3 3 1
4 2 4 1 5
…
2 3 1
Users
Movies
480,000
17,700

DATA DESCRIPTION OF NETFLIX PRIZE
• Each set(training set, test set) is a set of (i, j, Mij)s.
• i: user index
• j: item(or movie) index
• Mij: true rating of user u to item i.
• : the set of (i, j) corresponding to the training set(=The set of known entries).
• : the set of (i, j) corresponding to the test set(=The set of entries to predict).
• m is the number of users.
• n is the number of items.
!5
⌦<latexit sha1_base64="QevwxOiyXhYIQIu06UQswEZLGZ8=">AAAB7HicbVDLSgNBEOz1GeMr6tHLYBA8hV0R1FvAizcjuEkgWcLsZDYZM49lZlYIS/7BiwcVr36QN//GSbIHTSxoKKq66e6KU86M9f1vb2V1bX1js7RV3t7Z3duvHBw2jco0oSFRXOl2jA3lTNLQMstpO9UUi5jTVjy6mfqtJ6oNU/LBjlMaCTyQLGEEWyc1u3eCDnCvUvVr/gxomQQFqUKBRq/y1e0rkgkqLeHYmE7gpzbKsbaMcDopdzNDU0xGeEA7jkosqIny2bUTdOqUPkqUdiUtmqm/J3IsjBmL2HUKbIdm0ZuK/3mdzCZXUc5kmlkqyXxRknFkFZq+jvpMU2L52BFMNHO3IjLEGhPrAiq7EILFl5dJeF67rgX3F9W6X6RRgmM4gTMI4BLqcAsNCIHAIzzDK7x5ynvx3r2PeeuKV8wcwR94nz/GTI69</latexit><latexit sha1_base64="QevwxOiyXhYIQIu06UQswEZLGZ8=">AAAB7HicbVDLSgNBEOz1GeMr6tHLYBA8hV0R1FvAizcjuEkgWcLsZDYZM49lZlYIS/7BiwcVr36QN//GSbIHTSxoKKq66e6KU86M9f1vb2V1bX1js7RV3t7Z3duvHBw2jco0oSFRXOl2jA3lTNLQMstpO9UUi5jTVjy6mfqtJ6oNU/LBjlMaCTyQLGEEWyc1u3eCDnCvUvVr/gxomQQFqUKBRq/y1e0rkgkqLeHYmE7gpzbKsbaMcDopdzNDU0xGeEA7jkosqIny2bUTdOqUPkqUdiUtmqm/J3IsjBmL2HUKbIdm0ZuK/3mdzCZXUc5kmlkqyXxRknFkFZq+jvpMU2L52BFMNHO3IjLEGhPrAiq7EILFl5dJeF67rgX3F9W6X6RRgmM4gTMI4BLqcAsNCIHAIzzDK7x5ynvx3r2PeeuKV8wcwR94nz/GTI69</latexit><latexit sha1_base64="QevwxOiyXhYIQIu06UQswEZLGZ8=">AAAB7HicbVDLSgNBEOz1GeMr6tHLYBA8hV0R1FvAizcjuEkgWcLsZDYZM49lZlYIS/7BiwcVr36QN//GSbIHTSxoKKq66e6KU86M9f1vb2V1bX1js7RV3t7Z3duvHBw2jco0oSFRXOl2jA3lTNLQMstpO9UUi5jTVjy6mfqtJ6oNU/LBjlMaCTyQLGEEWyc1u3eCDnCvUvVr/gxomQQFqUKBRq/y1e0rkgkqLeHYmE7gpzbKsbaMcDopdzNDU0xGeEA7jkosqIny2bUTdOqUPkqUdiUtmqm/J3IsjBmL2HUKbIdm0ZuK/3mdzCZXUc5kmlkqyXxRknFkFZq+jvpMU2L52BFMNHO3IjLEGhPrAiq7EILFl5dJeF67rgX3F9W6X6RRgmM4gTMI4BLqcAsNCIHAIzzDK7x5ynvx3r2PeeuKV8wcwR94nz/GTI69</latexit><latexit sha1_base64="QevwxOiyXhYIQIu06UQswEZLGZ8=">AAAB7HicbVDLSgNBEOz1GeMr6tHLYBA8hV0R1FvAizcjuEkgWcLsZDYZM49lZlYIS/7BiwcVr36QN//GSbIHTSxoKKq66e6KU86M9f1vb2V1bX1js7RV3t7Z3duvHBw2jco0oSFRXOl2jA3lTNLQMstpO9UUi5jTVjy6mfqtJ6oNU/LBjlMaCTyQLGEEWyc1u3eCDnCvUvVr/gxomQQFqUKBRq/y1e0rkgkqLeHYmE7gpzbKsbaMcDopdzNDU0xGeEA7jkosqIny2bUTdOqUPkqUdiUtmqm/J3IsjBmL2HUKbIdm0ZuK/3mdzCZXUc5kmlkqyXxRknFkFZq+jvpMU2L52BFMNHO3IjLEGhPrAiq7EILFl5dJeF67rgX3F9W6X6RRgmM4gTMI4BLqcAsNCIHAIzzDK7x5ynvx3r2PeeuKV8wcwR94nz/GTI69</latexit>
⌦0
<latexit sha1_base64="AhOmQwf0oN8sVpt2aiAX3g06gEg=">AAAB7XicbVBNS8NAEJ3Ur1q/qh69LBbRU0lEUG8FL96sYGyhDWWznbRLd5OwuxFK6I/w4kHFq//Hm//GbZuDtj4YeLw3w8y8MBVcG9f9dkorq2vrG+XNytb2zu5edf/gUSeZYuizRCSqHVKNgsfoG24EtlOFVIYCW+HoZuq3nlBpnsQPZpxiIOkg5hFn1Fip1b2TOKCnvWrNrbszkGXiFaQGBZq96le3n7BMYmyYoFp3PDc1QU6V4UzgpNLNNKaUjegAO5bGVKIO8tm5E3JilT6JEmUrNmSm/p7IqdR6LEPbKakZ6kVvKv7ndTITXQU5j9PMYMzmi6JMEJOQ6e+kzxUyI8aWUKa4vZWwIVWUGZtQxYbgLb68TPzz+nXdu7+oNdwijTIcwTGcgQeX0IBbaIIPDEbwDK/w5qTOi/PufMxbS04xcwh/4Hz+ACiCju4=</latexit><latexit sha1_base64="AhOmQwf0oN8sVpt2aiAX3g06gEg=">AAAB7XicbVBNS8NAEJ3Ur1q/qh69LBbRU0lEUG8FL96sYGyhDWWznbRLd5OwuxFK6I/w4kHFq//Hm//GbZuDtj4YeLw3w8y8MBVcG9f9dkorq2vrG+XNytb2zu5edf/gUSeZYuizRCSqHVKNgsfoG24EtlOFVIYCW+HoZuq3nlBpnsQPZpxiIOkg5hFn1Fip1b2TOKCnvWrNrbszkGXiFaQGBZq96le3n7BMYmyYoFp3PDc1QU6V4UzgpNLNNKaUjegAO5bGVKIO8tm5E3JilT6JEmUrNmSm/p7IqdR6LEPbKakZ6kVvKv7ndTITXQU5j9PMYMzmi6JMEJOQ6e+kzxUyI8aWUKa4vZWwIVWUGZtQxYbgLb68TPzz+nXdu7+oNdwijTIcwTGcgQeX0IBbaIIPDEbwDK/w5qTOi/PufMxbS04xcwh/4Hz+ACiCju4=</latexit><latexit sha1_base64="AhOmQwf0oN8sVpt2aiAX3g06gEg=">AAAB7XicbVBNS8NAEJ3Ur1q/qh69LBbRU0lEUG8FL96sYGyhDWWznbRLd5OwuxFK6I/w4kHFq//Hm//GbZuDtj4YeLw3w8y8MBVcG9f9dkorq2vrG+XNytb2zu5edf/gUSeZYuizRCSqHVKNgsfoG24EtlOFVIYCW+HoZuq3nlBpnsQPZpxiIOkg5hFn1Fip1b2TOKCnvWrNrbszkGXiFaQGBZq96le3n7BMYmyYoFp3PDc1QU6V4UzgpNLNNKaUjegAO5bGVKIO8tm5E3JilT6JEmUrNmSm/p7IqdR6LEPbKakZ6kVvKv7ndTITXQU5j9PMYMzmi6JMEJOQ6e+kzxUyI8aWUKa4vZWwIVWUGZtQxYbgLb68TPzz+nXdu7+oNdwijTIcwTGcgQeX0IBbaIIPDEbwDK/w5qTOi/PufMxbS04xcwh/4Hz+ACiCju4=</latexit><latexit sha1_base64="AhOmQwf0oN8sVpt2aiAX3g06gEg=">AAAB7XicbVBNS8NAEJ3Ur1q/qh69LBbRU0lEUG8FL96sYGyhDWWznbRLd5OwuxFK6I/w4kHFq//Hm//GbZuDtj4YeLw3w8y8MBVcG9f9dkorq2vrG+XNytb2zu5edf/gUSeZYuizRCSqHVKNgsfoG24EtlOFVIYCW+HoZuq3nlBpnsQPZpxiIOkg5hFn1Fip1b2TOKCnvWrNrbszkGXiFaQGBZq96le3n7BMYmyYoFp3PDc1QU6V4UzgpNLNNKaUjegAO5bGVKIO8tm5E3JilT6JEmUrNmSm/p7IqdR6LEPbKakZ6kVvKv7ndTITXQU5j9PMYMzmi6JMEJOQ6e+kzxUyI8aWUKa4vZWwIVWUGZtQxYbgLb68TPzz+nXdu7+oNdwijTIcwTGcgQeX0IBbaIIPDEbwDK/w5qTOi/PufMxbS04xcwh/4Hz+ACiCju4=</latexit>

DATA DESCRIPTION OF NETFLIX PRIZE
• RMSE(Root Mean Square Error) was used for measuring accuracy.
• Rij: predicted rating of user i to item j.
• is the cardinality of the .
• Projection Operator
• M, R are rating matrices.
•
!6
⌦0
<latexit sha1_base64="AhOmQwf0oN8sVpt2aiAX3g06gEg=">AAAB7XicbVBNS8NAEJ3Ur1q/qh69LBbRU0lEUG8FL96sYGyhDWWznbRLd5OwuxFK6I/w4kHFq//Hm//GbZuDtj4YeLw3w8y8MBVcG9f9dkorq2vrG+XNytb2zu5edf/gUSeZYuizRCSqHVKNgsfoG24EtlOFVIYCW+HoZuq3nlBpnsQPZpxiIOkg5hFn1Fip1b2TOKCnvWrNrbszkGXiFaQGBZq96le3n7BMYmyYoFp3PDc1QU6V4UzgpNLNNKaUjegAO5bGVKIO8tm5E3JilT6JEmUrNmSm/p7IqdR6LEPbKakZ6kVvKv7ndTITXQU5j9PMYMzmi6JMEJOQ6e+kzxUyI8aWUKa4vZWwIVWUGZtQxYbgLb68TPzz+nXdu7+oNdwijTIcwTGcgQeX0IBbaIIPDEbwDK/w5qTOi/PufMxbS04xcwh/4Hz+ACiCju4=</latexit><latexit sha1_base64="AhOmQwf0oN8sVpt2aiAX3g06gEg=">AAAB7XicbVBNS8NAEJ3Ur1q/qh69LBbRU0lEUG8FL96sYGyhDWWznbRLd5OwuxFK6I/w4kHFq//Hm//GbZuDtj4YeLw3w8y8MBVcG9f9dkorq2vrG+XNytb2zu5edf/gUSeZYuizRCSqHVKNgsfoG24EtlOFVIYCW+HoZuq3nlBpnsQPZpxiIOkg5hFn1Fip1b2TOKCnvWrNrbszkGXiFaQGBZq96le3n7BMYmyYoFp3PDc1QU6V4UzgpNLNNKaUjegAO5bGVKIO8tm5E3JilT6JEmUrNmSm/p7IqdR6LEPbKakZ6kVvKv7ndTITXQU5j9PMYMzmi6JMEJOQ6e+kzxUyI8aWUKa4vZWwIVWUGZtQxYbgLb68TPzz+nXdu7+oNdwijTIcwTGcgQeX0IBbaIIPDEbwDK/w5qTOi/PufMxbS04xcwh/4Hz+ACiCju4=</latexit><latexit sha1_base64="AhOmQwf0oN8sVpt2aiAX3g06gEg=">AAAB7XicbVBNS8NAEJ3Ur1q/qh69LBbRU0lEUG8FL96sYGyhDWWznbRLd5OwuxFK6I/w4kHFq//Hm//GbZuDtj4YeLw3w8y8MBVcG9f9dkorq2vrG+XNytb2zu5edf/gUSeZYuizRCSqHVKNgsfoG24EtlOFVIYCW+HoZuq3nlBpnsQPZpxiIOkg5hFn1Fip1b2TOKCnvWrNrbszkGXiFaQGBZq96le3n7BMYmyYoFp3PDc1QU6V4UzgpNLNNKaUjegAO5bGVKIO8tm5E3JilT6JEmUrNmSm/p7IqdR6LEPbKakZ6kVvKv7ndTITXQU5j9PMYMzmi6JMEJOQ6e+kzxUyI8aWUKa4vZWwIVWUGZtQxYbgLb68TPzz+nXdu7+oNdwijTIcwTGcgQeX0IBbaIIPDEbwDK/w5qTOi/PufMxbS04xcwh/4Hz+ACiCju4=</latexit><latexit sha1_base64="AhOmQwf0oN8sVpt2aiAX3g06gEg=">AAAB7XicbVBNS8NAEJ3Ur1q/qh69LBbRU0lEUG8FL96sYGyhDWWznbRLd5OwuxFK6I/w4kHFq//Hm//GbZuDtj4YeLw3w8y8MBVcG9f9dkorq2vrG+XNytb2zu5edf/gUSeZYuizRCSqHVKNgsfoG24EtlOFVIYCW+HoZuq3nlBpnsQPZpxiIOkg5hFn1Fip1b2TOKCnvWrNrbszkGXiFaQGBZq96le3n7BMYmyYoFp3PDc1QU6V4UzgpNLNNKaUjegAO5bGVKIO8tm5E3JilT6JEmUrNmSm/p7IqdR6LEPbKakZ6kVvKv7ndTITXQU5j9PMYMzmi6JMEJOQ6e+kzxUyI8aWUKa4vZWwIVWUGZtQxYbgLb68TPzz+nXdu7+oNdwijTIcwTGcgQeX0IBbaIIPDEbwDK/w5qTOi/PufMxbS04xcwh/4Hz+ACiCju4=</latexit>
|⌦0
|<latexit sha1_base64="lqftWvWVOn1LrUfnslhQk2K3r3M=">AAAB8XicbVBNS8NAEJ3Ur1q/qh69LBbRU0lEUG8FL96sYGwhDWWz3bRLd7NhdyOUtD/DiwcVr/4bb/4bt20O2vpg4PHeDDPzopQzbVz32ymtrK6tb5Q3K1vbO7t71f2DRy0zRahPJJeqHWFNOUuob5jhtJ0qikXEaSsa3kz91hNVmsnkwYxSGgrcT1jMCDZWCsaocydoH5+icbdac+vuDGiZeAWpQYFmt/rV6UmSCZoYwrHWgeemJsyxMoxwOql0Mk1TTIa4TwNLEyyoDvPZyRN0YpUeiqWylRg0U39P5FhoPRKR7RTYDPSiNxX/84LMxFdhzpI0MzQh80VxxpGRaPo/6jFFieEjSzBRzN6KyAArTIxNqWJD8BZfXib+ef267t1f1BpukUYZjuAYzsCDS2jALTTBBwISnuEV3hzjvDjvzse8teQUM4fwB87nD58AkE4=</latexit><latexit sha1_base64="lqftWvWVOn1LrUfnslhQk2K3r3M=">AAAB8XicbVBNS8NAEJ3Ur1q/qh69LBbRU0lEUG8FL96sYGwhDWWz3bRLd7NhdyOUtD/DiwcVr/4bb/4bt20O2vpg4PHeDDPzopQzbVz32ymtrK6tb5Q3K1vbO7t71f2DRy0zRahPJJeqHWFNOUuob5jhtJ0qikXEaSsa3kz91hNVmsnkwYxSGgrcT1jMCDZWCsaocydoH5+icbdac+vuDGiZeAWpQYFmt/rV6UmSCZoYwrHWgeemJsyxMoxwOql0Mk1TTIa4TwNLEyyoDvPZyRN0YpUeiqWylRg0U39P5FhoPRKR7RTYDPSiNxX/84LMxFdhzpI0MzQh80VxxpGRaPo/6jFFieEjSzBRzN6KyAArTIxNqWJD8BZfXib+ef267t1f1BpukUYZjuAYzsCDS2jALTTBBwISnuEV3hzjvDjvzse8teQUM4fwB87nD58AkE4=</latexit><latexit sha1_base64="lqftWvWVOn1LrUfnslhQk2K3r3M=">AAAB8XicbVBNS8NAEJ3Ur1q/qh69LBbRU0lEUG8FL96sYGwhDWWz3bRLd7NhdyOUtD/DiwcVr/4bb/4bt20O2vpg4PHeDDPzopQzbVz32ymtrK6tb5Q3K1vbO7t71f2DRy0zRahPJJeqHWFNOUuob5jhtJ0qikXEaSsa3kz91hNVmsnkwYxSGgrcT1jMCDZWCsaocydoH5+icbdac+vuDGiZeAWpQYFmt/rV6UmSCZoYwrHWgeemJsyxMoxwOql0Mk1TTIa4TwNLEyyoDvPZyRN0YpUeiqWylRg0U39P5FhoPRKR7RTYDPSiNxX/84LMxFdhzpI0MzQh80VxxpGRaPo/6jFFieEjSzBRzN6KyAArTIxNqWJD8BZfXib+ef267t1f1BpukUYZjuAYzsCDS2jALTTBBwISnuEV3hzjvDjvzse8teQUM4fwB87nD58AkE4=</latexit><latexit sha1_base64="lqftWvWVOn1LrUfnslhQk2K3r3M=">AAAB8XicbVBNS8NAEJ3Ur1q/qh69LBbRU0lEUG8FL96sYGwhDWWz3bRLd7NhdyOUtD/DiwcVr/4bb/4bt20O2vpg4PHeDDPzopQzbVz32ymtrK6tb5Q3K1vbO7t71f2DRy0zRahPJJeqHWFNOUuob5jhtJ0qikXEaSsa3kz91hNVmsnkwYxSGgrcT1jMCDZWCsaocydoH5+icbdac+vuDGiZeAWpQYFmt/rV6UmSCZoYwrHWgeemJsyxMoxwOql0Mk1TTIa4TwNLEyyoDvPZyRN0YpUeiqWylRg0U39P5FhoPRKR7RTYDPSiNxX/84LMxFdhzpI0MzQh80VxxpGRaPo/6jFFieEjSzBRzN6KyAArTIxNqWJD8BZfXib+ef267t1f1BpukUYZjuAYzsCDS2jALTTBBwISnuEV3hzjvDjvzse8teQUM4fwB87nD58AkE4=</latexit>
train RMSE =
s
X
⌦
(Mij Rij)2
|⌦|
=
kP⌦(M R)kF
p
⌦
test RMSE =
s
X
⌦0
(Mij Rij)2
|⌦0|
=
kP⌦0 (M R)kF
p
⌦0
<latexit sha1_base64="/9IZniw0dk7d9UcO15y/+IJjYXs=">AAADHXiclVLLbhMxFPUMrxJeKSzZXBFB0gXRTBWJsqhUCYHYRIRAaKU4jDyOk5rOq/YdROT6S9jwK2xYAGLBBvE3OJkBQQsIrmXp6Nx77j1+xEUiNQbBV88/dfrM2XNr5xsXLl66fKW5fvWZzkvFxYjnSa72YqZFIjMxQomJ2CuUYGmciN344N4yv/tSKC3z7CkuCjFJ2TyTM8kZOipa93oUxSs0qJjMYNh/ct/CrW2g+lChobpMI0MfpWLOLBgwnX5k5AsLt8EMbQU3nm9aoLmb4QrgCKpqB+xqbYMxQI9g8KNPp+/kww1HRg8sfJdWA2u1ta4lbdTOhMa/GWv/n7P2n621/8Vb25lrRM1W0A1WASdBWIMWqWMQNT/Tac7LVGTIE6b1OAwKnBimUPJE2AYttSgYP2BzMXYwY6nQE7N6Xgs3HTOFWa7czhBW7M8Kw1KtF2nsKlOG+/p4bkn+LjcucbY1MTIrShQZrwbNygQwh+VfgalUgmOycIBxJZ1X4PtMMY7uRy0vITx+5JNgtNm92w0f91o7vfo21sh1coN0SEjukB3ykAzIiHDvtffWe+998N/47/yP/qeq1PdqzTXyS/hfvgGzoveX</latexit><latexit sha1_base64="/9IZniw0dk7d9UcO15y/+IJjYXs=">AAADHXiclVLLbhMxFPUMrxJeKSzZXBFB0gXRTBWJsqhUCYHYRIRAaKU4jDyOk5rOq/YdROT6S9jwK2xYAGLBBvE3OJkBQQsIrmXp6Nx77j1+xEUiNQbBV88/dfrM2XNr5xsXLl66fKW5fvWZzkvFxYjnSa72YqZFIjMxQomJ2CuUYGmciN344N4yv/tSKC3z7CkuCjFJ2TyTM8kZOipa93oUxSs0qJjMYNh/ct/CrW2g+lChobpMI0MfpWLOLBgwnX5k5AsLt8EMbQU3nm9aoLmb4QrgCKpqB+xqbYMxQI9g8KNPp+/kww1HRg8sfJdWA2u1ta4lbdTOhMa/GWv/n7P2n621/8Vb25lrRM1W0A1WASdBWIMWqWMQNT/Tac7LVGTIE6b1OAwKnBimUPJE2AYttSgYP2BzMXYwY6nQE7N6Xgs3HTOFWa7czhBW7M8Kw1KtF2nsKlOG+/p4bkn+LjcucbY1MTIrShQZrwbNygQwh+VfgalUgmOycIBxJZ1X4PtMMY7uRy0vITx+5JNgtNm92w0f91o7vfo21sh1coN0SEjukB3ykAzIiHDvtffWe+998N/47/yP/qeq1PdqzTXyS/hfvgGzoveX</latexit><latexit sha1_base64="/9IZniw0dk7d9UcO15y/+IJjYXs=">AAADHXiclVLLbhMxFPUMrxJeKSzZXBFB0gXRTBWJsqhUCYHYRIRAaKU4jDyOk5rOq/YdROT6S9jwK2xYAGLBBvE3OJkBQQsIrmXp6Nx77j1+xEUiNQbBV88/dfrM2XNr5xsXLl66fKW5fvWZzkvFxYjnSa72YqZFIjMxQomJ2CuUYGmciN344N4yv/tSKC3z7CkuCjFJ2TyTM8kZOipa93oUxSs0qJjMYNh/ct/CrW2g+lChobpMI0MfpWLOLBgwnX5k5AsLt8EMbQU3nm9aoLmb4QrgCKpqB+xqbYMxQI9g8KNPp+/kww1HRg8sfJdWA2u1ta4lbdTOhMa/GWv/n7P2n621/8Vb25lrRM1W0A1WASdBWIMWqWMQNT/Tac7LVGTIE6b1OAwKnBimUPJE2AYttSgYP2BzMXYwY6nQE7N6Xgs3HTOFWa7czhBW7M8Kw1KtF2nsKlOG+/p4bkn+LjcucbY1MTIrShQZrwbNygQwh+VfgalUgmOycIBxJZ1X4PtMMY7uRy0vITx+5JNgtNm92w0f91o7vfo21sh1coN0SEjukB3ykAzIiHDvtffWe+998N/47/yP/qeq1PdqzTXyS/hfvgGzoveX</latexit><latexit sha1_base64="/9IZniw0dk7d9UcO15y/+IJjYXs=">AAADHXiclVLLbhMxFPUMrxJeKSzZXBFB0gXRTBWJsqhUCYHYRIRAaKU4jDyOk5rOq/YdROT6S9jwK2xYAGLBBvE3OJkBQQsIrmXp6Nx77j1+xEUiNQbBV88/dfrM2XNr5xsXLl66fKW5fvWZzkvFxYjnSa72YqZFIjMxQomJ2CuUYGmciN344N4yv/tSKC3z7CkuCjFJ2TyTM8kZOipa93oUxSs0qJjMYNh/ct/CrW2g+lChobpMI0MfpWLOLBgwnX5k5AsLt8EMbQU3nm9aoLmb4QrgCKpqB+xqbYMxQI9g8KNPp+/kww1HRg8sfJdWA2u1ta4lbdTOhMa/GWv/n7P2n621/8Vb25lrRM1W0A1WASdBWIMWqWMQNT/Tac7LVGTIE6b1OAwKnBimUPJE2AYttSgYP2BzMXYwY6nQE7N6Xgs3HTOFWa7czhBW7M8Kw1KtF2nsKlOG+/p4bkn+LjcucbY1MTIrShQZrwbNygQwh+VfgalUgmOycIBxJZ1X4PtMMY7uRy0vITx+5JNgtNm92w0f91o7vfo21sh1coN0SEjukB3ykAzIiHDvtffWe+998N/47/yP/qeq1PdqzTXyS/hfvgGzoveX</latexit>
P⌦(M) =
⇢
Mij if (i, j) 2 ⌦,
0 otherwise.<latexit sha1_base64="h8sB+vFxh04Y/hYuefLrNWXXg2U=">AAACd3icbVFNa9wwEJXdr3T7tUmPuQzdJmwgGDsUmh4CgV56Cd1AtwmsFiNrx14lsmwkue0i/Bv633rrD+mlp8q7JrRJBwSPN+89STNZLYWxcfwzCO/df/Dw0dbjwZOnz56/GG7vfDZVozlOeSUrfZkxg1IonFphJV7WGlmZSbzIrt93/YsvqI2o1Ce7qnFeskKJXHBmPZUOv09SRz+WWLAWxmcHcAJUYm6pGwDQDAuhHNOarVonZes5gLPUiasWYB9omVXfnMjBW8UhXB0AFQo2aYdA6Voe3wgru0T9VRiMuiCKatFHD6gWxdJGnSUdjuIoXhfcBUkPRqSvSTr8QRcVb0pUlktmzCyJazv3yVZwiT67MVgzfs0KnHmoWIlm7taDa2HPMwvIK+2PsrBm/3Y4VhqzKjOvLJldmtu9jvxfb9bY/HjuhKobi4pvLsobCbaCbguwEBq5lSsPGNfCvxX4kmnGrd9VN4Tk9pfvgulR9C5Kzt+MTo/6aWyRXfKKjElC3pJT8oFMyJRw8ivYDV4He8HvEML9cLyRhkHveUn+qTD5A4QtuSk=</latexit><latexit sha1_base64="h8sB+vFxh04Y/hYuefLrNWXXg2U=">AAACd3icbVFNa9wwEJXdr3T7tUmPuQzdJmwgGDsUmh4CgV56Cd1AtwmsFiNrx14lsmwkue0i/Bv633rrD+mlp8q7JrRJBwSPN+89STNZLYWxcfwzCO/df/Dw0dbjwZOnz56/GG7vfDZVozlOeSUrfZkxg1IonFphJV7WGlmZSbzIrt93/YsvqI2o1Ce7qnFeskKJXHBmPZUOv09SRz+WWLAWxmcHcAJUYm6pGwDQDAuhHNOarVonZes5gLPUiasWYB9omVXfnMjBW8UhXB0AFQo2aYdA6Voe3wgru0T9VRiMuiCKatFHD6gWxdJGnSUdjuIoXhfcBUkPRqSvSTr8QRcVb0pUlktmzCyJazv3yVZwiT67MVgzfs0KnHmoWIlm7taDa2HPMwvIK+2PsrBm/3Y4VhqzKjOvLJldmtu9jvxfb9bY/HjuhKobi4pvLsobCbaCbguwEBq5lSsPGNfCvxX4kmnGrd9VN4Tk9pfvgulR9C5Kzt+MTo/6aWyRXfKKjElC3pJT8oFMyJRw8ivYDV4He8HvEML9cLyRhkHveUn+qTD5A4QtuSk=</latexit><latexit sha1_base64="h8sB+vFxh04Y/hYuefLrNWXXg2U=">AAACd3icbVFNa9wwEJXdr3T7tUmPuQzdJmwgGDsUmh4CgV56Cd1AtwmsFiNrx14lsmwkue0i/Bv633rrD+mlp8q7JrRJBwSPN+89STNZLYWxcfwzCO/df/Dw0dbjwZOnz56/GG7vfDZVozlOeSUrfZkxg1IonFphJV7WGlmZSbzIrt93/YsvqI2o1Ce7qnFeskKJXHBmPZUOv09SRz+WWLAWxmcHcAJUYm6pGwDQDAuhHNOarVonZes5gLPUiasWYB9omVXfnMjBW8UhXB0AFQo2aYdA6Voe3wgru0T9VRiMuiCKatFHD6gWxdJGnSUdjuIoXhfcBUkPRqSvSTr8QRcVb0pUlktmzCyJazv3yVZwiT67MVgzfs0KnHmoWIlm7taDa2HPMwvIK+2PsrBm/3Y4VhqzKjOvLJldmtu9jvxfb9bY/HjuhKobi4pvLsobCbaCbguwEBq5lSsPGNfCvxX4kmnGrd9VN4Tk9pfvgulR9C5Kzt+MTo/6aWyRXfKKjElC3pJT8oFMyJRw8ivYDV4He8HvEML9cLyRhkHveUn+qTD5A4QtuSk=</latexit><latexit sha1_base64="h8sB+vFxh04Y/hYuefLrNWXXg2U=">AAACd3icbVFNa9wwEJXdr3T7tUmPuQzdJmwgGDsUmh4CgV56Cd1AtwmsFiNrx14lsmwkue0i/Bv633rrD+mlp8q7JrRJBwSPN+89STNZLYWxcfwzCO/df/Dw0dbjwZOnz56/GG7vfDZVozlOeSUrfZkxg1IonFphJV7WGlmZSbzIrt93/YsvqI2o1Ce7qnFeskKJXHBmPZUOv09SRz+WWLAWxmcHcAJUYm6pGwDQDAuhHNOarVonZes5gLPUiasWYB9omVXfnMjBW8UhXB0AFQo2aYdA6Voe3wgru0T9VRiMuiCKatFHD6gWxdJGnSUdjuIoXhfcBUkPRqSvSTr8QRcVb0pUlktmzCyJazv3yVZwiT67MVgzfs0KnHmoWIlm7taDa2HPMwvIK+2PsrBm/3Y4VhqzKjOvLJldmtu9jvxfb9bY/HjuhKobi4pvLsobCbaCbguwEBq5lSsPGNfCvxX4kmnGrd9VN4Tk9pfvgulR9C5Kzt+MTo/6aWyRXfKKjElC3pJT8oFMyJRw8ivYDV4He8HvEML9cLyRhkHveUn+qTD5A4QtuSk=</latexit>
M, R 2 Rm⇥n
<latexit sha1_base64="9fnhV6oxrlHnJGzB7Tuj95171cQ=">AAACCHicbVBNS8NAEN34WetX1KOXxSJ4kJKIoN4KXrwItRhbaGLZbDft0t1N2N0IJeTqxb/ixYOKV3+CN/+NmzYHbX0w8Hhvhpl5YcKo0o7zbS0sLi2vrFbWqusbm1vb9s7unYpTiYmHYxbLTogUYVQQT1PNSCeRBPGQkXY4uiz89gORisbiVo8TEnA0EDSiGGkj9Wx4fQxb0KcC+hzpYRhmrfw+49DXlBMFRd6za07dmQDOE7ckNVCi2bO//H6MU06Exgwp1XWdRAcZkppiRvKqnyqSIDxCA9I1VCCzJ8gmn+Tw0Ch9GMXSlNBwov6eyBBXasxD01mcq2a9QvzP66Y6Og8yKpJUE4Gni6KUQR3DIhbYp5JgzcaGICypuRXiIZIIaxNe1YTgzr48T7yT+kXdvTmtNZwyjQrYBwfgCLjgDDTAFWgCD2DwCJ7BK3iznqwX6936mLYuWOXMHvgD6/MHZxCZDg==</latexit><latexit sha1_base64="9fnhV6oxrlHnJGzB7Tuj95171cQ=">AAACCHicbVBNS8NAEN34WetX1KOXxSJ4kJKIoN4KXrwItRhbaGLZbDft0t1N2N0IJeTqxb/ixYOKV3+CN/+NmzYHbX0w8Hhvhpl5YcKo0o7zbS0sLi2vrFbWqusbm1vb9s7unYpTiYmHYxbLTogUYVQQT1PNSCeRBPGQkXY4uiz89gORisbiVo8TEnA0EDSiGGkj9Wx4fQxb0KcC+hzpYRhmrfw+49DXlBMFRd6za07dmQDOE7ckNVCi2bO//H6MU06Exgwp1XWdRAcZkppiRvKqnyqSIDxCA9I1VCCzJ8gmn+Tw0Ch9GMXSlNBwov6eyBBXasxD01mcq2a9QvzP66Y6Og8yKpJUE4Gni6KUQR3DIhbYp5JgzcaGICypuRXiIZIIaxNe1YTgzr48T7yT+kXdvTmtNZwyjQrYBwfgCLjgDDTAFWgCD2DwCJ7BK3iznqwX6936mLYuWOXMHvgD6/MHZxCZDg==</latexit><latexit sha1_base64="9fnhV6oxrlHnJGzB7Tuj95171cQ=">AAACCHicbVBNS8NAEN34WetX1KOXxSJ4kJKIoN4KXrwItRhbaGLZbDft0t1N2N0IJeTqxb/ixYOKV3+CN/+NmzYHbX0w8Hhvhpl5YcKo0o7zbS0sLi2vrFbWqusbm1vb9s7unYpTiYmHYxbLTogUYVQQT1PNSCeRBPGQkXY4uiz89gORisbiVo8TEnA0EDSiGGkj9Wx4fQxb0KcC+hzpYRhmrfw+49DXlBMFRd6za07dmQDOE7ckNVCi2bO//H6MU06Exgwp1XWdRAcZkppiRvKqnyqSIDxCA9I1VCCzJ8gmn+Tw0Ch9GMXSlNBwov6eyBBXasxD01mcq2a9QvzP66Y6Og8yKpJUE4Gni6KUQR3DIhbYp5JgzcaGICypuRXiIZIIaxNe1YTgzr48T7yT+kXdvTmtNZwyjQrYBwfgCLjgDDTAFWgCD2DwCJ7BK3iznqwX6936mLYuWOXMHvgD6/MHZxCZDg==</latexit><latexit sha1_base64="9fnhV6oxrlHnJGzB7Tuj95171cQ=">AAACCHicbVBNS8NAEN34WetX1KOXxSJ4kJKIoN4KXrwItRhbaGLZbDft0t1N2N0IJeTqxb/ixYOKV3+CN/+NmzYHbX0w8Hhvhpl5YcKo0o7zbS0sLi2vrFbWqusbm1vb9s7unYpTiYmHYxbLTogUYVQQT1PNSCeRBPGQkXY4uiz89gORisbiVo8TEnA0EDSiGGkj9Wx4fQxb0KcC+hzpYRhmrfw+49DXlBMFRd6za07dmQDOE7ckNVCi2bO//H6MU06Exgwp1XWdRAcZkppiRvKqnyqSIDxCA9I1VCCzJ8gmn+Tw0Ch9GMXSlNBwov6eyBBXasxD01mcq2a9QvzP66Y6Og8yKpJUE4Gni6KUQR3DIhbYp5JgzcaGICypuRXiIZIIaxNe1YTgzr48T7yT+kXdvTmtNZwyjQrYBwfgCLjgDDTAFWgCD2DwCJ7BK3iznqwX6936mLYuWOXMHvgD6/MHZxCZDg==</latexit>

RMSE on Test Set
RMSE OF NETFLIX, PRIZE WINNER AND BIASED-MF
• RMSE of Cinematch (Original Netflix Recommendation Engine): 0.9514
• RMSE of Winner: 0.8553 (10% enhancement)
• They ensembled a lot of algorithms.
• RMSE of Biased-MF ONLY: 0.8799 (7.5% enhancement)
• Some materials said 6% enhancement only by Biased-MF.
• RMSE of Biased-MF(with tuned hyper-parameters): 0.844 (11.3% enhancement)
• From the AutoRec Paper in WWW’15.
!7

BIASED-MF IS FAST AND WORKS WELL
• Many Algorithms are presented after Biased-MF.
• Biased-MF is the fastest algorithm among them.
• The convex relaxation(nuclear norm minimization) algorithm is slower than Biased-MF.
• Biased-MF works quite well even these days.
• And, the some of algorithms with good performance are based on Biased-MF.
• SMA and MRMA are ensemble algorithms based on Biased-MF.
• LLoRMA and ABCF are modification of Biased-MF.
!8
Movielens 1M Movielens 10M Netflix(100M)
Biased MF(IEEE’09) 0.845 0.803 0.844
LLoRMA(ICML’13) 0.8333 0.7815 0.8337
AutoRec(WWW’15) 0.831 0.782 0.823
CF-NADE(ICML ’16) 0.829 0.771 0.803
SMA(ICML’16) - 0.7682 0.8036
ABCF(Neurocomputing’18) 0.836 0.766 0.795
MRMA(NIPS’17) - 0.7634 0.7973

FUNK SVD(SINGULAR VECTOR DECOMPOSITION)
• Before introducing Biased-MF, Let’s see Funk SVD(suggested by Funk).
• Goal of Netflix Prize: Minimize RMSE
• There is no way to solve this optimization form without any assumption.
• In rating recommendation, it is natural to assume the matrix is low-rank.
• New Optimization Form with Low-rank Assumption:
• p and q are low-rank matrices:
• To avoid overfitting:
!10
ˆrui<latexit sha1_base64="ExB9hem06ilaSw+uGS5343B2V7A=">AAAB8nicbVDLSgNBEOyNrxhfUY9eBoPgKeyKoN4CXjxGcE0gu4TZySQZMvtgpkcIy/6GFw8qXv0ab/6Nk2QPmljQUFR1090VZVJodN1vp7K2vrG5Vd2u7ezu7R/UD48edWoU4z5LZaq6EdVcioT7KFDybqY4jSPJO9HkduZ3nrjSIk0ecJrxMKajRAwFo2ilIBhTzFXRz40o+vWG23TnIKvEK0kDSrT79a9gkDIT8wSZpFr3PDfDMKcKBZO8qAVG84yyCR3xnqUJjbkO8/nNBTmzyoAMU2UrQTJXf0/kNNZ6Gke2M6Y41sveTPzP6xkcXoe5SDKDPGGLRUMjCaZkFgAZCMUZyqkllClhbyVsTBVlaGOq2RC85ZdXiX/RvGl695eNllumUYUTOIVz8OAKWnAHbfCBQQbP8ApvjnFenHfnY9FaccqZY/gD5/MHL/KR7A==</latexit><latexit sha1_base64="ExB9hem06ilaSw+uGS5343B2V7A=">AAAB8nicbVDLSgNBEOyNrxhfUY9eBoPgKeyKoN4CXjxGcE0gu4TZySQZMvtgpkcIy/6GFw8qXv0ab/6Nk2QPmljQUFR1090VZVJodN1vp7K2vrG5Vd2u7ezu7R/UD48edWoU4z5LZaq6EdVcioT7KFDybqY4jSPJO9HkduZ3nrjSIk0ecJrxMKajRAwFo2ilIBhTzFXRz40o+vWG23TnIKvEK0kDSrT79a9gkDIT8wSZpFr3PDfDMKcKBZO8qAVG84yyCR3xnqUJjbkO8/nNBTmzyoAMU2UrQTJXf0/kNNZ6Gke2M6Y41sveTPzP6xkcXoe5SDKDPGGLRUMjCaZkFgAZCMUZyqkllClhbyVsTBVlaGOq2RC85ZdXiX/RvGl695eNllumUYUTOIVz8OAKWnAHbfCBQQbP8ApvjnFenHfnY9FaccqZY/gD5/MHL/KR7A==</latexit><latexit sha1_base64="ExB9hem06ilaSw+uGS5343B2V7A=">AAAB8nicbVDLSgNBEOyNrxhfUY9eBoPgKeyKoN4CXjxGcE0gu4TZySQZMvtgpkcIy/6GFw8qXv0ab/6Nk2QPmljQUFR1090VZVJodN1vp7K2vrG5Vd2u7ezu7R/UD48edWoU4z5LZaq6EdVcioT7KFDybqY4jSPJO9HkduZ3nrjSIk0ecJrxMKajRAwFo2ilIBhTzFXRz40o+vWG23TnIKvEK0kDSrT79a9gkDIT8wSZpFr3PDfDMKcKBZO8qAVG84yyCR3xnqUJjbkO8/nNBTmzyoAMU2UrQTJXf0/kNNZ6Gke2M6Y41sveTPzP6xkcXoe5SDKDPGGLRUMjCaZkFgAZCMUZyqkllClhbyVsTBVlaGOq2RC85ZdXiX/RvGl695eNllumUYUTOIVz8OAKWnAHbfCBQQbP8ApvjnFenHfnY9FaccqZY/gD5/MHL/KR7A==</latexit><latexit sha1_base64="ExB9hem06ilaSw+uGS5343B2V7A=">AAAB8nicbVDLSgNBEOyNrxhfUY9eBoPgKeyKoN4CXjxGcE0gu4TZySQZMvtgpkcIy/6GFw8qXv0ab/6Nk2QPmljQUFR1090VZVJodN1vp7K2vrG5Vd2u7ezu7R/UD48edWoU4z5LZaq6EdVcioT7KFDybqY4jSPJO9HkduZ3nrjSIk0ecJrxMKajRAwFo2ilIBhTzFXRz40o+vWG23TnIKvEK0kDSrT79a9gkDIT8wSZpFr3PDfDMKcKBZO8qAVG84yyCR3xnqUJjbkO8/nNBTmzyoAMU2UrQTJXf0/kNNZ6Gke2M6Y41sveTPzP6xkcXoe5SDKDPGGLRUMjCaZkFgAZCMUZyqkllClhbyVsTBVlaGOq2RC85ZdXiX/RvGl695eNllumUYUTOIVz8OAKWnAHbfCBQQbP8ApvjnFenHfnY9FaccqZY/gD5/MHL/KR7A==</latexit>
arg min
R
v
u
u
t
X
(i,j)2⌦
(Mij Rij)2
|⌦|
= arg min
R
X
(i,j)2⌦
(Mij Rij)2
<latexit sha1_base64="CW8tcQThzWoM9yMmC0TflQgxgI4=">AAACknicdVHRTtswFHUyxrpubGFoT7xcUW0qEquSahqbpmlFvPAAgqEVkJoSOa5bTG0nsx2kyuSH+Bze+BvcNg8UxpUsH597zr32dZpzpk0Y3nn+i6WXy69qr+tv3q68ex+sfjjRWaEI7ZKMZ+osxZpyJmnXMMPpWa4oFimnp+l4d5o/vaJKs0z+NZOc9gUeSTZkBBtHJcHNZ4ixGkEsmEzscQmx/qeMdVshEttkW3C5CTGTEB8KOsIlWLDNg8SyyxIAvoCzzA+b521nzlwvJ7mu5HBdgtOV8GuhC5TP118sf/ygeBI0wlY4C3gKogo0UBVHSXAbDzJSCCoN4VjrXhTmpm+xMoxwWtbjQtMckzEe0Z6DEguq+3Y20xI+OWYAw0y5JQ3M2IcOi4XWE5E6pcDmQj/OTcn/5XqFGX7vWybzwlBJ5o2GBQeTwfSDYMAUJYZPHMBEMXdXIBdYYWLcN9bdEKLHT34Kuu3Wj1b052uj066mUUPraAM1UYS2UQftoSPURcQLvG/eb6/jf/R/+jv+7lzqe5VnDS2Ev38P3HjB6g==</latexit><latexit sha1_base64="CW8tcQThzWoM9yMmC0TflQgxgI4=">AAACknicdVHRTtswFHUyxrpubGFoT7xcUW0qEquSahqbpmlFvPAAgqEVkJoSOa5bTG0nsx2kyuSH+Bze+BvcNg8UxpUsH597zr32dZpzpk0Y3nn+i6WXy69qr+tv3q68ex+sfjjRWaEI7ZKMZ+osxZpyJmnXMMPpWa4oFimnp+l4d5o/vaJKs0z+NZOc9gUeSTZkBBtHJcHNZ4ixGkEsmEzscQmx/qeMdVshEttkW3C5CTGTEB8KOsIlWLDNg8SyyxIAvoCzzA+b521nzlwvJ7mu5HBdgtOV8GuhC5TP118sf/ygeBI0wlY4C3gKogo0UBVHSXAbDzJSCCoN4VjrXhTmpm+xMoxwWtbjQtMckzEe0Z6DEguq+3Y20xI+OWYAw0y5JQ3M2IcOi4XWE5E6pcDmQj/OTcn/5XqFGX7vWybzwlBJ5o2GBQeTwfSDYMAUJYZPHMBEMXdXIBdYYWLcN9bdEKLHT34Kuu3Wj1b052uj066mUUPraAM1UYS2UQftoSPURcQLvG/eb6/jf/R/+jv+7lzqe5VnDS2Ev38P3HjB6g==</latexit><latexit sha1_base64="CW8tcQThzWoM9yMmC0TflQgxgI4=">AAACknicdVHRTtswFHUyxrpubGFoT7xcUW0qEquSahqbpmlFvPAAgqEVkJoSOa5bTG0nsx2kyuSH+Bze+BvcNg8UxpUsH597zr32dZpzpk0Y3nn+i6WXy69qr+tv3q68ex+sfjjRWaEI7ZKMZ+osxZpyJmnXMMPpWa4oFimnp+l4d5o/vaJKs0z+NZOc9gUeSTZkBBtHJcHNZ4ixGkEsmEzscQmx/qeMdVshEttkW3C5CTGTEB8KOsIlWLDNg8SyyxIAvoCzzA+b521nzlwvJ7mu5HBdgtOV8GuhC5TP118sf/ygeBI0wlY4C3gKogo0UBVHSXAbDzJSCCoN4VjrXhTmpm+xMoxwWtbjQtMckzEe0Z6DEguq+3Y20xI+OWYAw0y5JQ3M2IcOi4XWE5E6pcDmQj/OTcn/5XqFGX7vWybzwlBJ5o2GBQeTwfSDYMAUJYZPHMBEMXdXIBdYYWLcN9bdEKLHT34Kuu3Wj1b052uj066mUUPraAM1UYS2UQftoSPURcQLvG/eb6/jf/R/+jv+7lzqe5VnDS2Ev38P3HjB6g==</latexit><latexit sha1_base64="CW8tcQThzWoM9yMmC0TflQgxgI4=">AAACknicdVHRTtswFHUyxrpubGFoT7xcUW0qEquSahqbpmlFvPAAgqEVkJoSOa5bTG0nsx2kyuSH+Bze+BvcNg8UxpUsH597zr32dZpzpk0Y3nn+i6WXy69qr+tv3q68ex+sfjjRWaEI7ZKMZ+osxZpyJmnXMMPpWa4oFimnp+l4d5o/vaJKs0z+NZOc9gUeSTZkBBtHJcHNZ4ixGkEsmEzscQmx/qeMdVshEttkW3C5CTGTEB8KOsIlWLDNg8SyyxIAvoCzzA+b521nzlwvJ7mu5HBdgtOV8GuhC5TP118sf/ygeBI0wlY4C3gKogo0UBVHSXAbDzJSCCoN4VjrXhTmpm+xMoxwWtbjQtMckzEe0Z6DEguq+3Y20xI+OWYAw0y5JQ3M2IcOi4XWE5E6pcDmQj/OTcn/5XqFGX7vWybzwlBJ5o2GBQeTwfSDYMAUJYZPHMBEMXdXIBdYYWLcN9bdEKLHT34Kuu3Wj1b052uj066mUUPraAM1UYS2UQftoSPURcQLvG/eb6/jf/R/+jv+7lzqe5VnDS2Ev38P3HjB6g==</latexit>
arg min
Û, ˆV
X
(i,j)2⌦
(Mij
Ûi
ˆV †
j )2
<latexit sha1_base64="sYlUpp/x2K67VgHp7EuvG0Zqvg8=">AAACUnicbVJdaxQxFM2sX3WtdrWPvlxchC3UZaYU1LeCL76IFZy2sNkOd7LZ2WyTzJBkhCXkP0qhD/4RX/qg2e0I2nohcHLOPdzkJGUjhXVp+iPp3bv/4OGjrcf9J9tPn+0Mnr84sXVrGM9ZLWtzVqLlUmieO+EkP2sMR1VKflpefFjrp9+4saLWX92q4VOFlRZzwdBFqhgsKZoKqBK68HSBzudhHzbgJEAAaltV+JHYh+UeUKGBfla8wgDgR/5TKLxYBngDnTNuwx9zsTz3dIZVxU2AvfODAMVgmI7TTcFdkHVgSLo6LgaXdFazVnHtmERrJ1nauKlH4wSTPPRpa3mD7AIrPolQo+J26jeZBHgdmRnMaxOXdrBh/3Z4VNauVBk7FbqFva2tyf9pk9bN30290E3ruGY3g+atBFfDOmCYCcOZk6sIkBkRzwpsgQaZi8/QjyFkt698F+QH4/fj7Mvh8OiwS2OLvCSvyIhk5C05Ih/JMckJI9/JT/IrIclVct2Lv+SmtZd0nl3yT/W2fwM5kLKW</latexit><latexit sha1_base64="sYlUpp/x2K67VgHp7EuvG0Zqvg8=">AAACUnicbVJdaxQxFM2sX3WtdrWPvlxchC3UZaYU1LeCL76IFZy2sNkOd7LZ2WyTzJBkhCXkP0qhD/4RX/qg2e0I2nohcHLOPdzkJGUjhXVp+iPp3bv/4OGjrcf9J9tPn+0Mnr84sXVrGM9ZLWtzVqLlUmieO+EkP2sMR1VKflpefFjrp9+4saLWX92q4VOFlRZzwdBFqhgsKZoKqBK68HSBzudhHzbgJEAAaltV+JHYh+UeUKGBfla8wgDgR/5TKLxYBngDnTNuwx9zsTz3dIZVxU2AvfODAMVgmI7TTcFdkHVgSLo6LgaXdFazVnHtmERrJ1nauKlH4wSTPPRpa3mD7AIrPolQo+J26jeZBHgdmRnMaxOXdrBh/3Z4VNauVBk7FbqFva2tyf9pk9bN30290E3ruGY3g+atBFfDOmCYCcOZk6sIkBkRzwpsgQaZi8/QjyFkt698F+QH4/fj7Mvh8OiwS2OLvCSvyIhk5C05Ih/JMckJI9/JT/IrIclVct2Lv+SmtZd0nl3yT/W2fwM5kLKW</latexit><latexit sha1_base64="sYlUpp/x2K67VgHp7EuvG0Zqvg8=">AAACUnicbVJdaxQxFM2sX3WtdrWPvlxchC3UZaYU1LeCL76IFZy2sNkOd7LZ2WyTzJBkhCXkP0qhD/4RX/qg2e0I2nohcHLOPdzkJGUjhXVp+iPp3bv/4OGjrcf9J9tPn+0Mnr84sXVrGM9ZLWtzVqLlUmieO+EkP2sMR1VKflpefFjrp9+4saLWX92q4VOFlRZzwdBFqhgsKZoKqBK68HSBzudhHzbgJEAAaltV+JHYh+UeUKGBfla8wgDgR/5TKLxYBngDnTNuwx9zsTz3dIZVxU2AvfODAMVgmI7TTcFdkHVgSLo6LgaXdFazVnHtmERrJ1nauKlH4wSTPPRpa3mD7AIrPolQo+J26jeZBHgdmRnMaxOXdrBh/3Z4VNauVBk7FbqFva2tyf9pk9bN30290E3ruGY3g+atBFfDOmCYCcOZk6sIkBkRzwpsgQaZi8/QjyFkt698F+QH4/fj7Mvh8OiwS2OLvCSvyIhk5C05Ih/JMckJI9/JT/IrIclVct2Lv+SmtZd0nl3yT/W2fwM5kLKW</latexit><latexit sha1_base64="sYlUpp/x2K67VgHp7EuvG0Zqvg8=">AAACUnicbVJdaxQxFM2sX3WtdrWPvlxchC3UZaYU1LeCL76IFZy2sNkOd7LZ2WyTzJBkhCXkP0qhD/4RX/qg2e0I2nohcHLOPdzkJGUjhXVp+iPp3bv/4OGjrcf9J9tPn+0Mnr84sXVrGM9ZLWtzVqLlUmieO+EkP2sMR1VKflpefFjrp9+4saLWX92q4VOFlRZzwdBFqhgsKZoKqBK68HSBzudhHzbgJEAAaltV+JHYh+UeUKGBfla8wgDgR/5TKLxYBngDnTNuwx9zsTz3dIZVxU2AvfODAMVgmI7TTcFdkHVgSLo6LgaXdFazVnHtmERrJ1nauKlH4wSTPPRpa3mD7AIrPolQo+J26jeZBHgdmRnMaxOXdrBh/3Z4VNauVBk7FbqFva2tyf9pk9bN30290E3ruGY3g+atBFfDOmCYCcOZk6sIkBkRzwpsgQaZi8/QjyFkt698F+QH4/fj7Mvh8OiwS2OLvCSvyIhk5C05Ih/JMckJI9/JT/IrIclVct2Lv+SmtZd0nl3yT/W2fwM5kLKW</latexit>
arg min
Û, ˆV
X
(i,j)2⌦
(Mij
Ûi
ˆV †
j )2
+ U k Ûk2
F + V k ˆV k2
F
<latexit sha1_base64="QbHbHTDK8iJKyzMIsXFV+BYgY7g=">AAACjnicbVFdixMxFM2MX2tdtauPvlwsQhfXMlMWVlFxQZB9EVdwugtNO6SZdJpukhmSjFBi/o4/yDf/jWl3KuuuFwIn59xDbs6d1YIbmyS/o/jW7Tt37+3c7zzYffjocXfvychUjaYso5Wo9PmMGCa4YpnlVrDzWjMiZ4KdzS4+rvWz70wbXqlvdlWziSSl4nNOiQ1U3v2JiS4BS65yhxfEuswfwAaMPHjAppG56/MDWO4D5grwF8lK4gFc3332ueNLD6+gdYar35rz5dThgpQl0x72p8NgeQlYhMkKkmeAf2xNAeafpsMr6uivOtqqebeXDJJNwU2QtqCH2jrNu79wUdFGMmWpIMaM06S2E0e05VQw38GNYTWhF6Rk4wAVkcxM3CZPDy8CU8C80uEoCxv2qsMRacxKzkKnJHZhrmtr8n/auLHz1xPHVd1YpujlQ/NGgK1gvRwouGbUilUAhGoeZgW6IJpQG1bYCSGk1798E2TDwZtB+vWwd3zYprGDnqHnqI9SdISO0Qk6RRmi0W40jN5G7+K9+Ch+H3+4bI2j1vMU/VPxyR/Vt8QD</latexit><latexit sha1_base64="QbHbHTDK8iJKyzMIsXFV+BYgY7g=">AAACjnicbVFdixMxFM2MX2tdtauPvlwsQhfXMlMWVlFxQZB9EVdwugtNO6SZdJpukhmSjFBi/o4/yDf/jWl3KuuuFwIn59xDbs6d1YIbmyS/o/jW7Tt37+3c7zzYffjocXfvychUjaYso5Wo9PmMGCa4YpnlVrDzWjMiZ4KdzS4+rvWz70wbXqlvdlWziSSl4nNOiQ1U3v2JiS4BS65yhxfEuswfwAaMPHjAppG56/MDWO4D5grwF8lK4gFc3332ueNLD6+gdYar35rz5dThgpQl0x72p8NgeQlYhMkKkmeAf2xNAeafpsMr6uivOtqqebeXDJJNwU2QtqCH2jrNu79wUdFGMmWpIMaM06S2E0e05VQw38GNYTWhF6Rk4wAVkcxM3CZPDy8CU8C80uEoCxv2qsMRacxKzkKnJHZhrmtr8n/auLHz1xPHVd1YpujlQ/NGgK1gvRwouGbUilUAhGoeZgW6IJpQG1bYCSGk1798E2TDwZtB+vWwd3zYprGDnqHnqI9SdISO0Qk6RRmi0W40jN5G7+K9+Ch+H3+4bI2j1vMU/VPxyR/Vt8QD</latexit><latexit sha1_base64="QbHbHTDK8iJKyzMIsXFV+BYgY7g=">AAACjnicbVFdixMxFM2MX2tdtauPvlwsQhfXMlMWVlFxQZB9EVdwugtNO6SZdJpukhmSjFBi/o4/yDf/jWl3KuuuFwIn59xDbs6d1YIbmyS/o/jW7Tt37+3c7zzYffjocXfvychUjaYso5Wo9PmMGCa4YpnlVrDzWjMiZ4KdzS4+rvWz70wbXqlvdlWziSSl4nNOiQ1U3v2JiS4BS65yhxfEuswfwAaMPHjAppG56/MDWO4D5grwF8lK4gFc3332ueNLD6+gdYar35rz5dThgpQl0x72p8NgeQlYhMkKkmeAf2xNAeafpsMr6uivOtqqebeXDJJNwU2QtqCH2jrNu79wUdFGMmWpIMaM06S2E0e05VQw38GNYTWhF6Rk4wAVkcxM3CZPDy8CU8C80uEoCxv2qsMRacxKzkKnJHZhrmtr8n/auLHz1xPHVd1YpujlQ/NGgK1gvRwouGbUilUAhGoeZgW6IJpQG1bYCSGk1798E2TDwZtB+vWwd3zYprGDnqHnqI9SdISO0Qk6RRmi0W40jN5G7+K9+Ch+H3+4bI2j1vMU/VPxyR/Vt8QD</latexit><latexit sha1_base64="QbHbHTDK8iJKyzMIsXFV+BYgY7g=">AAACjnicbVFdixMxFM2MX2tdtauPvlwsQhfXMlMWVlFxQZB9EVdwugtNO6SZdJpukhmSjFBi/o4/yDf/jWl3KuuuFwIn59xDbs6d1YIbmyS/o/jW7Tt37+3c7zzYffjocXfvychUjaYso5Wo9PmMGCa4YpnlVrDzWjMiZ4KdzS4+rvWz70wbXqlvdlWziSSl4nNOiQ1U3v2JiS4BS65yhxfEuswfwAaMPHjAppG56/MDWO4D5grwF8lK4gFc3332ueNLD6+gdYar35rz5dThgpQl0x72p8NgeQlYhMkKkmeAf2xNAeafpsMr6uivOtqqebeXDJJNwU2QtqCH2jrNu79wUdFGMmWpIMaM06S2E0e05VQw38GNYTWhF6Rk4wAVkcxM3CZPDy8CU8C80uEoCxv2qsMRacxKzkKnJHZhrmtr8n/auLHz1xPHVd1YpujlQ/NGgK1gvRwouGbUilUAhGoeZgW6IJpQG1bYCSGk1798E2TDwZtB+vWwd3zYprGDnqHnqI9SdISO0Qk6RRmi0W40jN5G7+K9+Ch+H3+4bI2j1vMU/VPxyR/Vt8QD</latexit>
max(rank( Û), rank( ˆV )) ⌧ dim(M)<latexit sha1_base64="fXL2aN8MX5rq1Tyzyywy1m+UmlE=">AAACJHicbVA9SwNBEN3z2/gVtbRZDEICEu5EUMFCsLERFDwVciHMbTZmye7esTsnCcf9GRv/io2FioWNv8VNTKHGBwOP92aYmRenUlj0/Q9vanpmdm5+YbG0tLyyulZe37i2SWYYD1kiE3Mbg+VSaB6iQMlvU8NBxZLfxL3ToX9zz40Vib7CQcqbCu606AgG6KRW+ThS0K9GyPuYG9C9ohp1AfOwqO3SCfW6qNVoJCWN2kJVz2utcsWv+yPQSRKMSYWMcdEqv0bthGWKa2QSrG0EforNHAwKJnlRijLLU2A9uOMNRzUobpv56MuC7jilTTuJcaWRjtSfEzkoawcqdp0KsGv/ekPxP6+RYeewmQudZsg1+17UySTFhA4jo21hOEM5cASYEe5WyrpggKELtuRCCP6+PEnCvfpRPbjcr5z44zQWyBbZJlUSkANyQs7IBQkJIw/kibyQV+/Re/bevPfv1ilvPLNJfsH7/AJcoaSl</latexit><latexit sha1_base64="fXL2aN8MX5rq1Tyzyywy1m+UmlE=">AAACJHicbVA9SwNBEN3z2/gVtbRZDEICEu5EUMFCsLERFDwVciHMbTZmye7esTsnCcf9GRv/io2FioWNv8VNTKHGBwOP92aYmRenUlj0/Q9vanpmdm5+YbG0tLyyulZe37i2SWYYD1kiE3Mbg+VSaB6iQMlvU8NBxZLfxL3ToX9zz40Vib7CQcqbCu606AgG6KRW+ThS0K9GyPuYG9C9ohp1AfOwqO3SCfW6qNVoJCWN2kJVz2utcsWv+yPQSRKMSYWMcdEqv0bthGWKa2QSrG0EforNHAwKJnlRijLLU2A9uOMNRzUobpv56MuC7jilTTuJcaWRjtSfEzkoawcqdp0KsGv/ekPxP6+RYeewmQudZsg1+17UySTFhA4jo21hOEM5cASYEe5WyrpggKELtuRCCP6+PEnCvfpRPbjcr5z44zQWyBbZJlUSkANyQs7IBQkJIw/kibyQV+/Re/bevPfv1ilvPLNJfsH7/AJcoaSl</latexit><latexit sha1_base64="fXL2aN8MX5rq1Tyzyywy1m+UmlE=">AAACJHicbVA9SwNBEN3z2/gVtbRZDEICEu5EUMFCsLERFDwVciHMbTZmye7esTsnCcf9GRv/io2FioWNv8VNTKHGBwOP92aYmRenUlj0/Q9vanpmdm5+YbG0tLyyulZe37i2SWYYD1kiE3Mbg+VSaB6iQMlvU8NBxZLfxL3ToX9zz40Vib7CQcqbCu606AgG6KRW+ThS0K9GyPuYG9C9ohp1AfOwqO3SCfW6qNVoJCWN2kJVz2utcsWv+yPQSRKMSYWMcdEqv0bthGWKa2QSrG0EforNHAwKJnlRijLLU2A9uOMNRzUobpv56MuC7jilTTuJcaWRjtSfEzkoawcqdp0KsGv/ekPxP6+RYeewmQudZsg1+17UySTFhA4jo21hOEM5cASYEe5WyrpggKELtuRCCP6+PEnCvfpRPbjcr5z44zQWyBbZJlUSkANyQs7IBQkJIw/kibyQV+/Re/bevPfv1ilvPLNJfsH7/AJcoaSl</latexit><latexit sha1_base64="fXL2aN8MX5rq1Tyzyywy1m+UmlE=">AAACJHicbVA9SwNBEN3z2/gVtbRZDEICEu5EUMFCsLERFDwVciHMbTZmye7esTsnCcf9GRv/io2FioWNv8VNTKHGBwOP92aYmRenUlj0/Q9vanpmdm5+YbG0tLyyulZe37i2SWYYD1kiE3Mbg+VSaB6iQMlvU8NBxZLfxL3ToX9zz40Vib7CQcqbCu606AgG6KRW+ThS0K9GyPuYG9C9ohp1AfOwqO3SCfW6qNVoJCWN2kJVz2utcsWv+yPQSRKMSYWMcdEqv0bthGWKa2QSrG0EforNHAwKJnlRijLLU2A9uOMNRzUobpv56MuC7jilTTuJcaWRjtSfEzkoawcqdp0KsGv/ekPxP6+RYeewmQudZsg1+17UySTFhA4jo21hOEM5cASYEe5WyrpggKELtuRCCP6+PEnCvfpRPbjcr5z44zQWyBbZJlUSkANyQs7IBQkJIw/kibyQV+/Re/bevPfv1ilvPLNJfsH7/AJcoaSl</latexit>

SOLUTION OF THE FUNK SVD
• Optimization form of the Funk SVD:
• If we know all entries (= contains all (u, i) pairs), we can solve by SVD (Singular vector
decomposition).
• However, we can’t know the all entries.
• If we don’t know all entries, this is NP-hard problem.
• Then, how to solve this optimization problem?
• One of approaches is convex relaxation.
• But, the Funk-SVD use another algorithm.
!11
⌦<latexit sha1_base64="CXYjJQmltPnYcikH2EiL4mSI51A=">AAAB7HicbVDLSgNBEOyNrxhfUY9eBoPgKewGQb0FvHgzgnlAsoTZyWwyZh7LzKwQlvyDFw8qXv0gb/6Nk2QPmljQUFR1090VJZwZ6/vfXmFtfWNzq7hd2tnd2z8oHx61jEo1oU2iuNKdCBvKmaRNyyynnURTLCJO29H4Zua3n6g2TMkHO0loKPBQspgRbJ3U6t0JOsT9csWv+nOgVRLkpAI5Gv3yV2+gSCqotIRjY7qBn9gww9oywum01EsNTTAZ4yHtOiqxoCbM5tdO0ZlTBihW2pW0aK7+nsiwMGYiItcpsB2ZZW8m/ud1UxtfhRmTSWqpJItFccqRVWj2OhowTYnlE0cw0czdisgIa0ysC6jkQgiWX14lzVr1uhrcX1TqtTyNIpzAKZxDAJdQh1toQBMIPMIzvMKbp7wX7937WLQWvHzmGP7A+/wBxuaOvw==</latexit><latexit sha1_base64="CXYjJQmltPnYcikH2EiL4mSI51A=">AAAB7HicbVDLSgNBEOyNrxhfUY9eBoPgKewGQb0FvHgzgnlAsoTZyWwyZh7LzKwQlvyDFw8qXv0gb/6Nk2QPmljQUFR1090VJZwZ6/vfXmFtfWNzq7hd2tnd2z8oHx61jEo1oU2iuNKdCBvKmaRNyyynnURTLCJO29H4Zua3n6g2TMkHO0loKPBQspgRbJ3U6t0JOsT9csWv+nOgVRLkpAI5Gv3yV2+gSCqotIRjY7qBn9gww9oywum01EsNTTAZ4yHtOiqxoCbM5tdO0ZlTBihW2pW0aK7+nsiwMGYiItcpsB2ZZW8m/ud1UxtfhRmTSWqpJItFccqRVWj2OhowTYnlE0cw0czdisgIa0ysC6jkQgiWX14lzVr1uhrcX1TqtTyNIpzAKZxDAJdQh1toQBMIPMIzvMKbp7wX7937WLQWvHzmGP7A+/wBxuaOvw==</latexit><latexit sha1_base64="CXYjJQmltPnYcikH2EiL4mSI51A=">AAAB7HicbVDLSgNBEOyNrxhfUY9eBoPgKewGQb0FvHgzgnlAsoTZyWwyZh7LzKwQlvyDFw8qXv0gb/6Nk2QPmljQUFR1090VJZwZ6/vfXmFtfWNzq7hd2tnd2z8oHx61jEo1oU2iuNKdCBvKmaRNyyynnURTLCJO29H4Zua3n6g2TMkHO0loKPBQspgRbJ3U6t0JOsT9csWv+nOgVRLkpAI5Gv3yV2+gSCqotIRjY7qBn9gww9oywum01EsNTTAZ4yHtOiqxoCbM5tdO0ZlTBihW2pW0aK7+nsiwMGYiItcpsB2ZZW8m/ud1UxtfhRmTSWqpJItFccqRVWj2OhowTYnlE0cw0czdisgIa0ysC6jkQgiWX14lzVr1uhrcX1TqtTyNIpzAKZxDAJdQh1toQBMIPMIzvMKbp7wX7937WLQWvHzmGP7A+/wBxuaOvw==</latexit><latexit sha1_base64="CXYjJQmltPnYcikH2EiL4mSI51A=">AAAB7HicbVDLSgNBEOyNrxhfUY9eBoPgKewGQb0FvHgzgnlAsoTZyWwyZh7LzKwQlvyDFw8qXv0gb/6Nk2QPmljQUFR1090VJZwZ6/vfXmFtfWNzq7hd2tnd2z8oHx61jEo1oU2iuNKdCBvKmaRNyyynnURTLCJO29H4Zua3n6g2TMkHO0loKPBQspgRbJ3U6t0JOsT9csWv+nOgVRLkpAI5Gv3yV2+gSCqotIRjY7qBn9gww9oywum01EsNTTAZ4yHtOiqxoCbM5tdO0ZlTBihW2pW0aK7+nsiwMGYiItcpsB2ZZW8m/ud1UxtfhRmTSWqpJItFccqRVWj2OhowTYnlE0cw0czdisgIa0ysC6jkQgiWX14lzVr1uhrcX1TqtTyNIpzAKZxDAJdQh1toQBMIPMIzvMKbp7wX7937WLQWvHzmGP7A+/wBxuaOvw==</latexit>
arg min
Û, ˆV
X
(i,j)2⌦
(Mij
Ûi
ˆV †
j )2
+ U k Ûk2
F + V k ˆV k2
F

SOLVE BY ALTERNATING MINIMIZATION
!12
M ≒ xm m
n
n
k
k
These can be solved by SVD!
(pseudo inverse)
Û<latexit sha1_base64="qEBNw/NF9I5Pysh/o7KQOMhnYcA=">AAAB7XicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEaL0VvHisYGyhDWWz3bRLN5uwOxFK6I/w4kHFq//Hm//GbZuDtj4YeLw3w8y8MJXCoOt+O6WNza3tnfJuZW//4PCoenzyaJJMM+6zRCa6G1LDpVDcR4GSd1PNaRxK3gknt3O/88S1EYl6wGnKg5iOlIgEo2ilTn9MMfdng2rNrbsLkHXiFaQGBdqD6ld/mLAs5gqZpMb0PDfFIKcaBZN8VulnhqeUTeiI9yxVNOYmyBfnzsiFVYYkSrQthWSh/p7IaWzMNA5tZ0xxbFa9ufif18swaga5UGmGXLHloiiTBBMy/50MheYM5dQSyrSwtxI2ppoytAlVbAje6svrxL+q39S9++taq1mkUYYzOIdL8KABLbiDNvjAYALP8ApvTuq8OO/Ox7K15BQzp/AHzucP4+6PcA==</latexit><latexit sha1_base64="qEBNw/NF9I5Pysh/o7KQOMhnYcA=">AAAB7XicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEaL0VvHisYGyhDWWz3bRLN5uwOxFK6I/w4kHFq//Hm//GbZuDtj4YeLw3w8y8MJXCoOt+O6WNza3tnfJuZW//4PCoenzyaJJMM+6zRCa6G1LDpVDcR4GSd1PNaRxK3gknt3O/88S1EYl6wGnKg5iOlIgEo2ilTn9MMfdng2rNrbsLkHXiFaQGBdqD6ld/mLAs5gqZpMb0PDfFIKcaBZN8VulnhqeUTeiI9yxVNOYmyBfnzsiFVYYkSrQthWSh/p7IaWzMNA5tZ0xxbFa9ufif18swaga5UGmGXLHloiiTBBMy/50MheYM5dQSyrSwtxI2ppoytAlVbAje6svrxL+q39S9++taq1mkUYYzOIdL8KABLbiDNvjAYALP8ApvTuq8OO/Ox7K15BQzp/AHzucP4+6PcA==</latexit><latexit sha1_base64="qEBNw/NF9I5Pysh/o7KQOMhnYcA=">AAAB7XicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEaL0VvHisYGyhDWWz3bRLN5uwOxFK6I/w4kHFq//Hm//GbZuDtj4YeLw3w8y8MJXCoOt+O6WNza3tnfJuZW//4PCoenzyaJJMM+6zRCa6G1LDpVDcR4GSd1PNaRxK3gknt3O/88S1EYl6wGnKg5iOlIgEo2ilTn9MMfdng2rNrbsLkHXiFaQGBdqD6ld/mLAs5gqZpMb0PDfFIKcaBZN8VulnhqeUTeiI9yxVNOYmyBfnzsiFVYYkSrQthWSh/p7IaWzMNA5tZ0xxbFa9ufif18swaga5UGmGXLHloiiTBBMy/50MheYM5dQSyrSwtxI2ppoytAlVbAje6svrxL+q39S9++taq1mkUYYzOIdL8KABLbiDNvjAYALP8ApvTuq8OO/Ox7K15BQzp/AHzucP4+6PcA==</latexit><latexit sha1_base64="qEBNw/NF9I5Pysh/o7KQOMhnYcA=">AAAB7XicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEaL0VvHisYGyhDWWz3bRLN5uwOxFK6I/w4kHFq//Hm//GbZuDtj4YeLw3w8y8MJXCoOt+O6WNza3tnfJuZW//4PCoenzyaJJMM+6zRCa6G1LDpVDcR4GSd1PNaRxK3gknt3O/88S1EYl6wGnKg5iOlIgEo2ilTn9MMfdng2rNrbsLkHXiFaQGBdqD6ld/mLAs5gqZpMb0PDfFIKcaBZN8VulnhqeUTeiI9yxVNOYmyBfnzsiFVYYkSrQthWSh/p7IaWzMNA5tZ0xxbFa9ufif18swaga5UGmGXLHloiiTBBMy/50MheYM5dQSyrSwtxI2ppoytAlVbAje6svrxL+q39S9++taq1mkUYYzOIdL8KABLbiDNvjAYALP8ApvTuq8OO/Ox7K15BQzp/AHzucP4+6PcA==</latexit>
ˆV †
<latexit sha1_base64="gToXFTlZVnuHTXJsUTeaDo+Bsko=">AAAB93icbVBNS8NAEN34WetHox69LBbBU0lEsN4KXjxWMG2hjWWz2aRLN5uwOxFq6C/x4kHFq3/Fm//GbZuDtj4YeLw3w8y8IBNcg+N8W2vrG5tb25Wd6u7e/kHNPjzq6DRXlHk0FanqBUQzwSXzgINgvUwxkgSCdYPxzczvPjKleSrvYZIxPyGx5BGnBIw0tGuDEYGiM30YhCSOmRradafhzIFXiVuSOirRHtpfgzClecIkUEG07rtOBn5BFHAq2LQ6yDXLCB2TmPUNlSRh2i/mh0/xmVFCHKXKlAQ8V39PFCTRepIEpjMhMNLL3kz8z+vnEDX9gsssBybpYlGUCwwpnqWAQ64YBTExhFDFza2YjogiFExWVROCu/zyKvEuGtcN9+6y3mqWaVTQCTpF58hFV6iFblEbeYiiHD2jV/RmPVkv1rv1sWhds8qZY/QH1ucPZl+TFg==</latexit><latexit sha1_base64="gToXFTlZVnuHTXJsUTeaDo+Bsko=">AAAB93icbVBNS8NAEN34WetHox69LBbBU0lEsN4KXjxWMG2hjWWz2aRLN5uwOxFq6C/x4kHFq3/Fm//GbZuDtj4YeLw3w8y8IBNcg+N8W2vrG5tb25Wd6u7e/kHNPjzq6DRXlHk0FanqBUQzwSXzgINgvUwxkgSCdYPxzczvPjKleSrvYZIxPyGx5BGnBIw0tGuDEYGiM30YhCSOmRradafhzIFXiVuSOirRHtpfgzClecIkUEG07rtOBn5BFHAq2LQ6yDXLCB2TmPUNlSRh2i/mh0/xmVFCHKXKlAQ8V39PFCTRepIEpjMhMNLL3kz8z+vnEDX9gsssBybpYlGUCwwpnqWAQ64YBTExhFDFza2YjogiFExWVROCu/zyKvEuGtcN9+6y3mqWaVTQCTpF58hFV6iFblEbeYiiHD2jV/RmPVkv1rv1sWhds8qZY/QH1ucPZl+TFg==</latexit><latexit sha1_base64="gToXFTlZVnuHTXJsUTeaDo+Bsko=">AAAB93icbVBNS8NAEN34WetHox69LBbBU0lEsN4KXjxWMG2hjWWz2aRLN5uwOxFq6C/x4kHFq3/Fm//GbZuDtj4YeLw3w8y8IBNcg+N8W2vrG5tb25Wd6u7e/kHNPjzq6DRXlHk0FanqBUQzwSXzgINgvUwxkgSCdYPxzczvPjKleSrvYZIxPyGx5BGnBIw0tGuDEYGiM30YhCSOmRradafhzIFXiVuSOirRHtpfgzClecIkUEG07rtOBn5BFHAq2LQ6yDXLCB2TmPUNlSRh2i/mh0/xmVFCHKXKlAQ8V39PFCTRepIEpjMhMNLL3kz8z+vnEDX9gsssBybpYlGUCwwpnqWAQ64YBTExhFDFza2YjogiFExWVROCu/zyKvEuGtcN9+6y3mqWaVTQCTpF58hFV6iFblEbeYiiHD2jV/RmPVkv1rv1sWhds8qZY/QH1ucPZl+TFg==</latexit><latexit sha1_base64="gToXFTlZVnuHTXJsUTeaDo+Bsko=">AAAB93icbVBNS8NAEN34WetHox69LBbBU0lEsN4KXjxWMG2hjWWz2aRLN5uwOxFq6C/x4kHFq3/Fm//GbZuDtj4YeLw3w8y8IBNcg+N8W2vrG5tb25Wd6u7e/kHNPjzq6DRXlHk0FanqBUQzwSXzgINgvUwxkgSCdYPxzczvPjKleSrvYZIxPyGx5BGnBIw0tGuDEYGiM30YhCSOmRradafhzIFXiVuSOirRHtpfgzClecIkUEG07rtOBn5BFHAq2LQ6yDXLCB2TmPUNlSRh2i/mh0/xmVFCHKXKlAQ8V39PFCTRepIEpjMhMNLL3kz8z+vnEDX9gsssBybpYlGUCwwpnqWAQ64YBTExhFDFza2YjogiFExWVROCu/zyKvEuGtcN9+6y3mqWaVTQCTpF58hFV6iFblEbeYiiHD2jV/RmPVkv1rv1sWhds8qZY/QH1ucPZl+TFg==</latexit>
1: Input: observed set ⌦, values P⌦(M)
2: Initialize ˆV 0
randomly.
3: for t = 1, · · · , T:
4: Ût
arg min
U2Rm⇥k
kP⌦(M U( ˆV t 1
)†
)k2
F + U kUk2
F
5: ˆV t
arg min
V 2Rk⇥n
kP⌦(M Ût
V †
)k2
F + V kV k2
F
6: Return R = Ût
( ˆV t
)†
<latexit sha1_base64="pQ+/bsDaW5e0HjtjvbKSiNfXu+o=">AAAEBXicfVNNb9NAEHUdPkr4auEIhxEVKBVtFIfyFalSJSQEh4pS1W6lbmKt15t0lfU62h0XgusTF/4KFw6AuPIfuPFvWDdxCW3F+vI0M2/2vfFsNJLCYKv1e86tXbh46fL8lfrVa9dv3FxYvBWYNNOM+yyVqd6LqOFSKO6jQMn3RprTJJJ8Nxq+KPO7h1wbkaodHI94N6EDJfqCUbShcNG9+4Agf4+514HXapRhB9LIcH3IYzAcoSBvEj6gK5MiOKQy4waKrXASb8AmLAMh9WmXdtlFoKBSfOCWfEAxD4pea0rXVMVpIsfNYobzqGMLS4SY91MNBSCseytA4hRNtAI7MKmzZTOstQ4Q+02u8ItejjYreR+p1uk7IFQPgCRChbkPRCiLKR5EUb5tSxPbUSTWx7CwpCOo3EBjE1bBh0alO8dVr1jukZgOBlxbo0fhy14bHtqb7IhjGvol3a/ihEAl7/GsvOB/8oIz8oaVPHWevMowQlAJg3OUBSUz+KusEvakA9scM63sNLdhfabdiWs8cRwuLLWareMDZ4E3BUvO9GyFC7/sX2NZwhUySY3Z91oj7OZUo2CSF3WSGT6ibEgHfN9CRa3Nbn68xgXct5EYyhXopwrhODrLyGlizDiJbGU5LXM6VwbPy+1n2H/WzUW53lyxyUX9TAKmUL4JiIXmDOXYAsq0XV8G7IBqytC+nLodgnfa8lngt5vPm97btaWN9nQa884d557TcDznqbPhvHK2HN9h7kf3s/vV/Vb7VPtS+177MSl156ac284/p/bzD1A7SFE=</latexit><latexit sha1_base64="pQ+/bsDaW5e0HjtjvbKSiNfXu+o=">AAAEBXicfVNNb9NAEHUdPkr4auEIhxEVKBVtFIfyFalSJSQEh4pS1W6lbmKt15t0lfU62h0XgusTF/4KFw6AuPIfuPFvWDdxCW3F+vI0M2/2vfFsNJLCYKv1e86tXbh46fL8lfrVa9dv3FxYvBWYNNOM+yyVqd6LqOFSKO6jQMn3RprTJJJ8Nxq+KPO7h1wbkaodHI94N6EDJfqCUbShcNG9+4Agf4+514HXapRhB9LIcH3IYzAcoSBvEj6gK5MiOKQy4waKrXASb8AmLAMh9WmXdtlFoKBSfOCWfEAxD4pea0rXVMVpIsfNYobzqGMLS4SY91MNBSCseytA4hRNtAI7MKmzZTOstQ4Q+02u8ItejjYreR+p1uk7IFQPgCRChbkPRCiLKR5EUb5tSxPbUSTWx7CwpCOo3EBjE1bBh0alO8dVr1jukZgOBlxbo0fhy14bHtqb7IhjGvol3a/ihEAl7/GsvOB/8oIz8oaVPHWevMowQlAJg3OUBSUz+KusEvakA9scM63sNLdhfabdiWs8cRwuLLWareMDZ4E3BUvO9GyFC7/sX2NZwhUySY3Z91oj7OZUo2CSF3WSGT6ibEgHfN9CRa3Nbn68xgXct5EYyhXopwrhODrLyGlizDiJbGU5LXM6VwbPy+1n2H/WzUW53lyxyUX9TAKmUL4JiIXmDOXYAsq0XV8G7IBqytC+nLodgnfa8lngt5vPm97btaWN9nQa884d557TcDznqbPhvHK2HN9h7kf3s/vV/Vb7VPtS+177MSl156ac284/p/bzD1A7SFE=</latexit><latexit sha1_base64="pQ+/bsDaW5e0HjtjvbKSiNfXu+o=">AAAEBXicfVNNb9NAEHUdPkr4auEIhxEVKBVtFIfyFalSJSQEh4pS1W6lbmKt15t0lfU62h0XgusTF/4KFw6AuPIfuPFvWDdxCW3F+vI0M2/2vfFsNJLCYKv1e86tXbh46fL8lfrVa9dv3FxYvBWYNNOM+yyVqd6LqOFSKO6jQMn3RprTJJJ8Nxq+KPO7h1wbkaodHI94N6EDJfqCUbShcNG9+4Agf4+514HXapRhB9LIcH3IYzAcoSBvEj6gK5MiOKQy4waKrXASb8AmLAMh9WmXdtlFoKBSfOCWfEAxD4pea0rXVMVpIsfNYobzqGMLS4SY91MNBSCseytA4hRNtAI7MKmzZTOstQ4Q+02u8ItejjYreR+p1uk7IFQPgCRChbkPRCiLKR5EUb5tSxPbUSTWx7CwpCOo3EBjE1bBh0alO8dVr1jukZgOBlxbo0fhy14bHtqb7IhjGvol3a/ihEAl7/GsvOB/8oIz8oaVPHWevMowQlAJg3OUBSUz+KusEvakA9scM63sNLdhfabdiWs8cRwuLLWareMDZ4E3BUvO9GyFC7/sX2NZwhUySY3Z91oj7OZUo2CSF3WSGT6ibEgHfN9CRa3Nbn68xgXct5EYyhXopwrhODrLyGlizDiJbGU5LXM6VwbPy+1n2H/WzUW53lyxyUX9TAKmUL4JiIXmDOXYAsq0XV8G7IBqytC+nLodgnfa8lngt5vPm97btaWN9nQa884d557TcDznqbPhvHK2HN9h7kf3s/vV/Vb7VPtS+177MSl156ac284/p/bzD1A7SFE=</latexit><latexit sha1_base64="pQ+/bsDaW5e0HjtjvbKSiNfXu+o=">AAAEBXicfVNNb9NAEHUdPkr4auEIhxEVKBVtFIfyFalSJSQEh4pS1W6lbmKt15t0lfU62h0XgusTF/4KFw6AuPIfuPFvWDdxCW3F+vI0M2/2vfFsNJLCYKv1e86tXbh46fL8lfrVa9dv3FxYvBWYNNOM+yyVqd6LqOFSKO6jQMn3RprTJJJ8Nxq+KPO7h1wbkaodHI94N6EDJfqCUbShcNG9+4Agf4+514HXapRhB9LIcH3IYzAcoSBvEj6gK5MiOKQy4waKrXASb8AmLAMh9WmXdtlFoKBSfOCWfEAxD4pea0rXVMVpIsfNYobzqGMLS4SY91MNBSCseytA4hRNtAI7MKmzZTOstQ4Q+02u8ItejjYreR+p1uk7IFQPgCRChbkPRCiLKR5EUb5tSxPbUSTWx7CwpCOo3EBjE1bBh0alO8dVr1jukZgOBlxbo0fhy14bHtqb7IhjGvol3a/ihEAl7/GsvOB/8oIz8oaVPHWevMowQlAJg3OUBSUz+KusEvakA9scM63sNLdhfabdiWs8cRwuLLWareMDZ4E3BUvO9GyFC7/sX2NZwhUySY3Z91oj7OZUo2CSF3WSGT6ibEgHfN9CRa3Nbn68xgXct5EYyhXopwrhODrLyGlizDiJbGU5LXM6VwbPy+1n2H/WzUW53lyxyUX9TAKmUL4JiIXmDOXYAsq0XV8G7IBqytC+nLodgnfa8lngt5vPm97btaWN9nQa884d557TcDznqbPhvHK2HN9h7kf3s/vV/Vb7VPtS+177MSl156ac284/p/bzD1A7SFE=</latexit>

WHAT IS BIASED MATRIX FACTORIZATION?
• The Biased Matrix Factorization is based on Funk SVD.
• Biased MF = Funk SVD + Bias Terms
• Some users tend to give higher ratings, and some users tend to give lower ratings.
• The tendencies of movies are similar to that of users.
• Some movies tend to get higher ratings, and some movies tend to get lower ratings.
• Biased Matrix Factorization wants to handle this phenomenon.
• To introduce bias terms in the optimization form.
!13

OPTIMIZATION FORM OF BIASED-MF
• Therefore, Biased-MF introduce biased terms related to user and item respectively.
• is the average rating of training set(constant).
• is the bias vector related to user.
• is the bias vector related to item.
• This optimization form can be solved by alternating minimization.
• Similar to the solution of Funk SVD.
• The Biased Matrix Factorization is also called SVD++.
!14
µ<latexit sha1_base64="VlnKo72qdPNK/XpDYE6WLCIIdz8=">AAAB6XicbVA9SwNBEJ2LXzF+RS1tFoNgFe5EULuAjWVEzwSSI+xt5pIlu3vH7p4QQn6CjYWKrf/Izn/jJrlCow8GHu/NMDMvzgQ31ve/vNLK6tr6RnmzsrW9s7tX3T94MGmuGYYsFalux9Sg4ApDy63AdqaRylhgKx5dz/zWI2rDU3VvxxlGkg4UTzij1kl3XZn3qjW/7s9B/pKgIDUo0OxVP7v9lOUSlWWCGtMJ/MxGE6otZwKnlW5uMKNsRAfYcVRRiSaazE+dkhOn9EmSalfKkrn6c2JCpTFjGbtOSe3QLHsz8T+vk9vkMppwleUWFVssSnJBbEpmf5M+18isGDtCmebuVsKGVFNmXToVF0Kw/PJfEp7Vr+rB7Xmt4RdplOEIjuEUAriABtxAE0JgMIAneIFXT3jP3pv3vmgtecXMIfyC9/ENxYONmA==</latexit><latexit sha1_base64="VlnKo72qdPNK/XpDYE6WLCIIdz8=">AAAB6XicbVA9SwNBEJ2LXzF+RS1tFoNgFe5EULuAjWVEzwSSI+xt5pIlu3vH7p4QQn6CjYWKrf/Izn/jJrlCow8GHu/NMDMvzgQ31ve/vNLK6tr6RnmzsrW9s7tX3T94MGmuGYYsFalux9Sg4ApDy63AdqaRylhgKx5dz/zWI2rDU3VvxxlGkg4UTzij1kl3XZn3qjW/7s9B/pKgIDUo0OxVP7v9lOUSlWWCGtMJ/MxGE6otZwKnlW5uMKNsRAfYcVRRiSaazE+dkhOn9EmSalfKkrn6c2JCpTFjGbtOSe3QLHsz8T+vk9vkMppwleUWFVssSnJBbEpmf5M+18isGDtCmebuVsKGVFNmXToVF0Kw/PJfEp7Vr+rB7Xmt4RdplOEIjuEUAriABtxAE0JgMIAneIFXT3jP3pv3vmgtecXMIfyC9/ENxYONmA==</latexit><latexit sha1_base64="VlnKo72qdPNK/XpDYE6WLCIIdz8=">AAAB6XicbVA9SwNBEJ2LXzF+RS1tFoNgFe5EULuAjWVEzwSSI+xt5pIlu3vH7p4QQn6CjYWKrf/Izn/jJrlCow8GHu/NMDMvzgQ31ve/vNLK6tr6RnmzsrW9s7tX3T94MGmuGYYsFalux9Sg4ApDy63AdqaRylhgKx5dz/zWI2rDU3VvxxlGkg4UTzij1kl3XZn3qjW/7s9B/pKgIDUo0OxVP7v9lOUSlWWCGtMJ/MxGE6otZwKnlW5uMKNsRAfYcVRRiSaazE+dkhOn9EmSalfKkrn6c2JCpTFjGbtOSe3QLHsz8T+vk9vkMppwleUWFVssSnJBbEpmf5M+18isGDtCmebuVsKGVFNmXToVF0Kw/PJfEp7Vr+rB7Xmt4RdplOEIjuEUAriABtxAE0JgMIAneIFXT3jP3pv3vmgtecXMIfyC9/ENxYONmA==</latexit><latexit sha1_base64="VlnKo72qdPNK/XpDYE6WLCIIdz8=">AAAB6XicbVA9SwNBEJ2LXzF+RS1tFoNgFe5EULuAjWVEzwSSI+xt5pIlu3vH7p4QQn6CjYWKrf/Izn/jJrlCow8GHu/NMDMvzgQ31ve/vNLK6tr6RnmzsrW9s7tX3T94MGmuGYYsFalux9Sg4ApDy63AdqaRylhgKx5dz/zWI2rDU3VvxxlGkg4UTzij1kl3XZn3qjW/7s9B/pKgIDUo0OxVP7v9lOUSlWWCGtMJ/MxGE6otZwKnlW5uMKNsRAfYcVRRiSaazE+dkhOn9EmSalfKkrn6c2JCpTFjGbtOSe3QLHsz8T+vk9vkMppwleUWFVssSnJBbEpmf5M+18isGDtCmebuVsKGVFNmXToVF0Kw/PJfEp7Vr+rB7Xmt4RdplOEIjuEUAriABtxAE0JgMIAneIFXT3jP3pv3vmgtecXMIfyC9/ENxYONmA==</latexit>
buser
<latexit sha1_base64="DXbG6GcW+vrGhhOT8KyvQGgSL1c=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUG8FLx4rGFtoY9lsJ+3SzSbuboQS+ie8eFDx6u/x5r9x2+agrQ8WHu/NzM68MBVcG9f9dkorq2vrG+XNytb2zu5edf/gXieZYuizRCSqHVKNgkv0DTcC26lCGocCW+Hoeuq3nlBpnsg7M04xiOlA8ogzaqzUDh/yTKOa9Ko1t+7OQJaJV5AaFGj2ql/dfsKyGKVhgmrd8dzUBDlVhjOBk0rXjk0pG9EBdiyVNEYd5LN9J+TEKn0SJco+achM/d2R01jrcRzaypiaoV70puJ/Xicz0WWQc5lmBiWbfxRlgpiETI8nfa6QGTG2hDLF7a6EDamizNiIKjYEb/HkZeKf1a/q3u15reEWaZThCI7hFDy4gAbcQBN8YCDgGV7hzXl0Xpx352NeWnKKnkP4A+fzB+UOkAM=</latexit><latexit sha1_base64="DXbG6GcW+vrGhhOT8KyvQGgSL1c=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUG8FLx4rGFtoY9lsJ+3SzSbuboQS+ie8eFDx6u/x5r9x2+agrQ8WHu/NzM68MBVcG9f9dkorq2vrG+XNytb2zu5edf/gXieZYuizRCSqHVKNgkv0DTcC26lCGocCW+Hoeuq3nlBpnsg7M04xiOlA8ogzaqzUDh/yTKOa9Ko1t+7OQJaJV5AaFGj2ql/dfsKyGKVhgmrd8dzUBDlVhjOBk0rXjk0pG9EBdiyVNEYd5LN9J+TEKn0SJco+achM/d2R01jrcRzaypiaoV70puJ/Xicz0WWQc5lmBiWbfxRlgpiETI8nfa6QGTG2hDLF7a6EDamizNiIKjYEb/HkZeKf1a/q3u15reEWaZThCI7hFDy4gAbcQBN8YCDgGV7hzXl0Xpx352NeWnKKnkP4A+fzB+UOkAM=</latexit><latexit sha1_base64="DXbG6GcW+vrGhhOT8KyvQGgSL1c=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUG8FLx4rGFtoY9lsJ+3SzSbuboQS+ie8eFDx6u/x5r9x2+agrQ8WHu/NzM68MBVcG9f9dkorq2vrG+XNytb2zu5edf/gXieZYuizRCSqHVKNgkv0DTcC26lCGocCW+Hoeuq3nlBpnsg7M04xiOlA8ogzaqzUDh/yTKOa9Ko1t+7OQJaJV5AaFGj2ql/dfsKyGKVhgmrd8dzUBDlVhjOBk0rXjk0pG9EBdiyVNEYd5LN9J+TEKn0SJco+achM/d2R01jrcRzaypiaoV70puJ/Xicz0WWQc5lmBiWbfxRlgpiETI8nfa6QGTG2hDLF7a6EDamizNiIKjYEb/HkZeKf1a/q3u15reEWaZThCI7hFDy4gAbcQBN8YCDgGV7hzXl0Xpx352NeWnKKnkP4A+fzB+UOkAM=</latexit><latexit sha1_base64="DXbG6GcW+vrGhhOT8KyvQGgSL1c=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUG8FLx4rGFtoY9lsJ+3SzSbuboQS+ie8eFDx6u/x5r9x2+agrQ8WHu/NzM68MBVcG9f9dkorq2vrG+XNytb2zu5edf/gXieZYuizRCSqHVKNgkv0DTcC26lCGocCW+Hoeuq3nlBpnsg7M04xiOlA8ogzaqzUDh/yTKOa9Ko1t+7OQJaJV5AaFGj2ql/dfsKyGKVhgmrd8dzUBDlVhjOBk0rXjk0pG9EBdiyVNEYd5LN9J+TEKn0SJco+achM/d2R01jrcRzaypiaoV70puJ/Xicz0WWQc5lmBiWbfxRlgpiETI8nfa6QGTG2hDLF7a6EDamizNiIKjYEb/HkZeKf1a/q3u15reEWaZThCI7hFDy4gAbcQBN8YCDgGV7hzXl0Xpx352NeWnKKnkP4A+fzB+UOkAM=</latexit>
bitem
<latexit sha1_base64="ihgt0y/lts5uDQPDa3MALu8LZCM=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUG8FLx4rGFtoY9lsJ+3SzSbuboQS+ie8eFDx6u/x5r9x2+agrQ8GHu/NMDMvTAXXxnW/ndLK6tr6RnmzsrW9s7tX3T+410mmGPosEYlqh1Sj4BJ9w43AdqqQxqHAVji6nvqtJ1SaJ/LOjFMMYjqQPOKMGiu1w4ecG4wnvWrNrbszkGXiFaQGBZq96le3n7AsRmmYoFp3PDc1QU6V4UzgpNLNNKaUjegAO5ZKGqMO8tm9E3JilT6JEmVLGjJTf0/kNNZ6HIe2M6ZmqBe9qfif18lMdBnkXKaZQcnmi6JMEJOQ6fOkzxUyI8aWUKa4vZWwIVWUGRtRxYbgLb68TPyz+lXduz2vNdwijTIcwTGcggcX0IAbaIIPDAQ8wyu8OY/Oi/PufMxbS04xcwh/4Hz+AMysj/M=</latexit><latexit sha1_base64="ihgt0y/lts5uDQPDa3MALu8LZCM=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUG8FLx4rGFtoY9lsJ+3SzSbuboQS+ie8eFDx6u/x5r9x2+agrQ8GHu/NMDMvTAXXxnW/ndLK6tr6RnmzsrW9s7tX3T+410mmGPosEYlqh1Sj4BJ9w43AdqqQxqHAVji6nvqtJ1SaJ/LOjFMMYjqQPOKMGiu1w4ecG4wnvWrNrbszkGXiFaQGBZq96le3n7AsRmmYoFp3PDc1QU6V4UzgpNLNNKaUjegAO5ZKGqMO8tm9E3JilT6JEmVLGjJTf0/kNNZ6HIe2M6ZmqBe9qfif18lMdBnkXKaZQcnmi6JMEJOQ6fOkzxUyI8aWUKa4vZWwIVWUGRtRxYbgLb68TPyz+lXduz2vNdwijTIcwTGcggcX0IAbaIIPDAQ8wyu8OY/Oi/PufMxbS04xcwh/4Hz+AMysj/M=</latexit><latexit sha1_base64="ihgt0y/lts5uDQPDa3MALu8LZCM=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUG8FLx4rGFtoY9lsJ+3SzSbuboQS+ie8eFDx6u/x5r9x2+agrQ8GHu/NMDMvTAXXxnW/ndLK6tr6RnmzsrW9s7tX3T+410mmGPosEYlqh1Sj4BJ9w43AdqqQxqHAVji6nvqtJ1SaJ/LOjFMMYjqQPOKMGiu1w4ecG4wnvWrNrbszkGXiFaQGBZq96le3n7AsRmmYoFp3PDc1QU6V4UzgpNLNNKaUjegAO5ZKGqMO8tm9E3JilT6JEmVLGjJTf0/kNNZ6HIe2M6ZmqBe9qfif18lMdBnkXKaZQcnmi6JMEJOQ6fOkzxUyI8aWUKa4vZWwIVWUGRtRxYbgLb68TPyz+lXduz2vNdwijTIcwTGcggcX0IAbaIIPDAQ8wyu8OY/Oi/PufMxbS04xcwh/4Hz+AMysj/M=</latexit><latexit sha1_base64="ihgt0y/lts5uDQPDa3MALu8LZCM=">AAAB7nicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEUG8FLx4rGFtoY9lsJ+3SzSbuboQS+ie8eFDx6u/x5r9x2+agrQ8GHu/NMDMvTAXXxnW/ndLK6tr6RnmzsrW9s7tX3T+410mmGPosEYlqh1Sj4BJ9w43AdqqQxqHAVji6nvqtJ1SaJ/LOjFMMYjqQPOKMGiu1w4ecG4wnvWrNrbszkGXiFaQGBZq96le3n7AsRmmYoFp3PDc1QU6V4UzgpNLNNKaUjegAO5ZKGqMO8tm9E3JilT6JEmVLGjJTf0/kNNZ6HIe2M6ZmqBe9qfif18lMdBnkXKaZQcnmi6JMEJOQ6fOkzxUyI8aWUKa4vZWwIVWUGRtRxYbgLb68TPyz+lXduz2vNdwijTIcwTGcggcX0IAbaIIPDAQ8wyu8OY/Oi/PufMxbS04xcwh/4Hz+AMysj/M=</latexit>
min
Û, ˆV ,buser,bitem
X
(i,j)2⌦
(Mij µ buser
i bitem
j
Ûi
ˆV †
j )2
+ U k Ûk2
F + V k ˆV k2
F + buser kbuser
k2
2 + bitem kbitem
k2
2<latexit sha1_base64="utwFR+JySkMZlz+0CF9UitgaWsY=">AAADEHicbVJNj9MwEHXC11I+tgtHLiMqlq7YrZJqJeC2EhLiglgk0l2pbiPHdVN34ySyHaTK5C9w4a9w4QCIK0du/BucNFlguyNZfp73xjNjT5QnXGnP++24V65eu35j62bn1u07d7e7O/dGKiskZQHNkkyeRkSxhKcs0Fwn7DSXjIgoYSfR2YuKP3nPpOJZ+k6vcjYRJE75nFOirSvccR5jwdPQ4AXRJij3oQYjC6KpKRSTa8Q1EyWUsAuAVSFC0+f7sNwDzFPAbwSLSQlg+q9Dw5clHAAWBdgtCnlzS31YthcdQJMv5G1CS+IZiWMmYW86LAHjzu4TwIltZUbCAPCHNsbC8OV0CH/Z0Tk7uoQ1bScVd95WpRtu6OryWt261kYXdnvewKsNNoHfgB5q7Djs/sKzjBaCpZomRKmx7+V6YojUnCas7GBbRk7oGYnZ2MKUCKYmpv7REh5ZzwzmmbQr1VB7/40wRCi1EpFVCqIX6iJXOS/jxoWeP5sYnuaFZildJ5oXCegMqvGAGZeM6mRlAaGS21qBLogkVNsh6thH8C+2vAmC4eD5wH972Ds6bF5jCz1AD1Ef+egpOkKv0DEKEHU+Op+dr84395P7xf3u/lhLXaeJuY/+M/fnH3m99G4=</latexit><latexit sha1_base64="utwFR+JySkMZlz+0CF9UitgaWsY=">AAADEHicbVJNj9MwEHXC11I+tgtHLiMqlq7YrZJqJeC2EhLiglgk0l2pbiPHdVN34ySyHaTK5C9w4a9w4QCIK0du/BucNFlguyNZfp73xjNjT5QnXGnP++24V65eu35j62bn1u07d7e7O/dGKiskZQHNkkyeRkSxhKcs0Fwn7DSXjIgoYSfR2YuKP3nPpOJZ+k6vcjYRJE75nFOirSvccR5jwdPQ4AXRJij3oQYjC6KpKRSTa8Q1EyWUsAuAVSFC0+f7sNwDzFPAbwSLSQlg+q9Dw5clHAAWBdgtCnlzS31YthcdQJMv5G1CS+IZiWMmYW86LAHjzu4TwIltZUbCAPCHNsbC8OV0CH/Z0Tk7uoQ1bScVd95WpRtu6OryWt261kYXdnvewKsNNoHfgB5q7Djs/sKzjBaCpZomRKmx7+V6YojUnCas7GBbRk7oGYnZ2MKUCKYmpv7REh5ZzwzmmbQr1VB7/40wRCi1EpFVCqIX6iJXOS/jxoWeP5sYnuaFZildJ5oXCegMqvGAGZeM6mRlAaGS21qBLogkVNsh6thH8C+2vAmC4eD5wH972Ds6bF5jCz1AD1Ef+egpOkKv0DEKEHU+Op+dr84395P7xf3u/lhLXaeJuY/+M/fnH3m99G4=</latexit><latexit sha1_base64="utwFR+JySkMZlz+0CF9UitgaWsY=">AAADEHicbVJNj9MwEHXC11I+tgtHLiMqlq7YrZJqJeC2EhLiglgk0l2pbiPHdVN34ySyHaTK5C9w4a9w4QCIK0du/BucNFlguyNZfp73xjNjT5QnXGnP++24V65eu35j62bn1u07d7e7O/dGKiskZQHNkkyeRkSxhKcs0Fwn7DSXjIgoYSfR2YuKP3nPpOJZ+k6vcjYRJE75nFOirSvccR5jwdPQ4AXRJij3oQYjC6KpKRSTa8Q1EyWUsAuAVSFC0+f7sNwDzFPAbwSLSQlg+q9Dw5clHAAWBdgtCnlzS31YthcdQJMv5G1CS+IZiWMmYW86LAHjzu4TwIltZUbCAPCHNsbC8OV0CH/Z0Tk7uoQ1bScVd95WpRtu6OryWt261kYXdnvewKsNNoHfgB5q7Djs/sKzjBaCpZomRKmx7+V6YojUnCas7GBbRk7oGYnZ2MKUCKYmpv7REh5ZzwzmmbQr1VB7/40wRCi1EpFVCqIX6iJXOS/jxoWeP5sYnuaFZildJ5oXCegMqvGAGZeM6mRlAaGS21qBLogkVNsh6thH8C+2vAmC4eD5wH972Ds6bF5jCz1AD1Ef+egpOkKv0DEKEHU+Op+dr84395P7xf3u/lhLXaeJuY/+M/fnH3m99G4=</latexit><latexit sha1_base64="utwFR+JySkMZlz+0CF9UitgaWsY=">AAADEHicbVJNj9MwEHXC11I+tgtHLiMqlq7YrZJqJeC2EhLiglgk0l2pbiPHdVN34ySyHaTK5C9w4a9w4QCIK0du/BucNFlguyNZfp73xjNjT5QnXGnP++24V65eu35j62bn1u07d7e7O/dGKiskZQHNkkyeRkSxhKcs0Fwn7DSXjIgoYSfR2YuKP3nPpOJZ+k6vcjYRJE75nFOirSvccR5jwdPQ4AXRJij3oQYjC6KpKRSTa8Q1EyWUsAuAVSFC0+f7sNwDzFPAbwSLSQlg+q9Dw5clHAAWBdgtCnlzS31YthcdQJMv5G1CS+IZiWMmYW86LAHjzu4TwIltZUbCAPCHNsbC8OV0CH/Z0Tk7uoQ1bScVd95WpRtu6OryWt261kYXdnvewKsNNoHfgB5q7Djs/sKzjBaCpZomRKmx7+V6YojUnCas7GBbRk7oGYnZ2MKUCKYmpv7REh5ZzwzmmbQr1VB7/40wRCi1EpFVCqIX6iJXOS/jxoWeP5sYnuaFZildJ5oXCegMqvGAGZeM6mRlAaGS21qBLogkVNsh6thH8C+2vAmC4eD5wH972Ds6bF5jCz1AD1Ef+egpOkKv0DEKEHU+Op+dr84395P7xf3u/lhLXaeJuY/+M/fnH3m99G4=</latexit>

THE ROLE OF BIAS TERMS IN BIASED-MF
• The bias terms serve the role of normalization.
• The bias terms serve to make the rating matrix r be zero mean.
• Meanwhile, the Biased-MF without bias terms(=Funk SVD) also works quite well.
• The RMSE performance on Netflix Dataset is similar to original recommendation engine of
Netflix.
• Therefore, knowing why Biased-MF works well is equivalent to 
knowing why Funk SVD works well.
• Now, we will investigate why Funk SVD works well on low-rank matrices.
!15

OPTIMIZATION FORM OF FUNK SVD
• Recall: Optimization form of Funk SVD
• Probabilistic Matrix Factorization (PMF, NIPS’08) interprets Funk SVD  
in the view point of posterior.
• Funk SVD can be interpreted as an algorithm that finds the most probable low-rank matrices
p and q when a star is observed.
!17
arg min
Û, ˆV
X
(i,j)2⌦
(Mij
Ûi
ˆV †
j )2
+ U k Ûk2
F + V k ˆV k2
F

PROBABILISTIC MATRIX FACTORIZATION
• Optimization form of PMF:
• To find and via MAP(Maximum A Posteriori).
• A low-rank assumption ( ) and Gaussian noise assumptions.
!18
Mij
i=1,…,m
j=1,…,n
σ
σU σV
arg max
Û, ˆV
Pr[ Û, ˆV |P⌦(M), ⌦, 2
, 2
U , 2
V ]
<latexit sha1_base64="W0+3TbimD40TlGA8JBqmkMPUT+k=">AAACWXicbZFfa9swFMVlb2uz7E+z9nEvYmHQwQh2KGx7K/SlL2MpzGkhds21cuOISrYnXY8G119yL2OwrzKo4mbQpbsg9OOce5F0lFVKWgqCX57/6PGTnd3e0/6z5y9e7g1e7U9tWRuBkShVaS4ysKhkgRFJUnhRGQSdKTzPrk7W/vl3NFaWxVdaVZhoyAu5kALISengWwwm57GG67SJl0BN1L7nHUzblseE19RMTDvb9m74JI2/aMyBH35+5+SO3W5lruFy/JfS6B5PL8cJTwfDYBR0xR9CuIEh29QkHfyI56WoNRYkFFg7C4OKkgYMSaGw7ce1xQrEFeQ4c1iARps0XTQtf+uUOV+Uxq2CeKfen2hAW7vSmevUQEu77a3F/3mzmhYfk0YWVU1YiLuDFrXiVPJ1znwuDQpSKwcgjHR35WIJBgS53+i7EMLtJz+EaDz6NArPjobHR5s0euw1e8MOWcg+sGN2yiYsYoL9ZH+8HW/X++37fs/v37X63mbmgP1T/sEtCBayuA==</latexit><latexit sha1_base64="W0+3TbimD40TlGA8JBqmkMPUT+k=">AAACWXicbZFfa9swFMVlb2uz7E+z9nEvYmHQwQh2KGx7K/SlL2MpzGkhds21cuOISrYnXY8G119yL2OwrzKo4mbQpbsg9OOce5F0lFVKWgqCX57/6PGTnd3e0/6z5y9e7g1e7U9tWRuBkShVaS4ysKhkgRFJUnhRGQSdKTzPrk7W/vl3NFaWxVdaVZhoyAu5kALISengWwwm57GG67SJl0BN1L7nHUzblseE19RMTDvb9m74JI2/aMyBH35+5+SO3W5lruFy/JfS6B5PL8cJTwfDYBR0xR9CuIEh29QkHfyI56WoNRYkFFg7C4OKkgYMSaGw7ce1xQrEFeQ4c1iARps0XTQtf+uUOV+Uxq2CeKfen2hAW7vSmevUQEu77a3F/3mzmhYfk0YWVU1YiLuDFrXiVPJ1znwuDQpSKwcgjHR35WIJBgS53+i7EMLtJz+EaDz6NArPjobHR5s0euw1e8MOWcg+sGN2yiYsYoL9ZH+8HW/X++37fs/v37X63mbmgP1T/sEtCBayuA==</latexit><latexit sha1_base64="W0+3TbimD40TlGA8JBqmkMPUT+k=">AAACWXicbZFfa9swFMVlb2uz7E+z9nEvYmHQwQh2KGx7K/SlL2MpzGkhds21cuOISrYnXY8G119yL2OwrzKo4mbQpbsg9OOce5F0lFVKWgqCX57/6PGTnd3e0/6z5y9e7g1e7U9tWRuBkShVaS4ysKhkgRFJUnhRGQSdKTzPrk7W/vl3NFaWxVdaVZhoyAu5kALISengWwwm57GG67SJl0BN1L7nHUzblseE19RMTDvb9m74JI2/aMyBH35+5+SO3W5lruFy/JfS6B5PL8cJTwfDYBR0xR9CuIEh29QkHfyI56WoNRYkFFg7C4OKkgYMSaGw7ce1xQrEFeQ4c1iARps0XTQtf+uUOV+Uxq2CeKfen2hAW7vSmevUQEu77a3F/3mzmhYfk0YWVU1YiLuDFrXiVPJ1znwuDQpSKwcgjHR35WIJBgS53+i7EMLtJz+EaDz6NArPjobHR5s0euw1e8MOWcg+sGN2yiYsYoL9ZH+8HW/X++37fs/v37X63mbmgP1T/sEtCBayuA==</latexit><latexit sha1_base64="W0+3TbimD40TlGA8JBqmkMPUT+k=">AAACWXicbZFfa9swFMVlb2uz7E+z9nEvYmHQwQh2KGx7K/SlL2MpzGkhds21cuOISrYnXY8G119yL2OwrzKo4mbQpbsg9OOce5F0lFVKWgqCX57/6PGTnd3e0/6z5y9e7g1e7U9tWRuBkShVaS4ysKhkgRFJUnhRGQSdKTzPrk7W/vl3NFaWxVdaVZhoyAu5kALISengWwwm57GG67SJl0BN1L7nHUzblseE19RMTDvb9m74JI2/aMyBH35+5+SO3W5lruFy/JfS6B5PL8cJTwfDYBR0xR9CuIEh29QkHfyI56WoNRYkFFg7C4OKkgYMSaGw7ce1xQrEFeQ4c1iARps0XTQtf+uUOV+Uxq2CeKfen2hAW7vSmevUQEu77a3F/3mzmhYfk0YWVU1YiLuDFrXiVPJ1znwuDQpSKwcgjHR35WIJBgS53+i7EMLtJz+EaDz6NArPjobHR5s0euw1e8MOWcg+sGN2yiYsYoL9ZH+8HW/X++37fs/v37X63mbmgP1T/sEtCBayuA==</latexit>
Ûi<latexit sha1_base64="DmZux6dYard+s7S159poIGMBQ98=">AAAB73icbVBNS8NAEJ3Ur1q/qh69LBbBU0mKoN4KXjxWMLbShrLZbtqlm03YnQgl9Fd48aDi1b/jzX/jts1BWx8MPN6bYWZemEph0HW/ndLa+sbmVnm7srO7t39QPTx6MEmmGfdZIhPdCanhUijuo0DJO6nmNA4lb4fjm5nffuLaiETd4yTlQUyHSkSCUbTSY29EMfenfdGv1ty6OwdZJV5BalCg1a9+9QYJy2KukElqTNdzUwxyqlEwyaeVXmZ4StmYDnnXUkVjboJ8fvCUnFllQKJE21JI5urviZzGxkzi0HbGFEdm2ZuJ/3ndDKOrIBcqzZArtlgUZZJgQmbfk4HQnKGcWEKZFvZWwkZUU4Y2o4oNwVt+eZX4jfp13bu7qDUbRRplOIFTOAcPLqEJt9ACHxjE8Ayv8OZo58V5dz4WrSWnmDmGP3A+fwBgoZBG</latexit><latexit sha1_base64="DmZux6dYard+s7S159poIGMBQ98=">AAAB73icbVBNS8NAEJ3Ur1q/qh69LBbBU0mKoN4KXjxWMLbShrLZbtqlm03YnQgl9Fd48aDi1b/jzX/jts1BWx8MPN6bYWZemEph0HW/ndLa+sbmVnm7srO7t39QPTx6MEmmGfdZIhPdCanhUijuo0DJO6nmNA4lb4fjm5nffuLaiETd4yTlQUyHSkSCUbTSY29EMfenfdGv1ty6OwdZJV5BalCg1a9+9QYJy2KukElqTNdzUwxyqlEwyaeVXmZ4StmYDnnXUkVjboJ8fvCUnFllQKJE21JI5urviZzGxkzi0HbGFEdm2ZuJ/3ndDKOrIBcqzZArtlgUZZJgQmbfk4HQnKGcWEKZFvZWwkZUU4Y2o4oNwVt+eZX4jfp13bu7qDUbRRplOIFTOAcPLqEJt9ACHxjE8Ayv8OZo58V5dz4WrSWnmDmGP3A+fwBgoZBG</latexit><latexit sha1_base64="DmZux6dYard+s7S159poIGMBQ98=">AAAB73icbVBNS8NAEJ3Ur1q/qh69LBbBU0mKoN4KXjxWMLbShrLZbtqlm03YnQgl9Fd48aDi1b/jzX/jts1BWx8MPN6bYWZemEph0HW/ndLa+sbmVnm7srO7t39QPTx6MEmmGfdZIhPdCanhUijuo0DJO6nmNA4lb4fjm5nffuLaiETd4yTlQUyHSkSCUbTSY29EMfenfdGv1ty6OwdZJV5BalCg1a9+9QYJy2KukElqTNdzUwxyqlEwyaeVXmZ4StmYDnnXUkVjboJ8fvCUnFllQKJE21JI5urviZzGxkzi0HbGFEdm2ZuJ/3ndDKOrIBcqzZArtlgUZZJgQmbfk4HQnKGcWEKZFvZWwkZUU4Y2o4oNwVt+eZX4jfp13bu7qDUbRRplOIFTOAcPLqEJt9ACHxjE8Ayv8OZo58V5dz4WrSWnmDmGP3A+fwBgoZBG</latexit><latexit sha1_base64="DmZux6dYard+s7S159poIGMBQ98=">AAAB73icbVBNS8NAEJ3Ur1q/qh69LBbBU0mKoN4KXjxWMLbShrLZbtqlm03YnQgl9Fd48aDi1b/jzX/jts1BWx8MPN6bYWZemEph0HW/ndLa+sbmVnm7srO7t39QPTx6MEmmGfdZIhPdCanhUijuo0DJO6nmNA4lb4fjm5nffuLaiETd4yTlQUyHSkSCUbTSY29EMfenfdGv1ty6OwdZJV5BalCg1a9+9QYJy2KukElqTNdzUwxyqlEwyaeVXmZ4StmYDnnXUkVjboJ8fvCUnFllQKJE21JI5urviZzGxkzi0HbGFEdm2ZuJ/3ndDKOrIBcqzZArtlgUZZJgQmbfk4HQnKGcWEKZFvZWwkZUU4Y2o4oNwVt+eZX4jfp13bu7qDUbRRplOIFTOAcPLqEJt9ACHxjE8Ayv8OZo58V5dz4WrSWnmDmGP3A+fwBgoZBG</latexit>
ˆVj<latexit sha1_base64="H+66y9gu8sYQVMmwNQcT8xMRc3A=">AAAB73icbVBNS8NAEJ3Ur1q/qh69LBbBU0mKoN4KXjxWMG2lDWWz3bZrd5OwOxFK6K/w4kHFq3/Hm//GbZuDtj4YeLw3w8y8MJHCoOt+O4W19Y3NreJ2aWd3b/+gfHjUNHGqGfdZLGPdDqnhUkTcR4GStxPNqQolb4Xjm5nfeuLaiDi6x0nCA0WHkRgIRtFKD90Rxaw57T32yhW36s5BVomXkwrkaPTKX91+zFLFI2SSGtPx3ASDjGoUTPJpqZsanlA2pkPesTSiipsgmx88JWdW6ZNBrG1FSObq74mMKmMmKrSdiuLILHsz8T+vk+LgKshElKTII7ZYNEglwZjMvid9oTlDObGEMi3srYSNqKYMbUYlG4K3/PIq8WvV66p3d1Gp1/I0inACp3AOHlxCHW6hAT4wUPAMr/DmaOfFeXc+Fq0FJ585hj9wPn8AY6qQSA==</latexit><latexit sha1_base64="H+66y9gu8sYQVMmwNQcT8xMRc3A=">AAAB73icbVBNS8NAEJ3Ur1q/qh69LBbBU0mKoN4KXjxWMG2lDWWz3bZrd5OwOxFK6K/w4kHFq3/Hm//GbZuDtj4YeLw3w8y8MJHCoOt+O4W19Y3NreJ2aWd3b/+gfHjUNHGqGfdZLGPdDqnhUkTcR4GStxPNqQolb4Xjm5nfeuLaiDi6x0nCA0WHkRgIRtFKD90Rxaw57T32yhW36s5BVomXkwrkaPTKX91+zFLFI2SSGtPx3ASDjGoUTPJpqZsanlA2pkPesTSiipsgmx88JWdW6ZNBrG1FSObq74mMKmMmKrSdiuLILHsz8T+vk+LgKshElKTII7ZYNEglwZjMvid9oTlDObGEMi3srYSNqKYMbUYlG4K3/PIq8WvV66p3d1Gp1/I0inACp3AOHlxCHW6hAT4wUPAMr/DmaOfFeXc+Fq0FJ585hj9wPn8AY6qQSA==</latexit><latexit sha1_base64="H+66y9gu8sYQVMmwNQcT8xMRc3A=">AAAB73icbVBNS8NAEJ3Ur1q/qh69LBbBU0mKoN4KXjxWMG2lDWWz3bZrd5OwOxFK6K/w4kHFq3/Hm//GbZuDtj4YeLw3w8y8MJHCoOt+O4W19Y3NreJ2aWd3b/+gfHjUNHGqGfdZLGPdDqnhUkTcR4GStxPNqQolb4Xjm5nfeuLaiDi6x0nCA0WHkRgIRtFKD90Rxaw57T32yhW36s5BVomXkwrkaPTKX91+zFLFI2SSGtPx3ASDjGoUTPJpqZsanlA2pkPesTSiipsgmx88JWdW6ZNBrG1FSObq74mMKmMmKrSdiuLILHsz8T+vk+LgKshElKTII7ZYNEglwZjMvid9oTlDObGEMi3srYSNqKYMbUYlG4K3/PIq8WvV66p3d1Gp1/I0inACp3AOHlxCHW6hAT4wUPAMr/DmaOfFeXc+Fq0FJ585hj9wPn8AY6qQSA==</latexit><latexit sha1_base64="H+66y9gu8sYQVMmwNQcT8xMRc3A=">AAAB73icbVBNS8NAEJ3Ur1q/qh69LBbBU0mKoN4KXjxWMG2lDWWz3bZrd5OwOxFK6K/w4kHFq3/Hm//GbZuDtj4YeLw3w8y8MJHCoOt+O4W19Y3NreJ2aWd3b/+gfHjUNHGqGfdZLGPdDqnhUkTcR4GStxPNqQolb4Xjm5nfeuLaiDi6x0nCA0WHkRgIRtFKD90Rxaw57T32yhW36s5BVomXkwrkaPTKX91+zFLFI2SSGtPx3ASDjGoUTPJpqZsanlA2pkPesTSiipsgmx88JWdW6ZNBrG1FSObq74mMKmMmKrSdiuLILHsz8T+vk+LgKshElKTII7ZYNEglwZjMvid9oTlDObGEMi3srYSNqKYMbUYlG4K3/PIq8WvV66p3d1Gp1/I0inACp3AOHlxCHW6hAT4wUPAMr/DmaOfFeXc+Fq0FJ585hj9wPn8AY6qQSA==</latexit>
Mij = Ûi
ˆV †
j<latexit sha1_base64="/xMSLN6PbfmwT3kOQ+z4TtUfAnI=">AAACDHicbZDLSsNAFIYn9VbrLerSzWAVXJWkCOpCKLhxI1QwbaGJYTKZpNNOLsxMhBLyAm58FTcuVNz6AO58G6dtFtr6w8DHf87hzPm9lFEhDeNbqywtr6yuVddrG5tb2zv67l5HJBnHxMIJS3jPQ4IwGhNLUslIL+UERR4jXW90Nal3HwgXNInv5DglToTCmAYUI6ksVz+6cXM6LOAltAdI5lbh0hl1Cnd4b/soDAl39brRMKaCi2CWUAel2q7+ZfsJziISS8yQEH3TSKWTIy4pZqSo2ZkgKcIjFJK+whhFRDj59JoCHivHh0HC1YslnLq/J3IUCTGOPNUZITkQ87WJ+V+tn8ng3MlpnGaSxHi2KMgYlAmcRAN9ygmWbKwAYU7VXyEeII6wVAHWVAjm/MmLYDUbFw3z9rTeapZpVMEBOAQnwARnoAWuQRtYAINH8AxewZv2pL1o79rHrLWilTP74I+0zx+WTJtt</latexit><latexit sha1_base64="/xMSLN6PbfmwT3kOQ+z4TtUfAnI=">AAACDHicbZDLSsNAFIYn9VbrLerSzWAVXJWkCOpCKLhxI1QwbaGJYTKZpNNOLsxMhBLyAm58FTcuVNz6AO58G6dtFtr6w8DHf87hzPm9lFEhDeNbqywtr6yuVddrG5tb2zv67l5HJBnHxMIJS3jPQ4IwGhNLUslIL+UERR4jXW90Nal3HwgXNInv5DglToTCmAYUI6ksVz+6cXM6LOAltAdI5lbh0hl1Cnd4b/soDAl39brRMKaCi2CWUAel2q7+ZfsJziISS8yQEH3TSKWTIy4pZqSo2ZkgKcIjFJK+whhFRDj59JoCHivHh0HC1YslnLq/J3IUCTGOPNUZITkQ87WJ+V+tn8ng3MlpnGaSxHi2KMgYlAmcRAN9ygmWbKwAYU7VXyEeII6wVAHWVAjm/MmLYDUbFw3z9rTeapZpVMEBOAQnwARnoAWuQRtYAINH8AxewZv2pL1o79rHrLWilTP74I+0zx+WTJtt</latexit><latexit sha1_base64="/xMSLN6PbfmwT3kOQ+z4TtUfAnI=">AAACDHicbZDLSsNAFIYn9VbrLerSzWAVXJWkCOpCKLhxI1QwbaGJYTKZpNNOLsxMhBLyAm58FTcuVNz6AO58G6dtFtr6w8DHf87hzPm9lFEhDeNbqywtr6yuVddrG5tb2zv67l5HJBnHxMIJS3jPQ4IwGhNLUslIL+UERR4jXW90Nal3HwgXNInv5DglToTCmAYUI6ksVz+6cXM6LOAltAdI5lbh0hl1Cnd4b/soDAl39brRMKaCi2CWUAel2q7+ZfsJziISS8yQEH3TSKWTIy4pZqSo2ZkgKcIjFJK+whhFRDj59JoCHivHh0HC1YslnLq/J3IUCTGOPNUZITkQ87WJ+V+tn8ng3MlpnGaSxHi2KMgYlAmcRAN9ygmWbKwAYU7VXyEeII6wVAHWVAjm/MmLYDUbFw3z9rTeapZpVMEBOAQnwARnoAWuQRtYAINH8AxewZv2pL1o79rHrLWilTP74I+0zx+WTJtt</latexit><latexit sha1_base64="/xMSLN6PbfmwT3kOQ+z4TtUfAnI=">AAACDHicbZDLSsNAFIYn9VbrLerSzWAVXJWkCOpCKLhxI1QwbaGJYTKZpNNOLsxMhBLyAm58FTcuVNz6AO58G6dtFtr6w8DHf87hzPm9lFEhDeNbqywtr6yuVddrG5tb2zv67l5HJBnHxMIJS3jPQ4IwGhNLUslIL+UERR4jXW90Nal3HwgXNInv5DglToTCmAYUI6ksVz+6cXM6LOAltAdI5lbh0hl1Cnd4b/soDAl39brRMKaCi2CWUAel2q7+ZfsJziISS8yQEH3TSKWTIy4pZqSo2ZkgKcIjFJK+whhFRDj59JoCHivHh0HC1YslnLq/J3IUCTGOPNUZITkQ87WJ+V+tn8ng3MlpnGaSxHi2KMgYlAmcRAN9ygmWbKwAYU7VXyEeII6wVAHWVAjm/MmLYDUbFw3z9rTeapZpVMEBOAQnwARnoAWuQRtYAINH8AxewZv2pL1o79rHrLWilTP74I+0zx+WTJtt</latexit>
Pr[M| Û, ˆV , 2
] =
mY
i=1
nY
j=1
[N[Mij| Ûi
ˆV †
j , 2
]]I⌦
ij ,
Pr[ Û| 2
U ] =
mY
i=1
N[ Ûi|0, 2
U I],
Pr[ ˆV | 2
V ] =
nY
j=1
N[ ˆVj|0, 2
V I],
I⌦
ij =
⇢
1 (i, j) 2 ⌦
0 otherwise.<latexit sha1_base64="4vyAlKNdd7n8kEzBLhheSN736jk=">AAAD7XicdVPLbhMxFJ1OCpTwaApLNldEVEWqopkKCVhUqsQGFoUgkbRSPBl5HGfixPOQ7UAjZz6CDQtAbPkfdvwN9mSSpk2xNNK5j3POvZYnyjmTyvP+brm17Vu37+zcrd+7/+DhbmPvUVdmU0Foh2Q8E+cRlpSzlHYUU5ye54LiJOL0LJq8sfWzz1RIlqWf1CynQYLjlA0Zwcqkwj13Gyl6oXRbFL3TORphpTvFIZSga4FkcYL7RwHsHwPKRTbo66QINTv2i2Wcmnhs4x6gBKsRwVy/N9GpaRsXMIdKNmRL3XDcRwMcx1TAukXQ1yuBd0UffUhojEsROwmqX45aKc4X3LDzvwHXx7mcYg7e0tZSF21RZDyDm3y6K5/uVZ/V4ps+dscrPt1NH6ivbVuuWa1cWnA6VEjXAVBEY5ZqLASeFZrzwuQAfIB9OGCHMH4OiKVQMc30tuqZIkqi7EJnakTFFyZpy/IQTQeVUh0JFo9Uy1LCRtNreeWBTeBXoOlUpx02/qBBRqYJTRXhWMqe7+UqMMqKEU6N9lTSHJMJjmnPwBQnVAa6fK4FPDOZAQwzYb5UQZldZ2icSDlLItNp70der9nkTbXeVA1fBZql+VTRlCyMhlMOKgP79mHABCWKzwzARDAzK5ARFpgo84fYS/Cvr7wJOket1y3/44vmiVfdxo7zxHnqHDi+89I5cd46bafjEHfifnW/uz9qee1b7Wft16LV3ao4j50rp/b7Hw+5QzI=</latexit><latexit sha1_base64="4vyAlKNdd7n8kEzBLhheSN736jk=">AAAD7XicdVPLbhMxFJ1OCpTwaApLNldEVEWqopkKCVhUqsQGFoUgkbRSPBl5HGfixPOQ7UAjZz6CDQtAbPkfdvwN9mSSpk2xNNK5j3POvZYnyjmTyvP+brm17Vu37+zcrd+7/+DhbmPvUVdmU0Foh2Q8E+cRlpSzlHYUU5ye54LiJOL0LJq8sfWzz1RIlqWf1CynQYLjlA0Zwcqkwj13Gyl6oXRbFL3TORphpTvFIZSga4FkcYL7RwHsHwPKRTbo66QINTv2i2Wcmnhs4x6gBKsRwVy/N9GpaRsXMIdKNmRL3XDcRwMcx1TAukXQ1yuBd0UffUhojEsROwmqX45aKc4X3LDzvwHXx7mcYg7e0tZSF21RZDyDm3y6K5/uVZ/V4ps+dscrPt1NH6ivbVuuWa1cWnA6VEjXAVBEY5ZqLASeFZrzwuQAfIB9OGCHMH4OiKVQMc30tuqZIkqi7EJnakTFFyZpy/IQTQeVUh0JFo9Uy1LCRtNreeWBTeBXoOlUpx02/qBBRqYJTRXhWMqe7+UqMMqKEU6N9lTSHJMJjmnPwBQnVAa6fK4FPDOZAQwzYb5UQZldZ2icSDlLItNp70der9nkTbXeVA1fBZql+VTRlCyMhlMOKgP79mHABCWKzwzARDAzK5ARFpgo84fYS/Cvr7wJOket1y3/44vmiVfdxo7zxHnqHDi+89I5cd46bafjEHfifnW/uz9qee1b7Wft16LV3ao4j50rp/b7Hw+5QzI=</latexit><latexit sha1_base64="4vyAlKNdd7n8kEzBLhheSN736jk=">AAAD7XicdVPLbhMxFJ1OCpTwaApLNldEVEWqopkKCVhUqsQGFoUgkbRSPBl5HGfixPOQ7UAjZz6CDQtAbPkfdvwN9mSSpk2xNNK5j3POvZYnyjmTyvP+brm17Vu37+zcrd+7/+DhbmPvUVdmU0Foh2Q8E+cRlpSzlHYUU5ye54LiJOL0LJq8sfWzz1RIlqWf1CynQYLjlA0Zwcqkwj13Gyl6oXRbFL3TORphpTvFIZSga4FkcYL7RwHsHwPKRTbo66QINTv2i2Wcmnhs4x6gBKsRwVy/N9GpaRsXMIdKNmRL3XDcRwMcx1TAukXQ1yuBd0UffUhojEsROwmqX45aKc4X3LDzvwHXx7mcYg7e0tZSF21RZDyDm3y6K5/uVZ/V4ps+dscrPt1NH6ivbVuuWa1cWnA6VEjXAVBEY5ZqLASeFZrzwuQAfIB9OGCHMH4OiKVQMc30tuqZIkqi7EJnakTFFyZpy/IQTQeVUh0JFo9Uy1LCRtNreeWBTeBXoOlUpx02/qBBRqYJTRXhWMqe7+UqMMqKEU6N9lTSHJMJjmnPwBQnVAa6fK4FPDOZAQwzYb5UQZldZ2icSDlLItNp70der9nkTbXeVA1fBZql+VTRlCyMhlMOKgP79mHABCWKzwzARDAzK5ARFpgo84fYS/Cvr7wJOket1y3/44vmiVfdxo7zxHnqHDi+89I5cd46bafjEHfifnW/uz9qee1b7Wft16LV3ao4j50rp/b7Hw+5QzI=</latexit><latexit sha1_base64="4vyAlKNdd7n8kEzBLhheSN736jk=">AAAD7XicdVPLbhMxFJ1OCpTwaApLNldEVEWqopkKCVhUqsQGFoUgkbRSPBl5HGfixPOQ7UAjZz6CDQtAbPkfdvwN9mSSpk2xNNK5j3POvZYnyjmTyvP+brm17Vu37+zcrd+7/+DhbmPvUVdmU0Foh2Q8E+cRlpSzlHYUU5ye54LiJOL0LJq8sfWzz1RIlqWf1CynQYLjlA0Zwcqkwj13Gyl6oXRbFL3TORphpTvFIZSga4FkcYL7RwHsHwPKRTbo66QINTv2i2Wcmnhs4x6gBKsRwVy/N9GpaRsXMIdKNmRL3XDcRwMcx1TAukXQ1yuBd0UffUhojEsROwmqX45aKc4X3LDzvwHXx7mcYg7e0tZSF21RZDyDm3y6K5/uVZ/V4ps+dscrPt1NH6ivbVuuWa1cWnA6VEjXAVBEY5ZqLASeFZrzwuQAfIB9OGCHMH4OiKVQMc30tuqZIkqi7EJnakTFFyZpy/IQTQeVUh0JFo9Uy1LCRtNreeWBTeBXoOlUpx02/qBBRqYJTRXhWMqe7+UqMMqKEU6N9lTSHJMJjmnPwBQnVAa6fK4FPDOZAQwzYb5UQZldZ2icSDlLItNp70der9nkTbXeVA1fBZql+VTRlCyMhlMOKgP79mHABCWKzwzARDAzK5ARFpgo84fYS/Cvr7wJOket1y3/44vmiVfdxo7zxHnqHDi+89I5cd46bafjEHfifnW/uz9qee1b7Wft16LV3ao4j50rp/b7Hw+5QzI=</latexit>
̂U ̂V

THE LOG POSTERIORI OF PMF
• The log posteriori of PMF is as follows:
• Hence, the following equation holds:
!19
arg max
Û, ˆV
Pr[ Û, ˆV |P⌦(M), ⌦, 2
, 2
U , 2
V ]
= arg min
Û, ˆV
X
(i,j)2⌦
(Mij
Ûi
ˆV †
j )2
+ U k Ûk2
F + V k ˆV k2
F
<latexit sha1_base64="XlBMvMAdTTRcsp7OF0gzCIi5a5E=">AAADAHicbZJNj9MwEIad8LWUry4cOHAZUbFqxVKlERJwQFoJCXFZUSSSXalOLdd1U3fjJLIdtFU2XPgrXDjAaq/8DG78G9xuikrLSFYeve+Mx554lCdCG8/77bhXrl67fmPnZuPW7Tt37zV374c6KxTjAcuSTB2PqOaJSHlghEn4ca44laOEH41O3iz8o09caZGlH80855GkcSomglFjJbLrPNzDVMWAJT0lJZ5SUwbVPiwhrCrAhp+asq+qwaZ3Bn2C30seU2gfdqy8ZPvVIpZ06K+IBGscDv0IMG683oO6rUi324LtqwtJyrbYh1kHsEjr7SuAsn1ISjGr4BnUdUSsCslsiMc0jrmCztCv4CngxM5ibA8B+GyVb5G8HfprbvjXDVcuaba8rrcM2IZeDS1UR580f+FxxgrJU8MSqvWg5+UmKqkygiW8auBC85yyExrzgcWUSq6jcvkHK3hilTFMMmVXamCprleUVGo9lyObKamZ6k1vIf7PGxRm8jIqRZoXhqfsstGkSMBksHgOMBaKM5PMLVCmhD0rsClVlBn7aBp2CL3NK29D4HdfdXsfnrcO/HoaO+gReozaqIdeoAP0DvVRgJjz2fnqfHd+uF/cb+65e3GZ6jp1zQP0T7g//wA+U+2w</latexit><latexit sha1_base64="XlBMvMAdTTRcsp7OF0gzCIi5a5E=">AAADAHicbZJNj9MwEIad8LWUry4cOHAZUbFqxVKlERJwQFoJCXFZUSSSXalOLdd1U3fjJLIdtFU2XPgrXDjAaq/8DG78G9xuikrLSFYeve+Mx554lCdCG8/77bhXrl67fmPnZuPW7Tt37zV374c6KxTjAcuSTB2PqOaJSHlghEn4ca44laOEH41O3iz8o09caZGlH80855GkcSomglFjJbLrPNzDVMWAJT0lJZ5SUwbVPiwhrCrAhp+asq+qwaZ3Bn2C30seU2gfdqy8ZPvVIpZ06K+IBGscDv0IMG683oO6rUi324LtqwtJyrbYh1kHsEjr7SuAsn1ISjGr4BnUdUSsCslsiMc0jrmCztCv4CngxM5ibA8B+GyVb5G8HfprbvjXDVcuaba8rrcM2IZeDS1UR580f+FxxgrJU8MSqvWg5+UmKqkygiW8auBC85yyExrzgcWUSq6jcvkHK3hilTFMMmVXamCprleUVGo9lyObKamZ6k1vIf7PGxRm8jIqRZoXhqfsstGkSMBksHgOMBaKM5PMLVCmhD0rsClVlBn7aBp2CL3NK29D4HdfdXsfnrcO/HoaO+gReozaqIdeoAP0DvVRgJjz2fnqfHd+uF/cb+65e3GZ6jp1zQP0T7g//wA+U+2w</latexit><latexit sha1_base64="XlBMvMAdTTRcsp7OF0gzCIi5a5E=">AAADAHicbZJNj9MwEIad8LWUry4cOHAZUbFqxVKlERJwQFoJCXFZUSSSXalOLdd1U3fjJLIdtFU2XPgrXDjAaq/8DG78G9xuikrLSFYeve+Mx554lCdCG8/77bhXrl67fmPnZuPW7Tt37zV374c6KxTjAcuSTB2PqOaJSHlghEn4ca44laOEH41O3iz8o09caZGlH80855GkcSomglFjJbLrPNzDVMWAJT0lJZ5SUwbVPiwhrCrAhp+asq+qwaZ3Bn2C30seU2gfdqy8ZPvVIpZ06K+IBGscDv0IMG683oO6rUi324LtqwtJyrbYh1kHsEjr7SuAsn1ISjGr4BnUdUSsCslsiMc0jrmCztCv4CngxM5ibA8B+GyVb5G8HfprbvjXDVcuaba8rrcM2IZeDS1UR580f+FxxgrJU8MSqvWg5+UmKqkygiW8auBC85yyExrzgcWUSq6jcvkHK3hilTFMMmVXamCprleUVGo9lyObKamZ6k1vIf7PGxRm8jIqRZoXhqfsstGkSMBksHgOMBaKM5PMLVCmhD0rsClVlBn7aBp2CL3NK29D4HdfdXsfnrcO/HoaO+gReozaqIdeoAP0DvVRgJjz2fnqfHd+uF/cb+65e3GZ6jp1zQP0T7g//wA+U+2w</latexit><latexit sha1_base64="XlBMvMAdTTRcsp7OF0gzCIi5a5E=">AAADAHicbZJNj9MwEIad8LWUry4cOHAZUbFqxVKlERJwQFoJCXFZUSSSXalOLdd1U3fjJLIdtFU2XPgrXDjAaq/8DG78G9xuikrLSFYeve+Mx554lCdCG8/77bhXrl67fmPnZuPW7Tt37zV374c6KxTjAcuSTB2PqOaJSHlghEn4ca44laOEH41O3iz8o09caZGlH80855GkcSomglFjJbLrPNzDVMWAJT0lJZ5SUwbVPiwhrCrAhp+asq+qwaZ3Bn2C30seU2gfdqy8ZPvVIpZ06K+IBGscDv0IMG683oO6rUi324LtqwtJyrbYh1kHsEjr7SuAsn1ISjGr4BnUdUSsCslsiMc0jrmCztCv4CngxM5ibA8B+GyVb5G8HfprbvjXDVcuaba8rrcM2IZeDS1UR580f+FxxgrJU8MSqvWg5+UmKqkygiW8auBC85yyExrzgcWUSq6jcvkHK3hilTFMMmVXamCprleUVGo9lyObKamZ6k1vIf7PGxRm8jIqRZoXhqfsstGkSMBksHgOMBaKM5PMLVCmhD0rsClVlBn7aBp2CL3NK29D4HdfdXsfnrcO/HoaO+gReozaqIdeoAP0DvVRgJjz2fnqfHd+uF/cb+65e3GZ6jp1zQP0T7g//wA+U+2w</latexit>
ln Pr[ Û, ˆV |P⌦(M), ⌦, 2
, 2
U , 2
V ]
=
1
2 2
mX
i=1
nX
j=1
I⌦
ij(Mij
Ûi
ˆV †
j )2 1
2 2
U
mX
i=1
Û†
i
Ûi
1
2 2
V
nX
j=1
ˆV †
j
ˆVj
1
2
((
mX
i=1
nX
j=1
I⌦
ij) ln 2
+ mk ln 2
U + nk ln 2
V ) + C
<latexit sha1_base64="FacZlGnfamKX+f7FRk0QGJSESm8=">AAAD2HicnVNNj9MwEE0TPpbysV04chlRUbVaWjUREnBYaaW9wAFRJNJdqWki13VTt3ESxQ6i8ubAARBXfho3fgV/ASdtttluT4wU+fmNZ+bNxJ7EAeWi3/9T041bt+/cPbhXv//g4aPDxtHjIY/SBBMbR0GUXEwQJwENiS2oCMhFnBDEJgE5nyzPcv/5Z5JwGoWfxComY4b8kM4oRkJR3lHtb8sJQnAE+SLkIMlGzhwJaWcvoADD7BIGnvOBER9B+31H0QVWK6c+Q65VIs+u4KFrjcFx6ict6EppZuBESgRI6yoMMkXylHmSnpiZK1m5XRTbUG0ZEnOMAvkuc9dF1dlFplSs1y5spHq01OotXGeKfJ8kHdeqd0GaV4W3IvcULvOU0ZXM+5IMK0m2cnclbJl8Eq1qJkt1Ae3/ax86UPywco7HwGBZofIWFRleJ5VkFXgMZ+A1mv1evzC4CcwNaGobG3iN3840wikjocAB4nxk9mMxligRFAckqzspJzHCS+STkYIhYoSPZXEzM3iumCnMokR9oYCCrUZIxDhfsYk6mffLd305uc83SsXs9VjSME4FCfG60CwNQESQX3OY0oRgEawUQDihSivgOUoQFuox1NUQzN2WbwLb6r3pmR9fNk+tzTQOtKfaM62tmdor7VR7qw00W8P6UL/Uv+nfjZHx1fhh/Fwf1WubmCfaNTN+/QMTIjkx</latexit><latexit sha1_base64="FacZlGnfamKX+f7FRk0QGJSESm8=">AAAD2HicnVNNj9MwEE0TPpbysV04chlRUbVaWjUREnBYaaW9wAFRJNJdqWki13VTt3ESxQ6i8ubAARBXfho3fgV/ASdtttluT4wU+fmNZ+bNxJ7EAeWi3/9T041bt+/cPbhXv//g4aPDxtHjIY/SBBMbR0GUXEwQJwENiS2oCMhFnBDEJgE5nyzPcv/5Z5JwGoWfxComY4b8kM4oRkJR3lHtb8sJQnAE+SLkIMlGzhwJaWcvoADD7BIGnvOBER9B+31H0QVWK6c+Q65VIs+u4KFrjcFx6ict6EppZuBESgRI6yoMMkXylHmSnpiZK1m5XRTbUG0ZEnOMAvkuc9dF1dlFplSs1y5spHq01OotXGeKfJ8kHdeqd0GaV4W3IvcULvOU0ZXM+5IMK0m2cnclbJl8Eq1qJkt1Ae3/ax86UPywco7HwGBZofIWFRleJ5VkFXgMZ+A1mv1evzC4CcwNaGobG3iN3840wikjocAB4nxk9mMxligRFAckqzspJzHCS+STkYIhYoSPZXEzM3iumCnMokR9oYCCrUZIxDhfsYk6mffLd305uc83SsXs9VjSME4FCfG60CwNQESQX3OY0oRgEawUQDihSivgOUoQFuox1NUQzN2WbwLb6r3pmR9fNk+tzTQOtKfaM62tmdor7VR7qw00W8P6UL/Uv+nfjZHx1fhh/Fwf1WubmCfaNTN+/QMTIjkx</latexit><latexit sha1_base64="FacZlGnfamKX+f7FRk0QGJSESm8=">AAAD2HicnVNNj9MwEE0TPpbysV04chlRUbVaWjUREnBYaaW9wAFRJNJdqWki13VTt3ESxQ6i8ubAARBXfho3fgV/ASdtttluT4wU+fmNZ+bNxJ7EAeWi3/9T041bt+/cPbhXv//g4aPDxtHjIY/SBBMbR0GUXEwQJwENiS2oCMhFnBDEJgE5nyzPcv/5Z5JwGoWfxComY4b8kM4oRkJR3lHtb8sJQnAE+SLkIMlGzhwJaWcvoADD7BIGnvOBER9B+31H0QVWK6c+Q65VIs+u4KFrjcFx6ict6EppZuBESgRI6yoMMkXylHmSnpiZK1m5XRTbUG0ZEnOMAvkuc9dF1dlFplSs1y5spHq01OotXGeKfJ8kHdeqd0GaV4W3IvcULvOU0ZXM+5IMK0m2cnclbJl8Eq1qJkt1Ae3/ax86UPywco7HwGBZofIWFRleJ5VkFXgMZ+A1mv1evzC4CcwNaGobG3iN3840wikjocAB4nxk9mMxligRFAckqzspJzHCS+STkYIhYoSPZXEzM3iumCnMokR9oYCCrUZIxDhfsYk6mffLd305uc83SsXs9VjSME4FCfG60CwNQESQX3OY0oRgEawUQDihSivgOUoQFuox1NUQzN2WbwLb6r3pmR9fNk+tzTQOtKfaM62tmdor7VR7qw00W8P6UL/Uv+nfjZHx1fhh/Fwf1WubmCfaNTN+/QMTIjkx</latexit><latexit sha1_base64="FacZlGnfamKX+f7FRk0QGJSESm8=">AAAD2HicnVNNj9MwEE0TPpbysV04chlRUbVaWjUREnBYaaW9wAFRJNJdqWki13VTt3ESxQ6i8ubAARBXfho3fgV/ASdtttluT4wU+fmNZ+bNxJ7EAeWi3/9T041bt+/cPbhXv//g4aPDxtHjIY/SBBMbR0GUXEwQJwENiS2oCMhFnBDEJgE5nyzPcv/5Z5JwGoWfxComY4b8kM4oRkJR3lHtb8sJQnAE+SLkIMlGzhwJaWcvoADD7BIGnvOBER9B+31H0QVWK6c+Q65VIs+u4KFrjcFx6ict6EppZuBESgRI6yoMMkXylHmSnpiZK1m5XRTbUG0ZEnOMAvkuc9dF1dlFplSs1y5spHq01OotXGeKfJ8kHdeqd0GaV4W3IvcULvOU0ZXM+5IMK0m2cnclbJl8Eq1qJkt1Ae3/ax86UPywco7HwGBZofIWFRleJ5VkFXgMZ+A1mv1evzC4CcwNaGobG3iN3840wikjocAB4nxk9mMxligRFAckqzspJzHCS+STkYIhYoSPZXEzM3iumCnMokR9oYCCrUZIxDhfsYk6mffLd305uc83SsXs9VjSME4FCfG60CwNQESQX3OY0oRgEawUQDihSivgOUoQFuox1NUQzN2WbwLb6r3pmR9fNk+tzTQOtKfaM62tmdor7VR7qw00W8P6UL/Uv+nfjZHx1fhh/Fwf1WubmCfaNTN+/QMTIjkx</latexit>

WHY WE USE ALTERNATING MINIMIZATION?
• We can just use Gradient Descent with random initialization without alternating
minimization.
• Why we should use alternating minimization?
• First of all, Gradient Descent can oscillate without alternating minimization.
• The MRMA paper(NIPS’17) said that it is possible to overfit without alternating
minimization.
• Alternating Minimization is not the only way to avoid overfitting.
• However, Besag(1986) said that it is a good way to avoid overfitting.
• Glendinning(1989) said that alternating minimization is robust to initial point empirically.
• However, the initial point is nonetheless important.
• Because it can fall into the local minima.
!21

LOCAL MINIMA PROBLEM OF AM
• In otherwise, Jain et al(2013) showed that AM has no local minima problem.
• Noiseless case ONLY. Exact Completion Setting.
• With the slightly modified algorithm.
• With some assumptions.
• Let’s look at the modified algorithm first.
!23

THE MODIFIED ALGORITHM
!25
Mini-batch

!26
Alternating Minimization
similar to Funk SVD

!27
When performing initialization using SVD + Clipping,
the local optimum found by the SVD method 
is close enough to the global optimum!

MOTIVATION OF THE INCOHERENCE ASSUMPTION
• Consider the rank-1 matrix M:
• Let |Ω| be the number of observed entries of M.
• Then, we can see only 0 with probability 1 - |Ω| / (mn).
• If sample set doesn’t contain 1, we can not complete matrix exactly.
• Therefore, it is impossible to recover all low-rank matrices.
!28
M = e1e⇤
n =
2
6
6
6
4
0 0 · · · 0 1
0 0 · · · 0 0
...
...
...
...
...
0 0 · · · 0 0
3
7
7
7
5
<latexit sha1_base64="BDwoBlvzuP9vRxmJB62/VzKKa7Y=">AAACp3ichVHfS9xAEN6kttrYH9f2sS+Dh6X4cCRFaH0QBF8EKZxg9OCSHpu9yd3iZhN2J8IR8rf1f/DN/8bNmYdWRT8Y+Pb7dmZnZ7JKSUtheOv5rzZev9ncehtsv3v/4ePg0+cLW9ZGYCxKVZpJxi0qqTEmSQonlUFeZAovs6vjzr+8RmNlqc9pVWFa8IWWuRScnDQb/P0Nh4CzyIX+swfBISQKc5pCk2S4kLrhxvBV24gObRDCN+gimZdks/4QQZI87YSdk1x3Ume9SJ6vg3retwMtJEYulpTOBsNwFK4Bj0nUkyHrMZ4NblxlUReoSShu7TQKK0pdXZJCYRsktcWKiyu+wKmjmhdo02Y96BZ2nTKHvDQuNMFa/Tej4YW1qyJzNwtOS/vQ68SnvGlN+a+0kbqqCbW4fyivFVAJ3dZgLg0KUitHuDDS9QpiyQ0X5HYbuCFED7/8mMQ/Rgej6Gx/eLTfT2OLfWU77DuL2E92xE7YmMVMeLveqXfuxf6eP/Yv/Mn9Vd/rc76w/+DzOybDwwE=</latexit><latexit sha1_base64="BDwoBlvzuP9vRxmJB62/VzKKa7Y=">AAACp3ichVHfS9xAEN6kttrYH9f2sS+Dh6X4cCRFaH0QBF8EKZxg9OCSHpu9yd3iZhN2J8IR8rf1f/DN/8bNmYdWRT8Y+Pb7dmZnZ7JKSUtheOv5rzZev9ncehtsv3v/4ePg0+cLW9ZGYCxKVZpJxi0qqTEmSQonlUFeZAovs6vjzr+8RmNlqc9pVWFa8IWWuRScnDQb/P0Nh4CzyIX+swfBISQKc5pCk2S4kLrhxvBV24gObRDCN+gimZdks/4QQZI87YSdk1x3Ume9SJ6vg3retwMtJEYulpTOBsNwFK4Bj0nUkyHrMZ4NblxlUReoSShu7TQKK0pdXZJCYRsktcWKiyu+wKmjmhdo02Y96BZ2nTKHvDQuNMFa/Tej4YW1qyJzNwtOS/vQ68SnvGlN+a+0kbqqCbW4fyivFVAJ3dZgLg0KUitHuDDS9QpiyQ0X5HYbuCFED7/8mMQ/Rgej6Gx/eLTfT2OLfWU77DuL2E92xE7YmMVMeLveqXfuxf6eP/Yv/Mn9Vd/rc76w/+DzOybDwwE=</latexit><latexit sha1_base64="BDwoBlvzuP9vRxmJB62/VzKKa7Y=">AAACp3ichVHfS9xAEN6kttrYH9f2sS+Dh6X4cCRFaH0QBF8EKZxg9OCSHpu9yd3iZhN2J8IR8rf1f/DN/8bNmYdWRT8Y+Pb7dmZnZ7JKSUtheOv5rzZev9ncehtsv3v/4ePg0+cLW9ZGYCxKVZpJxi0qqTEmSQonlUFeZAovs6vjzr+8RmNlqc9pVWFa8IWWuRScnDQb/P0Nh4CzyIX+swfBISQKc5pCk2S4kLrhxvBV24gObRDCN+gimZdks/4QQZI87YSdk1x3Ume9SJ6vg3retwMtJEYulpTOBsNwFK4Bj0nUkyHrMZ4NblxlUReoSShu7TQKK0pdXZJCYRsktcWKiyu+wKmjmhdo02Y96BZ2nTKHvDQuNMFa/Tej4YW1qyJzNwtOS/vQ68SnvGlN+a+0kbqqCbW4fyivFVAJ3dZgLg0KUitHuDDS9QpiyQ0X5HYbuCFED7/8mMQ/Rgej6Gx/eLTfT2OLfWU77DuL2E92xE7YmMVMeLveqXfuxf6eP/Yv/Mn9Vd/rc76w/+DzOybDwwE=</latexit><latexit sha1_base64="BDwoBlvzuP9vRxmJB62/VzKKa7Y=">AAACp3ichVHfS9xAEN6kttrYH9f2sS+Dh6X4cCRFaH0QBF8EKZxg9OCSHpu9yd3iZhN2J8IR8rf1f/DN/8bNmYdWRT8Y+Pb7dmZnZ7JKSUtheOv5rzZev9ncehtsv3v/4ePg0+cLW9ZGYCxKVZpJxi0qqTEmSQonlUFeZAovs6vjzr+8RmNlqc9pVWFa8IWWuRScnDQb/P0Nh4CzyIX+swfBISQKc5pCk2S4kLrhxvBV24gObRDCN+gimZdks/4QQZI87YSdk1x3Ume9SJ6vg3retwMtJEYulpTOBsNwFK4Bj0nUkyHrMZ4NblxlUReoSShu7TQKK0pdXZJCYRsktcWKiyu+wKmjmhdo02Y96BZ2nTKHvDQuNMFa/Tej4YW1qyJzNwtOS/vQ68SnvGlN+a+0kbqqCbW4fyivFVAJ3dZgLg0KUitHuDDS9QpiyQ0X5HYbuCFED7/8mMQ/Rgej6Gx/eLTfT2OLfWU77DuL2E92xE7YmMVMeLveqXfuxf6eP/Yv/Mn9Vd/rc76w/+DzOybDwwE=</latexit>

THE INCOHERENCE ASSUMPTION
• More generally, it is hard to recover if the singular vectors of the matrix M are similar to
standard basis.
• Because, information is highly concentrated on specific region.
• Hence, the singular vectors need to be sufficiently spread.
• This is why the paper introduce the incoherence assumption.
• The EMCCO paper said that the random low-rank (orthogonal) matrices satisfy the
incoherent assumption.
!29

MAIN RESULTS
• Required entries: |Ω| = O((k4.5 log k) n log n)
• Required steps: O(log (1/ε))
• The EMCCO paper said that the optimum number of required entries is O(n log n)
!30

COMPARISON TO NUCLEAR NORM MINIMIZATION
• Alternating Minimization
• Required entries: |Ω| = O((k4.5 log k) n log n)
• Required steps: O(log (1/ε))
• Nuclear Norm Minimization (Convex Relaxation)
• Required entries: |Ω| = O((k) n log n)
• Required steps: O(ε-1/2)
• Nuclear Norm Minimization requires smaller the number of samples.
• Alternating Minimization converges faster than Nuclear Norm Minimization.
• Theoretical bound is not tight in Nuclear Norm Minimization.
!31

INDUCTION STEP (INCOHERENCE)
!36

WHY RANDOM INITIALIZATION WORKS?
• In prior paper, initialization steps matters to guarantee global optimality.
• In practice, it is hard to determine hyper-parameters for clipping.
• However, random initialization works well practically.
• In this section, I’ll show you to why random initialization works.
• It can be view as removing hyper-parameters.
• Similar to motivation of WGAN-GP.
• WGAN is really hard to determine hyper-parameter that performance is highly sensitive to.
!38

PRELIMINARY: MATRIX SENSING PROBLEM
• Quite similar to the matrix completion problem.
• A Low-rank version of linear regression (y = xβ problem).
• RIP condition:
• Actually, the Jain et al’s work (2013) proved in low-rank matrix sensing problem first.
• And then, they extended their work to low-rank matrix completion problem.
!39

NO SPURIOUS LOCAL MINIMA
• Rong et al (NIPS 2016), Matrix Completion has No Spurious Local Minimum
• Matrix Completion problem with Semi-definite matrix assumption.
• Showed why random initialization works with proper regularizer.
• Srinadh et al (NIPS 2016), Global Optimality of Local Search for Low Rank Matrix
Recovery
• Matrix Sensing Problem.
• Showed all local minima are very close to a global optimum with noisy measurements.
• With a curvature bound (RIP condition),  
a polynomial time global convergence is guaranteed by SGD from random initialization.
• Rong et al (ArXiv 2017), No Spurious Local Minima in Non-convex Low Rank Problems:  
A Unified Geometric Analysis
• Extended Srinadh et al’s works from matrix sensing to matrix completion and robust PCA.
• Showed no high-order saddle point exists if being with proper regularizer.
!40

REFERENCES
[1] Koren,Yehuda, Robert Bell, and ChrisVolinsky. "Matrix factorization techniques for
recommender systems." Computer 8 (2009): 30-37.
[2] Mnih,Andriy, and Ruslan R. Salakhutdinov. "Probabilistic matrix factorization." Advances
in neural information processing systems. 2008.
[3] Jain, Prateek, Praneeth Netrapalli, and Sujay Sanghavi. "Low-rank matrix completion
using alternating minimization." Proceedings of the forty-fifth annual ACM symposium on
Theory of computing.ACM, 2013.
[4] Ge, Rong, Jason D. Lee, and Tengyu Ma. "Matrix completion has no spurious local
minimum." Advances in Neural Information Processing Systems. 2016.
[5] Bhojanapalli, Srinadh, Behnam Neyshabur, and Nati Srebro. "Global optimality of local
search for low rank matrix recovery." Advances in Neural Information Processing Systems.
2016.
[6] Ge, Rong, Chi Jin, andYi Zheng. "No spurious local minima in nonconvex low rank
problems:A unified geometric analysis." arXiv preprint arXiv:1704.00708 (2017)
[7] how does Netflix recommend movies?
[8] Li, Dongsheng, et al. "Mixture-Rank Matrix Approximation for Collaborative Filtering."
Advances in Neural Information Processing Systems. 2017.
!42

REFERENCES
[9] Candès, Emmanuel J., and Benjamin Recht. "Exact matrix completion via convex
optimization." Foundations of Computational mathematics 9.6 (2009): 717.
[10] Glendinning, R. H. "An evaluation of the ICM algorithm for image reconstruction."
Journal of Statistical Computation and Simulation 31.3 (1989): 169-185.
[11] http://sifter.org/~simon/journal/20061211.html
[12] https://www.slideshare.net/ssuser62b35f/exact-matrix-completion-via-convex-
optimization-slideppt
[13] Lee, Joonseok, et al. "Local low-rank matrix approximation." International Conference on
Machine Learning. 2013.
[14] Sedhain, Suvash, et al. "Autorec:Autoencoders meet collaborative
filtering." Proceedings of the 24th International Conference onWorldWideWeb.ACM, 2015.
[15] Zheng,Yin, et al. "A neural autoregressive approach to collaborative filtering." arXiv
preprint arXiv:1605.09477 (2016).
[16] Fu, Mingsheng, et al. "Attention based collaborative filtering." Neurocomputing (2018).
[17] Li, Dongsheng, et al. "Low-rank matrix approximation with stability." International
Conference on Machine Learning. 2016.
!43

Why biased matrix factorization works well?

Recomendados

Recomendados

Más contenido relacionado

La actualidad más candente

La actualidad más candente (20)

Similar a Why biased matrix factorization works well?

Similar a Why biased matrix factorization works well? (20)

Más de Joonyoung Yi

Más de Joonyoung Yi (9)

Último

Último (20)

Why biased matrix factorization works well?