Se ha denunciado esta presentación.
Utilizamos tu perfil de LinkedIn y tus datos de actividad para personalizar los anuncios y mostrarte publicidad más relevante. Puedes cambiar tus preferencias de publicidad en cualquier momento.

AIによるアニメ生成の挑戦

46.857 visualizaciones

Publicado el

「AIによるアニメ生成の挑戦」。DeNAの「構造的生成学習」技術開発により、従来AI生成で難しかった “複雑な構造での高品質生成”、"構造変化が大きい画像間の補完生成・中割” 等の課題を解消し、多様なキャラクター全体でのアニメ生成・中割生成、を実現しています。DeNA TechCon 2019での DeNA "アニメ生成プロジェクト" の活動紹介、発表資料です。

[AIによる生成アニメ例]
https://www.youtube.com/watch?v=tOZW_KWb8b0

---
"AIによるアニメ生成の挑戦".
濱田晃一 , 李天琦.
DeNA TechCon 2019.
---
"Challenges toward Anime Generation with Deep Generative Models".
Koichi Hamada and Tianqi Li.
In DeNA Technology Conference 2019.

Publicado en: Ingeniería
  • Sé el primero en comentar

AIによるアニメ生成の挑戦

  1. 1. #denatechcon #denatechcon
  2. 2. #denatechcon
  3. 3. #denatechcon
  4. 4. #denatechcon 00./ 0
  5. 5. #denatechcon 00./ 0
  6. 6. #denatechcon
  7. 7. #denatechcon
  8. 8. #denatechcon // . /
  9. 9. #denatechcon // . /
  10. 10. #denatechcon
  11. 11. #denatechcon
  12. 12. #denatechcon
  13. 13. #denatechcon
  14. 14. #denatechcon
  15. 15. #denatechcon AGENDA
  16. 16. #denatechcon (@hamadakoichi) Mobage 2010 6 Mobage2011 4 DeNA DeNA2014 10 : ( ) TokyoWebmining - 1500 - 2010 60 DeNA 8 AI AI AI
  17. 17. #denatechcon 50 /
  18. 18. #denatechcon
  19. 19. #denatechcon , , , , , . “ ”. 232 (7/20/2017). 0 3
  20. 20. #denatechcon 5 4 /0 / 5 4 /0 : 6 / .
  21. 21. #denatechcon Full-body High-resolution Anime Generation with Progressive Structure-conditional Generative Adversarial Networks Koichi Hamada, Kentaro Tachibana, Tianqi Li, Hiroto Honda, and Yusuke Uchida. In ECCV Workshop 2018. (ECCV: European Conference on Computer Vision)
  22. 22. #denatechcon AGENDA
  23. 23. #denatechcon AGENDA
  24. 24. #denatechcon
  25. 25. #denatechcon 1 3 5 7 2 4 6 8
  26. 26. #denatechcon 1 3 5 7 2 4 6 8
  27. 27. #denatechcon ProgressiveGAN (Karras et al., ICLR 2018) BigGAN (Brock et al., ICLR 2019) 1 3 5 7 2 4 6 8
  28. 28. #denatechcon ProgressiveGAN (Karras et al., ICLR 2018) BigGAN (Brock et al., ICLR 2019)
  29. 29. #denatechcon ProgressiveGAN (Karras et al., ICLR 2018) BigGAN (Brock et al., ICLR 2019)
  30. 30. #denatechcon ProgressiveGAN (Karras et al., ICLR 2018) BigGAN (Brock et al., ICLR 2019)
  31. 31. #denatechcon
  32. 32. #denatechcon Generative Adversarial Nets. Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde- Farley, Sherjil Ozair, Aaron Courville, Yoshua Bengio. arXiv:1406.2661. In NIPS 2014.
  33. 33. #denatechcon
  34. 34. #denatechcon ProgressiveGAN (Karras et al., ICLR 2018) BigGAN (Brock et al., ICLR 2019)
  35. 35. #denatechcon ProgressiveGAN (Karras et al., ICLR 2018) BigGAN (Brock et al., ICLR 2019)
  36. 36. #denatechcon Progressive Growing of GANs for Improved Quality, Stability, and Variation. Tero Karras, Timo Aila, Samuli Laine, Jaakko Lehtinen. In ICLR 2018. (1024X1024) (256x256)
  37. 37. #denatechcon .441 7 545 7 4 / Progressive Growing of GANs for Improved Quality, Stability, and Variation Tero Karras, Timo Aila, Samuli Laine, Jaakko Lehtinen. In ICLR 2018.
  38. 38. #denatechcon / 5. 44 5 Progressive Growing of GANs for Improved Quality, Stability, and Variation Tero Karras, Timo Aila, Samuli Laine, Jaakko Lehtinen. In ICLR 2018.
  39. 39. #denatechcon ProgressiveGAN (Karras et al., ICLR 2018) BigGAN (Brock et al., ICLR 2019)
  40. 40. #denatechcon ProgressiveGAN (Karras et al., ICLR 2018) BigGAN (Brock et al., ICLR 2019)
  41. 41. #denatechcon + Spectral Normalization on Generator + Self Attention + Two Time Scale Update Rule (512x512) + Spectral Normalization on Discriminator + Projection Discriminator SNGAN with Projection (Miyato+, ICLR’18) SAGAN (Zhang+, 18) BigGAN (Brock+, ICLR’19) + Large Batch Size (256→2048) + Large Channel (64→96) + Shared Embedding + Hierarchical Latent Space + Truncation Trick + Orthogonal Regularization + First Singular Value Clamp + Zero-centered Gradient Penalty Large Scale GAN Training for High Fidelity Natural Image Synthesis. Andrew Brock, Jeff Donahue, Karen Simonyan. arXiv:1809.11096. In ICLR 2018.
  42. 42. #denatechcon (512x512) Generator Typical Architecture Res Block Architecture for ImageNet at 512x512 Large Scale GAN Training for High Fidelity Natural Image Synthesis. Andrew Brock, Jeff Donahue, Karen Simonyan. arXiv:1809.11096. In ICLR 2019.
  43. 43. #denatechcon (512x512) Large Scale GAN Training for High Fidelity Natural Image Synthesis. Andrew Brock, Jeff Donahue, Karen Simonyan. arXiv:1809.11096. In ICLR 2019.
  44. 44. #denatechcon Large Scale GAN Training for High Fidelity Natural Image Synthesis. Andrew Brock, Jeff Donahue, Karen Simonyan. arXiv:1809.11096. In ICLR 2019. (512x512)
  45. 45. #denatechcon Large Scale GAN Training for High Fidelity Natural Image Synthesis. Andrew Brock, Jeff Donahue, Karen Simonyan. arXiv:1809.11096. In ICLR 2019. (512x512)
  46. 46. #denatechcon Large Scale GAN Training for High Fidelity Natural Image Synthesis. Andrew Brock, Jeff Donahue, Karen Simonyan. arXiv:1809.11096. In ICLR 2019. (512x512)
  47. 47. #denatechcon (512x512) Large Scale GAN Training for High Fidelity Natural Image Synthesis. Andrew Brock, Jeff Donahue, Karen Simonyan. arXiv:1809.11096. In ICLR 2019.
  48. 48. #denatechcon (512x512) Large Scale GAN Training for High Fidelity Natural Image Synthesis. Andrew Brock, Jeff Donahue, Karen Simonyan. arXiv:1809.11096. In ICLR 2019.
  49. 49. #denatechcon Large Scale GAN Training for High Fidelity Natural Image Synthesis. Andrew Brock, Jeff Donahue, Karen Simonyan. arXiv:1809.11096. In ICLR 2019. (512x512)
  50. 50. #denatechcon ProgressiveGAN (Karras et al., ICLR 2018) BigGAN (Brock et al., ICLR 2019)
  51. 51. #denatechcon ProgressiveGAN (Karras et al., ICLR 2018) BigGAN (Brock et al., ICLR 2019)
  52. 52. #denatechcon ProgressiveGAN (Karras et al., ICLR 2018) BigGAN (Brock et al., ICLR 2019)
  53. 53. #denatechcon ProgressiveGAN (Karras et al., ICLR 2018) BigGAN (Brock et al., ICLR 2019)
  54. 54. #denatechcon Full-body High-resolution Anime Generation with Progressive Structure-conditional Generative Adversarial Networks Koichi Hamada, Kentaro Tachibana, Tianqi Li, Hiroto Honda, and Yusuke Uchida. In ECCVW 2018. 00./ 0
  55. 55. #denatechcon Full-body High-resolution Anime Generation with Progressive Structure-conditional Generative Adversarial Networks Koichi Hamada, Kentaro Tachibana, Tianqi Li, Hiroto Honda, and Yusuke Uchida. In ECCVW 2018. 00./ 0
  56. 56. #denatechcon
  57. 57. #denatechcon
  58. 58. #denatechcon
  59. 59. #denatechcon
  60. 60. #denatechcon
  61. 61. #denatechcon
  62. 62. #denatechcon
  63. 63. #denatechcon
  64. 64. #denatechcon
  65. 65. #denatechcon
  66. 66. #denatechcon AGENDA
  67. 67. #denatechcon ProgressiveGAN (Karras et al., ICLR 2018) BigGAN (Brock et al., ICLR 2019)
  68. 68. #denatechcon
  69. 69. #denatechcon Full-body High-resolution Anime Generation with Progressive Structure-conditional Generative Adversarial Networks Koichi Hamada, Kentaro Tachibana, Tianqi Li, Hiroto Honda, and Yusuke Uchida. In ECCVW 2018. 00./ 0
  70. 70. #denatechcon Full-body High-resolution Anime Generation with Progressive Structure-conditional Generative Adversarial Networks Koichi Hamada, Kentaro Tachibana, Tianqi Li, Hiroto Honda, and Yusuke Uchida. In ECCVW 2018. 00./ 0
  71. 71. #denatechcon Full-body High-resolution Anime Generation with Progressive Structure-conditional Generative Adversarial Networks Koichi Hamada, Kentaro Tachibana, Tianqi Li, Hiroto Honda, and Yusuke Uchida. In ECCVW 2018. 0 0 0 0 0 0 0 0 0 0 0
  72. 72. #denatechcon Full-body High-resolution Anime Generation with Progressive Structure-conditional Generative Adversarial Networks Koichi Hamada, Kentaro Tachibana, Tianqi Li, Hiroto Honda, and Yusuke Uchida. In ECCVW 2018.
  73. 73. #denatechcon Full-body High-resolution Anime Generation with Progressive Structure-conditional Generative Adversarial Networks Koichi Hamada, Kentaro Tachibana, Tianqi Li, Hiroto Honda, and Yusuke Uchida. In ECCVW 2018.
  74. 74. #denatechcon Full-body High-resolution Anime Generation with Progressive Structure-conditional Generative Adversarial Networks Koichi Hamada, Kentaro Tachibana, Tianqi Li, Hiroto Honda, and Yusuke Uchida. In ECCVW 2018. 0 0 0 0 0 0 0 0 0 0 0
  75. 75. #denatechcon Full-body High-resolution Anime Generation with Progressive Structure-conditional Generative Adversarial Networks Koichi Hamada, Kentaro Tachibana, Tianqi Li, Hiroto Honda, and Yusuke Uchida. In ECCVW 2018.
  76. 76. #denatechcon Full-body High-resolution Anime Generation with Progressive Structure-conditional Generative Adversarial Networks Koichi Hamada, Kentaro Tachibana, Tianqi Li, Hiroto Honda, and Yusuke Uchida. In ECCVW 2018.
  77. 77. #denatechcon Full-body High-resolution Anime Generation with Progressive Structure-conditional Generative Adversarial Networks Koichi Hamada, Kentaro Tachibana, Tianqi Li, Hiroto Honda, and Yusuke Uchida. In ECCVW 2018.
  78. 78. #denatechcon Full-body High-resolution Anime Generation with Progressive Structure-conditional Generative Adversarial Networks Koichi Hamada, Kentaro Tachibana, Tianqi Li, Hiroto Honda, and Yusuke Uchida. In ECCVW 2018. 00./ 0
  79. 79. #denatechcon Full-body High-resolution Anime Generation with Progressive Structure-conditional Generative Adversarial Networks Koichi Hamada, Kentaro Tachibana, Tianqi Li, Hiroto Honda, and Yusuke Uchida. In ECCVW 2018. 00./ 0
  80. 80. #denatechcon Full-body High-resolution Anime Generation with Progressive Structure-conditional Generative Adversarial Networks Koichi Hamada, Kentaro Tachibana, Tianqi Li, Hiroto Honda, and Yusuke Uchida. In ECCVW 2018. 00./ 0 Full-body anime generation at 1024x1024 with Progressive Structure-conditional GANs
  81. 81. #denatechcon Full-body High-resolution Anime Generation with Progressive Structure-conditional Generative Adversarial Networks Koichi Hamada, Kentaro Tachibana, Tianqi Li, Hiroto Honda, and Yusuke Uchida. In ECCVW 2018. 00./ 0
  82. 82. #denatechcon Full-body High-resolution Anime Generation with Progressive Structure-conditional Generative Adversarial Networks Koichi Hamada, Kentaro Tachibana, Tianqi Li, Hiroto Honda, and Yusuke Uchida. In ECCVW 2018. // . 0/0 Adding action to full-body anime characters with Progressive Structure-conditional GANs
  83. 83. #denatechcon Full-body High-resolution Anime Generation with Progressive Structure-conditional Generative Adversarial Networks Koichi Hamada, Kentaro Tachibana, Tianqi Li, Hiroto Honda, and Yusuke Uchida. In ECCVW 2018.
  84. 84. #denatechcon Full-body High-resolution Anime Generation with Progressive Structure-conditional Generative Adversarial Networks Koichi Hamada, Kentaro Tachibana, Tianqi Li, Hiroto Honda, and Yusuke Uchida. In ECCVW 2018. (ICLR’18)
  85. 85. #denatechcon Full-body High-resolution Anime Generation with Progressive Structure-conditional Generative Adversarial Networks Koichi Hamada, Kentaro Tachibana, Tianqi Li, Hiroto Honda, and Yusuke Uchida. In ECCVW 2018. (ICLR’18)
  86. 86. #denatechcon Full-body High-resolution Anime Generation with Progressive Structure-conditional Generative Adversarial Networks Koichi Hamada, Kentaro Tachibana, Tianqi Li, Hiroto Honda, and Yusuke Uchida. In ECCVW 2018. (ICLR’18) (NIPS’17) (NIPS’17)
  87. 87. #denatechcon Full-body High-resolution Anime Generation with Progressive Structure-conditional Generative Adversarial Networks Koichi Hamada, Kentaro Tachibana, Tianqi Li, Hiroto Honda, and Yusuke Uchida. In ECCVW 2018. (ICLR’18) (NIPS’17) (NIPS’17)
  88. 88. #denatechcon Full-body High-resolution Anime Generation with Progressive Structure-conditional Generative Adversarial Networks Koichi Hamada, Kentaro Tachibana, Tianqi Li, Hiroto Honda, and Yusuke Uchida. In ECCVW 2018.
  89. 89. #denatechcon
  90. 90. #denatechcon
  91. 91. #denatechcon AGENDA
  92. 92. #denatechcon n e n 4 3 33 36 4 n 0231 33 36 4 n c b a )3 ( 0 06I J@ N LT A F ( D T
  93. 93. #denatechcon n n n
  94. 94. #denatechcon n n
  95. 95. #denatechcon n C n 0 1 . n 0:: /1:0 6 ..:. 21 )()
  96. 96. #denatechcon n CEAD A n ) 17: n 7 2 -( ) 17: 0/ 1 7 0 7 . :0. 0 7 1 7 :
  97. 97. #denatechcon AGENDA
  98. 98. #denatechcon
  99. 99. #denatechcon : ( . ) / .
  100. 100. #denatechcon • • • • : ( . ) / .
  101. 101. #denatechcon
  102. 102. #denatechcon O .10 2:: ( / /:7 10 * .) A L A
  103. 103. #denatechcon O .10 2:: ( / /:7 10 * .) A L A
  104. 104. #denatechcon • O .10 2:: ( / /:7 10 * .) A L A
  105. 105. #denatechcon • • O .10 2:: ( / /:7 10 * .) A L A
  106. 106. #denatechcon O .10 2:: ( / /:7 10 * .) A L A • •
  107. 107. #denatechcon
  108. 108. #denatechcon ) ) 3
  109. 109. #denatechcon ) ) 3
  110. 110. #denatechcon
  111. 111. #denatechcon
  112. 112. #denatechcon
  113. 113. #denatechcon
  114. 114. #denatechcon 8 1 1 1 Super SloMo: High Quality Estimation of Multiple Intermediate Frames for Video Interpolation. Huaizu Jiang, Deqing Sun, Varun Jampani, Ming-Hsuan Yang, Erik Learned-Miller, Jan Kautz. In CVPR 2018.
  115. 115. #denatechcon /30 480 6 2/81 4C + 60 2 • 8 , Super SloMo: High Quality Estimation of Multiple Intermediate Frames for Video Interpolation. Huaizu Jiang, Deqing Sun, Varun Jampani, Ming-Hsuan Yang, Erik Learned-Miller, Jan Kautz. In CVPR 2018. https://youtu.be/MjViy6kyiqs Research at NVIDIA: Transforming Standard Video Into Slow Motion with AI
  116. 116. #denatechcon 7 1 Video Frame Synthesis using Deep Voxel Flow. Ziwei Liu, Raymond A. Yeh, Xiaoou Tang, Yiming Liu, Aseem Agarwala. In ICCV 2017.
  117. 117. #denatechcon N I 7 B7 =: B = P 77 = 7: :=D • /0 (+ /0 , Video Frame Synthesis using Deep Voxel Flow. Ziwei Liu, Raymond A. Yeh, Xiaoou Tang, Yiming Liu, Aseem Agarwala. In ICCV 2017. BB F=CBC 67 ?. / 3: 1 B Video Frame Synthesis using Deep Voxel Flow
  118. 118. #denatechcon D F 6 + 23C • 1 76 , P SV P J IOM S R • ( ,24 c SV P J ,24 cP / ++ C • 1 76 , P J SV P • 4 8 4 0 L a Super SloMo(Adobe) Super SloMo Deep Voxel Flow Video Frame Synthesis using Deep Voxel Flow. Ziwei Liu, Raymond A. Yeh, Xiaoou Tang, Yiming Liu, Aseem Agarwala. In ICCV 2017. Super SloMo: High Quality Estimation of Multiple Intermediate Frames for Video Interpolation. Huaizu Jiang, Deqing Sun, Varun Jampani, Ming-Hsuan Yang, Erik Learned-Miller, Jan Kautz. In CVPR 2018. F
  119. 119. #denatechcon 2
  120. 120. #denatechcon
  121. 121. #denatechcon F 8 0 O T PL n S 4 ) n F 4 ( W4000301 32 ( 6 A 7
  122. 122. #denatechcon F F F 0 -1 12 W n n S L ***7* 7 A A OT 0 -1 12
  123. 123. #denatechcon
  124. 124. #denatechcon
  125. 125. #denatechcon
  126. 126. #denatechcon n n
  127. 127. #denatechcon n n n
  128. 128. #denatechcon n n n n F
  129. 129. #denatechcon n n n n F
  130. 130. #denatechcon G I0 I4 Optical Flow I1, I2, I3 (Generated)
  131. 131. #denatechcon G I0 I4 D Optical Flow I1, I2, I3 (Generated) Ground Truth
  132. 132. #denatechcon G I0 I4 D Optical Flow I1, I2, I3 (Generated) Ground Truth MSE
  133. 133. #denatechcon G I0 I4 D Optical Flow Ground Truth I1, I2, I3 (Generated) Ground Truth Generated MSE MSE
  134. 134. #denatechcon G I0 I4 D Optical Flow Ground Truth I1, I2, I3 (Generated) Ground Truth Generated MSE MSE
  135. 135. #denatechcon 2
  136. 136. #denatechcon Conv-BN-ReLU Conv-BN-ReLU Conv-BN-ReLU Conv-BN-ReLU Local Discriminator “Real” or “Fake” Local Patch (16×16pix)
  137. 137. #denatechcon Conv-BN-ReLU Conv-BN-ReLU Generated Image Sequense Conv-BN-ReLU Conv-BN-ReLU Conv-BN-ReLU Conv-BN-ReLU Conv-BN-ReLU Conv-BN-ReLU Conv-BN-ReLU FC Local Discriminator Temporal Discriminator “Real” or “Fake” Local Patch (16×16pix) Image Sequense “Real” or “Fake”
  138. 138. #denatechcon
  139. 139. #denatechcon Video
  140. 140. #denatechcon image0 image1 image2 image3 image4Video
  141. 141. #denatechcon ⁃ step size = 4 7FPS -> 30FPS 001.png, 005.png, 009.png, 013.png, 017.png ⁃ step size = 1 30FPS -> 120FPS 001.png, 002.png, 003.png, 004.png, 005.png
  142. 142. #denatechcon ⁃ step size = 4 7FPS -> 30FPS 001.png, 005.png, 009.png, 013.png, 017.png ⁃ step size = 1 30FPS -> 120FPS 001.png, 002.png, 003.png, 004.png, 005.png
  143. 143. #denatechcon
  144. 144. #denatechcon Frame Frame Deep Voxel FlowInput // . / Experimental Results: “Anime Frame Generation with Structure-consistent Prediction GANs”
  145. 145. #denatechcon step size = 1 step size = 4 step size = 7 step size = 10 Input SPGAN (Ours) Deep Voxel Flow 4 // . / Experimental Results: “Anime Frame Generation with Structure-consistent Prediction GANs”
  146. 146. #denatechcon Deep Voxel Flow Ours 1.average PSNR/SSIM on test dataset step size=4 PSNR SSIM Deep Voxel Flow 23.32 0.9294 SPGAN(Ours) 24.27 0.9407
  147. 147. #denatechcon AGENDA
  148. 148. #denatechcon ProgressiveGAN (Karras et al., ICLR 2018) BigGAN (Brock et al., ICLR 2019)
  149. 149. #denatechcon ProgressiveGAN (Karras et al., ICLR 2018) BigGAN (Brock et al., ICLR 2019)
  150. 150. #denatechcon ProgressiveGAN (Karras et al., ICLR 2018) BigGAN (Brock et al., ICLR 2019)
  151. 151. #denatechcon 00./ 0
  152. 152. #denatechcon 00./ 0
  153. 153. #denatechcon // . /
  154. 154. #denatechcon // . /
  155. 155. #denatechcon
  156. 156. #denatechcon * 10 2 1
  157. 157. #denatechcon
  158. 158. #denatechcon L0*** * 2 D 1 A 7 1 D 1 7
  159. 159. #denatechcon TZ ... ./ 0 KD SL KA N O N KD SL K N W (3 Frame) AI (x16 ) Input Frames Generated Frames 78 : : 102*0DeNA AI :
  160. 160. #denatechcon

×