Se ha denunciado esta presentación.
Utilizamos tu perfil de LinkedIn y tus datos de actividad para personalizar los anuncios y mostrarte publicidad más relevante. Puedes cambiar tus preferencias de publicidad en cualquier momento.

Multitask Adversarial Learning of Deep Neural Networks for Medical Imaging and Image Analysis

A comprehensive review and summary of some of the recent works in the area of adversarial learning of deep neural networks carried out at the Kharagpur Learning, Imaging and Visualization (KLIV) Research Group.

  • Inicia sesión para ver los comentarios

  • Sé el primero en recomendar esto

Multitask Adversarial Learning of Deep Neural Networks for Medical Imaging and Image Analysis

  1. 1. 1 Sept. 2019 Multitask Adversarial Learning of DNN for MedIA [Debdoot Sheet] [Talk at Midnapore College] 0 One out this pair of images is an original image of intravascular ultrasound (IVUS) and the other is generated by a special type of artificial neural network (ANN) known as generative adversarial network (GAN). Can you identify the original?
  2. 2. Multitask Adversarial Learning of Deep Neural Networks for Medical Imaging and Image Analysis Dr. Debdoot Sheet Assistant Professor, Department of Electrical Engineering Principal Investigator, Kharagpur Learning, Imaging and Visualization Group Indian Institute of Technology Kharagpur www.facweb.iitkgp.ac.in/~debdoot/
  3. 3. Disclosure 1 Sept. 2019 Multitask Adversarial Learning of DNN for MedIA [Debdoot Sheet] [Talk at Midnapore College] 2 • Conflicts and Interests – SkinCurate Research Pvt. Ltd. – Founder, stock options – Intel Technology India Pvt. Ltd. – Research Sponsor, Startup Incubation Mentor – Dept. of Biotechnology, Govt. of India – Research Sponsor – Sigtuple Technologies Pvt. Ltd. – Research Sponsor – Tata Steel Ltd. – Research Sponsor – Amazon Web Services (AWS) Inc. – Research Sponsor – Nesa Medtech Pvt. Ltd. – Research Sponsor – Samsung Inc. – Research Sponsor – Nvidia Inc. – Lab. Resources Sponsor – Microsoft – Collaborator and Lab. Resources Sponsor – Texas Instruments India Pvt. Ltd. – Lab. Resources Sponsor – Analog Devices India Pvt. Ltd. – Lab. Resources Sponsor – Indian Council of Medical Research, Govt. of India – Travel Grants – Dept. of Science and Technology, Govt. of India – Travel Grants, Startup Incubation Mentor – Biotechnology Industry Research Assistance Council (BIRAC) – Startup Incubation Grant – Society for Innovation and Entrepreneurship (SINE) IIT Bombay – Startup Incubation Grant
  4. 4. Learning? A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P, if its performance at tasks in T, as measured by P, improves with experience E -Tom Mitchell Multitask Adversarial Learning of DNN for MedIA [Debdoot Sheet] [Talk at Midnapore College] 31 Sept. 2019
  5. 5. Contributions to be Discussed 1 Sept. 2019 Multitask Adversarial Learning of DNN for MedIA [Debdoot Sheet] [Talk at Midnapore College] 4 Learning with Multitask Adversaries using Weakly Labelled Data for Semantic Segmentation in Retinal Images (MIDL 2019) Oindrila Saha, Rachana Sathish, Debdoot Sheet Simulating Patho-realistic Ultrasound Images using Deep Generative Networks with Adversarial Learning (ISBI 2018) Francis Tom, Debdoot Sheet Learning a Deep Convolution Network with Turing Test Adversaries for Microscopy Image Super Resolution (ISBI 2019) Francis Tom, Himanshu Sharma, Dheeraj Mundhra, Tathagata Rai Dastidar, Debdoot Sheet UltraCompression: Framework for High Density Compression of Ultrasound Volumes using Physics Modeling Deep Neural Networks (ISBI 2019) Debarghya China, Francis Tom, Sumanth Nandamuri, Aupendu Kar, Mukundhan Srinivasan, Pabitra Mitra, Debdoot Sheet
  6. 6. RECALL 1 Sept. 2019 Multitask Adversarial Learning of DNN for MedIA [Debdoot Sheet] [Talk at Midnapore College] 5
  7. 7. Statistically Informed Decision 1 Sept. 2019 Multitask Adversarial Learning of DNN for MedIA [Debdoot Sheet] [Talk at Midnapore College] 6 𝑃 Ω = 𝜔|𝐱 = 𝒙 = 𝑃 Ω = 𝜔, 𝐱 = 𝒙 𝑃 𝐱 = 𝒙 𝑃 Ω = 𝜔|𝐱 = 𝒙 = 𝑝 𝐱 = 𝒙|Ω = 𝜔 𝑃 Ω = 𝜔 𝑃 𝐱 = 𝒙 Posterior probability Likelihood Prior probability EvidenceBayes’ Rule
  8. 8. Decision 1 Sept. 2019 Multitask Adversarial Learning of DNN for MedIA [Debdoot Sheet] [Talk at Midnapore College] 7 𝑃 Ω = 𝜔|𝐱 = 𝒙 = 𝑝 𝐱 = 𝒙|Ω = 𝜔 𝑃 Ω = 𝜔 𝑃 𝐱 = 𝒙 Posterior probability Likelihood Prior probability EvidenceBayes’ Rule Shade of color Length 𝑥1 𝑥2 𝜔 = arg max 𝑃 Ω = 𝜔|𝐱 = 𝒙 Maximum aposteriori (MAP) Decision boundary
  9. 9. Challenges with a Decision Boundary 1 Sept. 2019 Multitask Adversarial Learning of DNN for MedIA [Debdoot Sheet] [Talk at Midnapore College] 8 𝑥1 𝑥2 𝑥1 𝑥2 𝑥1 𝑥2 Abundant samples Not-so abundant samples Scarce samples
  10. 10. Understanding these Challenges 1 Sept. 2019 Multitask Adversarial Learning of DNN for MedIA [Debdoot Sheet] [Talk at Midnapore College] 9 𝑥1 𝑥2 𝑥1 𝑥2 𝑃 Ω = 𝜔|𝐱 = 𝒙 = 𝑝 𝐱 = 𝒙|Ω = 𝜔 𝑃 Ω = 𝜔 𝑃 𝐱 = 𝒙 Likelihood 𝑝 𝒙 ~𝜙 𝒙3 Cubic 𝑝 𝒙 ~𝜙 𝒙2 Quadratic 𝑝 𝒙 ~𝜙 𝑥Linear 𝑥2 = 𝑚𝑥1 + 𝑐
  11. 11. AI from Heuristics to DL 1 Sept. 2019 Multitask Adversarial Learning of DNN for MedIA [Debdoot Sheet] [Talk at Midnapore College] 10 Input Hand Designed Rules Output Input Hand Designed Features Output Learned Decision Input Learned Features Output Learned Decision Input Learned Features Output Learned Decision Learned Abstract Features Rule based AI Classical ML Representation Learn. Deep Learning
  12. 12. Objectives of Machine Learning 1 Sept. 2019 Multitask Adversarial Learning of DNN for MedIA [Debdoot Sheet] [Talk at Midnapore College] 11 𝑝 𝒙 ~𝜙 𝒙3 Cubic 𝑝 𝒙 ~𝜙 𝒙2 Quadratic 𝑝 𝒙 ~𝜙 𝑥 Linear Increasing order of complexity 𝑥1 𝑥2 𝑥1 𝑥2 Data space plane MLE Mean Squared Error (MSE) Perception Loss (PL)
  13. 13. DL addressing these ML Objectives 1 Sept. 2019 Multitask Adversarial Learning of DNN for MedIA [Debdoot Sheet] [Talk at Midnapore College] 12 𝑥1 𝑥2 𝑥1 𝑥2 Data space plane Van Dyk, David A., and Xiao-Li Meng. “The art of data augmentation.” Journal of Computational and Graphical Statistics 10.1 (2001): 1-50. rotate flipud fliplr flipud
  14. 14. DL addressing these ML Objectives 1 Sept. 2019 Multitask Adversarial Learning of DNN for MedIA [Debdoot Sheet] [Talk at Midnapore College] 13 𝑝 𝒙 ~𝜙 𝒙3 Cubic 𝑝 𝒙 ~𝜙 𝒙2 Quadratic 𝑝 𝒙 ~𝜙 𝑥 Linear Increasing order of complexity 𝑥1 𝑥2 𝑥1 𝑥2 Learned Features
  15. 15. DL addressing these ML Objectives 1 Sept. 2019 Multitask Adversarial Learning of DNN for MedIA [Debdoot Sheet] [Talk at Midnapore College] 14 MLE Mean Squared Error (MSE) Perception Loss (PL) low MSE, high PL high MSE, low PL Sketch2Photo CNN Discriminator Neural Network Real vs. Fake Photograph Generated
  16. 16. 1 Sept. 2019 Multitask Adversarial Learning of DNN for MedIA [Debdoot Sheet] [Talk at Midnapore College] 15 Learning with Multitask Adversaries using Weakly Labelled Data for Semantic Segmentation in Retinal Images Oindrila Saha, Rachana Sathish, Debdoot Sheet International Conference on Medical Imaging with Deep Learning (MIDL), London, July 2019 Oindrila Saha Rachana Sathish
  17. 17. Preamble 1 Sept. 2019 Multitask Adversarial Learning of DNN for MedIA [Debdoot Sheet] [Talk at Midnapore College] 16 SemanticSegmentation Network Retinal vessels, Optic disc, Optic cup, Edema, Soft exudates, Hard exudates
  18. 18. Challenge 1 Sept. 2019 Multitask Adversarial Learning of DNN for MedIA [Debdoot Sheet] [Talk at Midnapore College] 17 Image size #768✕584 Dataset 1 Retinal vessels Train # 20, Test # 20 Image size # 3,504 ✕ 2,336 Retinal vessels, optic disc Images # 18 Image size # 1,500 ✕ 1,152 Hard exudates, soft exudates, Microaneurysms, Hemorrhages Images # 89 Dataset 2 Dataset 3
  19. 19. Solution 1 Sept. 2019 Multitask Adversarial Learning of DNN for MedIA [Debdoot Sheet] [Talk at Midnapore College] 18 Semantic Segmentation Network 3✕M✕N I(:,:,:) C✕M✕N Discriminator 2 C✕M✕N Manual vs. Synthetic ChannelShuffler(.)ChannelShuffler(.) Discriminator 1 Presence of Class 𝐲𝐱
  20. 20. Solution 1 Sept. 2019 Multitask Adversarial Learning of DNN for MedIA [Debdoot Sheet] [Talk at Midnapore College] 19 Semantic Segmentation Network 3✕M✕N I(:,:,:) Discriminator 2 C✕M✕N Manual vs. Synthetic ChannelShuffler(.)ChannelShuffler(.) Discriminator 1 Segmented Channel Idx C✕M✕N I(:,:,:) I(:,:,:) 𝐱 𝐲 𝐲 𝐲𝐱 𝐿 𝑠𝑒𝑔 ∙ = ∀𝐲 𝑐 ≠∅ 𝐵𝐶𝐸 𝐲 𝑐 , 𝐲 𝑐 𝛻𝐿 𝑠𝑒𝑔 ∙
  21. 21. Discriminator 1 Semantic Segmentation Network 1 Sept. 2019 Multitask Adversarial Learning of DNN for MedIA [Debdoot Sheet] [Talk at Midnapore College] 20 Discriminator 2 C✕M✕N Manual vs. Synthetic ChannelShuffler(.) Segmented Channel Idx 𝐲𝐱 3✕M✕N I(:,:,:) Solution ChannelShuffler(.) C✕M✕N 𝐱 1 𝐲𝐱 2 𝐱 3 𝐱 1 𝐱 2 𝐱 3 𝐲𝐲 𝑘 = 𝐲 𝑠ℎ𝑢𝑓𝑓𝑙𝑒 1,2, ⋯ , 𝐶
  22. 22. 1 Sept. 2019 Multitask Adversarial Learning of DNN for MedIA [Debdoot Sheet] [Talk at Midnapore College] 21 Semantic Segmentation Network 3✕M✕N I(:,:,:) C✕M✕N Discriminator 2 C✕M✕N Manual vs. Synthetic ChannelShuffler(.)ChannelShuffler(.) 𝐲𝐱 Solution Discriminator 1 Presence of Class 𝐱 1 𝐱 2 𝐱 3 𝐲 𝐲 𝐧1 = 1,0,0,0,1,1 𝐧1 = 1,1,0,1,1,1 𝐿 𝐷1 ∙ = 𝐵𝐶𝐸 𝐧1, 𝐧1 𝛻𝐿 𝐷1 ∙ 𝐧1 Ganin, Yaroslav, et al. "Domain-adversarial training of neural networks." The Journal of Machine Learning Research 17.1 (2016): 2096-2030.
  23. 23. Discriminator 1 Segmented Channel Idx 1 Sept. 2019 Multitask Adversarial Learning of DNN for MedIA [Debdoot Sheet] [Talk at Midnapore College] 22 Semantic Segmentation Network 𝐲𝐱 Solution 𝐿 𝐷2 ∙ = 𝐵𝐶𝐸 𝐧2, 𝐧2 Discriminator 2 𝛻𝐿 𝐷2 ∙ 3✕M✕N I(:,:,:) C✕M✕N ChannelShuffler(.) C✕M✕NChannelShuffler(.) 𝐧2 = 0, 1 𝐧2 = 1, 0 𝐧2 Ganin, Yaroslav, et al. "Domain-adversarial training of neural networks." The Journal of Machine Learning Research 17.1 (2016): 2096-2030. {Manual, Synthetic} vs. {Synthetic, Manual}
  24. 24. 1 Sept. 2019 Multitask Adversarial Learning of DNN for MedIA [Debdoot Sheet] [Talk at Midnapore College] 23 3✕M✕N C✕M✕N C✕M✕N Manual vs. Synthetic ChannelShuffler(.)ChannelShuffler(.) Segmented Channel Idx 𝐲𝐱 Semantic Segmentation Network I(:,:,:) Discriminator 2 Discriminator 1 Solution 𝛻𝐿 𝑠𝑒𝑔 ∙ 𝛻𝐿 𝐷2 ∙ 𝛻𝐿 𝐷1 ∙ 𝛻𝐿 𝑎𝑑𝑣 ∙ = 𝛼1 𝛻𝐿 𝐷1 ∙ −𝛼2 𝛻𝐿 𝐷2 ∙ 𝐿 𝑎𝑑𝑣 ∙ = 𝛼1 𝐿 𝐷1 ∙ +𝛼2 1 − 𝐿 𝐷2 ∙
  25. 25. Results 1 Sept. 2019 Multitask Adversarial Learning of DNN for MedIA [Debdoot Sheet] [Talk at Midnapore College] 24
  26. 26. Computed Super-Resolution Microscopy Learning a Deep Convolution Network with Turing Test Adversaries for Microscopy Image Super Resolution Francis Tom, Himanshu Sharma, Dheeraj Mundhra, Tathagato Rai Dastidar, Debdoot Sheet IEEE International Symposium on Biomedical Imaging (ISBI), 2019. Francis Tom Himanshu Sharma Dheeraj Mundhra Tathagato Rai Dastidar Computed Super-Resolution Microscopy
  27. 27. 1 Sept. 2019 Multitask Adversarial Learning of DNN for MedIA [Debdoot Sheet] [Talk at Midnapore College] 26 Turing Test 1 Turing Test 2 Region Proposals {Real, SR} vs. {SR, Real} {Real, SR} vs. {SR, Real}𝐽 𝑇1 ∙ 𝛻𝐽 𝑇1 ∙ 𝐽 𝑇2 ∙ 𝛻𝐽 𝑇2 ∙ Mask 𝛻𝐽 𝐴𝑑𝑣 ∙ = −𝛼𝛻𝐽 𝑇1 ∙ −𝛽𝛻𝐽 𝑇2 ∙ Super Resolution CNN 𝐽 𝑅𝑒𝑐𝑜𝑛 ∙ 𝛻𝐽 𝑅𝑒𝑐𝑜𝑛 ∙ Real LR SR
  28. 28. 1 Sept. 2019 Multitask Adversarial Learning of DNN for MedIA [Debdoot Sheet] [Talk at Midnapore College] 27 Super Resolution CNN Turing Test 2 Region Proposals 𝐽 𝑅𝑒𝑐𝑜𝑛 ∙ 𝛻𝐽 𝑅𝑒𝑐𝑜𝑛 ∙ {Real, SR} vs. {SR, Real} LR 𝐽 𝑇2 ∙ 𝛻𝐽 𝑇2 ∙ Mask 𝛻𝐽 𝐴𝑑𝑣 ∙ = −𝛼𝛻𝐽 𝑇1 ∙ −𝛽𝛻𝐽 𝑇2 ∙ Turing Test 1 SR {Real, SR} vs. {SR, Real}𝐽 𝑇1 ∙ 𝛻𝐽 𝑇1 ∙ Real
  29. 29. 1 Sept. 2019 Multitask Adversarial Learning of DNN for MedIA [Debdoot Sheet] [Talk at Midnapore College] 28 Super Resolution CNN Turing Test 1 𝐽 𝑅𝑒𝑐𝑜𝑛 ∙ 𝛻𝐽 𝑅𝑒𝑐𝑜𝑛 ∙ LR {Real, SR} vs. {SR, Real}𝐽 𝑇1 ∙ 𝛻𝐽 𝑇1 ∙ 𝛻𝐽 𝐴𝑑𝑣 ∙ = −𝛼𝛻𝐽 𝑇1 ∙ −𝛽𝛻𝐽 𝑇2 ∙ Turing Test 2 Region Proposals {Real, SR} vs. {SR, Real} Real SR 𝐽 𝑇2 ∙ 𝛻𝐽 𝑇2 ∙ Mask
  30. 30. 1 Sept. 2019 Multitask Adversarial Learning of DNN for MedIA [Debdoot Sheet] [Talk at Midnapore College] 29 Turing Test 1 Turing Test 2 Region Proposals 𝐽 𝑅𝑒𝑐𝑜𝑛 ∙ 𝛻𝐽 𝑅𝑒𝑐𝑜𝑛 ∙ {Real, SR} vs. {SR, Real} Real {Real, SR} vs. {SR, Real}𝐽 𝑇1 ∙ 𝛻𝐽 𝑇1 ∙ 𝐽 𝑇2 ∙ 𝛻𝐽 𝑇2 ∙ Mask Super Resolution CNN LR SR 𝛻𝐽 𝐴𝑑𝑣 ∙ = −𝛼𝛻𝐽 𝑇1 ∙ −𝛽𝛻𝐽 𝑇2 ∙
  31. 31. 1 Sept. 2019 Multitask Adversarial Learning of DNN for MedIA [Debdoot Sheet] [Talk at Midnapore College] 30 Super Resolution CNN Turing Test 1 Turing Test 2 Region Proposals 𝐽 𝑅𝑒𝑐𝑜𝑛 ∙ 𝛻𝐽 𝑅𝑒𝑐𝑜𝑛 ∙ {Real, SR} vs. {SR, Real} Real LR SR {Real, SR} vs. {SR, Real}𝐽 𝑇1 ∙ 𝛻𝐽 𝑇1 ∙ 𝐽 𝑇2 ∙ 𝛻𝐽 𝑇2 ∙ Mask 𝛻𝐽 𝐴𝑑𝑣 ∙ = −𝛼𝛻𝐽 𝑇1 ∙ −𝛽𝛻𝐽 𝑇2 ∙
  32. 32. Some Results 1 Sept. 2019 Multitask Adversarial Learning of DNN for MedIA [Debdoot Sheet] [Talk at Midnapore College] 31
  33. 33. … and our AI powers Digital Pathology in India 1 Sept. 2019 Multitask Adversarial Learning of DNN for MedIA [Debdoot Sheet] [Talk at Midnapore College] 32
  34. 34. 1 Sept. 2019 Multitask Adversarial Learning of DNN for MedIA [Debdoot Sheet] [Talk at Midnapore College] 33 One out this pair of images is an original image of intravascular ultrasound (IVUS) and the other is generated by a special type of artificial neural network (ANN) known as generative adversarial network (GAN). Can you identify the original?
  35. 35. Can Generative Adversarial Networks Model Imaging Physics? Some experiences with simulating patho-realistic ultrasound images Francis Tom DeepSIP DeepKLIV “Simulating Patho-realistic Ultrasound Images using Deep Generative Networks with Adversarial Learning” Francis Tom, Debdoot Sheet IEEE International Symposium on Biomedical Imaging (ISBI), 2018.
  36. 36. Simulating Ultrasound 1 Sept. 2019 Multitask Adversarial Learning of DNN for MedIA [Debdoot Sheet] [Talk at Midnapore College] 35 Jørgen Arendt Jensen and Peter Munk, ”Computer phantoms for simulating ultrasound B-mode and cfm images”, Acoustical Imaging”, vol. 23, pp. 75-80, Eds.: S. Lees and L. A. Ferrari, Plenum Press, 1997. J. C. Bambre and R. J. Dickinson, "Ultrasonic B-scanning: A computer simulation", Phys. Med. Biol., vol. 25, no. 3, pp. 463– 479, 1980. [http://dx.doi.org/10.1088/0031-9155/25/3/006]
  37. 37. Pseudo B-mode US Image Simulation 1 Sept. 2019 Multitask Adversarial Learning of DNN for MedIA [Debdoot Sheet] [Talk at Midnapore College] 36 𝑇 𝑥, 𝑦 = 𝐸 𝑥, 𝑦 𝐺 𝑛 𝐺 𝑛 ~𝒩 0,1 𝑘0 = 2𝜋𝑓0 𝑐 ℎ 𝑥 𝑥, 𝑦 = sin 𝑘0 𝑥 𝑒 − 𝑥2 2𝜎 𝑥 2 ℎ 𝑦 𝑥, 𝑦 = 𝑒 − 𝑦2 2𝜎 𝑦 2 𝑉 𝑥, 𝑦 = 𝑇 𝑥, 𝑦 ∗ ℎ 𝑥 𝑥, 𝑦 ∗ ℎ 𝑦 𝑥, 𝑦 𝑏 𝑥, 𝑦 = 𝐻𝑖𝑙𝑏𝑒𝑟𝑡 𝑉 𝑥, 𝑦 J. C. Bambre and R. J. Dickinson, "Ultrasonic B-scanning: A computer simulation", Phys. Med. Biol., vol. 25, no. 3, pp. 463–479, 1980. [http://dx.doi.org/10.1088/0031-9155/25/3/006]
  38. 38. Where do we stand with Simulations? 1 Sept. 2019 Multitask Adversarial Learning of DNN for MedIA [Debdoot Sheet] [Talk at Midnapore College] 37 Digital Phantom Pseudo B-mode US Real B-mode US How to bridge this disparity in appearance?
  39. 39. The Challenge 1 Sept. 2019 Multitask Adversarial Learning of DNN for MedIA [Debdoot Sheet] [Talk at Midnapore College] 38 𝑇 𝑥, 𝑦 = 𝐸 𝑥, 𝑦 𝐺 𝑛 𝐺 𝑛 ~𝒩 0,1 𝑉 𝑥, 𝑦 = 𝑇 𝑥, 𝑦 ∗ ℎ 𝑥 𝑥, 𝑦 ∗ ℎ 𝑦 𝑥, 𝑦 𝑏 𝑥, 𝑦 = 𝐻𝑖𝑙𝑏𝑒𝑟𝑡 𝑉 𝑥, 𝑦 J. C. Bambre and R. J. Dickinson, "Ultrasonic B-scanning: A computer simulation", Phys. Med. Biol., vol. 25, no. 3, pp. 463–479, 1980. [http://dx.doi.org/10.1088/0031-9155/25/3/006] Are these convolution kernels not descriptive enough to model the signal mixing process resulting in image formation? Can we learn the convolution kernels? 𝑘0 = 2𝜋𝑓0 𝑐 ℎ 𝑥 𝑥, 𝑦 = sin 𝑘0 𝑥 𝑒 − 𝑥2 2𝜎 𝑥 2 ℎ 𝑦 𝑥, 𝑦 = 𝑒 − 𝑦2 2𝜎 𝑦 2
  40. 40. Learning Convolution with a Neural Network 1 Sept. 2019 Multitask Adversarial Learning of DNN for MedIA [Debdoot Sheet] [Talk at Midnapore College] 39 * = * =f 𝐖𝑖 𝒙 𝒚 𝒛 𝐖𝑖+1 𝒑 𝐖𝑖 𝑘+1 = 𝐖𝑖 𝑘 − 𝜂 𝜕𝐽 𝐖 𝜕𝐖𝑖 𝐽 𝐖 = 𝒑 − 𝒑 𝜕𝐽 𝐖 𝜕𝐖𝑖 = 𝜕𝐽 𝐖 𝜕𝒑 𝜕𝒑 𝜕𝐖𝑖+1 𝜕𝒛 𝜕𝒚 𝜕𝒚 𝜕𝐖𝑖 # Training Samples?
  41. 41. Adversarial Transformation for US Simulation 1 Sept. 2019 Multitask Adversarial Learning of DNN for MedIA [Debdoot Sheet] [Talk at Midnapore College] 40 Generator Net 𝐺 𝜃 Discriminator Net 𝐷 𝜙 𝐿 𝐺 𝜃 Simulated vs. Real 𝐿 𝐷 𝜙 𝛻𝐿 𝐷 𝜙 𝛻𝐿 𝐺 𝜃 Digital Phantom Simulated Real
  42. 42. Adversarial Transformation for US Simulation 1 Sept. 2019 Multitask Adversarial Learning of DNN for MedIA [Debdoot Sheet] [Talk at Midnapore College] 41 Hu, Y., Gibson, E., Lee, L. L., Xie, W., Barratt, D. C., Vercauteren, T., & Noble, J. A. (2017). Freehand Ultrasound Image Simulation with Spatially-Conditioned Generative Adversarial Networks. In Molecular Imaging, Reconstruction and Analysis of Moving Body Organs, and Stroke Imaging and Treatment (pp. 105-115). Springer, Cham.
  43. 43. The Challenges 1 Sept. 2019 Multitask Adversarial Learning of DNN for MedIA [Debdoot Sheet] [Talk at Midnapore College] 42 Transducer • Non uniform sampling in Cartesian coordinate domain • Signal interpolated for image formation, leading to smearing of speckles
  44. 44. Our Solution 1 Sept. 2019 Multitask Adversarial Learning of DNN for MedIA [Debdoot Sheet] [Talk at Midnapore College] 43 Digital Phantom Digital Phantom cart2pol( ) Pseudo B-mode Stage 0 Stage I 𝐺𝐼 𝜃 Stage II 𝐺𝐼𝐼 𝜃 𝐿 𝐺 𝐼 𝜃 𝐿 𝐺 𝐼𝐼 𝜃 cart2pol() Stage I 𝐷𝐼 𝜙 Stage II 𝐷𝐼𝐼 𝜙 𝐿 𝐷 𝐼 𝜙 𝐿 𝐷 𝐼𝐼 𝜙 Stage I Sim Stage II Sim Real Real Simulated vs. Real 64 x 64 256 x 256256 x 256256 x 256256 x 256 𝛻𝐿 𝐺 𝐼 𝛻𝐿 𝐺 𝐼𝐼 𝛻𝐿 𝐷 𝐼 𝛻𝐿 𝐷 𝐼𝐼
  45. 45. 1 Sept. 2019 Multitask Adversarial Learning of DNN for MedIA [Debdoot Sheet] [Talk at Midnapore College] 44 Lets Quiz: Real vs. Simulated
  46. 46. More Resources • Neural Information Processing Systems (NeurIPS) • International Conference on Learning Representations (ICLR) • International Conference on Machine Learning (ICML) • Association for Advancement of Artificial Intelligence (AAAI) • Computer Vision and Pattern Recognition (CVPR) • International Conference on Medical Imaging with Deep Learning (MIDL) • IEEE Int. Symp. Biomed. Imaging (ISBI) • Journal of Machine Learning Research (JMLR) • Machine Learning • IEEE Trans. Pattern Analysis and Machine Intelligence (PAMI) • IEEE Trans. Neural Networks and Learning Systems (TNNLS) • IEEE Trans. Medical Imaging (TMI) • Medical Image Analysis (MedIA) • Medical Image Computing and Computer Assisted Intervention (MICCAI) 1 Sept. 2019 Multitask Adversarial Learning of DNN for MedIA [Debdoot Sheet] [Talk at Midnapore College] 45
  47. 47. Take home message 1 Sept. 2019 Multitask Adversarial Learning of DNN for MedIA [Debdoot Sheet] [Talk at Midnapore College] 46
  48. 48. 1 Sept. 2019 Multitask Adversarial Learning of DNN for MedIA [Debdoot Sheet] [Talk at Midnapore College] 47 Thank you from #iitkliv http://iitkliv.github.io

×