How AI and ‘Big Learning’ transforms life science research - AI Dev Days Closing Keynote - Dr. Denis Bauer
•
3 recomendaciones•26,199 vistas
This presentation was given by Dr. Denis Bauer as closing keynote in AI Dev Days conference on 9th March 2018 in Bangalore. URL: www.aidevdays.com -- The talk showcases how genomic research has leapfrogged to the forefront of BigData and Cloud solutions. I outline how Apache Spark helps identify genomic association on population-scale whole genome sequencing data, as well as how the accuracy of genome editing approaches can be improved with massively parallel server-less cloud functions.
Recomendados
Recomendados
Más contenido relacionado
Similar a How AI and ‘Big Learning’ transforms life science research - AI Dev Days Closing Keynote - Dr. Denis Bauer
Similar a How AI and ‘Big Learning’ transforms life science research - AI Dev Days Closing Keynote - Dr. Denis Bauer (20)
Más de CodeOps Technologies LLP
Más de CodeOps Technologies LLP (20)
Último
Último (20)
How AI and ‘Big Learning’ transforms life science research - AI Dev Days Closing Keynote - Dr. Denis Bauer
- 3. Source PwC
- 8. By 2025 it is estimated that 50% of the world population will have been sequenced. Bioinformatics | Denis C. Bauer | @allPowerde8 | Frost&Sullivan
- 10. Bioinformatics | Denis C. Bauer | @allPowerde10 | Genomics will outpace other BigData disciplines Stephens et al. PLOS Biology 2015 Astronomy Twitter YouTube Genomics
- 14. Bioinformatics | Denis C. Bauer | @allPowerde14 | Machine learning on 1.7 Trillion data points 80 Million features Individuals Genomic profile Disease status22,500 samples Disease genes
- 15. Bioinformatics | Denis C. Bauer | @allPowerde15 | Population-scale genomic data analysis requires BigData solutions Desktop compute High-performance compute cluster Hadoop/Spark compute cluster Focus small data Compute-intensive Data-intensive Fault tolerant No No Yes Node-bound Yes Yes No Parallelization 10 CPU 100 CPU 1000 CPU Parallelization procedure bespoke bespoke standardized CSIRO solution
- 16. Bioinformatics | Denis C. Bauer | @allPowerde BMC Genomics 2015, 16:1052 PMID: 26651996 (IF=4) 16 | In the top 5% of all research outputs scored by Altmetric Spark Core Spark ML MLlib Variant Spark RESEARCH
- 17. Bioinformatics | Denis C. Bauer | @allPowerde17 | Parallelization of Random forest to scale with samples • Spark ML’s RF was designed for ‘Big’ low dimensional data. • The full genome-wide profile does not fit into the executors memory rendering the approach infeasible. “Cursed” BigData: e.g. Genomics Moderate number of samples with many features Feature set too large to be handled by single executer
- 18. Bioinformatics | Denis C. Bauer | @allPowerde18 | Firas Abuzaid (Spark Summit 2016) YGGDRASIL: Faster Decision Trees Column Partitioning in SPARK Flip the matrix: partition by column (take 2) Parallelization of Random forest to scale with features
- 20. Runtime comparison Bioinformatics | Denis C. Bauer | @allPowerde20 | ● ● ● ● ● ●0 5 10 15 0.030.040.05 accuracy (out−of−bag error) trainingvariantspersecond(Million) Program ● ● ● ● ● ● VariantSpark SparkML Randomforest(R) ranger(R) ranger(C++) H2O low Accuracy high lowSpeed high
- 21. Machine learning on ‘wide’ data Bioinformatics | Denis C. Bauer | @allPowerde21 | 80 Million features Individuals Genomic profile Disease status 22,500 samples Disease genes Time series, Concatenated data, Sensor Data, Log files Churn-rate, occurrence of failure, attacks Predictive markers
- 22. Workflow patters Bioinformatics | Denis C. Bauer | @allPowerde22 | Business Problem Curate Data • Cleaning • Visualization Build MVP • Scope Tech • Prototypes • Iterate Prep for Production • Test at scale • Provide endpoint On premises Databricks AWS Sagemaker Execution Options
- 24. Building a community • Build a robust ‘wide’ Machine Learning library for business • Research and potentially cure genetic diseases along the way • Can you help? Bioinformatics | Denis C. Bauer | @allPowerde24 | YES ! Lynn Langit Performance comparison Python API
- 27. Transformational Bioinformatics | Denis C. Bauer | @allPowerde Improve performance SPEED • Each search can be broken down into parallel tasks - each takes seconds SCALE • Researchers might want to search the target for one gene or 100,000 Agility + Scalability =
- 31. Results – 4x Faster (80% improvement) Transformational Bioinformatics | Denis C. Bauer | @allPowerde 0 40 80 120 old new implementation runtime(s) factor(Name) onTargetScoring base 2 min 30 sec
- 32. Bioinformatics | Denis C. Bauer | @allPowerde32 | Using hypothesis-driven architecture Architecture as text Evolve Automatic performance measure Evaluate James Lewis https://www.epsagon.com/ 10 DevOps DevOps changes, enabling a faster evidence- based pace of architecture development
- 34. Bioinformatics | Denis C. Bauer | @allPowerde34 | Three things to remember • ‘Datafication’ will make data ‘wider’ -> paradigm shift for Machine Learning applications • Serverless architecture can cater for compute-intensive tasks • Business and life-science research are not that different: let’s build a community together!
- 35. Bioinformatics | Denis C. Bauer | @allPowerde35 | Bioinformatics | Denis C. Bauer | @allPowerde Transformational Bioinformatics Denis Bauer, PhD Oscar Luo, PhD Rob Dunne, PhD Piotr Szul Team Aidan O’BrienLaurence Wilson, PhD Collaborators News Software Kaitao Lai, PhD 35 | Arash Bayat Lynn Langit Natalie Twine, PhD Top 10 Australian IT stories of 2017