D&D for a Scalable, Server-less, Web UI to Querying Variants (Bio it world 2018)

1. bill@bioteam.net
Variant Query Tool
D&D for a Scalable, Server-less, Web UI to Querying
Variants
1

2. bill@bioteam.net
http://vqt.bioteam.net
Simple POC application with elegant architecture
2

3. bill@bioteam.net
http://vqt.bioteam.net
Variant Query Tool
3
Sample id
variant (chr-start-end-ref-alt) gene symbol dbSNP id
allele frequency zygosity Clinvar pathogenicity

4. bill@bioteam.net
http://vqt.bioteam.net
Variant Query Tool
4
Show SQL

5. bill@bioteam.net
http://vqt.bioteam.net
Variant Query Tool
5
Submit Query

6. bill@bioteam.net
http://vqt.bioteam.net
Variant Query Tool
6
UCSC
dbSNP
NCBI

7. bill@bioteam.net
3 Inspirations
Around this time last year… MSSNG, Ember, Athena
7

8. bill@bioteam.net
https://research.mss.ng
MSSNG Project
8
“Whole genome sequencing resource identifies 18 new candidate genes for autism spectrum disorder”
Nature Neuroscience 20, pages 602–611 (2017), March 2017

9. bill@bioteam.net
https://research.mss.ng
MSSNG Project
9
“Whole genome sequencing resource identifies 18 new candidate genes for autism spectrum disorder”
Nature Neuroscience 20, pages 602–611 (2017), March 2017
Google Cloud Google Bigquery
Linux VM
Server-side Rails Application

10. bill@bioteam.net
Client-side Web Application interface to REST-ful API
Some Other Project…
10

11. bill@bioteam.net
Server-less Interactive Query Service
Amazon Athena API Introduced
‣ Query files in S3 using
standard SQL
‣ SQL joins between files of
different formats
11

12. bill@bioteam.net
Server-less
Variant Query Tool
12
http://vqt.bioteam.net *.html
*.css
*.jsUI
API
*.vcf
*.csv.gz
Data
1000g chr22
Clinvar

13. bill@bioteam.net
Static S3 Web Site
13
HTML (25 lines)

14. bill@bioteam.net
Static S3 Web Site
14
HTML
CSS, JS

15. bill@bioteam.net
Model (58 lines)
Ember Application
15

16. bill@bioteam.net
View (70 lines)
Ember Application
16

17. bill@bioteam.net
Controller (71 lines)
Ember Application
17

18. bill@bioteam.net
index.js (112 lines)
Lambda Function calling Athena
18

19. bill@bioteam.net
Lambda Function calling Athena
19

20. bill@bioteam.net
The Drag & Drop Part
A little more involved…
20

21. bill@bioteam.net
Drag & Drop
21
VCF input lambda VCF.parquet
Athena Glue
Spark/Adam/Code
EMR cluster

22. bill@bioteam.net
YAML Template
Cloud Formation
22
VCF input lambda VCF.parquet
Athena Glue
Spark/Adam/Code
EMR cluster

23. bill@bioteam.net
Upload Template
Cloud Formation
23

24. bill@bioteam.net
Cloud Formation
24
Name Stack
Code Bucket
VCF Input Bucket
Parquet Output Bucket
Select VPC for EMR Cluster
Select Size for EMR Cluster

25. bill@bioteam.net
Cloud Formation
25

26. bill@bioteam.net
Add VCF file(s)
Drag & Drop
26

27. bill@bioteam.net
Triggers Cascade of Events
Drag & Drop
27
VCF input lambda VCF.parquet
Athena Glue
Spark/Adam/Code
EMR cluster

28. bill@bioteam.net
Standard SQL
Athena
28

29. bill@bioteam.net
http://vqt.bioteam.net
Variant Query Tool
29

30. 30
Variant Query Tool
http://vqt.bioteam.net
Thank You! bill@bioteam.net