The document summarizes discussions from a working group meeting regarding the selection and design of reference materials for genome sequencing. It was attended by approximately 25 people from federal agencies, clinical labs, and technology companies. Options discussed included plasmids with cancer mutations from NCI, tumor-normal cell line pairs, and cell lines from HorizonDx and Acrometrix that contain engineered variants. The working group aims to define testing protocols to evaluate reference material performance across labs and assess their ability to characterize sequencing platforms and tests.

1. Genome in a Bottle Working Group
Reference Material (RM) Selection and Design
NIST Workshop
August 15, 2013

2. Main Meetings – NIST Reference Materials
• April 13, 2012 (NIST)
– Genome in a Bottle consortium initiation
• August 16, 2012 (NIST)
– Intended uses of RMs
– RM selection strategies
• November 7, 2012 (ASHG)
– Status updates
• December 6, 2012
– Selection of initial RMs
• March 21, 2013 (XGEN Congress)
• August 15, 2016 (NIST)
| 2

3. Workgroup Attendees
NIST Workshop – Aug 16, 2012
• Approximately 25 attendees
– Federal, incl. FDA, CDA, NIST
– Lab accreditation
– Clinical reference labs
– Platform technologies
– Reference material / reagent providers
– Research
| 3

4. Reference Materials Needed For
• Clinical platform validation
– Sequencing System
– Bioinformatics/Analysis Pipeline
• Clinical test development and validation
– Whole genome
– Targeted
– Germline vs. tumor
– Other?
• Research
– Process development and QC
• Product development
– Sequencing Systems
– Software development
| 4

5. NY State Guidelines – Oncology NGS
Minimum Data Requirement - Validation
• Accuracy: Sequence a well-characterized reference sample (e.g.
HapMap DNA GM12878) to determine error rate across all
amplicons.
• Analytical sensitivity: Establish the analytical sensitivity of the
assay by interrogating all variants in the 3 amplicons with the
consistently poorest coverage, and all variants in 3 amplicons with
consistently good coverage. This can initially be established with
defined mixtures of cell line DNAs (not plasmids), but needs to be
verified with 3-5 patient samples.
• Analytical specificity: Establish the analytical specificity of the assay
by interrogating all variants in the 3 amplicons with the consistently
poorest coverage, and all variants in 3 amplicons with consistently
good coverage. This can initially be established with defined
mixtures of cell line DNAs (not plasmids), but needs to be verified
with 3-5 patient samples.
| 5

6. Reference Material – Intended Uses
• Characterize Platforms & Methods
– DNA sequencing
– Existing & upcoming NGS technologies
– Research applications
– Clinical diagnostics applications
• Not intended as reference material for
– Validation of specific mutations in a panel
| 6

7. Preferred Reference Standard Criteria
Reference Lab Perspective
• Commercially available (renewable and constant)
• Cost-effective
• Similar in performance to anticipated test samples
• Contain representative type and number of mutations tested
• Ability to adjust mutation frequency to monitor assay sensitivity
• Recognized by agencies responsible for recommending standards

8. Status Summary – RM Working Group
• Defined required RM characteristics
• Initial set of RM samples selected
– NA12878
• Many existing public and proprietary datasets
• Listed in clinical guidelines to establish validation parameters
• Consent limitations
– Commercial use, re-identification through sequence
• Under legal and IRB review by NIST
– Three PGP trios
• One trio already available at Coriell
• Consent without withdrawal option may not
meet ethical review standards
| 8

9. Reference Material Portfolio
Synthetic DNA
• What types are most useful
– Variant types, sequence context, phasing
• Assess strengths & weaknesses of platform
• Test validation for existing and medically relevant variants
• What size
• How many synthetic RMs
– Platform validation
– Test validation
• Which existing sources should be considered
– Available for research and for profit
• Source of commercial development and distribution
– Manufactured under quality system for diagnostic applications
| 9

10. Agenda
RM Selection and Design Working Group
• Presentations:
– NCI Cancer spike-ins – Jason Lih
– TGen-Illumina tumor-normal pair study – Stephanie Pond
– HorizonDx cell lines – Jonathan Frampton
– Acrometrix FFPE controls – Mona Shahbazian
• What role should synthetic spike-ins play in GIAB?
– Spike-ins could include NCI cancer spike-ins, TGen cancer rearrangement spike-ins, or new synthetic
DNA designs
• Should GIAB start to develop tumor-normal cell line pairs as Reference Materials
• What is the relative importance of having genomic Reference Materials from a
variety of ancestry groups
– What are the relative priorities of ancestral diversity, trios, and the broad consent of the
PGP for commercial redistribution
– Trios in PGP are currently all Caucasian except for one Asian (Chinese) trio. If PGP
cannot have trios from other ancestry groups, should we use PGP Caucasian trios, PGP
individuals without consented parents, or trios from other projects like 1000 Genomes

11. Workgroup Attendees
NIST Workshop – Aug 15, 2013
• Approximately 25 attendees
– Federal, incl. FDA, CDA, NIST
– Clinical reference lab
– Platform technologies
– Reference material / reagent providers
– Research
| 11

12. Summary
Jason Lih / NCI
• 53 plasmids with engineered cancer mutations
– 1kb fragments
– Mutation in center
– Alien barcode in vicinity for differentiation from human DNA
– Sequence is Sanger confirmed
• Low level mutations not observed in NGS verification
• Quantitation by qPCR
– Observed concentrations in dilution series are linear, similar to cell line
DNA controls
• Spike in controls for reference material or clinical samples
• Linearized, no chromatin structure or FFPE exposure
• No blocking I.P. for plasmid
| 12
Proposal: Interlab comparison of plasmid controls to assess
performance and resemblance of FFPE derived DNA
Possibility of QC Standard?

13. Summary
Stephanie Pond / Illumina
• Tumor / Normal Cell Line Pair
– COLO829B / COLO829
– Pleasance et al, Nature 2010, 463:191-196
– SNVs, InDels, CNVs, SVs (37)
• Confirmed by PCR & sequencing
– HiSeq 90x WGS at Illumina & TGEN
– Complete Genomics WGS
– Multiple dilution series analyzed
– Consent needs investigation
| 13
Monitor: Offers no clear advantage over existing selected
normal cell line trios

14. Summary
Stephanie Pond / Illumina
• 9 synthetic fusion constructs
– Spiked into COLO829 DNA
– One Spike-In not detected by any fusion caller
• Confirmed to be present in sequence
| 14
Opportunity: Assess and optimize fusion caller software
Recommendation: Recommend to make results available
in data repository for software optimization
Early status for fusion constructs, continue to monitor

15. Summary
Jonathan Frampton / Horizon Diagnostics
• Large set of engineered cancer mutations in 3 cancer
cell lines
– Proprietary engineered cell lines
– Confirmed identities of parental cell lines
– Digital PCR confirmed mutational frequencies
– FFPE embedded cells / DNA available
– Linear serial dilution response to 0.05%
– 11 core mutations are multiplexed with frequencies
between 1% - 25%
• digital PCR confirmed
– 19 additional mutations planned for multiplex with similar
range of frequencies in mixture
| 15
Assess performance in interlab comparison
Combine with assessment of synthetic plasmids and Acrometrix controls?
Opportunity for independently supplied reference materials

16. Summary
Kara Norman / AcroMetrix
• Engineered cell line controls
– 8 different multi mix controls
– 12-26 variants per control
• 1-7 COSMIC variants / control
– Standards produced under cGMP / Quality System
– Proprietary cell lines
– FFPE embedded cells / DNA available
| 16
Assess performance in interlab comparison
Combine with assessment of synthetic plasmids and Horizon Dx controls?
Opportunity for independently supplied reference materials

17. Next Steps
• Working group to define testing protocol(s)
and assess plasmid controls in interlab study
– Combine with assessment of Horizon &
Acrometrix cell line controls?
• Confirm that performance is similar to clinical
FFPE samples
| 17