Data Strategies in Testing

www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved
Why do we care about the data used by
the system under test?

Data Strategies
PAUL MERRILL
@dpaulmerrill

Agenda
Define Terms
Identify Constraints
Walk Through Examples

Define Terms

Define Terms
Data Strategy - the combination of code,
procedure and infrastructure that affect how
tests interact with data to stimulate the
System Under Test (SUT).

Define Terms
Signed Data - data which has been
modified so that it is easily recognizable.
For example, an account with a user name
like “TestUser” or an email address like
“automated@testing.com”

Define Terms
Constraint - a characteristic that applies a
force in a way that is limiting to an
automated test or its environment.

Define Terms
Constraints factor down to:
Policy
Cost
Test Expectations
Infrastructure
People

Identify Constraints

Examples of Constraints
Shared Environment
– Cost
– Policy
– Infrastructure
Downtime
– Policy
– Cost
– People
Technical Skills
– Cost
– People

Examples of Constraints
Data Source Growth
– Cost
– Policy
– Infrastructure
Repeatable Tests
– Policy
– Test Expectations
– People
– Infrastructure
Data Variance
– Policy

Example Data Strategies

Strategies Have 2 Parts:
Creational Strategy - how and when data
is created.
Cleanup Strategy - the method by which
a data source is returned to a benign
state.

Elementary Approach
Creation - Tests assume specific data in
SUT - no creational strategy
Clean up - No clean-up strategy, no
management of data

Elementary Approach
SUT
Execution 1
Execution 2
Execution 3
Data
Source

Elementary Approach
Pros:
Simple
No implementation
(quick)
Cons:
Tests are not
repeatable
No isolation of data
Does not work on
different systems

Refresh Data Source Approach
Creation - Tests assume specific data in
SUT
Cleanup - Data managed by refreshing the
data source prior to test execution

SUT
Execution 1
Data
Source
Refresh
Execution 2
Refresh
Execution 3
Refresh

Pros:
Clean test bed (data-
wise) per test
Uniform, one-size-fits-
all process
No variance of data
Less time/cost to
implement
Total control of data
Cons:
Timely
Requires Downtime
Blocks shared
environments
No variance of data
Stale data over time
Doesn’t work on
different systems

Selfish Data Generation
Creation - Tests create the data they need
in SUT
Clean up - No management of data after
generation
May be random (e.g. via Faker)
Create with batch process or in test
Create via SUT or directly in Data Source
Create In-test or pre-test

SUT
Execution 1
Execution 2
Execution 3
Data
Source
Generate Data
Generate Data
Generate Data

Pros:
Unique data per test
Does not block shared
environments
No downtime
No stale data
Tests can run anywhere
Tests become
repeatable
Cons:
Unchecked growth of
data source
Requires understanding
of data model
Tests need to know
about data

Data Generation & Batch Cleanup
Creation - Data generated for each test
Cleanup - Data cleaned up after test run
Clean up may happen in data source or
via SUT

SUT
Execution 1
Execution 2
Execution 3
Data
Source
Generate Data
Generate Data
Generate Data
Batch Clean up

Pros:
environments
Minimal downtime
No stale data
Growth of data source
kept in check
Tests run anywhere
Tests are repeatable
Cons:
Requires deep
understanding of data
model
Must know what to clean
Significant scaffolding
must be created

Data Generation & Immediate Cleanup
Creation - tests generate their own data
Clean up - for each test case immediately
after execution
Uses a TearDown method supplied by the
framework

SUT
Execution 1
Execution 2
Execution 3
Data
Source
Generate Data
Generate Data
Generate Data
Clean up
Clean up
Clean up

Pros:
Tests are repeatable
environments
Minimal downtime
No stale data
Growth of data source kept in
check
No data left over from other
tests
Tests run anywhere
Cons:
Requires deep
understanding of data model
Must know what to clean
Significant scaffolding must
be created

Creational Strategies
Elementary Refreshed Data
Source
Data Generation
Shared
Environment
Low Downtime
Repeatable
Tests
Data Variance
Minimal Data
Growth
Significant
Technical Skills

Cleanup Strategies
Elementary
(none)
Refresh
Data
Source
Batch Immediate (Signed,
Marked or Persisted)
Shared
Environment
Low
Downtime
Repeatable
Tests
Strong
Technical
Skills

How do we know what data to
clean up?

Signed Data
Record Name: “test_21323”
Field Name: “Record Name”
Format: “test_<random5digs>”
Persist in:
– a data source
– code

Marked Data
An “isTestData” field or
Data that is otherwise known to be test
data because of its format or type, for
example:
– A “test user” created it
– Linked to known test denotation

Internally Persisted Data
Keep a list in memory:
– userIds:
• 12312
• 4322
• 23423
– emailIds:
• 12312
• 423423
• 234

Externally Persisted Data
Keep a record in a data source:
– userIds:
• 12312
• 4322
• 23423
– emailIds:
• 12312
• 423423
• 234

In Review
Defined Terms
Identified Constraints
Walked Through Examples

Thank you!
PAUL MERRILL
@dpaulmerrill

Data Strategies in Testing

Recomendados

Recomendados

Más contenido relacionado

Similar a Data Strategies in Testing

Similar a Data Strategies in Testing (20)

Último

Último (20)

Data Strategies in Testing