What You Need to Know about Beta Management

What You Need to Know
Beta Management

Brought to you by the experts at Centercode

v1.1

WHAT YOU NEED TO KNOW • BETA MANAGEMENT

Introduction
About the Series
Centercode’s “What You Need to Know” series is designed to walk readers through the basics
of different kinds of beta tests. This chapter will provide a foundation on beta testing, while
future chapters will focus on specific types of beta tests such as hardware or mobile tests.

Audience
This series is intended for quality, marketing, project, and product managers who’ve been
tasked with running their first beta test. If you’re an experienced beta manager, it may
provide useful tips to help further improve your beta process.

Goals
This chapter will explore the core issues and terms found in beta test management, providing
you with a basic understanding of how to manage a beta test. It will also provide the
framework for future chapters in the series.

Download a simpler, print-friendly version of this chapter.

WWW.CENTERCODE.COM 2


About Us
• We’ve been focused on delivering great Beta Test Management Solutions since 2001.

• Our offerings include Centercode Connect™, a hosted beta test management platform that’s
been used to run thousands of beta tests globally, fully outsourced Managed Beta Tests, and
Onlinebeta.com, a community of more than 60,000 highly profiled test candidates.

• Feel free to contact us with feedback or questions at info@centercode.com.

• Our customers run the gamut of products, markets, and company size, including:



Table of Contents
Introduction 2 Planning 21 Managing 39
About Centercode 3 The Beta Plan 22 Beta Activities 40
Writing a Beta Plan 23 Bug Reports 41
Definition of a Beta Test 6 FAQ: How long to beta? 25 Feature Requests and Forums 42
Other Terms for Beta 7 Tasks and Journals 43
Beta Details 8 Recruiting 26 Surveys and Testimonials 44
Types of Beta Testing 9 Tester Recruitment 101 27 Wikis, Calls, and Visits 45
Benefits of Beta Testing 10 Why Do People Beta Test? 28 Tips for Tester Management 46
The Beta Message 11 Friends, Family, and Employees 29 FAQ: Tester Time Dedicated? 47
Major Beta Challenges 12 Advertising For Testers 30
Top 10 Beta Mistakes 13 Selecting Great Testers 31 Closing 48
FAQ: How Many Testers? 32 Keys to Closing a Beta Test 49
The Tools of Beta Testing 15 Incentives! 50
Productivity Tools 16 Kickoff 33 Developing a Closure Report 51
Non-Beta Software & Services 17 Getting Ready for Beta 34
In-House Beta Systems 18 Product Readiness Checklist 35 Improve 52
Commercial Beta Management Tools 19 Tester Readiness Checklist 36 Elements for Beta Success 53
Other Useful Tools for Beta 20 Team Readiness Checklist 37 Next Steps 54
Legal Agreements 38 Contact Us 55



{
Beta Management
Basics

{
Tools 1. Planning
2. Recruiting
Process 3. Kicking Off
4. Managing
Improve 5. Closing



Definition of a Beta Test

“A test of a product performed by
real users in real environments”

(basically your customers testing instead of your QA team)



Also Known As...
Beta testing goes by many names. Depending on the company,
sometimes they have slightly different applications, but they all focus on
customer feedback prior to product release. Here are a few:

• User Acceptance Testing (UAT)
• Customer Acceptance Testing (CAT)
• Customer Validation (CV)
• Field Trials (common term in Europe)
• Pre-Release

Note: While we use the term “beta” throughout this series, the same
principles, processes, and benefits apply, no matter what you call it.



A Beta Test... (the details)
• Is the final test prior to shipping a product

• Generally begins after an internal alpha test

• Begins when the product is at or near feature complete

• Includes existing and/or targeted prospective customers

• Commonly prioritizes quality improvement (i.e. squashing bugs)

• Establishes a final assessment of the complete product prior to release

• Extends QA testing (but should never replace it)

And Most Importantly...

• Differs greatly in challenges, execution, and benefits based on tools,
product type, target market, beta goals, and development phase



Types of Beta Testing
There are many different types of beta tests, each with its own set of challenges, goals, and
benefits. Any individual product test is likely comprised of a number of these, based on the type
of product, its target market, the goal the test is attempting to achieve, and the phase at which
it’s happening. For example, the following illustrates a traditional performance-based hardware
beta test aimed at the business market.

1. Product Type 2. Target Market 3. Primary Goal 4. Test Phase
FF Software FF Consumer FF Requirements FF Pre-Alpha
FF Hardware FF Children FF Performance FF Alpha
FF Video Game FF Business FF Focused FF Pre-Release
FF Mobile FF Enterprise FF Stress FF Agile
FF Web FF Government FF Awareness FF Post-Release

Each of these defining attributes (hardware, enterprise, etc.) will be covered in individual
chapters of this series, focusing on their unique challenges, goals, and benefits.



Benefits of Beta Testing
• Beta testing provides true customer validation and improves product quality
beyond the inherent constraints of the quality testing phase, resulting in:

1. Reduced Development Costs
2. Increased Sales
3. Reduced Support Costs
4. Reduced Returns
5. Increased Customer Satisfaction
6. Improved Product Planning

• Other benefits are unique to product type, market, beta goals, and phase (each will be
covered in further chapters in this series)

• Incredible return based on minimal investment (see our Beta Test ROI Kit for details)



The Beta Message
Everyone will see the value of beta differently. To gain buy-in, it helps to communicate how
beta will help achieve each group’s unique goals.

• Product Management • Beta testing uniquely covers every aspect of a product, including
support and documentation.

• Marketing • Beta testing is an opportunity to gain initial impressions from your
target market. This feedback can be used to improve the launch of the product.

• Engineering • Beta testing ensures the product is performing exactly as expected in the
hands of real customers.

• Executive Level • Beta testing provides a high level view of the entire product. This
information can be used to make crucial decisions about the launch of the product.

• Quality • Beta directly extends quality efforts to reach real-world environments that are
difficult or impossible to simulate in a test lab.

• Sales • Beta testing can generate testimonials and other assests useful in early sales efforts.
In addition, beta quality enhancements can lead to higher reviews resulting in more sales.



Major Beta Challenges
1. Dedicating Qualified Resources • Beta testing generally happens at the least
opportune time, right when the product team is frantically trying to close up every
loose end, and has little available time or energy to focus on beta.

2. Recruiting Appropriate Beta Testers • Not every interested applicant is the right
beta tester for your product. Finding an adequate pool of targeted but unbiased
testers is a difficult task.

3. Maintaining User Participation • The single most common challenge in beta is
achieving high participation levels from beta testers. The industry average is lower
than 30%, meaning that 70% or more will fail to provide any meaningful feedback.

4. Collecting Relevant Feedback • Not all feedback is valuable. Relevant feedback
is any feedback that either results in a direct improvement in the quality of your
product, or meets another specific goal set for your beta test.

5. Organizing and Distributing Feedback • Successful beta tests generate an
enormous amount of feedback. Distributing the right feedback to the right people
is a complicated and time consuming task.



Top 10 Beta Mistakes
1 Lack of universal beta program support
2 No well defined plan
3 Underestimating ramp-up time requirements
4 Releasing a non-viable product to beta test
5 Too few or too many beta testers
6 Test period is too short or too long
7 Poorly motivated and/or managed testers
8 Ineffective tools resulting in lost time and feedback
9 Failing to manage feedback effectively
10 Badly managed incentives



{
Beta Management
Basics

{
Tools 1. Planning
2. Recruiting
4. Managing
Improve 5. Closing



The Tools of Beta Testing
Beta testing is a highly unique process that includes individuals ranging a wide variety of disciplines,
including your customers. Effective tools can contribute greatly to the success of a beta test.

1. Shorten beta cycles • A pre-profiled pool of candidates can reduce your recruiting and ramp-
up period substantially, allowing you to achieve more, faster.

2. Reduce time investment • Great tools can cut the time you spend coordinating,
communicating, and reviewing feedback of your beta test by more than half.

3. Increase tester participation • Providing a clear path for feedback reduces the friction of
providing feedback, resulting in higher participation rates.

4. Impove feedback quality • Dedicated tools can help produce structured feedback that greatly
improves its clarity, providing much more actionable data.

5. Increase visibility • Great beta tools will integrate with your existing systems, ensuring that
your team is aware of feedback and able to react accordingly — fixing bugs and improving plans.



Productivity Tools
Examples
• Email, spreadsheets, word processors

Pros
• Extremely inexpensive (you probably already have them)

• No learning curve

Cons
• Can’t scale (more than ~10 participants is extremely difficult to manage)

• No means for communication among beta testers

• Not sufficient for a team environment (little to no visibility or accountability)



Non-Beta Software & Services
Examples
• BugZilla, Jira, Survey Monkey, Get Satisfaction, phpBB, WordPress, etc.

Pros
• Inexpensive

• Easily accessible

• Many choices

Cons
• Utilizing multiple systems is disjointed for testers (reducing participation)

• Difficult to manage / Produces fragmented data

• Participation is difficult to track



In-House Beta Systems
Examples
• A tool developed within an organization for managing beta tests

Pros
• Built for beta, customized for specific processes and needs

• Dedicated experience for testers reduces friction and increases feedback

• May include other custom system integrations

Cons
• Immensely expensive to develop and maintain

• The platform doesn’t evolve without ongoing investment

• Risks going end-of-life if internal support wanes



Commercial Beta Management Tools
Examples
• Centercode Connect, VOConline FMS, BetaEasy

Pros
• Built for beta (community, agreements, CMS, surveys, feedback, forums, reporting)

• Low learning curve for customers/testers

• Flexibility to meet evolving requirements, integration with existing tools

Cons
• Niche offering with few vendor options

• Likely more costly than single-function tools

• New application to adopt



Other Useful Tools for Beta
Project Planning Tools (ex. MS Project) – A beta is a project like any
other, with milestones, activities, and resources.

Spreadsheets (ex. Excel, Google Docs) – Spreadsheets contain
everything needed to extract actionable data from any beta test.

The Social Web (Portals, Social Networks, Forums) – Mined correctly, the
Internet offers an endless collection of great beta candidates.

The Centercode Library – We’ve produced a number of educational
resources and useful kits to help run successful beta tests, all available
for free at www.centercode.com/library.



{
Beta Management
Basics

{
Tools 1. Planning
2. Recruiting
4. Managing
Improve 5. Closing



The Beta Plan
• Every good project begins with a plan. In the case of Levantra Confidential Aurora 2.1 Beta Project

beta, the plan ensures that the project parameters, goals,
milestones, activities, and participant demographics are EXAMPLE HARDWARE BETA TEST PLAN TEMPLATE

well-defined and communicated to everyone involved.

• We’ve developed two kits which include everything Mobile Phone
needed to develop a comprehensive beta plan. Each kit Beta Test Plan
includes a complete walkthrough, a detailed sample,
Levantra, Inc.

and a template to create your own plan. Aurora 2.1 Beta Test

Prepared by Austin Stevens

Download the free kits: ALL DATA IS FICTITIOUS

• Hardware Beta Test Planning Kit
BETA TEST PLAN EXAMPLE 1

• Software Beta Test Planning Kit



Writing a Beta Plan (1)
While there are countless ways to structure a beta plan, at a very minimum any plan should
include the following components.

1. Budget • Identify any costs which will be associated with the beta, including tools, beta
units, shipping logistics, and incentives.

2. Product Definition • Construct an outline of the product itself, including information
regarding its current state (alpha, beta, beta2, etc.), high-level modules, etc.

3. Test Parameters • Outline the basic scope of the project, including total number of
beta testers, project schedule, and milestones.

4. Target Market Definition • Detail the target market for the product. If the market
is segmented, break down the types of users into profile pools. Include the tester
requirements (time, hardware, etc.) in this definition.



Writing a Beta Plan (2)
5. Test Goals • List the unique goals of the project, such as general quality improvement,
interface acceptance, testing real-world performance, testing support infrastructure,
collecting customer suggestions, and recording testimonials. Also include the areas
(modules) of the product that will be tested.

6. Project Stakeholders • Define the responsibilities of those involved in the project. Define
how feedback will be managed as it changes hands and progresses.

7. Initial Testing Activities • Plan out the initial activities which will be performed
throughout this beta (activities will likely change in response to the project progression).
This may include items such as bug reporting, feature requests, forum discussions, survey
completion, and assigned tasks.

8. Incentives • Plan out the awards for participation, as well as the participation levels
which they will be based on.



Frequently Asked Question

Q How long should my beta last?

A Beta tests generally run anywhere from 4-8 weeks of test
time, often split between multiple phases with short breaks in
between. With effective processes and tools, betas can produce
great results in as little as 2 weeks. Beta length should be
determined by the set of goals you are intending to accomplish.
Depending on those goals you may also be able to reduce your
test length by increasing your tester pool.



{
Beta Management
Basics

{
Tools 1. Planning
2. Recruiting
4. Managing
Improve 5. Closing



Tester Recruitment 101
Your tester team is the most valuable resource in your beta test. This team can make or break your
entire project. Therefore, finding the right testers is absolutely crucial.

Recruitment is generally split into two phases:

1. Advertise via various means to recruit a large pool
of interested candidates. Candidates are generally
asked to complete a qualification survey. If possible,
it’s best for this pool to be 5-10x the size of your
intended tester team.

2. Use whatever data you have available to select
the most qualified best testers from the pool of
candidates. The better job you do selecting effective
testers, the fewer total participants you’ll actually
need in your test.



Why Do People Beta Test?
Knowing why people become testers will help both recruit great participants and keep them engaged. Here are
some of the primary reasons people test:

Problem Solvers • Many products (or new revisions of previous products) exist to solve a problem or fill a void
of some sort. People feeling the pain of those problems are often eager to be the first to have them solved.

Early Adopters • Some people are enthusiasts who enjoy experiencing early versions of products. These testers
love technology and are excited for the opportunity to carry even a minor role in its development process.

Raving Fans • Many people are passionate about a brand or product line, and are interested in having a chance
to influence where those products go. Beta provides a rare opportunity to do so.

Career Minded • Business focused beta tests offer beta testers an opportunity to expand their skillset to
include knowledge and use of products that aren’t even on the market, gaining them a competitive edge.
Similarly, betas can provide an opportunity to network with peers.

Free Product • The opportunity to earn free products is one the most commonly recognized reasons to
participate (but generally not what you’ll use for recruitment).



Friends, Family, and Employees
The most readily available beta testers are often those closest to you. Unfortunately, these are
generally not the best choices.

Employees
Your co-workers are generally a bad choice for testers as they either have too much insight into
the product itself or are impartial because of their relationship with the company.

Friends
Usually friends make bad testers because they are inclined to feel that their insider relationship
negates the requirement to participate at the same level as other testers.

Family
Family members are also not the best choice for testers as their connection to you or your
team may inhibit them from being objective.



Advertising For Testers
• The Internet opens the door to an endless supply of beta testers. It’s important to
manage this source effectively to collect the right ones.

• Find portals/networks that interest the target market (every market has them). Ask
an administrator to help advertise for you.

• Clearly indicate the technical requirements for each tester. Wading through
unqualified candidates is a waste of time.

• Incentives deserve mention in a tester call, but be vague. Testers are most
effective when selected based on excitement rather than reward.

• It is not necessary to advertise specific features of the product (especially anything
confidential). Focus on the product category being tested.

• Be very clear and honest about the time requirements to participate in the project.



Selecting Great Testers
• Always keep your target market in mind. Selecting testers from the wrong
market is the easiest way to produce useless results.

• Look for responsive and communicative behaviors. These traits will likely
remain constant in the actual beta phase.

• Look for detail-oriented applications. Select candidates whose applications
resemble the style that would ultimately be most valuable in feedback.

• Recognize those who are excited by your product.

• Avoid those who refer to or question test incentives immediately.

• If possible, include a few previously successful beta testers. They’ll
appreciate the recognition and positively influence other testers.




Q How many testers should I use?

A It varies. The size of your tester team should depend on your
product, target market, prior experience, and the tools available.
As a baseline, consumer products should generally be tested
with anywhere from 50-200 people, while business products
commonly include only 10-50. Many variables can change these
numbers. Keep in mind that internal resources should drive
tester team size as well.



{
Beta Management
Basics

{
Tools 1. Planning
2. Recruiting
4. Managing
Improve 5. Closing



Getting Ready for Beta
Kicking off a successful beta test requires alignment of the following three key components:

• The product is in a state that is ready to be tested.
The product is stable and near feature complete. General
functionality should work, with the majority of testers not
discovering unknown bugs in their first hour or so of testing.
GETTING READY FOR
• Testers are both ready and able to test your product.
Testers have been selected, understand the schedule and what’s
BETA TESTING
expected of them, and meet any technical or knowledge
requirements to successfully test.

For many companies, beta test preparation is fraught with uncertainty.

• The team is ready to engage and react in real-time. Even knowing when you’ll start can be complicated. You might want to base the decision on
QA milestones, ensuring the product is stable enough for users. But if you’re facing a product
release window that can’t be missed, your beta schedule might be set by counting backwards on

Stakeholders have been identified and educated, and are ready to
the calendar. The goal of this whitepaper is to establish a set of best practices for beta readiness
(private beta tests, specifically). That way, you can be confident that you’re ready to launch an
effective beta test regardless of any looming uncertainty.

interact with testers and feedback.
In our experience, the best way to get ready for beta testing is to establish concrete guidelines
in three areas: product readiness, team readiness, and tester readiness. If these areas are out
of sync or underprepared, your beta will most likely suffer. But when they come together, they
enable higher participation, easier management, and better results.


Failing to align any of these areas will greatly reduce feedback quantity
and quality, ultimately reducing the value and accuracy of the beta test.

Free Resource: The following checklists are summarized from our Getting Ready for Beta Testing whitepaper.



Product Readiness Checklist
FF The engineering team has verified that all components of the product are
ready to begin beta testing.

FF Auxiliary components (documentation, etc.) have been assembled into a
single package which represents exactly what will be sent to testers.

FF The out of the box experience has been successfully reviewed, including
setup, installation, and documentation.

FF Basic product functionality has been successfully reviewed (all key features
are working) by product management.

FF Known bugs which could not be addressed prior to beta have been clearly
documented and communicated to testers.

FF The uninstall process (if applicable) has been successfully verified.



Tester Readiness Checklist
FF A sufficient number of testers meeting each required target market segment
have been selected and notified.

FF Accurate contact information and addresses (if testing hardware) have been
verified for all testers.

FF Non-disclosure and beta participation agreements have been explained in
plain English, signed, and returned by all testers.

FF Responsibilities and the project schedule have been clearly communicated
to the tester team.

FF Testers understand how to use the systems provided for feedback. If they
aren’t dead simple, training or documentation has been provided.

FF All resources needed by testers to carry out their responsibilities are easily
accessible and easy to understand.



Team Readiness Checklist
FF Core parameters and processes (e.g. testing goals, strategies and mechanisms
for collecting feedback, categories for bugs, incentives, etc.) have been defined
and communicated to all stakeholders and contributors.

FF Milestones and deadlines have been communicated and understood and all
necessary resources are readily available.

FF Stakeholders have delivered all pre-test deliverables (e.g. tools, documentation,
surveys, packaging, product keys, NDAs, beta units, etc.)

FF Contingency plans have been defined for any stakeholders with limited
availability (e.g. planned vacation, potential birth of a child, etc.)

FF Any infrastructure that will be relied upon during the beta (beta test
management tools, customer support, bug tracking, content delivery, servers,
etc.) is accessible and has been tested.



Legal Agreements
Given the pre-release nature of beta products, most tests require
a Non-Disclosure Agreement (NDA) forbidding participants
from openly discussing their experiences with the product. Beta
NDAs are similar to traditional NDAs, but generally include a few
attributes unique to beta.
Beta Test
Agreement Kit
In addition to NDAs, it is common to execute a separate Presented by Centercode

(or combined) agreement known as a Beta Participation
Agreement (BPA). This agreement outlines the expectations for
participating in the project.

Free Resource: We’ve developed a comprehensive Beta Test Agreement Kit, which includes
NDA and BPA templates, along with an explanation of their contents.



{
Beta Management
Basics

{
Tools 1. Planning
2. Recruiting
4. Managing
Improve 5. Closing



Beta Activities
There are a wide variety of activities your beta testers would be more than willing to participate in
with the goal of improving your product. The activities most relevant to your product will vary based
on the objectives outlined in the beta plan. Some of the most common activities are:

• Bug Reports
• Feature Requests
• Forum Discussions
• Assigned Tasks
• Daily Journals
• Surveys and Polls
• Testimonials
• Wiki Collaboration
• Tester Calls
• Site Visits

Over the next few pages we’ll cover each of these in more detail.



Bug Reports
Bug reports are collected in nearly all beta tests. This is generally considered to be the most valuable aspect of
beta testing, as correcting defects will lead to higher quality. The following are tips for collecting beta bugs:

• Bug report forms for beta testers are generally simpler than those completed by trained QA staff, but at
a minimum should include a title, description and/or steps to reproduce, and severity (the tester’s view of
severity may not match your own definition, but it’s important to capture the impression).

• Bug reports should allow testers to attach files (for screen shots, logs, etc.). These files often describe the
bug more clearly or concisely than the tester may be able to (saving a great deal of time).

• If details about the tester’s environment (operating system, hardware configuration, etc.) are not available,
this information should be collected. Note that collecting this information on every bug report will require
repetition, ultimately reducing feedback (i.e. if possible, collect it up front and reference it).

• A comment log should be used to communicate back and forth with the tester (to collect other details).

• Testers should be encouraged to report bugs not only about the product, but also the documentation and
any other auxiliary systems or services they are experiencing as part of the beta (support system, etc.). This
feedback should be distributed to the appropriate stakeholders.

• Bug reports should flow into your development team’s defect tracking system (Jira, BugZilla, etc.).
Automating this process saves a great deal of time, allows engineering to use familiar tools, and reduces the
opportunity for crucial feedback to be lost.



Feature Requests and Forums
Feature Requests
• Given the near feature complete state of most beta tests, the goal of feature requests is to capture early
insight into customer requirements for future revisions of the product. In this sense feature requests are
most useful for product planning.

• If an overwhelming majority of your testers are asking for a specific feature, it may be worth considering
including that feature in the final version of the product. This is especially true in business product tests.

• It’s a good idea to include a disclaimer on a feature request form which claims ownership of any ideas
generated by testers as part of the beta test.

Forum Discussions
• Social forums in beta tests (1) Increase participation by providing testers with a sense of community, and a
reason to keep using and discussing the product; (2) Reduce beta management and support requirements
by allowing testers to support each other; (3) Provide a simple mechanism for hosting directed discussions
(e.g. an online focus group); and (4) Help reduce confidentiality liabilities by providing testers with a
controlled outlet to express their excitement, reducing their urge to do so on the public Internet.

• While forums are a great tool in beta testing, they’re a very inefficient means of managing bugs and feature
requests. It’s important to educate testers on the proper channels for different types of feedback.



Tasks and Journals
Assigned Tasks
• Assigning specific tasks to groups of testers is an excellent way to both achieve goals and increase
participation. Tasks can initially kickstart a project and encourage exploration. This can ensure coverage
while giving testers an initial push to get started. Once the project is up and running, assigned tasks can be
used for focused regression testing or to assess the impact of an issue.

• If used incorrectly, tasks can backfire and stifle exploration, more closely emulating QA testing as opposed
to true customer beta testing. Early tasks should be focused on things that are valuable for all users to do,
while for regression, tasks should be assigned to as few testers as possible to get the job done.

Daily Journals
• Daily journals are simple forms that testers are expected to complete each day. Each one provides a short
overview of the tester’s recent experience with the product. If possible these should include a rating scale
that allows you to focus on the highly negative or positive experiences.

• Daily journals are a great way to increase participation, as they give less social users an outlet to discuss
their experiences with your product. They also provide a reason to return to your beta site, which will keep
the tester engaged with updates, builds, etc.



Surveys and Testimonials
Surveys and Polls
• Surveys and polls are a great way to both collect feedback and keep testers engaged throughout
your beta test. These are most commonly used to capture opinions (about features, functionality, or
performance), or to collect specific feedback about issues (bugs, etc.) identified throughout the project.

• Generally surveys should be limited to once per week, and less than 20 questions. If essay questions
require a great deal of effort this should further limit the number of questions. Polls are simpler and may
run multiple times per week.

Testimonials
• Most beta testers are more than willing to share their excitement for your product. These forms should
include instructions encouraging the tester to be open and honest, followed by a freeform text box. If
you’re looking for a specific length, it’s important to make this clear, as some testers will naturally be very
short, while others will be overly verbose.

• Testimonials are generally collected toward the end of a beta test (possibly part of a final closure survey)
and should include a disclaimer that describes how the testimonial will be used. Depending on your
audience, it may also be useful to include a checkbox confirming use of the users name and/or company
when referencing the testimonial.



Wikis, Calls, and Visits
Wikis
• Collaborative wikis are a very contemporary beta activity where testers are allowed to either directly
edit or suggest improvements for future customer facing assets, most commonly including support
(documentation, FAQs, etc.) and marketing materials. These are an excellent way to give customers
another means of feedback, while directly contributing to the improvement of your product.

Tester Calls
• Customer conference calls (including either single or multiple testers) offer direct real-time feedback,
similar to a focus group. These are generally scheduled both early and late in a beta test, offering
the product team the ability to converse directly with customers prior to release. Customer calls also
increase participation rates by demonstrating the high value of the beta tester and their feedback.

Site Visits
• When possible, visiting a beta customer is a great way gain a first-hand understanding of the customer
experience. Beyond the natural benefits of a face-to-face conversation, customer visits allow product
teams to watch customers perform tasks in their natural environment, providing valuable insight into
real-world usage. Similar to customer calls, customer visits can increase tester participation.



Tips for Tester Management
Achieving great beta tester participation rates is widely considered to be the most frustrating
challenge when running a beta test. The following tips will help raise your participation rates.

1. Clearly express expectations early on and throughout

2. Consistently encourage confidentiality (require NDAs, etc.)

3. Balance gratitude with expectations Reaching 90%
Beta Participation
4. Keep testers involved with ongoing activities
A Best Practices eBook for Getting the Most
5. Keep testers current regarding project progress Out of Your Beta Program

by Centercode

6. Offer a simple method for testers to provide feedback

7. Allow testers to communicate amongst themselves
www.centercode.com v1.1

8. Respond quickly to all issues and requests

9. Contact inactive testers directly (by phone if possible)

Free Resource: Check out our eBook on Reaching 90% Beta Participation.




Q How much time should testers dedicate?

A This depends highly on the complexity of the product and
goals of the project. Simple products may only require a few
hours a week, while more complex products can require 10-20
hours a week or more. Keep in mind that a properly targeted
beta tester should be fitting this product into their daily life,
therefore testing shouldn’t feel like “work”.



{
Beta Management
Basics

{
Tools 1. Planning
2. Recruiting
4. Managing
Improve 5. Closing



Keys to Closing a Beta Test
We’re in the home stretch now. The following steps will help close out your test.

1. Give your testers time to submit final issues. Generally it’s best to give your testers
at least three days’ notice before the end of a project.

2. Cut-off tester access to submit issues. Once your end date has been reached it’s
important to cut off feedback.

3. Close all open issues. Review and close any remaining feedback. This should be done
quickly in case additional information is needed before you lose touch with your testers.

4. Offer testers a simple means to return the product. If the product is hardware
based, be sure to provide return instructions (testers should never bear any expense).



Incentives!
Incentives are incredibly important to your beta test. You want to make sure the testers
that provided you with all that amazing and critical feedback are well rewarded. Doing so is
not only fair, but also sets a good example for the next time you need their help.

• Include multiple levels of incentives based on performance. Participation will
likely vary heavily between your beta testers. It’s important to recognize and thank
those who went above and beyond.

• Ensure incentives match participation requirements. If you expected a great deal
of time out of your participants, the incentive should match that effort.

• Reward all who meet those levels. Generally a “top 10” approach is not as successful
as levels. If everyone provided incredible feedback, everyone deserves an award.

• Distribute incentives quickly. The golden rule applies here: You expected your
testers to participate immediately, so it’s only fair for you to do the same. If possible,
have your incentives sent within a week of test closure.

• If possible, thank testers individually. A personal touch goes a long way.



Developing a Closure Report
Now that your testers are happy and on their way, it’s time to go back to your stakeholders. A
closure report provides an overview of the results of the beta test.

• Develop an executive summary of the project

• Document key issues found

• Document frequently requested features

• Document survey results (with pretty charts)

• Document top testers & incentives rewarded

• Ensure all departments obtain a report



{
Beta Management
Basics

{
Tools 1. Planning
2. Recruiting
4. Managing
Improve 5. Closing



8 Elements for Beta Success
1. A Detailed Plan • A solid plan provides a clear roadmap for everyone involved.

2. Universal Buy-In • A team that believes in beta will be much more willing to help.

3. Great Testers • The wrong testers can produce useless results.

4. Communication • Provide regular updates on timelines, responsibilities, and progress.

5. Responsiveness • Make testers feel involved as key members of the team.

6. Effective Tools • The right tools increase the efficiency of everyone involved.

7. Organization • Effectively manage the enormous amount of data generated in beta.

8. Great Incentives • Great testers deserve great rewards.

Free Resource: Check out our eBook, 100 Tips for Better Beta Tests, for a comprehensive
collection of practical tips from Centercode and beta managers at top technology companies.



Next Steps
We hope this resource has helped you on your quest to understanding beta. If you’d like
to improve your betas further, we’d recommend taking a free trial run of our complete
beta test management platform, Centercode Connect™.

• Recruit great candidates and select the best
beta testers who meet your own specific
target market.

• Collect and manage the feedback you need
(including bugs, suggestions, tasks, surveys,
forums, and wikis).

• Generate and automatically distribute custom
reports based on your own beta objectives.

We offer a fully-featured free trial, available at www.centercode.com/trial



Banner image courtesy of Flickr user xjrlokix (Thanks!)

www.centercode.com • 800-705-6540 • info@centercode.com • @centercode


What You Need to Know about Beta Management

Recomendados

Recomendados

Más contenido relacionado

La actualidad más candente

La actualidad más candente (9)

Similar a What You Need to Know about Beta Management

Similar a What You Need to Know about Beta Management (20)

Más de Centercode

Más de Centercode (9)

Último

Último (20)

What You Need to Know about Beta Management