1. Fantasy Football Commissioner – Usability Study November 2008
Initial Feedback
Overview: A total of 10 participants took part in a usability study of the Fantasy Football Commissioner product
on November 18-19, 2008. Approximately half of the participants were experienced Fantasy players (2+ years
using a Fantasy product of some sort) the other half had little or no experience but were interested in Fantasy
sports. The goal of this study was to examine the user experience of visiting CBSSports.com, finding the Fantasy
Football product offerings, creating a Commissioner League, and then setting up and managing teams.
Usability
Rating
# Top Level Observations and Findings
1. Registration:
Participants regarded general site Registration as simple and straightforward – standard.
Note: One of the main errors encountered during registration was choosing a site ID – one that was
not already used by another user. Providing a “Check Availability” option would be helpful. Also,
field by field error checking could be beneficial. Error prompting on page reloads (the action
invoked by the user after completing the registration form) received some negative reaction from
participants.
2. Home Page Clutter:
A general theme observed during usability sessions was that the Home page was cluttered -
especially for a first time visitor. It takes mental and visual effort to absorb the Home page and
determine the lay of the land. Users have a point of comparison about Home page layout and given
the exposure to other Sports web sites, it is important to examine how to improve the first time visit
on CBSSports.com in the future.
Comments from two of the participants:
Question: Your first impression?
Answer: “It’s pretty cluttered… there’s a lot of things going on at the same time.”
… “But [the site] is comprehensive about all the sports”.
Question: Your first impression?
Answer: “There’s a lot going on … little pictures that, a lot of text, giant ads, like…
I’m trying to decipher what is an ad and what is not… for the first time there’s a lot
on the page” … “hard to tell content from advertising.”
3. Fantasy Products Realization on Home Page:
Participants quickly ascertained that CBSSports offers Fantasy products. The button labeled
“Fantasy” near the top left of the Home page was frequently the first navigation option taken by
participants. Large product ads were also noticed.
Nonetheless, some participants mistakenly thought that the Fantasy News section was the main
Page 1

2. Fantasy Football Commissioner – Usability Study November 2008
area for Fantasy products. In terms of organization, Fantasy, News, Products, etc. do not have a
consistent home base and adds to the mental labor a user must perform in order to distinguish page
organization.
The page fantasy home page above (http://www.sportsline.com/fantasy) has the same labels (e.g.,
Fantasy Football) to describe Fantasy News and Fantasy Products. The centered, vertical menu for
Fantasy product is counter-intuitive
Some ideas: Move vertical, centered menu to the far left
Perhaps, label items in menus differently and specifically (e.g., ff news vs ff products) or integrate
the menus so there would be one menu for fantasy assets.
Page 2

3. Fantasy Football Commissioner – Usability Study November 2008
4. Commissioner Robustness vs. Price:
The Commissioner product was seen by participants as having a lot of features and flexibility. Participants
liked the mobile component and live scoring (something noted as distinguishing features for CBSSports in
comparison to other products). However few, if any participants, deemed these features worth the price
given what they can do for Free elsewhere with other products. True product differentiators should
probably be strongly incorporated (in some fashion) into the Creative for the Commissioner product.
It’s not easy to see the differences between products in the ff products above. For example, it is hard to
tell the differences between the top three ff products, except 14-day free trial for ff commissioner vs free
for ff free. For commissioner product, the current setup is to click on a product to get more details of the
product (can’t see the price—used to be ~ $149), which then have link to free trial. Once in the detailed
page, if viewers want more full details of the product, they need to look around (would find under
features and new features in resources section)
Issues:
1. Hard to differentiate f products
2. $149.95 is viewed as a very high price for ff commissioner
3. Not easy to get to learn all features of ff commissioner. Some effort to find its full feature list. 14-day
trial period may be too short
Some ideas:
1. Provide succinct descriptions to differentiate ff products in image above. Make the product you
want to promote (commissioner) more salient. Include a link to take viewers to product side-by-side
comparison table that allows them to see key differences between products.
2. In the product landing page (http://football.sportsline.com/splash/football/spln/mgmt?
source=MKTG_FFBC08_FSPOETH&refcode=FFBC08PP_FSPOE_TH), provide a link to a page that show
Page 3

4. Fantasy Football Commissioner – Usability Study November 2008
all features of the product instead of having viewers search around in Fantasy Resource section. This
link may be placed before “Start a Trial Now” button
3. For interested viewers who are new to ff commissioner, they will need to try out the product if they
want to know it full potential. Yet, 14-day trial period may be too short for a football season. May
consider lengthening it to 3-4 weeks.
4. $149.95 price is viewed as very high. May need to consider alternative pricing scheme. Perhaps a
small fixed price per owner instead of a single charge to commissioner. The current price will
probably be the biggest hurdle to getting new users.
Setup page:
League Rules vs Scoring System Page
Page 4

5. Fantasy Football Commissioner – Usability Study November 2008
The Leagues Rules and Scoring System pages contain many different settings. Yet, they are organized
differently. Leagues Rule partition items under different tabs, whereas Scoring System (has a lot more
info than being shown above) is divided under different headings. Not consistent way of organizing info.
It could be better if they are organized consistently. The tab organization in League Rules is preferred by
viewers. Would be of minor importance as users can still work with both.
Page 5

6. Fantasy Football Commissioner – Usability Study November 2008
Submit Payment
In Submit Payment, it is not clear what function “Security Question”, “Security Answer”, and “Password”
serve. Perhaps, they are all required to validate accounts, but there is no instruction to inform users. Why
three? They introduce more obstacles for us to get $. The goal should be to make it very easy to user to
submit payment. Should require as little inputs as possible. Those fields should not be there, if possible.
Page 6

7. Fantasy Football Commissioner – Usability Study November 2008
5. Navigation on SPOE page:
When asked to go to specific Fantasy products on the SPOE page, some participants mistakenly clicked
on the hyperlinks above the main product selection area thinking that this was the navigation path to
Fantasy products (e.g., Fantasy Football). Participants were surprised that it was a Fantasy News area for
the selected sport. Competing horizontal navigation bars play a part in this confusion.
Some participants did not immediately understand that navigation to the different Fantasy sports could
be accomplished by clicking on any sport just to the right of the main splash image. Once pointed out, it
was deemed simple. It was a case of once experienced it was a learned behavior.
6. Multiple Horizontal Navigation:
The dual and triple horizontal navigation bars interfere with usability. Multiple horizontal navigation
options compete for attention and potentially confuse or slow down users as they carry out tasks. There
is some redundancy in the navigation as well (Add/Drop) and other actions that can effectively be
provided in one feature navigation area. Better organization of action buttons and navigation is
warranted.
Page 7

8. Fantasy Football Commissioner – Usability Study November 2008
7. Add/Drop Players:
While it was easy for participants to find the link to Add/Drop players, manipulation of the fields to
actually add or remove a player presented a problem. The labeling and orientation of buttons/links
confused some participants.
The “OK” button carries out the operation for both “Add” and “Drop” yet not all participants realized this
to be the case. The “clear” hyperlink adds to the confusion. Several participants thought this was the
way to invoke adding or dropping players and that the “OK” button merely accepted the actions taken on
the page.
Page 8

9. Fantasy Football Commissioner – Usability Study November 2008
While not discussed by participants, the “Add Unlisted Player” and “Edit Service Time” features lower on
the Add/Drop page seem curiously color coded and positioned. Why are these links red – a color choice
normally reserved for error conditions. And, given that the links are below the “OK” button, they largely
go unnoticed despite their coloring.
Player pool is a long list of names, many of which may not mean much to viewers in terms of their past
performance. May want to provide some stats on each viewer when viewer mouse over the player, so
that viewers can quickly see association between players and their performances. Same feature may be
considered for swap player section as well.
Page 9

10. Fantasy Football Commissioner – Usability Study November 2008
8. Initial Fantasy Team Page:
Once logged into the site and a Fantasy team selected, the page presented was not expected by all
participants. When clicking on a Fantasy team, there was an expectation to see the “My Team” page
rather than the general Fantasy News page.
Page participant expected Page participant got
9. Help Text and Hints
The Help text and Hints (light bulbs) provided on most screens were considered to be very helpful by
all participants. During most tasks, participants read and seemed to rely on the hints in order to
proceed or to confirm assumptions about the information or action they were about to take on the
screen.
10. Ease of Setup vs. Competition:
In terms of setting up a Fantasy League, CBSSports may exhibit an edge over the competition in
terms of ease of use. One participant specifically mentioned that the Commissioner product was
much easier to setup in comparison to EA Sports and Yahoo. Much of the success was due to helpful
hints along the way.
Page 10