Findings of the eXtension Ask an Expert Task Force, June 2009.

1. eXtension Ask an Expert Task Force:
Findings and Decisions
Final Report
June 3, 2009
Commitee Membership:
Kevin Gamble, Chair
Rick Durham
Mike Lambur
Larry Lippke
Ben MacNeill
Terry Meisenbach
Beth Raney
Craig Wood

2. Ask an Expert Task Force Report
Introduction
Since its beginnings, eXtension has included an Ask an Expert (AaE) function as part of its portfolio of tools
available for public information gathering. Concern has been expressed as to the success of AaE over time
and its sustainability for the future. eXtension has focused resources on AaE and as a part of that focus has
determined a need to study where AaE is today and where it can go in the future. With the addition of the
Ask eXtension widget as a supplement to AaE, especially at the local/county level, AaE from the national
perspective has garnered even greater attention. Staffing of the AaE system at the national level has been
formally limited to launched Communities of Practice (CoP). At the same time, however, questions
requiring expertise outside the launched CoPs also come to eXtension. The public knows that the
Cooperative Extension System (CES) has expertise that extends far beyond that of the publicly launched
CoPs. The growth in questions outside of staffed areas of expertise has created further questions regarding
the sustainability of AaE in the long term.
This Task Force was charged with reviewing the current status of eXtension's AaE system, and to provide
direction for its future.
Objectives of project
• Assess the Cooperative Extension System's ability to operate an AaE system.
• Assess what CES faculty and educators think are acceptable levels of performance: time to
respond, depth of answers, etc.
• Assess the current performance (human) to date: timeliness, capacity, correctness, coherence,
completeness, etc.
• Determine clientele expectations for an AaE system: timeliness, capacity, correctness, coherence,
completeness.
• Determine the level of staffing and resources required to operate a fully functioning AaE system.
• Prepare report to the Governing Committee for consideration.

3. How is the public best served?
Cooperative Extension's "stock and trade" for nearly a century has been via a community presence;
Extension educators and agents located within a local community as an educational source for new practices
in agriculture, family sciences, youth development, and community development. As more and more
Americans turn to the Internet for information, they also find communities... people with similar interests
who formally and informally band together in a social context. It stands to reason that Cooperative
Extension should consider this online community as an equally important and relevant public with as much
or more value as a public constituent group. In the traditional model, clients come to the Extension office,
ask a question, and are answered by an expert supplemented with publications, fact sheets, and relevant
additional educational resources. The new model via eXtension can and should operate with the same robust
service and educational model.
Americans have grown more and more sophisticated in their Internet use habits and are also more
demanding as they seek information from myriad sources online. Previous research and metrics from
eXtension in general show that 83% of clients come to eXtension from search engines (Google in
particular), and that their visits are to specific pages deep within the site. They look at an average 2.1 pages
and spend an average of 1:26 minutes per visit. General data on Internet use show that the public is
interacting with the eXtension Web site much as they do other "destination" sites that are content centric.
Increasingly they are visiting a few large sites, and staying longer. The total number of sites they are visiting
continues to steadly decrease. According to Steve Reubel citing March 2009 data from Nielsen Online 1:
After years of erosion it now it appears the destination web era is drawing to a close. This is a
trend that digital thinkers like Om Malik have long noted. In fact, the numbers prove it.
In March the average American visited a mere 111 domains and 2,500 web pages, according
to Nielsen Online. What’s worse, our attention across these pages is highly fragmented. The
average time spent per page is a mere 56 seconds. Portals and search engines dominate,
capturing approximately 12 of the 75 hours spent online in March. However, people-powered
sites like Wikipedia, Facebook and YouTube are not far behind, snagging nearly 4.5 hours of
our monthly attention.

4. Current Conditions: a Snapshot
Numbers served: Since its inception, the AaE system has received 12,205 questions. In May of 2009 it
received 1,435. So far, in 2009 the number of questions received has been increasing at 29% per month.
Figure 1 depicts the number of questions received and answered over the past year. It also shows the pattern
as to when experts answer questions, with a significant drop-off in weekend activity and a flurry of
answering on Mondays. The average time to respond to a question is 70 hours (3 days) with a median
response time of 39 hours (1.6 days). Questions are "retired" after being in the AaE system without being
answered after 10 days. Recipients are sent an apology and asked to contact their local Cooperative
Extension offices. On average, 3.5% of the questions received are going unanswered either for non-
performance, or because there are no experts in a subject matter available or willing to answer. In addition,
11.5% of the questions received are for subject matter areas where no current CoP exists to generate
responses.
Figure 1: Questions asked and answered
How the public comes to Ask an Expert: Overall 83% of the people visiting www.extension.org enter
directly from one of the major search engines. This is reflected in where people come from when coming to
the Ask an Expert form on the eXtension web site. Eighty-six percent of the people coming to the AaE form
on the eXtension site come from a content page deep within the site. They initially arrive via search, don't
find the specific information they are seeking, and redirect to the AaE to begin asking their questions.

5. Who is being served: The audience being served is not particularly diverse or representative of the general
population (Figure 2.) A quick glance tells us that the majority of users are female (63%), white (92%), well
educated (36% with bachelors, and another 39% with advanced degrees), older (66% > than age 50), and
financially better-off (72% reporting incomes > than the median family income).
Figure 2. Demographics of users of the AaE system

6. Nonetheless, the AaE system is reaching new clientele (Figure 3). Twenty-nine percent of the people asking
questions had never heard of Cooperative Extension prior to using the AaE system. Another 27% were
aware of Cooperative Extension but had never accessed it previously.
Figure 3: Previous Awareness/Use of Cooperative Extension
Who is Answering Questions: The number of Extension faculty and educators answering questions is what
you would expect from this sort of system. It is a classic Pareto Distribution (Figure 4) where 20% of the
individuals in the AaE system are responding to 80% of the questions. Of the 656 Extension professionals
who have answered questions, 131 of those individuals are providing the bulk of the effort. Less than 1% of
the estimated 15,000 Extension professionals are currently active in the AaE system.
Figure 4: Who's answering questions?

7. What Was Studied
Four studies were conducted as a part of this review:
Directors and Administrators: A sample of 15 Directors and Administrators were interviewed in face-to-
face meetings and by phone. The questions used in the interviews were validated and reviewed at a meeting
of the eXtension Governing Committee in March of 2009.
Public survey: A complexity-based approach was used to examine the various factors that affect users’
perceptions of the Ask An Expert service 2. A survey was conducted of the 6,557 unique users of the AaE
system. The Cognitive Edge SenseMaker™ software was used for the analysis: SenseMaker™ contains a
range of analytical and interrogation tools that allow both recall and interpretation of sense-making items. 3 It
makes extensive use of visualizations to allow complex patterns and exceptions to be discovered. The entire
population of question askers were sent an email survey. Subsequent follow-up surveys were sent to non-
responders. Five hundred and four individuals completed the survey for a response rate of 7.7%. This was a
qualitative research design, and 200 responses were needed at the lower range to begin to see meaningful
patterns in the data4. If sampling were to be done with this population a sample size of 376 would be
required for a probability of .05.5 The research methodology used was a pre-hypothesis narrative-based
approach6.
Faculty/Educator Focus Groups: Five focus groups were conducted using the Adobe Connect Pro web
conferencing system with selected CoPs, highly active AaE responders, and a group of less-than-enthusiastic
CoP members, to gain insight to each of the studied questions (see What Was Observed). The polling
features of Connect were used to determine the extent to which the questions identified the significance of
the issues in the minds of the participants, and to collect qualitative information through discussion.
Log Analysis: A review of logs was conducted, and other data analyzed to get a clear picture of how the
public was interacting with the www.extension.org site. This was to answer the questions as to whether the
public was able to find the page and easily enter their questions. Additionally, an analysis of the internal logs
was conducted to determine the responders' use of the AaE system as they interacted with it to provide
answers.
What Was Observed
Following are the major findings from the four studies:
The public believes that the answers provided by the AaE system are science-based (figure 5). This is a
remarkable finding when you consider that 56% of the respondents had no previous experience working
with Cooperative Extension. In addition, they believed the answers were from a trusted and unbiased source
(figure 6). These two strong preferences would indicate that the Cooperative Extension System is perceived
as being true to its mission even with people who had little or no previous familiarity with the system.

8. Figure 5: Public perception as to the science-based nature of AaE answers.
In addition, they trusted the answers that were provided and believed them to be from an unbiased source.
Figure 6: Public perception of Answers from eXtension as Not trusted to Trusted
What the clientele said:
“I am glad to have my other research confirmed however by a trustworthy source.”
“I got a reply that, although not what I wanted to hear, I trusted”
“I had a very prompt reply, with very accurate information plus direction to a paper.”
"A Google search provided an answer but I needed someone with authority and experience to
corroborate/validate that answer so I could do my calculations."
Clientele are primarily asking questions that they consider to be of personal and economic importance. The
AaE system is not being used to ask trivial questions or to inquire about things considered a curiosity (in
stark contract to Yahoo! Answers: answers.yahoo.com). The questions were also skewed toward being of
economic importance as well. When the responses were filtered to eliminate consumer horticulture, then the
economic importance of the questions moved even further towards being of significant economic need.

9. Figure 7: Public description of their question as to importance.
Figure 8: Public description of their question as to economic need.
What the clientele said:
“I was worried about food safety for my family if I planted my vegetable garden as planned.”
“I wanted to know about safety concerns when cleaning up a rat/mouse infested house as it
related to hanta virus.”
"Squirrels have been eating my house. We tried several suggestions, and the situation is
improving.”
“He gave practical information about how to decide what is best for my family, as opposed to
necessarily doing what's best for the tree. Very helpful."
The one variable that impacts the utility of the answers provided is the rapidity with which an answer is
provided. After two days, people indicated that the usefulness of the answer in helping them to solve their
problem was diminished (Figures 9 and 10). If the Cooperative Extension System desires to make a
difference in people's lives, to provide answers that are timely, and also of practical use, then it simply must
respond more quickly.

10. Figures 9 and 10: Public's Perception of Utility by Response Time (What role did the answer play in solving the
problem?)
What the clientele said:
“We were supplied with the information we wanted in an expedient, competent and
professional manner (which is so unbelievably rare these days)."
“I do know it took four days for me to get an answer and I thought at the time if the answer
was needed was urgent, I would have been in trouble.”
“To be frank, it’s kinda hit or miss and I’m not sold on the 'quick' response promised.”
Timeliness of the response comes into play once again when measuring the public's satisfaction with the
service. Satisfaction was at 80% when they received an answer within a day. Their satisfaction begins to
decrease after two days, and drops to just 68% if the answer isn't provided within a week. Satisfaction falls
dramatically when it takes longer than a week for the experts to respond.

11. Figure 11: Clientele satisfaction and timeliness of response
What the clientele said:
“I didn’t expect an answer so quickly. In the matter of an hour or so..the response was very
helpful. I am thrilled with this service”
“I will not keep a client waiting a week for a response. My suggestion would be that if you are
going to provide this question and answer service, you need to respond in a timely fashion
(within 2 days)”
These findings are in stark contrast to what faculty and educators determined were realistic response times
given other demands of their appointments. From the focus group findings:
Experts should not be expected to respond to questions during weekends and holidays. Such
time should be subtracted from the time triggering escalation reports. "Office hours" should
be published so questioners do not have false expectations on when to expect a response. But,
during such office hours, 48 hours seems an appropriate amount of time.
Exacerbating this is the public's desire for answers that are deeper and more detailed than the experts are
providing. They believed that the answers provided leaned toward being too simple, lacked context, and
specificity. Providing more detailed answers is likely to lead to a larger commitment of time in providing
more in-depth and researched answers. Figure 12 shows the tendency for the responses to be considered too
simplistic with the preponderance of responses falling below the mid-point on the Z scale.

12. Figure 12: Visualization of quality of answer. Red: science-based; Green: trusted; Black: simple/complex
What the clientele said:
“I did not really get an answer...I did ask a fairly long, drawn out and complicated question. I
was rather disappointed in the short answer.”
"The answer was well-written and included even more information that I had expected.”
“The ‘Expert’ forwarded an article but it did not address the specifics of my question. I felt
rather disappointed and ended up doing more research on my own... even re-emailed the
‘Expert’ a 2nd time to clarify but she did not return my email.”
“As a person who has been involved in Dairy business, I knew what we should do but I wanted
to get a confirmation on what we are going to do, but unfortunately what I got was very simple
concepts."
Faculty and Staff Focus Group:
Tenure track faculty observed that answering the public's questions is not considered scholarly work, and is
not rewarded. They acknowledged that it is an important component of the Extension mission, but that it is
best valued internally when carried out by individuals in non-tenure leading positions. One strategy is to
provide the opportunity for responding to AaE questions at an appropriate organizational level where it will
be adequately acknowledged and rewarded. This will differ by state Cooperative Extension Systems.

13. Faculty, educators and staff also indicated that they needed additional training opportunities to improve
awareness and to enhance performance of the AaE.
Administrative Interviews
Administrators' perception of faculty and staff involvement in working with AaE was very positive. There
were few concerns voiced regarding faculty and staff working or serving clientele outside of their assigned
geographic responsibilities. It was often cited as a positive aspect of eXtension, and an area that was more
critical than ever given the current budget situation. Virtually every administrator mentioned eXtension and
AaE as a key strategy for dealing with expertise gaps and personnel shortages.
Administrators observed that AaE is not considered scholarly work, but saw value in reaching clientele
through this technology. There was a general acknowledgment that involvement of faculty and staff in AaE
varies widely between states. Perception and awareness seem to depend on the state's administrative views
and promotion of AaE and eXtension in general. In states where the administration encourages and supports
involvement, AaE is perceived as positive. In those states where encouragement does not occur, there is
little engagement and faculty and staff are reluctant to participate.
What Needs to Happen
As noted earlier, Americans want engagement with online communities. They also want a more interactive
experience in that engagement process. To be a viable and valuable resource, eXtension must pay attention
to the needs of clientele and work with Cooperative Extension leaders to prepare for the current and future
demands of AaE.
There is a need for more people at the appropriate level within and outside Cooperative Extension to
answer questions. CoP's primary focus needs to be on building their communities, recruiting new people to
answer questions, and providing professional development opportunities to better prepare experts to answer
questions in an appropriate and timely manner. With the potential to receive questions from many more
people (orders of magnitude larger), Cooperative Extension is going to need to grow its capacity to serve
ever increasing audience numbers. Far more than the current group of 131 experts will be needed if the CES
is to even begin to service the public's demand for this type of system. Indeed, the system is struggling to
perform given the moderate number of questions received currently, and this will only be alleviated by
growing the number of active experts responding to questions.
Regardless of the information and educational content provided on the public website from launched CoPs,
when engaging with the site the public asks questions that fall outside the expertise of the current CoPs.
Clientele know that the CES has expertise outside of the areas of the launched CoPs and will ask their
questions. Indeed, 11.5% of the questions received are in areas where eXtension is not currently staffed to
respond. It is not acceptable to not respond to these questions. Each state needs at least one designated
Question Wrangler to handle the assignment of questions emanating from their state and to assist in
locating experts to answer incoming questions in areas where there are no current CoPs. Additional experts
in each state need to be identified. Working through Institutional Teams each state should develop a

14. strategy to have their staff flesh-out their profiles in the eXtension directory application found at
people.extension.org (People) so that they can more easily be identified to assist in responding to questions.
Performance metrics must match clientele expectations. The results are clear; customer satisfaction and
the utility of the answers provided drop significantly when it takes longer than two days to receive a
response. To speed responses, the timing of non-performance escalation reports to CoP leaders needs to be
reduced from the current 48 hours to 24. Unanswered questions need to be reassigned after 24 hours.
Multiple people need to be alerted to questions in their areas of expertise to mitigate the problems caused by
the current system of assigning each question to an individual creating a single-point-of-failure. The soon to
be released version of the AaE system that allows for multiple responses to a question should help to
alleviate some of the issues of experts not always being available.
Create AaE only CoPs. Using the profiling capabilities of for CES staff found in People, work to
proactively form new CoPs with an initial single focus of answering clientele questions. Provide tailored
professional development opportunities to assist them in getting organized, and in the mechanics of using
the AaE system. Assist in helping to recruiting new members.
Continue to advance the use of widgets to involve more local staff with AaE. It has been noticed that
many local educators who get involved with answering questions through their own county widgets also get
involved in answering questions coming in from the www.extension.org site. Provide additional professional
development opportunities on using the widgets. Work with state level IT staff on the use of the AaE
system, and on installation and maintenance of widgets.
Provide additional AaE training and professional development to raise expertise across the entire CES.
This needs to include not only the mechanics of using the AaE system, but also proper etiquette, answering
strategies, and awareness concerning clientele expectations.
Continue to focus on online engagement. To engage or not to engage with online audiences is not a
choice. We must engage-- our clientele expect it. AaE is the place within eXtension where this is done, and
it needs to be done well. It can no longer be viewed as an add-on, but needs to be a top priority. Even
though faculty get less credit…this is an opportunity for the CES to be relevant in ways heretofore
unimagined.
Eighty-three percent of the people coming to the AaE system are arriving from an Internet search engine. It
is the content of the www.extension.org site which brings them. One concern is the lack of diversity in the
clientele using AaE. Consideration needs to be given to creating CoPs that will appeal to a more diverse
and younger demographic.
Recognize that AaE is reaching new, online audiences; and this needs to be promoted. The AaE
represents a strategic opportunity for the CES that should not be squandered.
Vision
From the beginning, one of eXtension's promises has been to reach new audiences and to bring new people
to the CES. Through the eXtension brand study, through the ECOP Marketing Task Force brand
development process, and through this AaE task force, it has been learned that Americans who don't know

15. Cooperative Extension but who ultimately interact with it, are pleased with their experience. They believe
Cooperative Extension is reliable and trustworthy. They believe the land-grant university connection adds
credibility and trustworthiness to the information they find. And, in the case of the studies done with this
effort, many of those, over half, who are currently using the AaE system are unfamiliar with the Cooperative
Extension System.
So, where does Cooperative Extension go from here? From these studies it is known that people value and
demand Ask an Expert type services. Therefore it is important to meet this challenge as an opportunity for
further engagement with current audiences, and new engagement with people who have yet to discover
Cooperative Extension. There is an opportunity to offer a service on the Internet that is both unique and in
demand: scientifically-based knowledge from trusted sources. This is a unique niche, and and opportunity to
be seized.
As evidenced in the data above, demand both in volume and in time, can quickly dwarf Cooperative
Extension's ability to respond to Ask an Expert questions. Cooperative Extension must look to alternative
strategies including broadening the expertise base to answer questions. It must look at the use of trained
volunteers; incorporating eXtension's outstanding professional development capabilities and subject matter
specialists to provide training. It must take advantage of social networking strategies available now with
Web 2.0 to bring experts to a more granular level engaging at the community level where Cooperative
Extension has done its best work for nearly a century. It must also "grow" its expertise bank through a
mentor/protege approach where current experts share their knowledge one-on-one to bring new and younger
faculty and staff into this effort. The "graying" of America will not stop at the doors to America's land-grant
universities, and thus Cooperative Extension must look toward empowering and enabling new faculty and
staff to build these experts as soon as possible.
It has long been the goal of eXtension to put Cooperative Extension into the homes of more and more
Americans. The AeE system truly meets that goal. It is an interactive approach; an engaging approach; a
value-added approach. It takes full advantage of CES' reputation for reliability, credibility, and for
providing research-based expertise in a customer-focused manner. That trust, coupled with a responsive,
well-trained bank of experts will serve CES well into the future.
References
1. Reubel, Steve. "The End of the Destination Web Era." Micropersuasion,
http://www.micropersuasion.com/2009/05/the-end-of-the-destination-web-era.html, May 3, 2009.
2. Elliott, Jane. Using Narrative in Social Research: Qualitative and Quantitative Approaches.
Thousand Oaks, California: Sage Publications, 2005.
3. Sense-Maker Suite version 3, http://www.sensemaker-suite.com
4. Lorenz, M. O. "Methods of measuring the concentration of wealth." Publications of the American
Statistical Association. 9 (1905) : 209–219.
5. Chelveldave, Michael. (March 24, 2009). Personal conversation.
6. Israel, Glenn. Determining Sample Size: PEOD6. IFAS, University of Florida, 1992.

16. 7. Snowden, David and Mary Boone. Leader's Framework for Decision Making. Harvard Business
Review (November 2007): 10pp.