5. YOUR PEER LIBRARIES
• Service area population or enrollment
• Total operating expenditures
• Number of outlets
• Ethnicity
• Poverty level
• Education level
• Average income
6.
7.
8. Public Library Benchmark Sources
• LJ Index of Public Library Service
• Hennen’s American Public Library Ratings
(HAPLR)
• Public Library Data Service (PLDS)
• Institute of Museum and Library Services (IMLS)
• Urban Libraries Council Technology Benchmarks
• International City/County Mgmt Assn. (ICMA)
• US IMPACT Survey (IMLS, Gates, U-WA)
9. Academic & School Library
Benchmark Sources
Academic School
• National Center for Education • School Libraries Count!
Statistics (NCES) • State and consortium reports
• State and consortium reports • State guidelines and standards
10.
11. PER CAPITA
Visits
Circulation
Program
Attendance
Public Internet
Use
12.
13. HENNEN WEIGHTS
Expenditure Per Capita 3
Cost Per Circ 3
Visits Per Capita 3
Materials % of Budget 2
Collection Turnover 2
Circ Per FTE Staff Hour 2
Circ Per Capita 2
Circ Per Hour 2
Reference Per Capita 2
Materials $ Per Capita 2
FTE staff Per 1000 Pop 2
Periodicals Per 1000 Pop 1
Volumes Per Capita 1
Visits Per Hour 1
Circ Per Visit 1
17. Develop a library strategic plan that achieves a dynamic school
library program and includes a mission and vision, measurable
goals, operational management procedures, instructional
strategies and an evaluation component.
18.
19.
20. LIBRARY POP SCORE %ILE
Hotwings Mem PL 25K 559 58%
Beefstock Mem PL 25K 464 43%
Bacon Mem PL 25K 444 40%
21. LIBRARY POP SCORE %ILE
Hotwings Mem PL 25K 559 58%
Beefstock Mem PL 25K 464 43%
Bacon Mem PL 25K 444 40%
2 week loan period, 2 renewals
= 3 circs for 6 weeks
3 week loan period, 1 renewal
= 2 circs for 6 weeks
22. PUBLIC LIBRARY SUCCESS STORIES
Lower poverty
Higher
education
$$$$
Large collection
More outlets
23.
24. Large Suburban
County Library 30% fewer staff
< 2 books per
capita
Less expenditures
per capita
Highest program
attendance
25. Urban Library
Peers better
funded
Lowest education
and income
demographic
Highest diversity
demographic
Peer had more
visible marketing
How do you know if your library is good, or great? How can you measure effectiveness? What does success look like? One tool for measuring performing is the process of benchmarking. Today, you’ll get an introductory look to comparison tools available for academic and public libraries. Comparisons are just one way to assess community and library performance, in addition to gathering customer feedback, analyzing long-term data, and other measurement activities. How many of you look at Hennen or LJ ratings? Do ICMA or IMLS benchmarks? Other comparisons? Benchmarking tends to focus on what you have done in the past, and compares it to what others have done in the past. It doesn’t indicate quality, or a lot about your users. It measures outputs, such as children’s program attendance, but doesn’t fully measure outcomes, such as the impact of Summer Reading Program attendance on reading scores or of bibliographic instruction on research paper grades. So why do it?
This is why. When I became the director of my library a few years ago, my initial staff interviews showed a lot of exhaustion. 2 FTEs were cut the year before at a time when library use was at an all time high. Benchmarking with peers allowed me to show elected officials our staffing compared to other libraries – very low – and resulted in some small staffing gains. That is an example of what benchmarking can do for you. If you are understaffed, your first reaction is that you don’t have time for benchmarking. But it isn’t a major project. Keep it simple, stick to comparing key indicators that your funders and stakeholders and decision-makes are interested in. For example: our library is not obsessed with reference question counts…we help people in many different ways…. BENEFITS: Find peer libraries for deeper comparisons and sharing of best practices Objective data for advocacy, grants – elected officials respond to comparisons Accountability for public and private dollars given to the library Set targets for improvement Marketing ---Particularly large impact when library board or elected officials don’t know excellence; are complacent or self-satisfied; or library has poor self-image
What can benchmarking do for you? Here’s another example. One library we worked with was doing a strategic plan and had a vague impression that its bookmobile was underperforming. They had entrenched outreach staff, no changes or excitement, little data on performance. Found peers with bkms and benchmarked and collected data for the first time ever. The comparison found their bookmobile was underperforming; low circ, high cost per circ compared to peers. Their new plan addressed the issues. Then they looked at community data and reports to see if they could put send the bookmobile to areas of growth, or special needs. This is how numbers can contribute to improvement in your operations.
Finding peer libraries is a critical step in benchmarking and evaluation. They provide opportunities for sharing, learning, exchanges of best practices. Schools and academics might want to look at enrollment instead of population, some kind of budget figure, and number of outlets is important if you are looking at the entire campus or school system, otherwise you should look to other institutional markers of your context. The libraries should ideally be within about 20% of your stats in these areas to be similar. Library of Norman, Arkansas, not operating but on National Register of Historic Places, originally part of the old waterworks building. Shed is in Williamsburg, Brooklyn and phone booth in Belbroughton, UK
There is a compare academic libraries on the NCES website. The academics can find peers using this feature. Academics, schools and specials may want to look at peers their institution already identifies as peers. School libraries may use surveys such as AASL’s School Libraries Count and match up against schools or districts with similar enrollments, budgets, number of schools.
Benchmarking: Lots of data out there. http://www.ala.org/aasl/researchandstatistics/slcsurvey/slcsurvey
Community Benchmarks and Comparisons ICMA – International City/County Management Association is big on benchmarking government services – “Center for Performance Measurement.” They benchmark libraries, too. Digitally Inclusive Communities ICMA partnering with Gates Foundation on library public computing data. To help community leaders make strategic decisions about technology investments, they are working with IMLS and the University of Washington Information School to develop tools to help communities assess their current capabilities, work across institutional boundaries and create plans, using a common set of principles, that would help business, anchor institutions (schools, hospitals, libraries), public safety and cultural institutions work together to fully and effectively realize the value of a digitally inclusive community. Studies show early tech adopter libraries – more use, $. Your local government staff may be active and participate in these benchmarks. Good source of performance data for local government. Measure financials, public works expenditures, capital improvements, etc. They collect various kinds of data including results of citizen feedback surveys that can be used for comparison. Can be pricey to belong – partner with your municipality.
The LJ Index, identifies “America’s Star Libraries.” Authored by librarian/statistician Ray Lyons and Keith Curry Lance, groups libraries by total operating expenditures instead of population. It uses these 4 per capita outputs from IMLS data weighed equally : circulation, program attendance, visits, and public computer use. Critiques went back and forth between authors of the 2 systems. What do you think of your library being ranked by only four variables? Which system is best? The LJ authors state emphatically: “this should be one among several sources of information…decide how to incorporate it into a more comprehensive assessment process.” Or, as I said before, use them to start you on the benchmarking path. The ratings will prompt questions and concerns and drive research that may help improve library performance.
Hennen and LJ trends show NY and OH having many highly rated libraries. Here is the LJ map from the Nov. 1, 2011 article by Keieth Curry Lance and Ray Lyons. Sparks the question- what are we doing wrong in PA? We have relatively high state funding for public libraries which is eroding annually, and poor local funding, you can see the effect that has on us.
Hennen’s American Public Library Ratings are administered by Thomas Hennen, a Wisconsin library director who first published his formulas and ratings in 1999 in American Libraries . He was an independent researcher looking for some way to compare and benchmark libraries in a meaningful manner. Hennen uses the data you submit in your state annual report, that is forwarded to the Federal Government. It is now received and processed by the federal Institute for Museum and Library Services (IMLS) and the data is published, unfortunately, 1-2 years after you submit it, much like census data takes time to be processed. Hennen groups libraries by population served and then looks at 15 inputs, such as funding and staffing, and outputs such as circulation and visits per capita that have varying weights in the formula, producing a final numerical score. You can find the scores of libraries by state on Hennen’s website. He also, for a fee, provides a customized report comparing your libraries to others with some additional data not found on the website.
Hennen and LJ scores group libraries into large cadres which limit their value. The rating systems both have generated much debate. High-ranked libraries celebrate their success and low-ranking ones agonize over the reasons behind their numbers. American Libraries ’ observed a few years ago that they “have never fielded as many media inquiries over an AL article as the Hennen ratings.” So what should we make of these systems? We’ll take a closer look at them, but our advice is: despite their shortcomings, use them to start you on the benchmarking path. The results will prompt questions and concerns and drive research that may help improve library performance.
Public Library Association Statistical Report The Public Library Association conducts an annual survey and publishes results roughly a year earlier than IMLS for more timely comparisons. Published annually, the PLDS Statistical Report collects information on finances, library resources, annual use figures and technology from more than 800 US/Canada public libraries who voluntarily participate. BONUS: Each year there is a special, different add-on survey highlighting one service area like children’s programs, salaries, etc. It costs $135.00 for the print version, but for $200/year you can have online access which is much, much, more useful. Online database users can view all tables, export them into Excel, compose graphs, etc. “ Counting Opinions” just took over from U-IL, new look and functions.
Academics and Schools can use NCES website. The Association of Research Libraries uses a service called LibQUAL+ for collecting and interpreting user feedback and benchmarking with peers. Fee of $3,200.
OH Dept. of Education had nice document for school libraries about benchmarks but no data connected to it. Your benchmarks are complicated because you must work with educators to first define college or school learning outcomes, and then benchmark the library activities that affect those outcomes.
If Hennen and LJ have whetted your appetite for more comparison data, turn to IMLS, source of their data. Libraries annually report their output measures to their state libraries, and they pass the information on to the federal Institute of Museum and Library Services (IMLS). After processing the data, it is posted online for anyone to use. The most recent data available is from fiscal year 2009. Some states, like NJ, have more recent info online for their own states. Better yet, IMLS has its own comparison tool online which provides a more detailed and useful benchmark than the 2 previous ratings.
Here’s an article from a 2007 posting on a blog, “The Well Dressed Librarian.” This is the part where I tell you DON”T PANIC about your numbers. Comparing stats with peer libraries allows you to flag any extreme performance, and then you have to dig deeper to get the reason behind the numbers.
Case study of how Hennen sparks question: Here are 3 public libraries in a midwestern state that serve populations of 25,000 to 49,999. Bacon Memorial is really puzzled as to their low scores. They dug into the statistics. Hotwings had much higher circulation but a similar materials budget and collection size. What was causing the high circ? Better books? Large building and shelf capacity? What can it be? The answer was: different loan periods. Bacon could adjust circ period or – lower the status of circ as an important indicator.
These qualities of LJ and Hennen high ranked libraries are related – not necessarily a cause and effect, but many “winners” have 3 or more qualities. If you put a well-managed library in a new setting what different outcomes would you see? If you lack these qualities – is it possible to have high scores? And what score or ranking indicates success? We have no accepted benchmarks except the 12% materials and 70% personnel guidelines. No answers yet.
Hennen, LJ, and other benchmarks don’t, and can’t, measure customer service, leadership, management, collection quality and other factors that play a role in a library’s community success. The proof of that is that winners of the Bill & Melinda Gates Foundation and Library Journal “ The Best Small Library in America ” often have ratings in the bottom half. MILANOF-SCHOCK LIBRARY MOUNT JOY 10 K 454 41% So why look at them? It’s a rough “place-marker” as where a library stands among others, Lyons - “I think of them as a kind of filter…we do it as an advocacy tool and predictor of potential or additional success.” Leads to exploration of values, additional data.
Large suburban county library with 18 outlets. Culturally diverse, mixed income and education and good Hennen and LJ scores. Bond approved for capital projects – branch construction and renovation. But the economic downturn threatened operating income to staff the new or enlarged branches. How could they build their case for more funding in a very competitive county government budget process? IMLS data showed: 30% less staff than peers. Fewer than 2 books per capita, lower than peers, which affect circ. Expenditures per capita were less than peer libraries. Didn’t realize their program attendance was extraordinary and something to brag about. They had many famous author visits which put them over the top. Numbers allowed them to: cheerleaders for staff productivity; have neutral objectives; presented funding comparison to county commissioners; trustees had some 1-on-1 meetings with elected officials and released data to the press. Commissioners funded additional staff positions. Library commentary became more objective because it was backed by data and true descriptions of other libraries.
Urban library case study: LJ and Hennen scores in top quarter, but not “stars.” In addition to these attributes: Surprises: Program attendance for library outstanding and was not promoted enough; one of the top in the country and they didn’t realize or celebrate it. Library had fewer service hours than peers, which reduces circulation and other outputs. Helped to light a fire under a complacent board.
There are other things to measure such as economic impact, ROI, customer satisfaction, that can help explain/expand benchmarks and define goals. Combine with other processes: High staff turnover – benchmarking showed low personnel expenditures, but exit interviews needed to determine extent of bad work environment or management. Balanced scorecard – how can we improve? Decide what the goal is – to get in top half, above mean/median of peers?
When you produce a report, don’t snow them with tons of data.
