WebJunction

Hey Sharon, I will be interested in the answers to this as well. I am interested in libraries doing OBE evaluation on any project.

Does anyone know of an outcomes/indicators database for public libraries? I ran across this database for the Casey Foundation's Making Connections project and thought to myself how cool it would be if libraries could pull together their measurements and indicators. The closest thing that I could find to any such project is IBEC's Outcomes Toolkit (for libraries) from the University of Washington, Seattle. No specific iinfo there on evaluating library services to older adults, but maybe something to work with there.

Here's a link to their Sample Surveys page as well.

Message was edited by:
chrystiehill

I feel like I have to wade into the discussion despite lacking much substantive info.

As I'm involved with CT's LSTA grants, I struggle with OBE daily basis. Making 'real' (e.g., properly formatted) outcomes that are also 'meaningful' (e.g., actually measure something of merit) AND that aren't too labor-intensive on the part of the librarian and AND that the subject/target won't freak out about is...hard.

I find myself going back to methods, questions, and material that 'worked' in the past simply because it worked smoothly for everyone and find it difficult to assign or create outcomes retroactively for projects that are a round hole for the square peg.

I've heard from one (unreliable) source that OBE and outcomes will come and go; they simply represent the latest hoop that the Feds want folks to jump through for money. Whether it's goals and objectives or OBE or sigma six, it all waxes and wanes.

I don't know about any agenda and haven't heard anything like that, but maybe that's because I'm a hermit and unsocial to boot.

I also think that, as a soft science measure, OBE could be great IF it is done properly. IMPACT is what it tries to measure; does it succeed? I bemoan the lack of formal, effective training out there (unless I'm missing something, and for Pete's sake let me know if I am). What we need are examples, best practices, tools to make the technique work for us. Heck, a database like Chrystie suggests would be pretty darned useful.

Am I contributing anything here? Other than air my doubts and dream my little dreamy dreams, I doubt it. But forward-thinking on OBE stuff would probably help everyone, yah?

Doug

http://www.tsl.state.tx.us/outcomes/
seems to be Texas' version of the database we're talking about here.

The "resources section (http://www.tsl.state.tx.us/outcomes/resources.html) offers some of the standard sources from United Way, IMS, and the same consultant we've used in CT, Rhea Joyce Rubin.

The 'Examples' (http://www.tsl.state.tx.us/outcomes/examples.php) section has a few examples: a summer reading, a head start partnership, family literacy.

There is an example of outcome based evaluation of a senior deposit collection in Durrance and Fisher (2005). Chapter 12 of this book is titled "Putting the pieces together: an outcome study of the Ypsilanti District Library's Senior Deposit Collection Program" (Walker & Fisher, 2005, p. 150).

Durrance and Fisher (2005) proposed a model for evaluating library programs called the “How Libraries and Librarians Help” (HLLH) outcome model (p. 24). The four parts of the model are 1) the staff input and activities; 2) the specific information about the users which determines how they interact with the service; 3) the output as measured by the number of patrons served by the program; and 4) the outcomes associated with the program. The outcomes include all the ways that the program improved the quality of patron’s lives. Data collection for this outcome model is generally collected through focus groups, interviews, and surveys (Durrance & Fisher, 2005).

Walker & Fisher (2005) used this model to measure the outcome of a deposit collection which was provided by the Ypsilanti District Library for a senior center. They stressed the importance of choosing the right data collection method to use for a particular user population. For example, in the case of elderly people, phone interviews can be problematic due to the hearing problems experienced by many senior citizens. In addition, physical infirmity can limit the amount of time available to interview seniors. Based upon previous studies of senior outreach programs, Walker & Fisher (2005) concluded that personal interviews and focus groups would be the best forms of data collection, since social interaction is very important to the elderly and may encourage participation in the study. A problem with this method of data collection is that it is very time consuming and thus will yield smaller sample sizes.

Walker & Fisher (2005) recommended grouping the outcomes into categories. Categories included “access to reading, social connections, connection to the larger world, sustains sense of identity, life enhancement, and added emotional support” (p.161).

Reference:
Durrance, J. C. & Fisher, K. E. (2005). How libraries and librarians help: A guide to identifying user-centered outcomes. Chicago: American Library Association.

Walker, D. & Fisher, J. (2005). Putting the pieces together: an outcome study of the Ypsilanti District Library's Senior Deposit Collection Program. In J. C. Durrance and K. E. Fisher (Eds.) How libraries and librarians help: A guide to identifying user-centered outcomes (p. 150-178). Chicago: American Library Association.

Hi Doug,

Walker and Fisher (2005) asked the Seniors and the participating staff members open-ended questions about the deposit collection program. For example, one question was "Can you recall a time when the collection had an impact on yourself or another resident?" (p.158). Then, they took this qualitative data and organized it into the categories, such as increasing "social connections". The list of outcomes on p. 167-177 could be modified into survey questions using a Likert scale (as you mentioned) for a homebound program. Some of the outcomes may not apply, since a deposit collection may have more effect on social interaction than a homebound service would. This survey would yield quantitative results and entail much less work than focus groups or individual interviews.

Reference:
Walker, D. & Fisher, J. (2005). Putting the pieces together: an outcome study of the Ypsilanti District Library's Senior Deposit Collection Program. In J. C. Durrance and K. E. Fisher (Eds.) How libraries and librarians help: A guide to identifying user-centered outcomes (p. 150-178). Chicago: American Library Association.

You might also want to take a look at the recent book from Dresang and Holt about outcomes with youth. I know you are looking at seniors, but from the articles that I have read from Dresang and Holt, I thought their programs were well-structured to produce useful outcomes and the lessons and tips they provide would apply to any program developed regardless of audience.

I know they have recently released a book through ALA - tracked it down - http://www.alastore.ala.org/SiteSolution.taf?_sn=catalog2&_pn=product_detail&_op=1995

I handle some of the LSTA funding in Texas and deal with similar issues. One other resource that you might be interested in is the online Outcomes Based Evaluation and Planning course that is being offered by IUPUI through a grant from IMLS. I very briefly looked through the case studies this morning here for one on seniors in case there was something that might help you, but again, if you are looking for resources, it is another starting point.

http://www.shapingoutcomes.org/course/cases/index.htm

I thought it was really interesting that they have started to emphasize the planning aspect of OBE by adding that to the title of the course as I know that is something that is supposed to happen, but really rarely does - people look at the outcomes, total them up, and then do not ask any additional questions about why they either succeeded or did not.

At one IMLS OBE training they discussed creating an online community where they had wanted people to submit OBE logic models so that others could see what people were doing and you could search by program area or audience, so this exact scenario of looking for outcomes would not result in reinventing the wheel. I am not sure what ever happened with that - have you heard anything about that online community/service?

Outcomes-based planning and evaluation (OBPE) has emerged as a best practice in museum and library services - valuable for grant applications or planning any kind of project.

“[url http://www.shapingoutcomes.org ]Shaping Outcomes[/url]” is an online course designed to educate librarians and museum professionals in this concept. Thanks to a grant from the federal Institute of Museum and Library Services ([url http://www.imls.gov ]IMLS[/url]), the [url http://www.slis.indiana.edu/ ] Indiana University School of Library and Information Science, Indianapolis[/url], Indianapolis, offers the [url http://www.shapingoutcomes.org ]Shaping Outcomes[/url] course to library and museum professionals. The online course, accessible from any state and relevant to any library type, will teach participants the OBPE skills needed to design project evaluation suitable for [url http://ct.webjunction.org/do/DisplayContent?id=6159 ]LSTA[/url] and other library grant sources. IMLS scholarship funds are available for participants from libraries and other institutions serving economically challenged populations.

For more information, or to register for the course, visit the website at www.shapingoutcomes.org, or email outcomes@iupui.edu.

~ Doug