Association of Research Libraries

University of Washington Libraries

Library Assessment Conference

Building Effective, Sustainable, Practical Assessment


Seattle, Washington
August 4–6, 2014
#lac14


2014 Conference Workshops

Sunday, August 3

Cube: Unlocking the Value from Your Library's Data
(12:15PM–4:15PM)

Collection Assessment Strategies in Context: Using Data to Inform the Shift from Print to Electronic Collections
(12:15PM–4:15PM)

Library Value: Conceptualizing, Capturing, & Communicating Impact
(12:15PM–4:15PM)

Thursday, August 7

Practical Strategies for Building Assessment Capacity in Libraries
(8:30AM–12:30PM)

Closing the Loop: Evaluating Your Key Scholarly Communication Programs and Services
(8:30AM-12:30PM)

Getting the Message Out: Creating a Multi-Directional Approach to Communicating Assessment
(8:30AM-12:30PM)

Improving Library Service Quality by Using LibQUAL+® Effectively and Strategically
(8:30AM-3:30PM full-day)


Cube: Unlocking the Value from Your Library's Data

Leader: Margie Jantti (University of Wollongong)

Sunday, August 3, 12:15pm–4:15pm

Fee: $80
Register by July 6

Unlocking the value from your library’s data. Learn how the University of Wollongong built the ‘Cubes’ and its efforts to leverage enterprise and third party data platforms for institutional impact.

Now more than ever, libraries need to seek out new methods for maximizing the data that they are invested in and make it more readily accessible and comprehensible to their key stakeholders; the senior leadership teams of the library and that of the institution. But how equipped are we to do this efficiently and effectively, while at the same time show value, relevance and alignment to the institution’s aims?

Using data to demonstrate the worth and value of libraries has, up until recently, been heavily reliant on discrete evaluation projects, the use of surveys and other qualitative assessments. While these activities continue to be worthy, many are questioning as to whether they are sufficient for recognition of the library as an indispensible partner to the university; that through our endeavours there is a demonstrable benefit to the university’s current and aspirational aims. For example, many of us wish to believe that through the utilisation of Library resources and services that there is a positive effect on student performance; the challenge lies with the availability and visibility of evidence to underscore such claims. This is despite the fact that libraries generate and/or collect lots of data.

Library data is typically under-represented in institutional enterprise data and reporting systems. Yet these are the ‘go to’ destinations for the senior planning staff, the University Executive, Deans etc. to gather essential data and information to understand how the institution is performing across its many learning, teaching, research and business functions.

Couple these dilemmas with data from third party sources, which are often large, cumbersome and require not inconsiderable manipulation to produce meaningful reports for the end-user, for example research impact and quality reports. They require time and expertise, thus artificially limiting the ability to scale to demand.

Through this workshop, participants will learn of the University of Wollongong Library’s (UWL) approaches to:

  • Partnership with the University’s Performance Indicator Unit to build new data and reporting models to join library usage data with students’ demographic and academic performance data to test correlations of library use to the student experience and their academic success – the Value Cube.
  • The Marketing Cube, a cube designed to monitor student usage behaviour and the impact of planned communications, promotions and engagement initiatives the consumption of Library owned and subscribed information resources.
  • A new initiative designed to simplify the manipulation of complex, third party research impact datasets delivered at point of need through tablet devices, e.g. the research consultation; and to improve internal efficiencies as demand for services scale.

The Library Cubes and research impact reporting services will be demonstrated to showcase their application and the richness of data and information generated. The workshop is not intended to be technical or require participants to have significant systems knowledge.

Who should attend?

  • This workshop will of be of interest to leaders of libraries exploring new methods for the effective and efficient management and integration of usage data with enterprise systems
  • Quality and assessment librarians seeking models to leverage data management and reporting systems that may be available within their institutions

Workshop Leader Biography:

Margie Jantti, as the Director of Library Services at the University of Wollongong (UOW) Australia, provides leadership and direction for library services spanning six onshore campus locations and guidance for offshore library partnerships in the UAE and Asia. She is an active member of UOW policy and governance committees. Within the Council of Australian University Librarians (CAUL), she chairs the CAUL Quality Assessment Advisory Committee and the CAUL Leadership Institute and is also a member of the CAUL Executive. Key outcomes at UOW Library for the past three years include: significant restructuring of the organisation to extend capacity to support the research community, revitalisation of the physical space to support student learning, the creation of the Library Cube (an enterprise reporting system focussing on the impact of library resource usage and students’ academic performance). She is currently exploring, through third party arrangements, the efficient delivery and export of otherwise complex research impact datasets through handheld devices at point of client need. The repositioning of the Library to further improve alignment with University priorities has resulted in a revitalised approach to organisational performance monitoring and reporting frameworks. Margie holds a Master of Business Administration (distinction) and a BA in Library and Information Science.


Collection Assessment Strategies in Context: Using Data to Inform the Shift from Print to Electronic Collections

Leaders: Anne Osterman (VIVA) and Michael Matos (American University)

Sunday, August 3, 12:15pm–4:15pm

Fee: $80
Register by July 6

As libraries face the continuing challenge of migration to the electronic format while maintaining legacy print collections, collection assessment becomes a valuable tool in helping librarians make smart decisions with their staffing, space, time, and money. What print books should you send to storage or weed? How can you use historical usage data to inform future purchases? Are your collections in line with the research needs of the faculty and curricula they support? How much have print acquisitions dropped over time, and should that change your staffing models? Is the answer to sustainable preservation a broader collaboration with other libraries? This workshop provides an overview of the concepts and techniques of collection assessment with a focus on creating practical and actionable results.

Topics will include:

  • Planning for and designing an effective collection assessment project.
  • Outlining the full range of data available for collection assessment, including usage statistics, expenditure data, the size of the program or population that the collection supports, results from user surveys, and even data about collections that you don’t have but that your users want, shown through interlibrary loan and OpenURL resolver statistics.
  • The importance of reviewing trends over time and putting data in context.
  • The trend of partnership among libraries to create distributed repositories and the role that collection assessment plays in these projects.
  • Tips and techniques for analyzing library data in common software packages.
  • Applying a visual approach to the presentation of results.

Participants are encouraged to bring laptops.

Workshop Leader Biography:

Anne C. Osterman is Deputy Director of the Virtual Library of Virginia (VIVA), the consortium of the nonprofit academic libraries within the Commonwealth of Virginia. Anne has a Master’s in Library Science from the University of North Carolina at Chapel Hill and a Master’s in Statistics from American University, and she is co-author of Electronic Resource Management: Practical Perspectives in a New Technical Services Model. She has worked in the libraries of American University, the University of North Carolina at Charlotte, and Western Piedmont Community College, most recently serving as the Director of Information Delivery Services at American University.

Michael A. Matos is the Business and Economics Librarian and Adjunct Professor of Information Technology at American University in Washington, DC. Michael teaches introductory Information Technology courses for undergraduates and an IT Tools for Managers course for MBA students. His courses include extensive hands-on training in productivity software, including MS Excel. In the Library, Michael has served at the collection manager for the disciplines of business and economics since 2009, and he recently was the chair of the Collection Management Team. Michael has a Master’s in Library Science from Florida State University and has worked in the libraries of Florida State University, Georgetown University, and the Smithsonian Institution.


Library Value: Conceptualizing, Capturing, & Communicating Impact

Leader: Megan Oakleaf (iSchool, Syracuse University)

Sunday, August 3, 12:15pm–4:15pm

Fee: $80
Register by July 6

Need to define, demonstrate, and develop the institutional impact of your library? Overwhelmed by the challenges ahead? Seeking clarity for the way forward?

Intended for librarians engaging an academic library value project, initiative, or research agenda. This half-day workshop will include mini-lectures, discussion, and hands-on activities to engage participants in answering the following questions:

  • What is academic library value, when viewed through an institutional lens?
  • What library services, expertise, and resources have institutional value on your campus?
  • How can you capture evidence of that value?
  • What can you do with evidence of value once you have it? What decisions can you make? What actions can you take?

Workshop Leader Biography:

Megan Oakleaf is an Associate Professor in the iSchool at Syracuse University. She is the author of the Value of Academic Libraries Comprehensive Review and Report and has earned recognition and awards for articles published in top library and information science journals including College and Research Libraries, Portal, Reference and User Services Quarterly, and Journal of Documentation. Megan has presented at numerous conferences, including the American Library Association (ALA), Association of College and Research Libraries (ACRL), Association of American Colleges and Universities (AAC&U), and Association for the Assessment of Learning in Higher Education (AALHE) national conferences, the Association of Research Libraries (ARL) Library Assessment Conferences, the IUPUI Assessment Institute, the Texas A&M Assessment Conference, and EDUCAUSE. Her research areas include outcomes assessment, evidence-based decision making, information literacy instruction, and academic library impact and value.


Practical Strategies for Building Assessment Capacity in Libraries

Leader: Megan Oakleaf (iSchool, Syracuse University) and Steve Hiller (University of Washington)

Thursday, August 7, 8:30am–12:30pm

Fee: $80
Register by July 6

Do you need to build assessment capacity in your library? Tasked with bringing the Library Assessment Conference home? Wondering how to get started with assessment training for your colleagues? Hoping to teach them to initiate, plan, or manage assessment projects?

Intended for those seeking to develop the assessment skills of their colleagues (or those who need to develop assessment abilities on their own), this half-day workshop will include mini-lectures, discussion, and hands-on activities to engage participants in answering these questions:

  1. What does your library need to know to "do assessment"?
  2. Where does assessment fit within the organization?
  3. How can you determine what your colleagues (or you) most need to learn?
  4. What strategies, techniques, and tools help them acquire assessment skills?
  5. How can assessment fears, challenges, and barriers be overcome?

Workshop Leader Biographies:

Megan Oakleaf is an Associate Professor in the iSchool at Syracuse University. She is the author of the Value of Academic Libraries Comprehensive Review and Report and has earned recognition and awards for articles published in top library and information science journals including College and Research Libraries, Portal, Reference and User Services Quarterly, and Journal of Documentation. Megan has presented at numerous conferences, including the American Library Association (ALA), Association of College and Research Libraries (ACRL), Association of American Colleges and Universities (AAC&U), and Association for the Assessment of Learning in Higher Education (AALHE) national conferences, the Association of Research Libraries (ARL) Library Assessment Conferences, the IUPUI Assessment Institute, the Texas A&M Assessment Conference, and EDUCAUSE. Her research areas include outcomes assessment, evidence-based decision making, information literacy instruction, and academic library impact and value.

Steve Hiller has been active in library assessment for nearly 20 years, presenting and publishing widely on a number of assessment-related topics during that period. His current areas of interest are in user needs assessment, strategic planning, library performance metrics, and developing organizational capacity for sustainable assessment. Prior to his current appointment in 2006, Steve was Head of the Sciences Libraries at UW for more than 20 years. Steve also serves as a library evaluator for the Northwest Commission on Colleges and Universities, a regional accrediting body, and since 2004 has been a Visiting Program Officer for the Association of Research Libraries working on library assessment with Martha Kyrillidou. He is one of three founding organizers and a co-chair of the biennial Library Assessment Conference.


Closing the Loop: Evaluating Your Key Scholarly Communication Programs and Services

Leader: Catherine Brown (Principal, Integrative Solutions)

Thursday, August 7, 8:30am–12:30pm

Fee: $80
Register by July 6

How can we maximize the impact of the scholarly communication programs and services we offer? How do we know we've achieved our intended outcomes for our target audiences and what tools do we need to even begin to measure those outcomes? This half day workshop will provide attendees with basic evaluation concepts and frameworks, introducing the vocabulary of evaluation, and suggesting a step-by-step process for creating and implementing one's own logic models or theory of change. Examples will be drawn from both the field of academic and research libraries, and the wider non-profit sector, to help attendees understand the broader concepts behind specific measurement tools. The session time frame will allow for several real-world examples provided by attendees, and a highly participatory format, including group work. The session is appropriate both for those with some experience in program evaluation, and those who are new to it.

Sponsored by the ARL/ACRL Institute on Scholarly Communication

Workshop Leader Biography:

Catherine Brown joined the Chicago Public Library Foundation as its Chief Operating Officer in 2013. She has more than 25 years of experience across the nonprofit sector in areas such as fundraising, grant making, program design and evaluation, strategic planning, and internal and external communications. From 1994 until 2010, Brown held positions of increasing responsibility at the McCormick Foundation, one of Chicagoland’s largest public charities. She launched Integrative Solutions, her consulting practice, in the summer of 2010 and has worked with a variety of organizations. Prior to joining the Foundation, she was the Director of Individual Giving at the Lincoln Park Zoological Society in Chicago. She serves on the board of Friends of Writers.


Getting the Message Out: Creating a Multi-Directional Approach to Communicating Assessment

Leaders: Kathy Ball (McMaster University), Vivian Lewis (McMaster University), and Donna Tolson (University of Virginia)

Thursday, August 7, 8:30–12:30pm

Fee: $80
Register by July 6

Even the most important and interesting assessment work will be meaningless if no one knows about it. Unfortunately, good assessment data sometimes has minimal impact because it is shared in ways that make it hard to use, difficult to understand, or puts people to sleep!

Round out your conference experience by learning effective techniques for presenting assessment data that resonates with your audience. Explore strategies for communicating the results of your work to colleagues, library leadership, institutional stakeholders, and others.

Topics will include:

  • Understanding your audience
  • Choosing visuals with impact
  • Connecting assessment data with strategy
  • The importance of implications

The workshop will include exercises and examples from two universities, with an emphasis on communicating data in meaningful ways to library and institutional leadership.

Workshop Leader Biographies:

Kathy Ball is currently the Director of Assessment and Accountability at McMaster University where she has previously served as Reference Librarian, Business Librarian and Director of the H.G. Thode Library of Science & Engineering. She led the implementation of the Balanced Scorecard at McMaster University Library and continues to have responsibility for its maintenance. She is also a key member of the Library's strategic planning committee. Kathy is a frequent speaker at provincial and national conferences on a variety of assessment topics.

Vivian Lewis is the University Librarian at McMaster University in Hamilton, Ontario. She came to McMaster in 1991 as a government information specialist. In 2003 she became the Associate University Librarian. She holds a master's degree in Library and Information Science from the University of Toronto and is a graduate of the Harvard Institute for Academic Librarians, the Frye Leadership Institute, and the ARL Research Library Leadership Fellows program. Her expertise in strategic planning, library assessment, and information literacy led the successful implementation and design of McMaster's award-winning Mills Learning Commons.

Donna Tolson is the Library Strategist for the University of Virginia Library. In this capacity she works with the Dean and Deputy Librarian to set the Library's strategic direction, leads assessment and data gathering efforts, and incubates new strategic initiatives at the Library. She has extensive experience with the Balanced Scorecard approach to strategic management, and has taught workshops on the Balanced Scorecard at the national library conferences in 2008, 2010, and 2012. In prior roles, she gained operational experience running Clemons Library and led data services in the Scholars' Lab and Geostat Center. Before joining the University Library in 2002, she worked for many years as a demographic analyst for the Weldon Cooper Center for Public Service. She holds an M.A. and B.A. in Sociology from the University of Virginia.


Improving Library Service Quality by Using LibQUAL+® Effectively and Strategically

Leaders: Raynna Bowlby (ARL Consultant), Lisa Hinchliffe (University of Illinois at Urbana-Champaign), and Martha Kyrillidou (Association of Research Libraries)

Thursday, August 7, 8:30am–3:30pm

Fee: $130 Full-day
Register by July 6

LibQUAL+ is a survey tool that solicits users’ opinions of library service quality. The survey has been used by over 1,200 libraries worldwide and has been a foundational element of library assessment initiatives. Increasingly, academic and research libraries recognize that assessment is not an end in itself; rather, assessment is part of the library’s overall management strategy. This workshop will enable staff with responsibilities for planning or assessment to gain a better understanding of their library’s most recent LibQUAL+ survey results, how to use those results effectively to achieve service-quality improvements, and how to integrate this tool into the library’s strategic planning and other organizational performance efforts.

As a result of attending this workshop, participants will:

  • Understand their library’s most recent LibQUAL+ survey results
  • Mine LibQUAL+ data using three different interpretation frameworks
  • Identify specific targets for service-quality improvement
  • Consider ways to develop knowledge and accountability throughout the library in order to improve library service based on the survey results
  • Connect LibQUAL+ to the organization’s strategic plan and other organizational performance systems

Workshop Leader Biographies:

Raynna Bowlby serves as a Consultant to ARL’s Statistics & Service Quality Programs. As a consultant, Raynna develops and delivers consulting and training support for libraries in areas including organizational development, strategic planning, work analysis, human resource allocation and training, implementation of process improvement and new work processes, and assessment initiatives. She was responsible for library assessment activities and was the administrator of Brown’s LibQUAL+® survey in 2005. Raynna has an MLS from Simmons College and an MBA, with a specialization in General Management and Organizational Behavior, from the University of Rhode Island.

Lisa Janicke Hinchliffe is Professor, University Library and Coordinator for Information Literacy Services and Instruction at the University of Illinois at Urbana-Champaign as well as an adjunct faculty member in the university's library school. As Coordinator for Information Literacy Services and Instruction, Lisa oversees information literacy programs emanating from over 20 campus libraries serving more than 25,000 learners/year. A past-president of the Association of College and Research Libraries, ACRL launched the Value of Academic Libraries Initiative during her presidency. With Debra Gilchrist, Lisa is also the lead designer for ACRL's training program for the Standards for Libraries in Higher Education and the IMLS-funded Assessment in Action project. Lisa received her Master of Education and Master of Library and Information Science degrees from the University of Illinois at Urbana-Champaign and a BA in philosophy from the University of St. Thomas in Minnesota.

Martha Kyrillidou, Senior Director of the ARL Statistics and Service Quality Programs, is responsible for supporting strategic change by identifying and developing tools for measuring the organizational performance and effectiveness of academic and research libraries. Martha has worked in focusing library assessment to user defined priorities and nurturing a community of practice on demonstrating the value of the library. She has led the development of the StatsQUAL® suite of services that includes assessment tools such as ARL Statistics®, LibQUAL+®, ClimateQUAL®, and MINES for Libraries®. Martha has an MLS and an MEd with specialization in Evaluation and Measurement from Kent State University; and a PhD in library and information science from the University of Illinois at Urbana-Champaign.