March 15th, 2012 Brian Herzog
Presenter was a branch manager in the DC public library system. He was given six weeks in May-June to pull together a secret shopper program and run it over the course of six weeks during June-July.
Goals of secret shopping
- evaluate patron experience, for different types of patrons (using different types of shoppers)
- evaluate how well staff was trained on a particular product or service
- evaluate library's space, traffic flow, signage, etc
- evaluate collection and merchandising
- just get fresh eyes on the library
However: A Goals Caveat
- are you doing this to really find out something you don't already know?
- are you doing this to find proof of something you already believe to be the case?
- if problems are identified, are you in any position (financially, staffing, politically), to do anything about it?
DC used volunteers (teens, adult volunteers, and Friends of the Library), and developed their own tools; retail secret shopping ~$25-$35/shopping trip (~1 hour). Good to use non-librarians, so they don't already know the jargon (but nice to partner with other libraries because they won't be recognized by staff and each library benefits).
One great resource for them is ALA Publication's Assessing Service Quality. The shopper questionnaire [pdf] they created was all yes/no question (no "rate 1-5" scales, so as to be less subjective), and they had three specific uses cases:
- Ask staff help in finding a book on [ancient Egypt, trucks, other options given] for a seven year old [son, daughter, younger sibling, nephew - whatever fit the shopper's age]
- Ask staff help in finding a good book to read
- Ask staff help in creating a resume on the computer
Also included was calling in to ask for directions, impression of outside of library, parking lot, landscaping, etc.
Results were sort of disappointing: not enough shopping results to really have any kind of scientific impact. They did learn that 50% of patrons aren't greeted when they enter the library, and often there are no paper towels in the bathrooms.
Staff were all informed of the shopping beforehand, but only the timeframe - they didn't know exactly when or where. Afterward, a summary of the results were shared with all staff, too. Shoppers were not trying to connect individual staff with actions or experience - this was not designed to be a punitive exercise. There was no pushback from staff on the idea, and managers felt that six weeks was long enough so staff couldn't "fake it" the entire time. They never considered not telling staff, because they didn't want it to appear like a spying or "gotcha" program.
Was it worth it?
Not really - they just didn't get enough data to justify the amount of time that went into it. But it was a good exercise for managers to think about it. And they have lots of groundwork done, so it will be much better next time.
Other ideas presented as possibilities:
- do "exit interviews" with patrons as they leave the library, to get their immediate reaction
- do focus group of volunteers afterward, to see how they felt about it (and get them talking to each other)
- do website/catalog usability check - informal, 10-20 patrons in a lab, 15 questions/tasks (such as, what is the director's name and email?), maybe 2 hours on a Saturday morning, and give them a gift card for participating (use Steve Krug's books as guides)
- have shoppers ask for things they should not be able to get
- use app isecretshop, because people typing on a phone/ipad is less obvious than people walking around with clipboards
- do community polling outside the library, to find out why "unpatrons" don't use the library in the first place
June 9th, 2009 Brian Herzog
Another year has passed, which means another round of staff reviews and setting goals for the next fiscal year. Bleh.
In contrast to past years, I was encouraged to be brief. So this years goals are a bit more quantifiable, and a bit less "well, that's part of the job description anyway." You know, the way goals should be.
GOAL #1: Improve access to information resources and library services
- Weed the reference collection, refine the ref standing order list, and reevaluate how the reference shelving area is used and begin to develop a plan for alternate uses1
- Work with Tech Services to refine standing order list and evaluate reclassification of subjects to better group similar topics together
- Continue with staff-assigned sections for weeding, straightening and order suggestions
GOAL #2: Expand and improve the library's technology offerings
- Work with web committee to migrate website to new content management system2
- Add more website subject guides to tie together print and electronic resources, and link to expanded offerings of BPL and other MVLC libraries3
- Work with IT staff towards expanding technology offerings, such as wireless printing and loaning laptops
- Review current offerings utilizing new technologies, prioritize those needing ongoing maintenance, and document procedures to support maintenance by other staff
GOAL #3: Maintain and/or support web-based resources beyond the library's core collection
- CommInfo: utilize staff to contact and update organizations every Jan-Feb
- ChelmsfordVolunteers.org: work with other departments and organizations to keep listings up to date
- ChelmsfordHistory.org: provide leadership for the Chelmsford History project, coordinating with other organizations and volunteers to locate and index Chelmsford's historical resources
- Look for ways to better organize and provide access to the library's historical collections, such as the Vertical File, microfilm records, etc.
Admittedly, much of this still falls into the "continuing things we're already doing" category, but that is a large part of my job. And something else covered elsewhere in my review is encouraging all staff to attend at least 5 hours of some kind of training or professional development.
I think it's all doable. I can probably even manage to squeeze in helping patrons at the reference desk, too.
I'd like to interfile the ref books with the circulating non-fiction, and put into the reference area more quiet study rooms or subject tables --Back to Goals
2. Right now we're using Dreamweaver, but I'd like to see us move to a real CMS - NELA-ITS' CMS Day workshop is this Friday, so yay for good timing --Back to Goals--
3. Patrons like our genealogy subject guide, so I want to make more, incorporating Delicious bookmarks, and also linking to resources at other library to supplement what we offer --Back to Goals--
Tags: annual, department goal, evaluation, evaluations, fy10, fy2010, goals, libraries, Library, performance, public, reference, review, reviews, staff
May 22nd, 2008 Brian Herzog
The end of May is always staff performance review time in my library, and it seems unpleasant for everyone. Staff doesn't like it, department heads don't like it, and town hall doesn't seem to like it. So why do we do it?
One reason is because we always have done it. But other reason, as cheesy as it sounds, is because it really can help. Whether it provides an opportunity to address an ongoing problem, a pat on the back for a staff person who otherwise might go unnoticed, or just to make you sit down and really think about the way things are going in the library, as awkward and uncomfortable as they are, performance reviews and setting goals are beneficial.
That being said, I loath them. And I'm even lucky - I only have four staff people at the reference desk, and they're all pretty good.
This year, the union requested we use a new, much more simplified form for staff reviews (down from eight pages to three). As department heads, we modified a form that the town already used in other departments and customized it for library staff. The most challenging part was defining which each job criteria entailed, but I think the result works pretty well:
And as ever year, I developed goals for the reference department for the coming fiscal year (if anyone is keeping track, you might notice much repetition from last year):
Reference Department Goals for FY2009
- Continue to improve patron access to information resources1
- Look at ways to improve access to the collection
- Continue to weed and refine print reference materials to coordinate with non-fiction collection
- Continue to work with Tech Services to improve standing order list and recataloging of computer books to better group similar topics together2
- Finish weeding and shifting of the non-fiction collection, and then maintain collection by implementing a continual review through assigning sections to staff members for weeding, shelf-reading, straightening, order suggestions, etc.
- Continue to add online access listing to the website's comprehensive print periodicals listing
- Add more website "Subject Guides" to tie together print and electronic resources3
- Continue promoting database usage through existing methods (bookmarks, signs, links, staff training, etc.) as well as new technologies, and link to expanded offerings of BPL and other MVLC libraries
- Continue with website improvements, both to Reference section and all of website
- Work as part of web committee to revamp entire website to meet new design goals and accessibility standards
- Assist with development of town-wide events calendar
- Provide better access to the Vertical File, now that indexing is complete and files reorganized4
- Maintain Chelmsford listings in MVLC's Comm Info database and try to improve data and access
- Work with Childrens, Teen and Community Services departments to finally launch and maintain a local online volunteer resource5
- Work with IT on patron-related technology issues, such as timer software for public computers, internet access issues, printing from wireless computers, public faxing, etc.
- Help coordinate town-wide historical records project to improve access to historical resources both inside and outside of the library
- Continue to refine desk area and operations
- Ensure all procedures and policies are documented and easily available to desk staff
- Review new technologies and tools to see if paper-based methods can be improved upon
- Concentrate on professional development
- Keep current on journals, literature and blogs, for articles, book reviews and library trends
- Attend MVLC, NMRLS, NELA and other regional reference meetings
- Participate in staff, local, regional and national training opportunities
- Promote training opportunities to staff
Head of Reference
- Like last year, my base assumption is that there is no different between print and electronic resources - back to note 1
- Such as, we want to make sure all "powerpoint" books are together, all "sql" books are together, etc., instead of being shelved by dewey and then by author - back to note 2
- By this I mean creating subject pages that list research databases, print resources, and links to websites (via del.icio.us), to display all related materials in one place (like this) - back to note 3
- Reference staff went through and indexed everything in the vertical file into an Excel spreadsheet, and now we just need to find a way to make that easily searchable by patrons - back to note 4
- Our Chelmsford Volunteers resource has been in the works for almost a year; any comments are suggestions are welcome - back to note 5
Tags: department, evaluation, evaluations, goal, goals, libraries, Library, performance, public, reference, review, reviews, staff, supervisor