Presenter was a branch manager in the DC public library system. He was given six weeks in May-June to pull together a secret shopper program and run it over the course of six weeks during June-July.
Goals of secret shopping
- evaluate patron experience, for different types of patrons (using different types of shoppers)
- evaluate how well staff was trained on a particular product or service
- evaluate library's space, traffic flow, signage, etc
- evaluate collection and merchandising
- just get fresh eyes on the library
However: A Goals Caveat
- are you doing this to really find out something you don't already know?
- are you doing this to find proof of something you already believe to be the case?
- if problems are identified, are you in any position (financially, staffing, politically), to do anything about it?
DC used volunteers (teens, adult volunteers, and Friends of the Library), and developed their own tools; retail secret shopping ~$25-$35/shopping trip (~1 hour). Good to use non-librarians, so they don't already know the jargon (but nice to partner with other libraries because they won't be recognized by staff and each library benefits).
One great resource for them is ALA Publication's Assessing Service Quality. The shopper questionnaire [pdf] they created was all yes/no question (no "rate 1-5" scales, so as to be less subjective), and they had three specific uses cases:
- Ask staff help in finding a book on [ancient Egypt, trucks, other options given] for a seven year old [son, daughter, younger sibling, nephew - whatever fit the shopper's age]
- Ask staff help in finding a good book to read
- Ask staff help in creating a resume on the computer
Also included was calling in to ask for directions, impression of outside of library, parking lot, landscaping, etc.
Results were sort of disappointing: not enough shopping results to really have any kind of scientific impact. They did learn that 50% of patrons aren't greeted when they enter the library, and often there are no paper towels in the bathrooms.
Staff were all informed of the shopping beforehand, but only the timeframe - they didn't know exactly when or where. Afterward, a summary of the results were shared with all staff, too. Shoppers were not trying to connect individual staff with actions or experience - this was not designed to be a punitive exercise. There was no pushback from staff on the idea, and managers felt that six weeks was long enough so staff couldn't "fake it" the entire time. They never considered not telling staff, because they didn't want it to appear like a spying or "gotcha" program.
Was it worth it?
Not really - they just didn't get enough data to justify the amount of time that went into it. But it was a good exercise for managers to think about it. And they have lots of groundwork done, so it will be much better next time.
Other ideas presented as possibilities:
- do "exit interviews" with patrons as they leave the library, to get their immediate reaction
- do focus group of volunteers afterward, to see how they felt about it (and get them talking to each other)
- do website/catalog usability check - informal, 10-20 patrons in a lab, 15 questions/tasks (such as, what is the director's name and email?), maybe 2 hours on a Saturday morning, and give them a gift card for participating (use Steve Krug's books as guides)
- have shoppers ask for things they should not be able to get
- use app isecretshop, because people typing on a phone/ipad is less obvious than people walking around with clipboards
- do community polling outside the library, to find out why "unpatrons" don't use the library in the first place