or, The Hitchhiker’s Guide to Fear and Loathing at a Public Library Reference Desk


Update on our Evergreen Migration

   June 30th, 2011 Brian Herzog

Evergreen logoOver Memorial Day weekend, the Chelmsford Library (where I work), along with the rest of the Merrimack Valley Library Consortium (MVLC), migrated to the Evergreen ILS (from SirsiDynix's Horizon). Now that we've been up and running for a month, and the dust of the initial shock has mostly settled, I thought I'd answer the one question I keep getting: "so, what do you think of Evergreen?"

To hopefully makes things more clear, and perhaps be a resource for future migrations, I'm making separate sections for "Evergreen" comments and "Migration" comments. Every migration is bound to be different, and some of what we experienced may not apply elsewhere.

Also, I have a feeling this post is going to sound very negative - I don't mean it that way. It's much easier to talk about the "to do" list and the sore thumbs that stick out, than the "done" list and things that aren't a problem. I'll try to present both, but just keep in mind that I'm a whiny reference librarian who wants everything his own way.

Also, let me say this: I am a huge fan of open source, and am very happy we moved to Evergreen. I think I'll be much happier in a year or so when a lot of what I'm about to complain about has been worked out, but I'm happy that open source software gives us potential, which is not always an option with vendor software.

I'd also like to congratulate and thank the PINES developers (and all open source developers) for building this tool out of thin air and giving the library community something incredibly valuable.

Oh, and my comments below are numbered only for reference purposes, not for a ranked hierarchy or really any particular order.

Comments Relating to Evergreen

  1. Being that Evergreen is open source, one of the biggest advantages of it (and any OSS) is that everything is changeable. This is a selling point we heard all through the ILS selection and development process. And it's true, and it's great. However, it can also be a huge problem: saying that anything can be customized leads to pie-in-the-sky expectations, many of which are entirely unrealistic.
  2. Because Evergreen is so customizable, I have the feeling that just about every Evergreen installation is different. Because of this, I don't really know how much of what we use is "Evergreen" and how much is "MVLC Evergreen." This, I think, is especially true of our public catalog - since there are various "skins" available, I don't know which features are available in general, which are special to us, where potential for development lies, and what might just simply not be possible. I won't comment much on it because it is so context-dependent.
  3. The ability to customize and make changes has been evident during our migration process. On both our demo version and our live version, staff from all the libraries have been able to submit problems and suggestions, both minor and major, and sometimes see improvements on the same day. Sometimes it's week, or longer, and sometimes the answer is no, or we don't know, but the fact that library staff can be in direct communication with developers, and developers have the ability to make significant changes, is huge. Granted, since we just launched there have been a ton of problems and suggestions, which takes prioritization and some research, but still, just the fact that it's possible is incredible (it shouldn't be, but it is).
  4. Like most open source software, there is a user community all contributing to the development of features and functions - and the Evergreen community seems particularly strong. This is great for libraries who may not be able to do much development work themselves, as they may be able to freely adopt the work of someone else after that someone else releases their code to the community. However, the flipside is that, if you're waiting for someone else, you never know how long you'll have to wait, or what something may ultimately look like, or how it will work. If you can't do it yourself, you're still at the mercy of other people (like with vendor software), but if you have the money you can pay developers like Equinox for custom development.
  5. One of the most notable things about Evergreen, from a user's point of view, is how s-l-o-w it is. I don't know why this is the case, or if other Evergreen libraries also see it, but it is one of the biggest drawbacks to our system. Something else weird is that about 1/3 of the time, pages don't load, and you have to click the "Reload" button to try again (which, again, doesn't work about 1/4 of the time). This is incredibly frustrating when helping a patron at the reference desk - and I can only imagine the patrons' confusion at home.
  6. I know I am still very new to using Evergreen, but on the whole I find the staff client interface a little clunky. Different modules seem to be in different phases of development - the Circulation module seems like a very mature and powerful interface, whereas Cataloging and Searching seem to be lagging behind (we also previewed the Acquisition module, which appeared to be still in the very early stages of development - at least, based on our processes and needs). This will inevitably improve over time, and probably so will everything I'm complaining about, but I am glad that the Circulation module is the strongest, because that is the busiest desk with the least amount of time to fool around.
  7. One module that seems missing entirely is a reporting module. Right now, we are totally unable to run shelf lists, weeding reports, missing item reports, etc. This is major omission, since many of these reports our staff ran and pulled on a weekly basis.
  8. Something that I think is very weird about Evergreen is the way item records are structured. In our old catalog, there would be one "master" record for a book, and then individual "item" records for each of the physical copies we owned. So if we owned five copies of a Harry Potter book, there would be one main record that would show up in searches, which would indicate we had five copies of the book. In the example below, we have two copies of the book in Childrens and three copies in YA:

    + Harry Potter and the Chamber of Secrets, by J.K. Rowling
      - J/Fic/Rowling - 31480000000001
      - J/Fic/Rowling - 31480000000002
      - Y/Fic/Rowling - 31480000000003
      - Y/Fic/Rowling - 31480000000004
      - Y/Fic/Rowling - 31480000000005

    The two levels caused a few problems, but makes sense and I don't know how to simplify it.

    Evergreen however, actually has three levels. The two in the same hierarchy described above, but with a third level in between them. This third level is the "volume" level, which seems to serve no other purpose than to store the call number.

    + Harry Potter and the Chamber of Secrets, by J.K. Rowling
      + J/Fic/Rowling
        - 31480000000001
        - 31480000000002
      + Y/Fic/Rowling
        - 31480000000003
        - 31480000000004
        - 31480000000005

    It's an entirely different way of thinking, and definitely takes some getting used to - for catalogers, that is: the way items display in the public catalog is pretty much the same, so this is just a records management issue. But does mean that data interrelates in different ways, so you have to be careful what gets changed when, and always be aware of editing Items versus editing Volumes. It also means that if you wanted to change the call number for just one copy, then you have to create an entirely new Volume level to accommodate it

  9. Evergreen is based on Mozilla Firefox, which is both good and bad. I've been using Firefox for a long time, so I am familiar with the interface and keyboard shortcuts. However, Firefox isn't without problems, and the same is true of Evergreen - things don't always work right all of the time. Having to frequently "reload" pages that didn't load is one example.

    But something else about the interface is that it is very much a Windows interface, and requires frequent use of the mouse. I'm usually fine with using the mouse or the keyboard, but I find that having to switch between the two slows me down - but that is necessary with Evergreen: some thing you can do just by typing, but some things require mouseclicks. It's minor, but enough to be noticed (especially by our cataloger, who is a primarily a keyboard user. She also wasn't placated by the keyboard shortcuts, as some require Olympic-level finger gymnastics).

  10. Oh, but even though this is a Windows program, something that I use all the time - right-click - isn't always active. It does work in many places, but something as basic as the search box in the staff client is not right-clickable. This frustrates me to no end, because I often copy text from websites or emails (titles, authors, keywords, whatever) and want to paste them into the catalog to search. Instead of right-click > Paste, I have to click in the search box, and then Ctrl+V or Edit > Paste - which again is a nagging keyboard/mouse combination or involves extra steps that just takes that much more time to accomplish.
  11. Something else about clicking is that I find I do a lot of clicking. A great deal of information is buried layers and layers down, and it takes many clicks to drill down to find it. I'm sure this will get better as we learn the pathways to the information we need, but it seems like a lot of this could be streamlined.

    Another click-related annoyance (for me) is the number of confirmation dialog boxes that seem to always pop up. This scenario happens frequently:

    • I click to do something
    • Box pops up: "You are about to do something" OK - CANCEL
    • I click OK
    • Box pops up: "Are you sure you want to do something?" OK - CANCEL
    • I click OK
    • Box pops up: "You just did something" OK
    • I click OK

    Not horrible, I know, but a couple clicks too many for my taste. Plus, I know we're talking seconds here, but all those extra dialog boxes and mouseclicks take time - especially since Evergreen is slow to begin with.

  12. Another interface oddity is the randomly ambiguous wording: some screens have a "Reload" button while others have a "Refresh" button; for Loan Duration options, you have Short, Normal, Long; similarly, for Fine Levels the options are Low, Normal, High; and instead of a consistent "Save" button, that same function is alternately labeled All Modify, Modify copies, Modify/Create copies, Edit then create, Create with Defaults, Apply - all with the same effect.

    A funny anecdote: one of my coworkers was having difficulty canceling holds. She'd find the hold she'd want to cancel, click "Cancel Hold," a confirmation box would come up, but then the hold wouldn't disappear from the screen. It turns out, the confirmation box has two buttons: "Cancel" and "Apply." Since she wanted to cancel the hold, she (logically) clicked the "Cancel" button - but that was actually cancelling her action, not the hold. To cancel the hold, she had to click the "Apply" button. When I pointed this out, she was not amused.

    Using relative words like this allows different libraries to build in whatever meaning they want, but since our "Normal" loan period is always "3 weeks," it would be more helpful to just spell out "3 weeks" instead of staff needing to remember what the code "Normal" means in this case. Although, considering how rigidly precise library stuff is most of the time, the ambiguous wording does kind of make me laugh when I encounter it.

  13. A major customer-service-related complaint I have is that staff doesn't have access to all settings of a patron's record. For one thing, Evergreen stores all patron passwords/PINs in an encrypted format, and they cannot be viewed by staff. Meaning, if a patron calls in and asks us to look up their password because they forgot what it is, we can't - all we can do is reset it for them. Which is okay, and blah blah blah internet security, but it's annoying to say, "sorry, they system won't let me do that."

    Another thing the system won't let staff do is set the patron's preferred method of contact for holds notifications. Right now, patrons can get notification of their holds by phone, email, or both - and annoying, the default method is both. As in, unless a patron logs into their online account and chooses one of the other, they are going to get both a phone call and an email message letting them know it's ready for pickup. It's nice to offer both as an option, but I don't like that it's forced as the default - and that staff can't change it. There's no option for this in the staff interface, and we can't log into patrons' accounts because we can't just lookup their password.

    But something good (and new for us) about patron accounts is that Evergreen allows patrons to create a custom username to log into their account it. So, instead of having to remember their barcode to log in, they can create something memorable, like iamthebestartist, and use that. Neat.

  14. One of my favorite Evergreen features is that, in the staff client, you can adjust on the fly which columns appear in your display. So, if you're looking up an item that someone has on hold, the standard columns might be title, author, position in hold queue, etc. But in many screens, there are 20 or more additional columns of data you can display. So, when you're helping a patron and they ask something obscure about their item, often you can check out what other data is available for viewing and answer their question right then - unlike vendor software, where usually what you see is what you get

Comments Relating to the Migration

Just like I think most Evergreen installations are different, the whole migration process probably varies widely from library to library - perhaps even moreso. So again, take everything I say with a grain of salt, because it may or may not apply to anyone besides us.

An example: the same weekend MVLC migrated from Horizon to Evergreen, the Bibliomation library system in Connecticut also had a major migration of their BibliOak project - also moving libraries from Horizon to Evergreen. I know staff of the two systems were in communication before the migrations, but it turned out that our migration took three days (as planned) but they encountered problems and were down for over a week. Why? Because migrations are highly localized.

Although our migration process - planning-and-training beforehand to learning-as-we-go with the live system - has had problems (which I will complain about shortly), the hands-down shining moment of the entire process was the actual migration process itself. Our development team planned to be down for three days - Memorial Day weekend, since all our libraries would be closed on Monday anyway - and that is exactly how long it took. Tuesday morning, our system was up and running, and that feat can't be underestimated.

But as I said, we did have our problems, so here is a bit of hindsight that might be beneficial to other migrations.

  1. Know what you're getting into - which is especially critical with OSS. When you use vendor software, all you're really in control of is the data (item and patron information) and local rules (loan periods, borrowing policies, etc). The vendor dictates the interface, and there isn't much you can do or change about it.

    With open source software though, especially one that is as new and evolving as Evergreen, you're not just in charge of the data and local rules, but you're also responsible for the interface - both staff and patron. Most of our data and rules migrated well to the new system (one notable problem was that all reference materials migrated as circulating items), but as soon as we went live, we got slammed with patron complaints and usability problems, because our interface wasn't very high on the priority list during development.

    You can't get very far without good data and rules, but migrations also need to spend considerable time developing the interface - if people have trouble using the catalog, the data and rules don't mean much. Interface development is an entirely different mindset from data and rules management, so if you need to, hire new staff specifically for this purpose - it's hard to just pile the huge responsibility of interface development on existing staff who have never before been responsible for an interface. And if you do hire new staff, do it sooner rather than later. The new person will take time to adjust and contribute, which really is only possible in the early stages of development. Once you're just a couple months out from launch and you realize you need help, it's too late to hire someone because the existing development staff won't have the time to train the new person.

  2. When it comes to the interface, ask front-line librarians and patrons what features are important to them. This is especially true of patrons, because they might be using features in your existing system that you don't think much of, and which might not be replicated in the new system. Find out from the people who actually use the system what they use and how they use it, and then make sure those features are available in the new system.
  3. We over-trained patrons, but under-trained staff. My library spent a lot of time and effort focusing on making the public aware we were migrating to a new catalog, how it would work, what some new features were, etc. On the whole, it seemed like few people noticed or cared, and mainly seemed perfectly capable of figuring out the new system once it was launched (that is, except for when the features they had used heavily in the old system were not available in the new one - then we heard about that, over and over and over).

    Definitely make an effort to let people know things will be changing, but do it at normal catalog touchpoints - as in, put obvious notices in the old catalog that things will be changing. We may have (unknowingly) gone overboard with newspaper articles, webpages, tweets, blog posts, email newsletters, posters in the building, signs on computer monitors - all of that took a great deal of time and effort that may have been better spent.

    Another anecdote: During the first few days after the launch, I fielded many phone calls from patrons asking if "the system" was back up. When I said yes, their response was always something like, "oh great, so I can come in and use the internet?"

    This told me that we definitely got peoples' attention with our announcements, but we communicated the wrong message. We tried to clearly say that our materials catalog would be unavailable, but to many patrons, anything computer-related means "the internet," which they took to mean they couldn't get on the internet. That wasn't true, and it was an unexpected failure on our part.

    We also did a lot of staff training, but by the time we launched, it didn't feel like enough. Most staff were mostly ready, but on launch day, no one was feeling 100% confident that could help patrons.

    I think this was due to the limitations we faced with training: primarily, that no library staff ever saw the actual version we would launch with until it went live to the public. For months beforehand we had a demo server set up with an earlier version of Evergreen. Staff was encouraged to use this to get to know the system, but since it was a demo and always in development, it was never stable and was always changing, and we were never certain if something was acting oddly because that's was just how Evergreen worked or because it was in the process of being changed. Also, we contracted with Equinox for hands-on training, but instead of using even our demo version during that training, we were trained on their version of Evergreen - which was even more foreign to us and had many components that we wouldn't have.

    So when it comes to training, it would be best to provide staff access to train on the stable launch version before the launch. Even though our staff handled things very well, their confidence level was way below where it should have been, because the actual launch version was essentially a mystery to them - and I'm sure their fear, uncertainty, and doubt unintentionally got conveyed to patrons.

    Maybe the goal here should be: train staff to handle anything, and develop a system that any patron can use without training.

  4. Focus on providing clear and open communication. This is true for library staff communicating with patrons, as well as library staff communicating with development staff. Patrons/library staff know what is necessary, and development staff know what is possible, and there needs to be a continuous feedback loop so everyone is on the same page.

    It is also important to establish these communication lines during development before the launch, and then continue to use them after the launch. Things will be coming up all the time, so patrons need a clear way to communicate issues with library staff, library staff need a way to report issue to development staff, and development staff need a clear way to communicate system updates back up to library staff.

    One of the communication methods we used was a help desk ticketing system. Any library staff could "open a help desk ticket," which essentially posted whatever problem/question they were having to a forum. Development staff (i.e., the Help Desk) would check the forum and respond to tickets (although as launch time approached, response time flagged as the help desk staff got overwhelmed with the number and complexity of tickets). The forum was also open for browsing by any library staff, so people could see what issues other libraries were having, and what the resolutions were.

    Our problem is that our ticketing system is somewhat primitive. The ideal system would make it easy to search for existing tickets before submitting a new one, make it easy to keep up on what tickets were being submitted, make it easy to follow tickets of interest, and make it easy to "+1" or "like" an open ticket if you were having the same problem (instead of having to submit a new, identical ticket). This system needs to be easy to use from start to finish, because some people submitting tickets won't have much computer expertise beyond using the library ILS. Another requirement is that the ticketing system be scalable - once the system goes live, the number of tickets submitted per day is going to shoot up, and can quickly become overwhelming if there aren't enough development staff to address them.

    Another key area of communication is in establishing development priorities. Very, very early on in the development process, we held focus groups with library staff from each department to find out what their needs and wishes were. Many people expected the results of those meetings to be the development priorities moving forward, but quickly development began to focus just on what the system could do, not what we needed it to do. Right from the beginning, and carrying through after launch day, there should be a development oversight committee that considers all the development needs, and is responsible for setting development goals. This was almost entirely lacking in our migration, and caused unnecessary animosity and tension.

  5. Expect development to continue after the launch. It would be ideal to have everything ready to go by launch day, but that probably isn't realistic (especially with a continuously-developing product like Evergreen). A better scenario would be to establish the absolutely required features and functions necessary for launch, and then a development schedule for all the bells and whistles to be developed after the launch. The launch should not happen until the minimum launch requirements are met, regardless of timetables - even if it means delaying the launch. There is nothing more discouraging to patrons and staff than to launch with a brand new catalog that doesn't work right, and requires staff to apologize constantly to patrons and promise, "it'll get better eventually - we don't know when, but they tell us they're working on it." Patrons will resent it and resist the migration, constantly asking why they can't go back to using the old one. I think it's better to launch late with a stable and working product than to launch on time with one that doesn't work well.

    And the same is true of specific features and functions - less is more in this case. If the catalog has some really neat feature, but it's not quite working perfectly, turn it off until it is working perfectly. Patrons and staff will have enough to learn in the new interface without also having to face problems and learn workarounds on day one. Slowly rolling out additional features as they become stable and available spreads out the amount of new things patrons and staff are forced to learn all at once.

I think the most important takeaway is the need for communication. Developers need to know what staff needs are, and staff needs to know what developers can deliver. Otherwise, there will be tension and inaccurate expectations, which leads to unnecessary conflicts and problems.

A new catalog should be exciting, because the occasion should be a huge leap forward in library services. I know we'll get there with Evergreen, and I'm looking forward to it.



Tags: , , , , , , , , , , ,