or, The Hitchhiker’s Guide to Fear and Loathing at a Public Library Reference Desk

Reference Question of the Week – 1/2/11

   January 8th, 2011 Brian Herzog

Monty Python Scene 2I only got involved with this question towards the end - which was good, because that was the funny part.

An older patron came in and asked one of my coworkers:

Can you help me find an obituary? A friend told me that a woman we know died, but we're not sure when. The last time either of us spotted her alive was in October 2010.

That's straight-forward enough. The patron gave my coworker the deceased woman's name, and said she lived in Duxbury, MA. But, no matter what my coworker did, she couldn't find an obituary - that's when she asked me if I could think of anything else.

I basically retraced her steps - checking America's Obituaries and Dead Notices, Legacy.com, and the Duxbury Clipper newspaper website. When all of those failed to provide any information, we had both turned to general internet searching.

We started searching with "[name with middle initial]" obituary, but all we found was an obiturary for someone with this name in Souix City and another in Connecticut - they had the right dates of death, but were not the right people.

Next we dropped the woman's middle initial from our search and added the word "Duxbury," and this produced something very interesting: a woman by that name had left a comment on another Duxbury resident's death announcement in December 2010.

It was at this moment that it occurred to me that the person for who we were searching for an obituary may not actually have died.

My coworker changed tactics and looked for the presumed-dead woman's address phone number, and found that of her and her husband in Duxbury. When she called the patron and explained that we couldn't find an obituary and that the person may still be alive, the patron said,

Well, that is possible - the person who told me she died isn't always reliable.

My coworker gave her the person's phone number, and she said she intended to call her. But wow, what an awkward phone call that could be.

Tags: , , , , , ,

The Resurrection of Newspaper Obituaries

   September 7th, 2010 Brian Herzog

Yahoo PipesLast week I started talking about newspaper obituaries. Today's post details how we're improving access to the obituaries we do have in our newspaper microfilm records, using an online index created with Yahoo Pipes.

Our microfilm records of the local papers go back to 1940. But microfilm is primarily an archival format, rather than an accessible format, so it can be cumbersome to use. Our biggest impediment was that we didn't know what was there - when a patron contacted the Reference Desk asking for someone's obituary, it was very time-consuming for us to search the microfilm for an obituary, which may or may not have even appeared in the paper - we wouldn't even know until we checked.

So we created an online searchable index to the newspaper's obituaries - not the text of the obituaries, just a name/date/page index. Patrons and staff can use this to know whether someone's obituary appeared in our newspaper, instead of having to check the microfilm every time.

Here's how we did it: first, for about the past 10 months, volunteers have been going through every microfilm reel we have, page by page, and building an Excel spreadsheet with the following information:

Newspaper Year Month Day Page FirstName MiddleInitial LastName Maiden-Jr-Sr

The first column is necessary because we have records for both the Chelmsford Newsweekly (1940-1993) and the Chelmsford Independent (1986-present). The middle columns are reference and retrieval information. In the last column, we included extra information, like maiden name, whether a person was a "Jr." or "Sr." etc., and anything else that was random and didn't fit into another column.

The spreadsheet itself is useful, but I wanted to put this online so anyone could search it. The tool I chose was Yahoo Pipes, which has both pros and cons:


  • It's easy to play with and learn (like most Web 2.0 tools), but is also very powerful so we can grow into it
  • It can use a csv file for the data, which is easy to create with Excel
  • Beyond a simple search, it also provides fancy features like RSS feeds and tie-ins with other social media tools
  • Using Yahoo Pipes is covered in Chapter 7 of Library Mashups, written by Nicole Engard
  • The data is easy to update as the file continues to grow
  • It worked


  • Searching a database is not what Pipes is intended to do, so it's probably not the best tool out there (I wanted to use DabbleDB, but they're in transition right now)
  • The csv file must be ftp'ed to the webserver, which will be increasingly problematic - right now the file is 17,000+ lines and over 1MB. It will only get bigger, and the entire thing needs to be uploaded each time it's updated
  • Pipes has funny rules that you don't know about until something breaks. For instance, field names must be single words (hence "FirstName" and "Maiden-Jr-Sr"), you can't use certain characters in the data (like /), the search doesn't let you combine keywords (so far - I'm sure there must be some kind of fancy loop setup that will allow it, but right now people can only search either by first name or last name or year)
  • There isn't an easy way to embed the search box back into our website (there are Badge options, but only for search output) - you have to use the Pipe interface to search
  • There doesn't seem to be a wildcard for search
  • The results can't not link to something - I wanted the names and dates just to be displayed, but the way Pipes works requires the results to link to something

The last point was initially a pain, but it forced me to be creative, and I think the solution is actually more helpful for patrons than what I originally wanted. Now, when a patron finds the obituary listing they'd like to read, they click the link, and it automatically fills the obituary information into an email contact form on our website. That request gets sent to Reference staff, who then have an easy time of retrieving the obituary from the microfilm. Unfortunately, our microfilm machine isn't connected to a computer, so we'll just print and mail or fax the obituary to the patron. When possible we'll type them in and email them, and of course that will go into the searchable database too.

To make the connection from the Pipes listing to our email form, I had to use some javascript (which introduced another glitch: javascript makes names like O'Conner problematic, because it stops at the ', but I'll worry about this later).

Here's what the whole Pipe's source code looks like:
Yahoo Pipe for Obituary Search

Here's what it does:

  • The "Fetch CSV" module is the path to the csv file on our webserver
  • The module to the right of that controls what the patron search input box looks like. The "Label" field is "Enter EITHER a First Name, Last Name OR Year:" and you can see where that displays on the Pipe page
  • Both of those modules feed into "Filter" module - this one takes what the patron enters into the search box and filters the data from the csv file to create a subset of just matching records. Whatever the patron enters gets searched for in all the fields listed in the "Filter" module
  • The next module is "Rename" and I'm not sure I'm using it properly - I needed to create two new fields, so I'm just taking two existing fields, copying them, and renaming them so I can work with them later. The fields that got copied still exist untouched
  • Next is the "Regex" module, which is the most complicated and powerful, and I use it to create what the patron sees for the search results. The "Title" field is one I created, and here I'm replacing the contents from when I copied it to display what the patron will see on the screen - the code for it is "${FirstName} ${MiddleInitial} ${LastName} ${Maiden-Jr-Sr} - ${Newspaper}, ${Month} ${Day}, ${Year}, Page ${Page} ${Obituary}" which also includes punctuation formatting. So, for example, the result looks like this:

    Katherine M. Polley - Chelmsford Newsweekly, December 31, 1940, Page 7

    Because this field has to be a link, I also had to define what it links to, which is what I'm doing in the "Link" field. The value for that field is being written as


    which carries the data over to the library's website and some javascript pulls the data from the url and puts it in an email form. The patron can fill in their name and contact info into the form and submit it to us as an email message

  • The "Sort" module is self-explanatory, and I chose to list them with most recent first

This feels far more complicated than it should be, and I'm sharing it here to both save someone else from having to figure it all out again on their own, and to hopefully get suggestions on how to simplify/improve it.

Although, speaking of improving it, I do have one idea for future development: the local Cemetery Department has spreadsheet online listing complete burial locations - it would be neat to mashup up that data, so the obituary is linked to the cemetery plot location.

That's down the road a bit, so in the meantime I just keep adding whatever new obituaries appear in the paper to the csv data file - I had planned to do that weekly, but lately there have been many weeks without any obituaries in the paper (see my previous post). Anyway, we'll see how this works - it only went live last week, but already patrons have been using it, and it certainly does save a lot of staff time.

Tags: , , , , , , , , , , , , , , , ,

The Death of Newspaper Obituaries

   September 2nd, 2010 Brian Herzog

Newspaper MONEY SectionOne function of public libraries is to be a repository for community history. The extent to which a library can do this will vary, but at the very least, the library has holdings of the local newspaper, which patrons can use to look up obituaries of local residents.

But the reality of this is changing. As newspapers struggle to stay alive, they're exploring new revenue streams - our local paper recently started charging families to list obituaries, instead of providing that service for free. The paper is only published once a week for a town of 32,000 residents, but you can still see the effect below:

Year #/Obits
2000 444
2001 527
2002 523
2003 566
2004 556
2005 479
2006 500
2007 220
2008 215
2009 80
2010 26 (as of Aug.)

And of the 80 obituaries in 2009, only 12 were from June-December. With dramatically fewer obituaries appearing in the paper, the long-term research value of a library's newspaper holdings is diminished. There must be other factors at play too, but hopefully newspaper revenues will stabilize and this downward obituary trend will be reversed. Regardless, there will always at least be a gap for anyone doing genealogical research or just looking up a friend of family member.

And this doesn't seem to be just a local thing. A Slashdot post describes the same thing on a bigger scale. There's also a Boing Boing post that looks into Legacy.com, the company many newspapers are using to outsource obituary listings. The bottom line in both posts is that obituaries and death notices are turning into a cash cow business - and as it becomes more and more expensive to run an obituary, there are going to be fewer and fewer of them.

So, all of that is sad news - doubly so since it's out of the control of libraries (unless we start publishing family-written obituaries on our own websites for free). But at my library, we have been working to improve access to what we do have. Tune in next week for Part Two of this post, detailing how we created an online index to the obituaries in our newspaper microfilm records, to make then easier for patrons and staff to locate.

Tags: , , , , , , , , ,

Reference Question of the Week – 5/27

   June 2nd, 2007 Brian Herzog

Marine Corps insigniaLast week I received a letter in my mailbox at work. It started:

Dear Sir/Madam:

I work for the U. S. Marine Corp Casualty Section. I am looking for a copy of three obituaries that might help us locate a next of kin. I am hoping you have a service that could help us get a copy of these obituaries.

The letter then went on to list three peoples' names, their birth and death dates (death dates ranging from 1977 to 1995), and their social security numbers, as well as the contact information to use to send her what we find.

Now, we get genealogical research requests like this all the time, but never from the Marine Corps. Hmm. And, the contact information she cited was a street address in Murray, Utah, and a Yahoo.com email address - both seemed unMarinelike.

I checked the Marine Corps website and didn't see any listing for a Casualty Section, but did eventually find a form to request casualty information. The resulting email was basically about recently injured Marines, and the Corps' next-of-kin notification policy.

Not, I need to say, that any of this matters. Of course we researched these three names and sent the patron what we found. Unfortunately, we found that none of the three died locally enough or recently enough for us to be able to find their obituaries. As part of the response, I gave the patron the contact information for the libraries in the communities where these people died, hoping that those libraries would have their local newspapers on microfilm.

But I still found this particular request just... odd. If it weren't for that line about the Marine Corps, I wouldn't have thought twice about it. Even though the information was already sent to the patron, I felt a need to keep digging a little further.

At the bottom of the casualty information email from the Marine Corps website, there was a phone number for the Corps' Personal and Family Readiness Division. I thought that if I called it, at least I could find out if the Casualty Section actually existed, and if they used freelance researchers based in Utah to find next of kin.

After speaking with a Public Affairs Specialist in that office for a few minutes, I found out that no, the Marine Corps does not have an office in Murray, UT, does not use freelancers for this type of work, and would never (in his estimation) do any kind of official work with a Yahoo.com email address.

So what does all of this mean? Nothing, I guess. We answered this question just like any other, because why patrons request certain information is largely irrelevant. I don't know why this person claimed to be from the Marines, because, true or not, it was not necessary.

genealogical research, genealogy, libraries, library, marine corps, marines, obituaries, public libraries, public library, reference question

Tags: , , , , , , , , , ,