Reference Question of the Week – 6/15/08
June 21st, 2008
This is a reference question I've been holding onto for awhile, hoping I'd have an answer to share. I don't, so now I'm hoping someone else might.
A patron came to the desk asking for help with YouTube. He's one of our regulars, and has a bit of a compulsive personality. He's also a big fan of The Doors: he's working on a book, buys whatever merchandise he can from eBay, and watches any related video on YouTube - or rather, tries to.
One day, he came to the desk and said:
When I search for "the doors" on YouTube, there are over 79,000 videos. However, It only shows the first 50 pages of search results, which is only the first 500 videos. How can I watch the rest?
I had never clicked this far into any search returns in my life. So I tried it out, and sure enough, he was right. I played a bit, but couldn't find any way to get past this barrier to the rest of the videos.
I searched their Help Center with no success, and so sent in the question via their Contact Form. I also searched the general internet, but couldn't find anything relating to this issue.
This was on April 25th, 2008. So far, I haven't heard anything back from YouTube or Google. I resubmitted the question a couple weeks later, but again, no response.
I've played with this search limit again recently, and it looks like now YouTube cuts off the returned videos in the 540's, which is on page 28. The pagination shows out to page 31, and implies there is more, but when you click beyond page 28 the pagination and video numbering starts over at 1.
I can understand the technical limitations and the necessity of an upper cap on returned search matches. But with no explanation or message that there is a limit, and this confusing/resetting pagination, this patron feels YouTube is teasing him personally, and cheating him out of these other 78,500+ videos.
Does anyone have an answer I can pass on to the patron? Thanks.
June 21st, 2008 at 11:25 am
I don’t have an answer for you unfortunately, but maybe your patron could do a search with some other keywords related to the doors, eg, “Jim Morrison” “Riders on the storm”, etc. etc..
It’s a bit annoying, but this way he might be able to see some of the videos that have been eluding him through YouTube silliness.
Happy searching (and watching)!
June 21st, 2008 at 12:44 pm
Perhaps there is a clue here in the YouTube APIs Developer Forum: http://groups.google.com/group/youtube-api-gdata/browse_thread/thread/c2198cf4fadd8cdb
This conversation addresses the search results limits as well as a method of querying the database.
June 21st, 2008 at 1:11 pm
http://groups.google.com/group/youtube-api-gdata/browse_thread/thread/c2198cf4fadd8cdb
has some answers about the search limit and different numbers returned
June 21st, 2008 at 9:07 pm
Hi Brian: About a year ago, I was taking an online PHP class that used the YouTube API. After the class, one of the other students referenced a search limit in YouTube in the comments as we were working on things. Here is his comment:
“the problem is youtube doesnt service more than a thousand videos per search. So if you have 20 per page you can only go to page 50. At 51 it will error out. So the scripting that Im going to work on will limit the results to 1000 per search. So keep that in mind on how many you choose to have on your page.”
The class and its comments are at: http://waxjelly.com/2007/03/08/youtube-api-class-redeaux/.
As for why YouTube displays the hits (even though you can’t see the actual videos)… I’m not sure. Probably to impress searchers. Walt Crawford has an interesting discussion of search hit results in the latest “Cites and Insights” (http://citesandinsights.info/).
June 22nd, 2008 at 4:40 pm
There’s certainly something funny going on with undocumented limits.
Your patron will do a little bit better using Google Video Search at http://video.google.com. The search you want is “the doors” site:youtube.com
Video Search allows you to search by three different durations, which will increase the total number of videos you see.
I’d also look at the RSS feed of results (link at the upper right of the video search results page), and Google Alerts (link at the bottom of the results page). Over time, you may end up with more videos.
Finally, if you happen to have a programmer handy, you might look into using the Google API, which now supports YouTube (at http://code.google.com/ with some useful-looking information at http://apiblog.youtube.com/).
June 23rd, 2008 at 10:48 am
In addition to Google Video Search, you might also try http://www.blinkx.com. You can limit to YouTube and, though I didn’t look too closely, it looks like the pages of search results just keep coming.