Discovery Services: Over-Hyped and Under-Performed *updated*

This post was written by Abe on December 2, 2010
Posted Under: Federated Search,Marketing Announcements,Reviews,View from Inside

**(12/2/2010) Updated section “Full-text or metadata only”

This blog article is written by Abe Lederman, President and CTO of Deep Web Technologies:

A couple of weeks ago I had the pleasure of attending the Charleston Conference. It was my first time attending this conference and also my first time in Charleston, South Carolina, a city deservedly named last week the friendliest city in America by Travel & Leisure magazine.

On Thursday afternoon, Nov 4, 2010, I attended the 45 minute face-off between Serials Solutions’ Summon and EBSCO’s EDS (aka “EBSCO Discovery Service”) gladiators (see Carl Grant’s pre-face-off blog post).  This was a well attended session attracting several hundred other conference attendees.

The face-off was disappointing and did not live up to the hype that had been generated for the face-off and for Discovery Services over the last year. The format of the face-off, much like a presidential debate, was negotiated between the conference organizers, Serial Solutions and EBSCO, and left much to be desired. Representatives of Summon and EDS were given the questions in advance and, in the case of EDS, the prepared answers were read out loud to the audience. There was no opportunity for the audience to ask questions or to engage the Summon and EDS representatives.

Don Hawkins of Information Today has summarized the Moderator questions and the Summon and EDS responses in this blog post.

In the brief demonstration part of the face-off both Summon and EDS were each given 2 searches to conduct as a way to show off their respective products. As opposed to the Q&A portion of the face-off, these searches were not given to Summon and EDS in advance as evidenced by the fumbling of both sides. From my perspective, these demonstrations highlighted the fact that Discovery Services are not really any better when compared to Deep Web Technologies’ Explorit in terms of the quality of results that they return to the user.

There are a number of issues with Discovery Services, particularly with Summon and EDS, that are not being addressed in the press, although I am now starting to see them discussed by some of the librarians that I met in Charleston and elsewhere. In this blog article I will focus on 4 issues:

One size fits all  — Discovery Services provide uniform access to their index. Whether you are a student doing research for an astronomy paper, a psychology paper, or a history paper, the index being searched is the same. Discovery Services do not have the ability to allow the student to search a selected subset (chosen by a librarian or by the professor teaching the course) of the available content through the particular Discovery Service.

Not everything is going to be in the index – Discovery Services index content that is of general enough interest such that it makes business sense for these products to index. Discovery Services also need to establish business relationships with the owners of content in order to index it and may also require the owner of the content to expend effort in making their content available to the Discovery Service. This means that not all content is going to be in their indices, particularly niche (long-tail) content.

So, if a library desires to provide their patrons with comprehensive, one-stop access to all the content that they subscribe to, the use of Federated Search is still required. EBSCO acknowledges this while Serials Solutions does not.

Full-text or metadata only — As I point out later in this article, Discovery Services does not tell us when they are indexing the full-text of articles and when they are only indexing metadata. If the full-text of an article is not indexed by the Discovery’s service then a user’s search won’t find results that only mention terms of interest in the body of the article, regardless if the article is highly relevant! Federated search, on the other hand, searches the index created by the content holder, the full-text, and in many cases returns some gems that may otherwise remain undiscovered.

Vendor neutral — Both Summon and EDS are Discovery Services provided by multi-billion dollar publisher/aggregator companies whose main business is making sure that libraries purchase their content. Shouldn’t librarians worry whether results returned by Summon might be biased towards higher ranking of ProQuest results? Or even worse, shouldn’t librarians worry whether content from a competitor to the Discovery Service they are considering subscribing to is going to be missing. Carl Grant’s blog post addresses this issue in more detail.

I want to use this post to challenge all providers of Discovery Services, not just Summon and EDS, to become completely transparent by clearly listing on their web sites the content that has been indexed for use by your Discovery Service. For each publisher’s content that you index please indicate what journals / databases you are indexing, the period of coverage, whether you have indexed only an item’s meta-data or also its full-text, and how much your index typically lags behind current content available from each publisher. It would also be great if someone would create a table of the top 100 or 200 publishers and list the above information cross-indexing each publisher and each Discovery Service.

It might be eye opening to discover (pun intended) what major publishers’ content are missing from the various Discovery Services. I have been told, but have not had a chance to confirm, that EBSCO content is not in Summon and ProQuest content is not in EDS.  Current lack of transparency makes this extremely difficult.

Only when all of this information becomes readily available can librarians make informed decisions whether to purchase a Discovery Service or purchase a Federated Search solution such as Deep Web Technologies’ next generation Explorit product.

Let’s hope that the next face-off is more conducive to a fair and open evaluation of research solutions and includes not only Discovery Services solutions but also leading Federated Search solutions, as well.

-Abe Lederman

President and CTO

Deep Web Technologies

Reader Comments

Add a Comment

required, use real name
required, will not be published
optional, your blog address