In the article, he very briefly discusses federated search and the value to newspapers.  He says, “A near relative of enterprise search is federated search. It is an effort to collect all forms of content –articles, archives, advertising and classifieds — and organize them so they are all displayed in response to a user’s query (i.e. searching for “pizza” yields results that include reviews, ratings advertising for pizza restaurants and even job listings for pizza delivery. Seattle Times.com, Boston.com and StarTribune.com are all using federated search.)”

He goes on to say, “Federated search is especially helpful for newspapers that have multiple websites.”  With all due respect to Mr. Krasilovsk, he is promoting a common misconception about federated search.  Federated search makes it possible to search other search engines, aggregating all the results, de-duplicating them and producing a ranked list from multiple search engines.

This is an important distinction, because by adding true federated search to their websites, online newspapers have a tremendous opportunity to increase the value of the search results to their readers.  It’s no longer about just the information located within the news websites themselves, but about the topics of interest to their readers.  For example, a news story about breast cancer can turn up articles about the topic from the NIH, governmental sources, etc.

This makes for a richer experience for the user, giving them incentives to come back and stay longer, ultimately increasing ad revenues for the online newspaper.

Thank you, Mr. Krasilovsk. for bringing up the concept of federated search.  If you truly harness the power of federated search for newspapers, the opportunities are boundless.

I can’t believe that a whole year has gone by since my first Year in Review post. A lot has happened in 2008. In this post I’d like to highlight a few of the most significant events of the past year.

First of all I would be remiss if I didn’t mention that in early December the Federated Search blog turned a year old. The blog is going strong and has become the most authoritative, comprehensive resource on federated search on the Web. Sol has written a wonderful article detailing his experiences authoring the blog. Sol’s article is being published by FUMSI this month.

One of the more significant events of 2008 was the acquisition in February of WebFeat by ProQuest, parent company of Serials Solutions federated search vendor. I remember arriving at my hotel in Boston on Thursday evening, February 14 (yes, I was traveling on business on Valentine’s Day) and starting up my email and seeing a flurry of messages about the acquisition of WebFeat. I quickly got on the phone with Sol and we brainstormed on the significance of the announcement. A couple of hours later Sol published his assessment in the federated search blog.

Today, the WebFeat and Serials Solutions products are yet to be fully integrated although I hear that’s coming later this year. This event was significant because it decreased the number of low-cost options for federated search to be deployed in lots of libraries to mostly just one vendor. The acquisition of WebFeat by ProQuest also meant that a major federated search vendor (i.e. unaffiliated with a content publisher) was no longer independent. Sol’s post on the acquisition speculated that we might see federated search costs rise following the acquisition. This hasn’t been the case as far as I can tell.

Microsoft recently introduced federated search in Windows Vista and in Windows Server 2008. Before that, Microsoft announced federated search features in Microsoft Search Server 2008. I wrote about this last year. I believe that Microsoft is trying to further position itself as having the federated search of choice (in the enterprise), in direct competition with Google’s Search Appliance. For the industry, I think it’s a good thing that Microsoft aims to make “federated search” a household term. A potential downside, however, is that those not familiar with the technology may view federated search as something that should be much cheaper than it is. Neither Microsoft nor Google’s federated search capabilities are much of a threat in the library.

This year saw renewed interest in the “deep web”. In April Google announced that they were unlocking the “deep web” by filling out forms and harvesting retrieved content into their index. Since the announcement came from Google my inbox was inundated for the next several weeks with copies of the Google press release and/or articles commenting on this Google effort. The reality as explained by Sol’s blog post is that there is little chance of Google being able to systematically and comprehensively add significant portions of the “deep web” to their index. Later in the year Infovell, which soon afterwards became DeepDyve, announced that they were unleashing the “deep web”. For some reason DeepDyve marketing folks claimed that they were the first to make deep web content available to researchers. Neither Google nor DeepDyve are using federated search technology to access the deep web. However, federated search is the best technology available to unlock the treasure troves of content in the deep web, so this renewed interest in the deep web is really positive.

This fall we saw an unprecedented meltdown of the U.S. economy. It is not clear yet what the impact of the economy will be to the federated search industry in 2009. We have seen public libraries being shut down (or attempted to be shutdown). It is unfortunate that federated search is viewed by some as a nice to have, and by others as not even necessary. Carl Grant, in one of my favorite blog posts of 2008, argues against the fallacy that federated search (or metasearch as referred by Carl) is not really needed and that users would be better served by learning to use the native interfaces of the information sources they are searching.

I believe that we in the federated search industry, particularly in this economic climate, must make a stronger case for the value of federated search. We must show that federated search provides a strong ROI to organizations that acquire federated search solutions. We must show that federated search significantly reduces the amount of time that researchers spend looking for information. Not only will researchers spend less time looking for information but they are much less likely to miss finding important results, either because they didn’t search the source containing the critical item of information, or because this critical item was buried in one of the databases being searched.

2008 was a good year for Deep Web Technologies. Software engineering was kept busy making significant enhancements to our technology, including developing a new Ajax-based User Interface, adding a clustering capability to Explorit, and releasing a major upgrade to our Alerts product. Major upgrades were made to a number of the portals that we’ve developed including Science.gov, WorldWideScience.org, Science Accelerator, Scitopia.org and DTIC MultiSearch. In June, Sol and I attended a ceremony in Seoul, Korea where WorldWideScience.org was turned over to ICSTI. Late in the year we launched the first two of our vertical portals, Biznar and Mednar. I was excited to learn while on vacation, just last week, that Mednar was chosen as the best new health search engine by AltSearchEngines.com.

I wish you all a happy, healthy and prosperous 2009.