Deep Web Technologies Blog

Abe Lederman gets in the (mobile) groove

Sol — Mon, 16 Apr 2012 16:31:19 +0000

Abe Lederman, our founder and CEO, recently got a couple of exposures at MobileGroove, a site which provides analysis and commentary on mobile search, mobile advertising, and social media. The two MobileGroove articles cover Deep Web Technologies’ Biznar mobile federated search app.

Biznar is a free service, sponsored and provided by Deep Web Technologies, to business people everywhere. It uses an advanced form of federated search technology, enabling specialized access to more than 70 freely available deep web collections related to business.

The first of the MobileGroove articles is a review by Charles Knight who has made a reputation for himself seeking out and reviewing the “alternative” search engines, beyond Google, Yahoo!, and Bing.

The second article is a follow-on article in which MobileGroove founder and chief analyst, Peggy Anne Salz, in which she probes Abe about the company’s motivation for developing mobile apps.

Have your own experience of Biznar mobile. Check out the app at the iTunes Store.

How do the millennials search?

Sol — Tue, 27 Mar 2012 15:04:41 +0000

Here’s a paper worth reading: “A study of the information search behaviour of the millennial generation.” No, not because there are any earth-shattering conclusions, but you may want to read the article to confirm that what you already suspect to be true really is true. Here’s the introduction from the paper’s abstract:

Introduction. Members of the millennial generation (born after 1982) have come of age in a society infused with technology and information. It is unclear how they determine the validity of information gathered, or whether or not validity is even a concern. Previous information search models based on mediated searches with different age groups may not adequately describe the search behaviours of this generation.

Here’s the conclusion:

Conclusions. These findings indicate that the search behaviour of millennial generation searchers may be problematic. Existing search models are appropriate; it is the execution of the model by the searcher within the context of the search environment that is at issue.

Beyond telling us what we already know the paper gives insights as to how librarians can help students to become more sophisticated researchers. Areas in which librarians can add value include:

Verification of quality of Web information sources
A shift of focus from filtering content to first verifying its quality and then filtering
Developing an orderly methodology for performing research

The paper might provide insights that search engine developers could someday roll into their offerings targeted at students.

Our own Nick Urban contributes to Open Source movement

Sol — Fri, 23 Mar 2012 22:04:43 +0000

Deep Web Technologies software engineer Nick Urban was developing a software library to implement some basic functionality in Ruby. He realized that his code could prove useful to other software developers. From this inspiration, well_rested was born.

In Nick’s own words:

Well_rested is a Ruby library that takes takes JSON data and it turns it into Ruby objects that you can call methods on and manipulate. It increases the efficiency of manipulating data by supporting http caching.

The code is freely available at Github. Full details are here.

Nick’s code is also included at RubyGems.org.

Our Explorit integrates with Article Galaxy

Sol — Thu, 08 Mar 2012 21:03:45 +0000

Deep Web Technologies has integrated its Explorit federated search application with Reprints Desk’s Article Galaxy. Our company and Reprints Desk share the goal of facilitating literature discovery so integrating our technology with theirs was natural.

The recent Reprints Desk press release describes the feature:

Federated search, powered by Deep Web Technologies, for simultaneously searching across multiple online databases covering more than 52 million records, with easy ordering from search results that have been aggregated, ranked, and de-duplicated

The partnership between Deep Web Technologies and Reprints Desk began last October with a press release which reads in part:

Scott Ahlberg, Head of Corporate Services at Reprints Desk, stated, “The integration with Explorit enables Reprints Desk to help its growing base of document delivery customers maximize the value of their journal subscriptions, while filling content coverage gaps. It also allows us to meet our customers’ need for integrating search into the literature retrieval workflow, while enriching the growing number of research services that are bundled into our hosted article management software platform. With the Reprints Desk implementation of Explorit, native publisher databases will be ranked first followed by PubMed and other secondary sources.”

The integration includes RSS feeds for Alerts results so that users can easily see, within the Article Galaxy application, new articles that meet their interests. Future plans include a unified login to our Alerts system so that users can easily switch between creating alerts with Explorit and accessing the Article Galaxy accounts.

The Charleston Advisor gives Deep Web Technologies high marks

Sol — Tue, 28 Feb 2012 03:53:56 +0000

The highly regarded Charleston Advisor, known for its “Critical reviews of Web products for Information Professionals,” has given Deep Web Technologies 4 3/8 of 5 possible stars for its Explorit federated search product. The individual scores forming the composite were:

Content: 4 1/2 stars
User Interface/Searchability: 4 1/2 stars
Pricing: 4 1/2 stars
Contract Options: 4 stars

The scores were assigned by two reviewers who played a key role in bringing Explorit to Stanford University:

Grace Baysinger, Head Librarian and Bibliographer at the Swain Chemistry and Chemical Engineering Library at Stanford University
Tom Cramer, Chief Technology Strategist at Stanford University Libraries and Academic Information Resources

The review upon which the scores are based, is available at Stanford. (Click on the purl.stanford.edu link for access to the full text.) At just six pages, the review makes for a quick read. The first four pages describe the Explorit features, infrastructure and support, and makes the case for the partnership between Deep Web Technologies and Stanford University that led to the development of the locally branded xSearch federated search product. Pages five and six provide the reviewers’ critical evaluation of Explorit, references, and their bios.

Key points from the critical evaluation include:

“Compared to other federated search products, Stanford found that DWT offered the most compelling package of performance, features, and design.”
“While federated search engines’ performance is inherently limited by the performance of its target sites, DWT’s progressive delivery of results gives researchers near real-time response with the first set of results while the application assembles a complete set of hits from all sources.” More information about how near-real time response works is available at the Federated Search Blog.
Explorit was “the only service that included alerts, and the only service that allowed us to create customized “search engines” locally.
“DWT’s performance, good relevance ranking, and faceting capabilities are very helpful to users.”
“Because Abstracting and Indexing tools contain controlled vocabulary terms, when a user is searching xSearch, there are more discovery points than if they were searching Google Scholar or a publisher’s site.”

More observations are available in the review. More information about xSearch is available at Stanford. Our own press release about the review is available on our website. An Explorit overview is also available at our web-site.

NFAIS on discovery services

Abe — Tue, 14 Feb 2012 17:48:21 +0000

There’s an NFAIS draft Discovery Service Code of Practice that’s up for review.

The National Federation of Advanced Information Services (NFAIS™) is releasing a draft Discovery Service Code of Practice for review and comment by March 16, 2012. NFAIS believes that discovery services have the potential to provide ease of information discovery, access, and use, benefiting not only its member organizations, but also the global community of information seekers. However, the relative newness of these services has generated questions and concerns among information providers and librarians as to how these services meet expectations with regard to issues related to traditional search and retrieval services; e.g. usage reports, ranking algorithms, content coverage, updates, product identification, etc. Accordingly, the NFAIS Code Development Task Force has developed this draft document to assist those who choose to use this new distribution channel through the provision of guidelines that will help avoid the disruption of the delicate balance of interests involved.

I recently got an email raising concerns about the draft paper and why services such as my company’s Explorit Federated Search were being excluded by NFAIS. I don’t want to go further into the specific concerns of the email but I do want to comment on some things related to Discovery Services.

First, I want to say that, despite all the marketing, the concept of a Discovery Service is not new. Two or three years ago OCLC coined the term Web Scale Discovery to refer to their WorldCat index. Soon thereafter ProQuest started referring to Summon as a Discovery Service and put a lot of marketing muscle behind that. As a consequence of that the term Discovery Service has now become synonymous with a large centralized index even though federated search application such as my company’s Explorit can certainly be considered to be a Discovery Service, and in fact I have been talking about Explorit being a Discovery tool much longer than OCLC, Summon and EBSCO have adopted the Discovery Service label for just what they do.

Large centralized indexes, mostly of metadata have been around for decades now. Consider Web of Science, Scopus, Infotrieve, Ingenta and many others. So OCLC and ProQuest didn’t invent something new, they just developed an improved version of something that’s been around for a long time, stuck a new label on it, and through substantial marketing efforts have gotten the term – Discovery Service – to have become associated exclusively with large centralized indexes.

On the very positive side, for what we do and the issues I have raised about Discovery Services in the past, this NFAIS effort at developing a Code of Practice for Discovery Services (i.e. WorldCat, Summon, EDS, and Primo) raises a fairly substantial set of issues with these Services that Deep Web Technologies can address. I can see this paper as useful in highlighting issues with these Discovery Services. Three big issues that come to mind (I can think of others) are:

Muddiness about who owns content and inadvertent access to content (including meta data). “Lack of user authentication/verfiication and/or the inability to reliably identify an institution’s holdings” is cited by the NFAIS paper as a concern.
The ranking algorithm. Whose content is shown first in Discovery Service search results? There are issues with conflicts of interest among Discovery Service providers that makes me uneasy.
Coverage. What sources are available to users of Discovery Services? Who gets to decide that? What about access to the very specialized sources that Discovery Services are less likely to have access to?

Take a look at the NFAIS draft Code of Practice and raise your concerns.

A greater need than ever for multilingual federated search

Sol — Thu, 09 Feb 2012 03:16:10 +0000

Multilingual federated search, the ability to search and to view results from foreign language sources in your own language, may be just an interesting idea to some but there is a strategic value to the technology. Consider this article published by the BBC in March of 2011: China ‘to overtake US on science’ in two years. If the prediction of the UK’s national science academy, the Royal Society, proves true then sometime next year China will produce scientific research papers at a faster rate than the current leader, the U.S.

Researchers in the English-speaking world have mostly been restricted to searching only English language sources since the tools for simultaneously searching foreign language sources and for performing the translations haven’t existed until recently. Thus, opportunities to search scholarly journals in Chinese, Japanese, Portuguese and other languages associated with countries producing a great volume of science output are being missed. In an economic climate where performing research and getting products to market quickly translates to that competitive edge that leads to greater profits, being able to scour the research Web quickly, effectively, efficiently, and on an ongoing basis is critical to developing and maintaining a competitive edge.

Blog sponsor Deep Web Technologies has developed a patent pending multilingual search version of its Explorit federated search application that integrates the search and translation technologies making for a seamless and productive research environment for scientists, engineers, and researchers in business, science, and technology.

A publicly searchable deployment of Deep Web Technologies’ multilingual federated search is WorldWideScience.org. The site’s About page describes the application:

One-stop Searching of WorldWideScience Sources

WorldWideScience.org is a global science gateway comprised of national and international scientific databases and portals. WorldWideScience.org accelerates scientific discovery and progress by providing one-stop searching of databases from around the world (Architecture: What is under the Hood). Multilingual WorldWideScience.org provides real-time searching and translation of globally-dispersed multilingual scientific literature.

A recent article by search pundit Stephen Arnold at Beyond Search, Deep Web Technologies: Cracking Multilingual Search, provides a good overview of the history, challenge, and applications of the technology. Arnold is the creator of the Wizards series, in which he has interviewed dozens of leaders in the search industry, including Deep Web Technologies founder, president, and CTO Abe Lederman.

More information about Deep Web Technologies multilingual federated search technology is available in this blog’s archives on the subject. Articles include:

On Our Way – Charleston, South Carolina

Darcy Pedersen — Tue, 25 Oct 2011 16:35:36 +0000

Deep Web Technologies’ President, Abe Lederman will be attending the 2011 Charleston Conference this year in…you guessed it… Charleston, South Carolina.

The annual Charleston Conference hosts a variety of appealing activities but before you do anything, please be sure to dig into your conference bag for your free copy of the Charleston Advisor. Inside you will find a spectacular review of Deep Web Technologies’ Federated Search (xSearch) at the Stanford Library. This review neatly summarizes the strengths of the application in four different categories: Content, User Interface/Searchability, Pricing and Contract Options. We can’t tell you the score here – you’ll need to read the article – but we were very pleased to see our composite score (although not surprised!).

We are also pleased to announce that our Partner, Swets, will present Swetswise Searcher, powered by Explorit, at the Vendor Showcase, Table #26. Abe will be around and will happily demonstrate the power of next-generation federated search.

We hope to see you there!

Reducing Research Time and Costs in the Corporate Environment

Brian Despain — Thu, 28 Jul 2011 20:58:53 +0000

One of the many unseen costs in corporations is the actual time spent doing research for the company. Researchers, product managers and scientists use a significant time researching a wide variety of sources to bring a new product to market. There are multiple costs in research. First, there is the cost of the research materials themselves, namely the journals and subscription content that every technology driven organization uses to keep ahead of developments in the field. Maximizing utilization of subscription content is important to an enterprise. No one wants to buy content and then have it sit in the equivalent of a digital closet because the interface is too hard to use or there are too many interfaces to search for multiple sources. Enterprises want to get the maximum value for their subscription dollar.

The cost of the content isn’t the only cost to be considered either. Research scientists are paid on average $84,000 annually. A principal research scientist earns on average $120,000. In organizations that are primarily research driven such as aerospace, semiconductors, chemical manufacturing, law and engineering, time spent in research is time not spent developing a product, improving a process or inventing the next great widget. Research needs to get done, and quickly, without any sacrifice on breadth or depth of research.

Federated search addresses these two cost areas of research while providing a third benefit to organizations. By providing a single point to searching hundreds of sources, researchers can issue a single search request through a simple Google-like interface and get thousands of results sorted by semantically related concepts. Compare this with issuing multiple, individual search requests, then collating the results across multiple applications, de-duping the results and then getting the full text; federated search speeds this process up with every source that a researcher needs. Furthermore federated search decreases the likelihood of missing an important document. By extending the search across multiple sources, you give researchers more time to go in depth into the results. This significantly cuts the time to result for a researcher to begin analysis and to do the real job he or she is paid to do – build that next generation product, improve that process or design that industrial process.

Deep Web Technologies Explorit is used by some of the world’s leading research organizations to speed research. Our corporate customers are world leaders such as Boeing in aerospace, Intel in semiconductors and BASF in chemicals. These organizations have chosen Explorit to provide a competitive advantage in today’s world. Isn’t it time your firm looked at improving it’s bottom line and competitiveness? If you are interested in reducing research time and research costs and improving the effectiveness of your knowledge enterprise, contact Brian Despain, VP of Sales, toll free: 866-388-1407 x235 or visit Deep Web Technology and click on the chat link during business hours.

The Age of Discovery

Darcy Pedersen — Fri, 24 Jun 2011 16:06:22 +0000

Abe Lederman is heading to the ALA Annual Conference this weekend in New Orleans to take part in a fascinating panel discussion: The Age of Discovery: Understanding Discovery Services, Federated Search and Web Scale. Here’s a brief description:

Findability, discovery services, federated search, web scale—ways to discover content are increasing all the time, but how do we discover which discovery mechanism is appropriate? Join us to learn more about the discovery landscape. When is it appropriate to use federated search over a discovery service? How does this differ by type of researcher? What kinds of resources should be included in discovery tools? Learn discovery implementation from two librarians in the trenches; learn about “web scale” and how federated search and discovery are evolving from the experts; and how the rest of us can sort out this tangle of access methods!

Join Abe and the other panelists Sunday, June 26, 2011, 4:00-5:30 pm, at the Hilton Riverside – Grand Salon C.

Abe’s presentation is available here: http://www.deepwebtech.com/ala2011.ppt.