January 12, 2011
NIST Investment Significantly Improved Search Engines
RESEARCH TRIANGLE PARK, N.C.—An investment by the National Institute of Standards and Technology has significantly improved search engines' ability to retrieve electronic information, according to an evaluation conducted by researchers at RTI International.
NIST, a federal technology agency under the U.S. Department of Commerce, established the Text REtrieval Conference (TREC) in 1992 with two primary goals—to provide researchers with larger data sets to test information retrieval systems, more commonly called search engines, and to develop and disseminate evaluation methodologies to enable the standardized comparison of search engines.
According to the evaluation report, TREC helped to improve search engines like Google significantly. RTI's assessment found that about one-third of the improvement in web search engines from 1999 to 2009 is attributable to TREC. Those enhancements likely saved up to 3 billion hours of time using web search engines.
The research also showed that TREC made the search engine industry more efficient and effective. Search engine companies said that they would have delayed product improvements had it not been for TREC.
Additionally, the report showed that for every $1 that NIST and its partners invested in TREC, at least $3.35 to $5.07 in benefits were accrued to U.S. information retrieval researchers in both the private sector and academia.
"TREC has proven to be a success," said Brent Rowe, a senior economist at RTI and the report's lead author. "In the future, our research suggests that TREC will continue to be highly beneficial in improving information retrieval systems as long as TREC continues to focus on its key benefits—providing new data sets, developing methodologies to approach new information retrieval topics and providing a venue for information retrieval researchers to compare both their successful and their unsuccessful techniques in a safe environment."