why Uses for extraction tools

The most popular applications for information extraction tools remain competitive intelligence gathering and market research, but there are some new applications emerging as organizations learn how to better use the functionality in the new generation of tools.

Deep Web price gathering The explosion of e-tailing, e-business, and e-government makes a plethora of competitive pricing information available on Web sites and government information portals. Unfortunately, price lists are difficult to extract without selecting product categories or filling out Web forms. Also, some prices are buried deep in .pdf documents. Automated forms completion and automated downloading are necessary features to retrieve prices from the deep Web.

Primary research Message boards, e-pinion sites, and other Web forums provide a wealth of public opinion and user experience information on consumer products, air travel, test drives, experimental drugs, etc. While much of this information can be found with a search engine, features like simultaneous board crawling, selective content extraction, task scheduling, and custom output reformatting are only available with extraction tools.

Content aggregation for information portals Content is exploding and available from Web and non-Web sources. Extraction tools can crawl the Web, internal information sources, and subscription services to automatically populate portals with pertinent content such as competitive information, news, and financial data.

Supporting CRM systems The Web is a valuable source of external data to selectively populate a data warehouse or a CRM database. To date most organizations focus on aggregating internal data for their data warehouses and CRM systems. Now, however, some organizations are realizing the value of adding external data as well. In the book Web Farming for the Data Warehouse from Morgan Kaufman Publishers, Dr. Richard Hackathorn writes, “It is the synergism of external market information with internal customer data that creates the greatest business benefit."

Scientific research Scientific information on a given topic (such as a gene sequence) is available on multiple Web sites and subscription services. An effective extraction tool can automate the location and extraction of this information and aggregate it into a single presentation format or portal. This saves scientific researchers countless hours of searching, reading, copying, and pasting.

Business activity monitoring web mining can continuously monitor dynamically changing information sources to provide real time alerts and to populate information portals and dashboards.