Combining Information from Multiple Internet Sources

Table search_engine:
This table contains definitions of search engines. It contains all necessary
data to construct specific queries which in turn can be used to extract
answers from the search engines.
Columns:
• id – Primary Key
• name – Name of the search engine
• query_parameter – Contains base URL of the search engine, used when
constructing queries
• result_count_parameter – Contains name of the HTTP request parameter
used to manipulate number of results displayed per page
• cookie_based_parameter – Some of the search engines store user
preferences in cookies rather than, for instance, allow the user to
supply HTTP request parameters to manipulate the results
Table search_engine_ignore:
This table contains lists of pairs (Link Name, HREF) which should be ignored
when parsing page of search engine with results on given query. When parsing
the HTML document system should not consider all buttons, URLs, etc. which
are not connected to the query (for instance on Google page one can find
hyper links to Google Maps, Google News, etc. which should not be considered
as a result)
Columns:
• id – Primary Key
• href – URL to be ignored (if empty system will ignore all pairs with
the given link name)
• link_name – Name of a link to be ignored (if empty system will ignore
all pairs with given HREF)
• search_engine_id – (Foreign key) Specifies which search engine should
use the given pair
Listing 2.2.2 Search_engine and search_engine_ignore tables descriptions
19