Heuristic Learning of Rules for Information Extraction from Web Documents

The efficacy of an information extraction system is mostly determined by the quality of the extraction rules. Building these extraction rules is time-consuming and difficult to implement by hand. Hence, we propose a Heuristic Rule Learning (HRL) algorithm which can automatically and efficiently acquire highquality extraction rules from a user labeled training corpus. Moreover, these extraction rules are maintained at the most suitable generalization level to enhance information extraction efficacy. In HRL, we use a Dynamic tErm eXtraction Technique (DEXT) to construct terms and extraction rules at different generalization levels. The conditional entropy model is used to evaluate the suitability of these different generalization levels of the extraction rules so as to maintain them at a high-quality level. Experimental results show the algorithm’s efficacy of acquiring extraction rules at different generalization levels and the efficacy of these extraction rules in the information extraction tasks.