User Requirements Specifically, we focus on performance Sample Clauses

User Requirements Specifically, we focus on performance processing speed, feedback provided about the crawl progress, error handling, and quality of the documentation (Table 3). The BootCat toolkit (Xxxxxx et al., 2004) is a well-known suite of Perl scripts for bootstrapping specialized language corpora from the web. Bootcat initially creates random tuples from a seed term list and runs a query for each tuple (on the Yahoo! search engine). After keeping the first 10 results from each query, it constructs a URL list, downloads corresponding web pages and removes boilerplate. Two software applications that integrate the BootCat tools for constructing simple web corpora are the WebBootCat (Xxxxxx et al., 2006) and the BootCat front-end, a web service front-end and a graphical user interface to the core tool, respectively. Although these applications are primarily designed for end users, they can be employed for certain initial (off- line) tasks (e.g. construction and testing of a seed URL list in a specific domain). In addition, a modified version of the BootCat toolkit can be used as an alternative tool for acquisition of monolingual corpora in specific domains. Heritrix (Xxxx et al., 2004) is an open-source and extensible web crawler. It is implemented in Java and the main interface is accessible using a web browser. Heritrix is one of the most configurable tools for crawling. To the best of our knowledge, it does not include functions about focused crawling based on a predefined list of terms of a specific topic. Combine (Ardo, 2005) is an open system, implemented in Perl, for crawling Internet resources. It is based on a combination of a general Web crawler and an automated topic classifier. The classification is provided by a focus filter using a topic definition implemented as a list of terms describing this topic. One critical issue is the fact that, Combine, in its current implementation, does not sort and follow the most promising URLs in the frontier (i.e. links within pages with high relevance to the topic). In other words, it is a breadth-first crawler followed by a topic classifier. We believe that a modification to the Combine‘s strategy, which will include sorting URLs in the frontier, would be beneficial for Panacea purposes. It is worth mentioning that Combine: a) is an ―active project‖, b) includes modules for language identification and topic classification, c) is modular and open-source and d) allows monitoring of the crawl progress by logging its actions in a relational database. HTTrack (Roch...