Construction of a Manually Labelled Reference Set Sample Clauses

Construction of a Manually Labelled Reference Set. Because of how we constructed the labels from the data, it would be impossible to ensure that all files within a randomly sampled sub-set hold only correct labels. This means that even our test set would include an unknown percentage of incor- rectly labelled documents. Naturally, this is undesirable, as no valid conclusions can be drawn from a flawed test set. In order to deal with this issue, we created a manually labelled reference test set (Brandsen, 2020), of which we are certain that it consists of correctly labelled documents only. As manual labelling is very time consuming, this test set consists of ‘only’ 100 files. Figures 4.4 and 4.5 show the frequencies for each of the categories captured within the classification of time periods and site types, respectively. The average number of labels per document is 13.9 for time periods (median: 11, max: 53) and 2.79 for site types (median:2, max:13).

Construction of a Manually Labelled Reference Set. Because of how we constructed the labels from the data, it would be impossible to ensure that all files within a randomly sampled sub-set hold only correct labels. This means that even our test set would include an unknown percentage of incor- rectly labelled documents. Naturally, this is undesirable, as no valid conclusions can be drawn from a flawed test set. In order to deal with this issue, we created a manually labelled reference test set (Brandsen, 2020), of which we are certain that it consists of correctly labelled documents only. As manual labelling is very time consuming, this test set consists of ‘only’ 100 files. Figures 4.4 and 4.5 show the frequencies for each of the categories captured within the classification of time periods and site types, respectively. The average number of labels per document is 13.9 for time periods (median: 11, max: 53) and 2.79 for site types (median:2, max:13). The distributions of the test set are similar to those of the training set, as were shown in Figures 4.2 and 4.3. The only exception is the category of label xxx (unknown) for the site types. This is because all files in our test set are labelled by at least one time period, and many files labelled by xxx, i.e., reports about sites with no finds, are not labelled by any time period. A complete overview that includes all the frequencies of all main and sub-categories can be found in Appendix C.