Data Split Sample Clauses

Data Split. ‌ In order to train the statistical models, we use 10-fold cross-validation [47] to split our corpus. In the 10-fold cross-validation, the original sample is randomly partitioned into 10 equal size subsamples. Out of the 10 subsamples, a single subsample is used as the validation data to test the statistical model, whereas the remaining 9 subsamples are used as training data to train the model. The cross-validation process is repeated 10 times and each time a different subsample is taken out as test data and the remaining is used for training. The 10 results from the 10 folds are averaged to generate a single score, which is the final result for the 10-fold cross-validation. The advantage of the method is to make use of all annotations for both training and validation, with each annotation used exactly once. To make results comparable for different models, we set a constant seed value for randomization so that every time we train and test a model, the same splits are generated by the random number generator.