Intra-collection Links Sample Clauses

Intra-collection Links. D2.2. contains the same intra-collection links as in D2.1 We have evaluated these links with a manually created dataset, and concluded that a simpler method would also perform well, but given the small difference we decided to keep them (cf. Section 8.2 on evaluation). In addition we are preparing datasets and algorithms for identifying why two items are related, producing an inventory of relevant relations, in a process we dubbed “typed-similarity”. The items enriched with similarity and typed-similarity information will be especially useful for personalization and recommendation in the second prototype.

Intra-collection Links. We now turn our attention to the intra-collection links (cf. Section 6). To create a dataset of items in Europeana, we randomly selected 295 pairs of items from Culture Grid and Scran because these collections are in English. They contain different types of items such as objects, archives, videos and audio files. Each item corresponds to a metadata record consisting of textual information together with a URI. For each item, we extracted its title, description and subjects. The concatenation of these data is the textual information of the item. After generating the dataset, we need to get reliable judgements of similarity on the pairs of items. This can be done by asking humans to give similarity scores so that to create a set of similarity scores to measure the performance of our similarity methods. To do so, we collect human judgements by making use of Crowdflower10, a crowdsourcing platform. We collected 3261 annotations from 99 participants. Each participant was presented with a page of 10 pairs of items and was asked to rate similarity between pairs using the following scale: • 4: Completely related or identical items. • 3: Related items. 10 xxxx://xxxxxxxxxxx.xxx/ • 2: Partly related items. • 1: Unrelated items. • 0: Completely unrelated items. Participants were free to rate as many pages as they wanted to a maximum of 30 pages. In addition each pair must be rated from 10 different participants at least. To ensure that annotations were not given randomly, we inserted one question with a predefined answer in each page. Annotations from participants that failed to answer these questions or participants that have given same rating to all of their answers were removed. The final gold-standard set were generated by averaging the ratings of each pair. Moreover, we computed the inter-annotator agreement to test the level of the agreement between human responses as the average of the Xxxxxxx correlation between the ratings of each participant and the average ratings of the other participants. The higher the agreement, both qualitative is the gold-standard. The agreement of our gold-standard is rho=+0.553. In related work, (Xxxxxxx, 2011) reported an agreement of rho=+0.507. We pre-process the data by removing stop words and applying stemming. For both tf.idf and LDA the training corpus was a total of 759896 Europeana items. We have filtered out all items that have no description and have either title length shorter than 4, or have a title which has been repea...