Collective intelligence. AI for expanding the information content offered to users: the SBN Sommerso Project

Abstract

The SBN Sommerso project was developed with the aim of integrating bibliographic heritage not yet described in the centralised database into the collective catalog of the National Library Service (Servizio Bibliotecario Nazionale, SBN). To enrich the catalog, the project involves an automated process based on the use of artificial intelligence techniques for the processing of UNIMARC records which, when compared with the data in the SBN, are recognised as identical, similar or new, thus making new bibliographic data or new information relating to library holdings available, while ensuring that the uniqueness of the bibliographic record within the SBN Index is maintained. The automation of activities, which is necessary considering the amount of data managed, is achieved through the use of specific machine learning algorithms, designed and developed for the I.PaC infrastructure and adapted within SBN Sommerso through training on data and metadata specific to the bibliographic domain.

For the clustering and deduplication of entities represented in the records, namely Manifestation, Agent, Series, Work, Subject and Printer’s Mark, the algorithms took into account all the specific metadata for each element. The output required from the system involves presenting the user with groups of identical entities and identifying new ones. The system also groups entities that are assessed as similar and presents them to the domain expert user for verification of the results and correct modulation of the evaluative choices made by the AI, for an increasingly rich, accurate and reliable collective catalog.