Easy use and creation of vocabularies: the LoCloud vocabulary microservice

Microservices? An introduction

The past years have seen a move from monolithic applications towards microservice architectures. Microservice architectures are generally described as suites of small and independent services compiled into single applications, in opposite to monolithic applications that are built as single units, with a client-side user interface, a database, and a server-side application. While any changes to monolithic applications require the re-building and deployment of the entire applications, in microservice architectures it is quite easy to introduce new versions of the individual services or integrate new services. This will become even more important in the future, as applications will increasingly be deployed to the cloud.

Figure 1 – Monolithic versus microservices architecture

Microservices are independently deployable and scalable and they could even be written in different programming languages by different software teams. Usually. the services are built upon business capabilities and communicate via web service requests or remote procedure calls.

The LoCloud microservices have been developed by different project partner teams and are being deployed in the LoCloud MORe [1] aggregator. Additionally, some of the services provide front-end applications. The LoCloud vocabulary microservice provides the business capability for vocabulary management and enrichment of metadata.

Vocabulary Microservice

Vocabulary services support the enrichment of metadata (catalogue data) by adding vocabulary terms to metadata records. This can be done right “at the beginning” when the object is registered in the local cataloguing system. Alternatively, the data can be enriched by an automated service “after” cataloguing has taken place. The second possibility often prevails when data is ingested into joint virtual catalogues (like Europeana) where common vocabularies provide a means to semantically link data and to support easy browsing through the entire repository.

The requirement for the LoCloud vocabulary service team was to implement a cloud-based vocabulary web service and vocabulary application for the LoCloud network. The vocabulary application should support the development of multilingual, semantic thesauri for local heritage content and the vocabulary webservice had to be based on international standards such as SKOS [2] and the ISO thesaurus norms [3].

Three uses cases for vocabulary provision: Use the vocabulary microservice…

a)            …in the various enrichment workflows automatically through the generic enrichment service

b)           …through the Aggregator User interface by choice

c)            …in local cataloguing systems via web services.

One use case for vocabulary creation including import of existing vocabularies: Use the service

d)           …with a cloud based online tool.

There was a six-month implementation period and a three-month testing period for the service scheduled in the LoCloud workplan. A systematic research of already existing applications revealed that the open source tool TemaTres (http://sourceforge.net/projects/tematres/) was the best starting point for a rapid development and implementation of such a service.

TemaTres [4] supports the handling of vocabularies in accordance with the ISO standard thesaurus norms. It allows for import and export of data as simple text files or in SKOS format. The tool was installed in the cloud testlab of LoCloud and then adapted to the project’s needs. These needs concerned mainly the implementation of a simplified administration and import facility. Additionally, we implemented a note extension that allows storing coordinates for place names. Two new web service [5] calls “import” and “linkTerm” were added to the extensive list of already available TemaTres web services. For the import process of multilingual vocabularies a new identifier for concepts was introduced, connecting automatically the terms in the various languages. This identifier also carries the link information of the publicly published vocabulary and is later added to the metadata during the enrichment process.

The LoCloud microservice “Generic enrichment” automatically receives the vocabularies available in the vocabulary tool and uses them during the automated enrichment process conducted in the MORe aggregator.

To date around 30 vocabularies are imported to the vocabulary tool. A list of these vocabularies can be found here  > LoCloud Vocabularies.

Figures 2 and 3 show how the microservices Vocabulary and Vocabulary Matching (available currently for  English and Spanish language) are added to a data source enrichment plan in the the MORe aggregator.

Consequently, the EDM metadata is checked against the LoCloud vocabularies and where appropriate the vocabulary links are added.

Use Case 2 – Match vocabulary terms in the aggregator.

Instead of matching the vocabularies automatically to the metadata it is possible to selected individual terms from the LoCloud vocabularies that should be added to the metadata. These selections are called Subject Collections in MORe.

Use case 3 – Use the vocabularies in your local cataloguing systems via web services

One of the benefits of microservices is that these small and independent services may be plugged in into other application and can be used via the APIs [6] they provide. Figure 6 depicts an example of a local cataloguing system for music archiving that calls the vocabulary web service in the metadata field “Genre(s)”. The technical information on how the services can be integrated is published on the LoCloud online support centre. There are currently fifteen different web service calls for vocabulary integration available.

Use case 4 – The vocabulary experimental application.

The LoCloud vocabularies have been imported into the Tematres tool in order to use them via web services in the aggregation process. But

Tematres can also be used to create vocabularies from the scratch or to import already existing vocabularies. The advantage of importing vocabularies to the tool is clear: The vocabularies become available in SKOS format with an own web presence and can be used for semantic linking afterwards. With the consent of the vocabulary owner, the vocabulary might then become available for other LoCloud partners in the MORe aggregator. Moreover, the LoCloud Tematres installation allows collaborating online in creating and extending vocabularies (for example in order to create new translations of existing vocabularies).

The vocabulary application is accessible through this link (see figure 7). Please check the LoCloud support portal for information on a test user account which is necessary to access and create a new vocabulary.

 

***

[1]https://support.locloud.eu/MORE 26 January, 2015

[2]http://www.w3.org/2004/02/skos/ 26 January, 2015

[3]http://www.niso.org/schemas/iso25964/ 26 January, 2015

[4]http://www.vocabularyserver.com/ 26 January, 2015

[5]http://en.wikipedia.org/wiki/Web_service 26 January, 2015

[6]http://en.wikipedia.org/wiki/Application_programming_interface 26 January, 2015