Launching the LoCloud competition

The LoCloud competition is underway!

Are you a curator or volunteer from a local cultural institution, a local historian or student? Have you used Europeana to explore your interests in local heritage? Then why not share your story in a web page, a blog or a short video and enter the My Local History strand of our competition.

My Local History is being promoted across Europe by our partners who have set up websites to encourage entries from people within their countries. Its focus is on local heritage and culture, so we anticipate entries in many languages with some great visual resources!  You can find out more about the competition “in your country” on the Competition pages of the project website.  Winners of each national competition will be nominated to the international jury with a chance of winning a tablet computer and attending our final project event.  We will publicise all the winning entries via LoCloud and Europeana #mylocalhistory.

Are you a developer, aggregator or cultural institutions using LoCloud services and applications? Have you used LoCloud Collections to create an exhibition?  Are you using the LoCloud APIs for your application?  Then why not showcase your results in the My LoCloud Services strand of our competition with the chance of winning a smart watch #mylocloudservices .

The finalists will be invited to present their entries at the LoCloud final event, which will be held in Amersfoort, Netherlands on 5th February 2016.  All short-listed entries will be promoted on the LoCloud website.

Competition Information

LoCloud is launching an international competition focussing on local heritage and culture.  The competition has two strands:

  • My Local Heritage – invites people to explore a favourite place or part of their local history through Europeana and to present their experience online in webpages, a blog or video.
  • My LoCloud Services – invites developers, aggregators and individual cultural institutions to showcase their use of LoCloud services.

The finalists will be invited to present their entries at the LoCloud final event, which will be held in Amersfoort, Netherlands on 5th February 2016.

We attended the LoCloud Hackathon

Aleksandra Nowak, Team Leader, Data Aggregation and Enrichment Team, PSNC
writes:
On 11 February 2015 we participated in the LoCloud Hackathon in Paris, France. The event took place at the Google Cultural Institute. This gave us the opportunity to find out what Google is working on in the cultural heritage preservation domain. We could learn a lot about Google Art Project and see the its art pieces on an amazing gigapixel screen.
IMG_0557
Hackathon in progress at the Google Cultural Institute
At the beginning of the event all the participants were introduced to the LoCloud project. LoCloud representatives showed us its components and introduced us to the API of the LoCloud services. They also explained how to use the MORE aggregator and its API.The lunch break gave us  a chance to meet the other participants and talk over our project ideas for the hackathon. We started to work after lunch.
We decided to work in a team of three participants from PSNC – Mateusz Matela,  Aleksandra Nowak and Artur Pecyna – to share responsibilities and use the short time available as much effectively as possible. Our project was called LoTrips and we wanted to utilise geo-localisation data accessible in LoCloud’s collections.
The main idea was to create an application where users could see some objects from LoCloud collections on a map, choose the objects that are most interesting to them and filter them using additional criteria. The application would then create a route between these points as a proposition for sightseeing trips in real life.
We thought this could be a good tool to show the geographical or historical context of objects stored in LoCloud libraries and museums. Cultural institutions would also be able to choose the objects they want to promote using LoTrips, present the maps on their web pages or even organise trips for their users.We started our work with downloading the data using the MORE API. The data was indexed using Solr, to enable the creation of search queries and faceting on it.
The following stage was to access the data stored in Solr via its HTTP API and then show it on a map using Google Maps API. An example route is shown on the picture below.

 

IMG_0575

Aleksandra Nowak presenting at the Hackathon

Unfortunately the time for “hacking” was quite limited and we could not implement all the functionalities that we aimed to. However, we believe that a further development of this application will be quite easy since the main points have been developed. We also think that the tool for indexing the data into Solr might be integrated into the MORE workflow so that the data would automatically appear in the index.

After the coding was finished we presented our idea to the jury and to the other participants.The jury made its choice and the winning project was announced.

Several teams of developers took part in the hackathon. Most of them focused on extending the MORE aggregator. In particular, they presented a community extension for the MORE aggregator that instantly shows tasks performed by other users, a plugin which could move cultural objects closer to social media, an idea for integrating MORE with a cloud sourcing platform that improves geolocalization metadata, and the winning project – a metadata validator where users can define specific rules to check the validity of their metadata.

As a general suggestion, it would be good to extend the duration of future hackathons by a few hours or even a couple of days. This would allow us to present not just a prototype but possibly a working application.

We had great fun at the LoCloud Hackathon. It gave us the opportunity to find out more about the project and to work with new tools and APIs. It was a great experience for us and we look forward to taking part in future LoCloud hackathon.

 

Creative Commons licenses: how to use them

Creative Commons (CC) is a nonprofit organisation that enables the sharing of creative works, through a set of free to use licenses which enable organisations to modify their copyright terms to best suit their needs.

Europeana greatly encourages cultural institutions to use Creative Commons licenses as a tool to open up their collections.

One third of the content you can find in Europeana is ‘open’. This means that the material is either in the public domain and free from any copyright restrictions, or the rights holder has made the material available using one of the two open Creative Commons licenses (CC BY and CC BY-SA).

Currently nine million objects are licensed using one of the six licenses provided by Creative Commons. However, there is also a lot of misuse of these licenses. CC licenses can be applied only by the rights holder in the first place or by an institution holding a permission from the rightsholder.

Read the full article on the Europeana Pro Blog and access the CC report.

Easy use and creation of vocabularies: the LoCloud vocabulary microservice

Gerda and Walter Koch of AIT Applied Information Technology Graz, Austria write:

LoCloud provides support services and tools to regional and local heritage institutions. These tools and services simplify ingestion of data into Europeana and help to improve metadata quality with the use of cloud-based services. One of the LoCloud services developed during the second project year is the vocabulary microservice.

Microservices? An introduction

The past years have seen a move from monolithic applications towards microservice architectures. Microservice architectures are generally described as suites of small and independent services compiled into single applications, in opposite to monolithic applications that are built as single units, with a client-side user interface, a database, and a server-side application. While any changes to monolithic applications require the re-building and deployment of the entire applications, in microservice architectures it is quite easy to introduce new versions of the individual services or integrate new services. This will become even more important in the future, as applications will increasingly be deployed to the cloud.

Monolithic versus microservices architecture

Figure 1 – Monolithic versus microservices architecture

Microservices are independently deployable and scalable and they could even be written in different programming languages by different software teams. Usually. the services are built upon business capabilities and communicate via web service requests or remote procedure calls.

The LoCloud microservices have been developed by different project partner teams and are being deployed in the LoCloud MORe [1] aggregator. Additionally, some of the services provide front-end applications. The LoCloud vocabulary microservice provides the business capability for vocabulary management and enrichment of metadata.

Vocabulary Microservice

Vocabulary services support the enrichment of metadata (catalogue data) by adding vocabulary terms to metadata records. This can be done right “at the beginning” when the object is registered in the local cataloguing system. Alternatively, the data can be enriched by an automated service “after” cataloguing has taken place. The second possibility often prevails when data is ingested into joint virtual catalogues (like Europeana) where common vocabularies provide a means to semantically link data and to support easy browsing through the entire repository.

The requirement for the LoCloud vocabulary service team was to implement a cloud-based vocabulary web service and vocabulary application for the LoCloud network. The vocabulary application should support the development of multilingual, semantic thesauri for local heritage content and the vocabulary webservice had to be based on international standards such as SKOS [2] and the ISO thesaurus norms [3].
Four main use cases for the vocabulary services exist:
Three uses cases for vocabulary provision: Use the vocabulary microservice…
a)            …in the various enrichment workflows automatically through the generic enrichment service
b)           …through the Aggregator User interface by choice
c)            …in local cataloguing systems via web services.
One use case for vocabulary creation including import of existing vocabularies: Use the service
d)           …with a cloud based online tool.
There was a six-month implementation period and a three-month testing period for the service scheduled in the LoCloud workplan. A systematic research of already existing applications revealed that the open source tool TemaTres (http://sourceforge.net/projects/tematres/) was the best starting point for a rapid development and implementation of such a service.
TemaTres [4] supports the handling of vocabularies in accordance with the ISO standard thesaurus norms. It allows for import and export of data as simple text files or in SKOS format. The tool was installed in the cloud testlab of LoCloud and then adapted to the project’s needs. These needs concerned mainly the implementation of a simplified administration and import facility. Additionally, we implemented a note extension that allows storing coordinates for place names. Two new web service [5] calls “import” and “linkTerm” were added to the extensive list of already available TemaTres web services. For the import process of multilingual vocabularies a new identifier for concepts was introduced, connecting automatically the terms in the various languages. This identifier also carries the link information of the publicly published vocabulary and is later added to the metadata during the enrichment process.
How the LoCloud vocabulary services may be used.
Use Case 1 – Automatic usage through the generic enrichment service.

The LoCloud microservice “Generic enrichment” automatically receives the vocabularies available in the vocabulary tool and uses them during the automated enrichment process conducted in the MORe aggregator.

To date around 30 vocabularies are imported to the vocabulary tool. A list of these vocabularies can be found here  > LoCloud Vocabularies.

Figures 2 and 3 show how the microservices Vocabulary and Vocabulary Matching (available currently for  English and Spanish language) are added to a data source enrichment plan in the the MORe aggregator.

vocabo2-3_campaign

vocabo4_campaign

Figure 4 – Use Case 1 – Automatic usage through the generic enrichment service

Consequently, the EDM metadata is checked against the LoCloud vocabularies and where appropriate the vocabulary links are added.

Use Case 2 – Match vocabulary terms in the aggregator.

vocabo5_reference
Figure 5 – Selection of individual vocabulary terms for a Subject Collection in MORe

Instead of matching the vocabularies automatically to the metadata it is possible to selected individual terms from the LoCloud vocabularies that should be added to the metadata. These selections are called Subject Collections in MORe.

Use case 3 – Use the vocabularies in your local cataloguing systems via web services

vocabo6_reference
Figure 6 – Integration of a vocabulary to a local cataloguing tool via webservice

One of the benefits of microservices is that these small and independent services may be plugged in into other application and can be used via the APIs [6] they provide. Figure 6 depicts an example of a local cataloguing system for music archiving that calls the vocabulary web service in the metadata field “Genre(s)”. The technical information on how the services can be integrated is published on the LoCloud online support centre. There are currently fifteen different web service calls for vocabulary integration available.

Use case 4 – The vocabulary experimental application.

The LoCloud vocabularies have been imported into the Tematres tool in order to use them via web services in the aggregation process. But

vocabo7_reference
Figure 7 – LoCloud Vocabularies, Website

Tematres can also be used to create vocabularies from the scratch or to import already existing vocabularies. The advantage of importing vocabularies to the tool is clear: The vocabularies become available in SKOS format with an own web presence and can be used for semantic linking afterwards. With the consent of the vocabulary owner, the vocabulary might then become available for other LoCloud partners in the MORe aggregator. Moreover, the LoCloud Tematres installation allows collaborating online in creating and extending vocabularies (for example in order to create new translations of existing vocabularies).

The vocabulary application is accessible through this link (see figure 7). Please check the LoCloud support portal for information on a test user account which is necessary to access and create a new vocabulary.

 

***

[1]https://support.locloud.eu/MORE 26 January, 2015

[2]http://www.w3.org/2004/02/skos/ 26 January, 2015

[3]http://www.niso.org/schemas/iso25964/ 26 January, 2015

[4]http://www.vocabularyserver.com/ 26 January, 2015

[5]http://en.wikipedia.org/wiki/Web_service 26 January, 2015

[6]http://en.wikipedia.org/wiki/Application_programming_interface 26 January, 2015

Hackathon winner!

MoRe Quality by Vangelis Banos (Future LIbrary – Greece) is the winning prototype application of the LoCloud hackathon which took place on the 11th of February 2015 at the premises of the Google Cultural Institute in Paris, France.The hackathon was organised by LoCloud and Europeana in the context of EuropeanaTech 2015.

Image of Vangelis Banos at the Google culture institute
Vangelis Banos winner of the Locloud hackathon
Vangelis Banos, winner of the LoCloud Hackathon

Concept

The Metadata & Object Repository (MoRe) is an easy and powerful tool to aggregate information and harvest metadata from multiple sources in multiple schemas. Such aggregation schemas usually create problematic situations regarding the quality of the harvested metadata.

Metadata may pass the standard Europeana XML validity tests but they may include problematic metadata values. For instance:

* a dc:date value could be formatted in the wrong way:

<dc:date>approximately 18th century</dc:date>

This format is not correct according to established date formats.

*  an author name could be incomplete according to bibliographic standards.

Example: <dc:creator>Mike</dc:creator>.

* a URL may be invalid. E.g.: <ese:isShownAt>http://invalidurl.com/error-url</ese:isShownAt>

The aim of the MoRe Quality tool is to implement a validation system which could be able to catch these errors and produce useful reports to the collection administrators.

MoRe Quality application –The prototype functionality

MoRe Quality communicates with the LoCloud MoRe instance using a specific user API Key, retrieves the ingested metadata and performs evaluations to identify common errors such as:

* Invalid date formats (ISO 8601 standard)

* Invalid hyperlinks

* Invalid language codes (ISO 639 standard)

* Invalid author names

The results are presented to the user in a simple report.

The application is implemented in such a way that enables developers to add extra evaluation rules in an easy and intuitive way by implementing simple functions – plugins.

Technical information

MoRe Quality is implemented using linux and python 2.7.

Some common python modules are utilised:

* Virtual environments

* Flask

* Python Requests

* BeautifulSoup4

* pycountry

* iso8601

The prototype is not currently running on a production server but the full source code freely available at: https://bitbucket.org/vbanos/more-quality/

Anyone interested in MoRe Quality should feel free to contact the author for more information.

LoCloud and MINT

The MINT service is  a web based platform  designed and developed by  NTUA (the National Technical University of Athens – partner in LoCloud) to facilitate the aggregation of  digital cultural heritage content and metadata in Europe.

The service includes all the various  steps of  workflows, from the ingestion, mapping and aggregation of metadata records to  the implementation of a variety of remediation approaches for the resulting repository. The platform offers  users an organisation management system enabling the deployment and operation of different aggregation schemes (thematic or cross-domain, international, national or regional) and corresponding access rights. Registered organisations can upload (http, ftp, oai-pmh) their metadata records in xml or csv serialisation in order to manage, aggregate and publish their collections.

A reference metadata model serves as the aggregation schema to which the ingested (standard or proprietary) schemata are aligned to. Users can define their metadata crosswalks with the help of a visual mappings editor for the XSL language. The mapping is performed with simple drag-and-drop or input operations, which are then translated into the corresponding code. The mapping editor visualises both the input and target XSD, in an intuitive interface that provides access and navigation of the structure and data of the input schema, as well as the structure, documentation and restrictions of the target one. It supports string manipulation functions for input elements in order to perform 1-n and m-1 (with the option between concatenation and element repetition) mappings between the two models. Additionally, structural element mappings are allowed, as well as constant or controlled value (target schema enumerations) assignment, conditional mappings (with a complex condition editor) and value mappings between input and target value lists. Mappings can be applied to ingested records, edited, downloaded and shared as templates between users of the platform.

Preview interfaces present to users the steps of the aggregation including the current input xml record, the XSLT of their mappings, the transformed record in the target schema, subsequent transformations from the target schema to other models of interest (e.g. Europeana’s metadata schema), and available html renderings of each xml record. Users can transform their selected collections using complete and validated mappings in order to publish them in available target schemas for the required aggregation and remediation steps.

The MINT platform has been deployed for a variety of aggregation workflows corresponding to the whole or parts of the backend services. Specifically, it has served the aggregation of a significant amount of museum content for Europeana through the ATHENA project, that has ingested and aligned to the LIDO format over 4 million items from 135 organisations. The resulting repository offers an OAI-PMH interface presenting the records in the Europeana Semantic Elements schema (ESE). The use of a reference model allowed the rapid support of updated ESE versions that were introduced during  the project (2008-2011), with minimal input from providers. The users’ effort to align their data to an adopted domain model also motivated them to update their collection management systems and improve the quality of their annotations in order to take advantage of a well defined, machine understandable model and, subsequently, control and enrich their organisation’s contribution and visibility through the aggregator and Europeana.

Τhe MINT ingestion platform that is used in the LoCloud project is meant for large-scale ingestion of metadata with the final aim of  delivering  to Europeana a significant content from small and medium cultural institutions. Τhe development of MINT started within the ATHENA project when the NTUA team integrated all the necessary components for ingesting, mapping and publishing metadata to Europeana into a common technology platform, while it evolved through its use in other Europeana-feeder projects such as Linked Heritage, EuScreen, ECLAP, Carare, Europeana Fashion, Europeana Photography and others. The MINT platform provides content holders with the ability to perform the required mapping of their own metadata schemas into LIDO, Carare2.0 and EDM. It enables the ingestion of metadata from multiple sources, the mapping of the imported records to a target metadata schema and the transformation and storage of the metadata in a repository. Although its deployment is also guided by expediency, the system has been developed using established tools and standards and embodying best practices in order to animate familiar content provider procedures in an intuitive and transparent way also for newcomers.

Vassilis Tzouvaras
Senior Researcher
National Technical University of Athens

Cloud computing take-up in the public sector workshop

On 20 May 2014 Walter Koch, AIT, Austria, has presented LoCloud at the Cloud computing uptake in the public sector workshop, held at the European Commission’s premises in Brussels.

The objective of the  Cloud computing uptake in the public sector workshop was to bring together organisations that have expressed clear interest in cloud computing and which could benefit from interaction. The workshop also aimed to provide input for identifying possible future actions in this area.

Below is a short report by Walter Koch (AIT, Austria). All presentations made at the event are available for download from the workshop website.

 

Report of the workshop Cloud Computing Uptake In The Public Sector, Brussels, 20 May 2014

The workshop was introduced by Ken Ducatel, European Commission, Head of Unit: Software and Services.

Mr. Ducatel presentation focused on cloud computing within the framework of the European Commission programme Horizon 2020. He addressed issues such as like Service Level Agreements, Security Services, Standard Terms and pointed out to the final report Establishing a Trusted Cloud Europe. This document presents the results of a collaborative work by public administrations, cloud businesses, and data protection advocates who have joined forces through the European Cloud Partnership in order to establish a roadmap for European leadership in the cloud.

The first presentation at the workshop was made by Dirk van Rooy, Head of Sector European Cloud Partnership, European Commission DG Connect E2, who provided an overview of ICT8-2015: Boosting public sector productivity and innovation through cloud computing services, one of the challenges of the Horizon 2020 Work Programme. This specific challenge is divided into two areas:

a. Pre-commercial procurement for public sector cloud computing services (PCP)

b. Public procurement of innovative cloud computing solutions (PPI)

The PCP and PPI activities have been presented in detail by Lieve Bos, DG Connect Unit F2: Innovation. An overview of PCP and PPI related calls for proposals is included in the presentation (available from the event web site). The EC funding for the next call is € 9 million.

A session on Practical Examples Of Public Sector Cloud Use/Procurement followed. The final session of the workshop included presentations about cloud initiatives in various societal domains. This session was devoted to a series of themes including science, education, European cities, culture and transport.  Europeana was presented by Pavel Kats while Walter Koch outlined some technical aspects of the LoCloud services.  The workshop was mainly focused on procurement of cloud computing services and cloud computing solutions. The system presented by Maryline Lengert and Bob Jones, CERN may be of interest to Europeana. They presented the Helix Nebula System which could provide an infrastructure to Europeana developments and services.

 

Univ.-Prof.em.Dr.Walter Koch
Angewandte Informationstechnik Forschungsgesellschaft mb, Austria