Tag Archives: open data

Posted - 19-09-2018

Collections / Digitisation / News / Research

Aberystwyth shipping records

Creating linked open data for Victorian shipping registers

Volunteers at the National Library of Wales have been transcribing 19th century shipping records for Aberystwyth and these are now being shared openly on Wikidata by the Library’s National Wikimedian.

 

For the first time it is now possible to visualise and query this rich historical record giving us a glimpse of life in 19th century Aberystwyth.

 

In the 18th and 19th Century the Welsh ‘interior’ was not easy to reach. Before the coming of the train and the invention of tarmac, the best way to get goods in and out of West Wales was by boat. Shipping was a booming industry in towns and villages along the West Wales coast and Aberystwyth was no exception. Records for more than 500 ships registered in Aberystwyth survive at the National Library of Wales and Ceredigion County Archive.

 

Aberystwyth Harbour by Alfred Worthington

 

Volunteers at the National Library began transcribing the Aberystwyth shipping records in 2012. The data they extracted contained information about the ships, their crew and the voyages they undertook.

 

In 2016 the library began to explore the possibility of enriching some elements of the data using Wikidata as a platform to share this data. If you are unfamiliar with Wikidata, it is part of the Wikimedia family of websites, which includes Wikipedia, and is a massive database of free to use data. It isn’t even six years old but it already contains 50 million data items about all sorts of places, people, things and concepts, all added by volunteers and organisations wishing to share their data with the world. The library’s Wikimedian collaborated with Ceredigion County Archives, who held additional information about the ships in order to create linked data about the ships themselves. This data included details such as the type and size of each ship, the date and location of construction and, where known, their fate.

 

From this, we were able to begin digging around in the data, and creating revealing visualisations. If you wanted to see the most popular names for ships registered in Aberystwyth, for example, we can easily retrieve and present this information. A map of where the ships were built revealed some interesting facts too. As you might expect, many ships were build locally in Aberystwyth, Borth and Aberdyfi, for example, but the data also reveals that dozens of ships were built in Canada. A little more research revealed that the government of the day was so concerned about a French invasion that they deliberately established ship building yards in safer lands, such as Prince Edward Island off the Canadian Coast, in order to safeguard the ability to move good around the uk by boat.

Word map of most popular ship names

 

Left; ships built in eastern Canada. Right; Ships built in Aberystwyth and Aberdyfi

 

We were also able to plot all the shipwrecks mentioned in the records. This not only highlights the perils of 19th century shipping, but reveals how ships from West Wales villages were traveling the world. From India, China and Africa to South America and even the South Pole, Welsh sailors were very well traveled.

 

The location of shipwrecks recorded in the shipping records

 

After the initial transcription work, many of the volunteers who had worked on the collection were keen to do more, to collect more information about the ships, their crew and their owners, so in 2017 a series of new tasks were set. Volunteers began searching for photographs and paintings of the ships, investigating the fate of more of the vessels, recording the owners of each vessel and they began the mammoth task of researching the lives of every ship’s master mentioned in the records.

 

Whilst the task of identifying all the ships masters will take some time yet, the first of the tasks  has now been completed. Data about the owners of each ship exists in the original shipping records, but was not within the scope of the initial project, so two of the volunteers who worked on the original project, Lilian and Myfanwy kindly went back through the records, and other sources such as the Crew List Index Project and extracted the the data. Much of this has now been incorporated with the rest of the data for each ship on Wikidata. Apart from providing an easy way to search and explore the data held within the collection the improved Wikidata allows us to query and visualize the data in new ways, which helps us better understand what these records tell us.

 

The new data now means that for many ships, we can chart its ownership throughout its life on the seas. We have also been able to create data items for each of the ships owners, be they individuals or established shipping companies. We know where the companies were based, and where individuals lived, and we know, from their names whether they were men or women.

 

For example we know that of the 630 owners identified, 47 were women. More research would be need, but at first glance it would appear that most of those 47 took ownership following the death of their husbands.

 

The records show how the ships often changed hands regularly. If we take the rather appropriately named ‘Volunteer’ we can plot a chart which shows all of its owners, the other ships those people owned, and the other owners of those ships – painting a complex picture of the business of ship ownership in West Wales. And it should be stated that the 630 owners identified will, in many cases, simply be the majority shareholders, or the appointed owner/manager. Many of these ships had multiple shareholders, meaning people from many walks of life could afford to invest in the busy shipping trade.

 

Owners of the ‘Volunteer’ with other connected ships and their owners

 

We can also see who the big players were in Aberystwyth by querying ship owners by the number of ships they owned. Thomas Jones, an Aberystwyth shipbuilder comes top of the pyle, owning more than 20 vessels at one time of another.

 

Ship owners, ordered by the number of ships they have owned

 

Timeline showing the ships owned by Thomas Jones

 

Wikidata, like Wikipedia, is a platform which anyone can edit so any one can now help to improve the data. If they spot mistakes, or have extra information it can be easily added directly to Wikidata. Our volunteers are still working hard to collect even more data so the amount of data connected to the Aberystwyth Shipping records will continue to grow over the coming months and years. Everyone is free to explore and reuse the data, so for the technically minded among you, please feel free to hack, create, mash and re-work our data, and be sure to share the results with us!

 

Jason Evans

National Wikimedian

Tags: , , , , , ,

Posted - 27-06-2018

Collections / Digitisation / News / News and Events / Research

Welsh Portrait Collection

4800 Welsh portraits added to Wikimedia Commons and Wikidata

Over the last 4 years the National Library of Wales has worked with Wikimedia to provide open access to more than 10,000 public domain images. These include the Welsh Landscape Collection, photographs, maps and manuscripts.

 

This partnership has led to more than 455 million views of Wikipedia articles containing National Library images to date.

 

Images

Now the Library is pleased to announce that nearly 5000 portrait prints, photographs and paintings have been placed in the public domain on Wikimedia Commons.

 

Along with the images, the Library’s National Wikimedian has also shared rich metedata for every image as linked open data on Wikidata.

 

The Library’s main goal in releasing such content is to increase access to our collections and to contribute to the creation and sharing of knowledge about Wales and its people.

 

It is now hopped that the Wikimedia community will begin to use these images to illustrate Wikipedia articles. The National Library also plans to run a project to increase engagement with this collection, and hopes that volunteers will be encouraged to create Wikipedia articles about the Welsh sitters, artists, printers and photographers involved in the collection.

 

Because all these images are freely downloadable and in the public domain, we also encourage others to reuse them for any purpose they see fit, from education to the creative industries this is a free resource for everybody.

Data

The creation of linked data for the collection also offers interesting opportunities for researchers and academics. For the first time we can properly disambiguate (untangle) the names of the artists and sitters in order to better understand the makeup of the collection. For example 12 different individuals named John Jones have been identified in the collection, and we now know who they all are, and many are now connected via Wikidata to Wikipedia articles or Dictionary of Welsh Biography entries.

 

We can query and visualize the data in a number of ways using a Sparql query service. For example, we can analyze which engravers copied works by specific artists, and we can see the most frequently depicted types of people (clerics, by a country mile) and features, such as coats of arms, and border decoration.

visualisation of the data showing which printers copied work by certain artists
Visualization of the most frequently depicted things in the collection

We can easily visualize the sitters who appear most in the images using Wikidata’s ‘Main subject’ property. General Thomas Picton, a Welsh born war hero is depicted most often, with 32 portraits. Interestingly his Wikipedia article reveals he was not such a great hero after all, having been convicted of abusing women.

Visualization of the most frequently depicted sitters

We can also explore the collection chronologically and a first look reveals a clear correlation between the popularity of certain types of portrait and historical events. For example the number of images of preachers and clergymen increase dramatically at times of Religious revival.

A timeline of the most frequently depicted things in the collection over time

Language

Wikidata is a multilingual platform, so it also allows us to utilize the multilingual nature of Wikidata’s descriptive labels to view our data in dozens of languages. The Metadata held by the library for this collection was only available in English, however, by converting it to Wikidata 83% of the 40,000 data items were automatically available in Welsh, thanks to the work of Wikidata volunteers, who have added Welsh language labels to many Wikidata items. We hope to engage with Welsh speaking volunteers in order to make 100% of the data available in Welsh.

 

Linking our heritage

Another advantage of sharing our data on a public platform like Wikidata is that many other institutions have done the same, and this means that we can begin to build an extensive network of connected data. The data allows us to connect our own collections together, so for example we can see which publishers have published works in both the Welsh Portrait Collection but also the Welsh Landscape Collection. We have also been able to quickly identify over 400 portraits of people featured in the dictionary of Welsh Biography, and we are now connecting those portraits to the Welsh Biography Website.

All images by one publisher. Blue denotes images in the Welsh Portrait Collection and yellow shows images published by the same publisher which now form part of the Welsh Landscape Collection

Beyond our own institution, we can see which of our sitters also have portraits at the National Portrait Gallery, and we can identify the artists and sitters in our collection who have an Oxford Dictionary of National Biography entry. In this way the worlds cultural heritage can be connected together to provide the public with easy access, in one place, to a rich and diverse range of sources.

 

Jason Evans, National Wikimedian

Tags: , , , , , , , , , ,

Posted - 20-04-2018

#LoveMaps / Collections / Digitisation / Research

Old Periodicals, a New Datatype and Spiderfied Query Results in Wikidata

Several years ago the National Library of Wales appointed the Worlds first Wikidata Visiting Scholar. The volunteer, Simon Cobb, has worked with the library ever since to share, enrich and explore the library’s data using Wikidata – a massive open access linked data repository which contains tens of millions of pieces of data on just about every subject imaginable. In this guest blog, Simon presents his recent work, using open library data on periodicals, publishers and printers in conjunction with the latest Wikidata visualization tools. Jason Evans, National Wikimedian.

Recent developments in Wikidata have made it possible to display more items from the National Library of Wales’ collections on a map. A cluster feature in the Wikidata Query Service map view has enhanced geolocation data visualisation and the new geoshape datatype provides access points for content discovery.

Previously, the display of geolocated images was hampered by only one item per coordinate location being shown on the map. Since a SPARQL query can return multiple results with an identical location this was never an ideal situation. The problem is, in fact, inherent to linked data because every item with a specific relationship to a place will appear at exactly the same point on the map. This is due to the coordinates being attached to the place rather than each individual item.

If the item is an image of Aberystwyth Castle, it depicts the castle and, conversely, the castle is depicted by the image. This is a semantic relationship between the subject (image) and object (castle). The location of the castle is recorded as latitude and longitude coordinates, and thus a query to show on a map the location that the image depicts will use these coordinates. Other images that depicts Aberystwyth Castle will also have these exact coordinates.

 

The marker cluster plugin was implemented to address this problem and it is now possible to view two or more items with the same coordinates. Nearby map markers are grouped using an animated clustering functionality to display an increasing number of clusters, with fewer markers in each, as one zooms in closer. The real gamechanger, however, is the spiderfied markers. Rather than having a solitary marker at a shared location, multiple markers now spiral outwards from a central point, with legs being used to retain their attachment to the precise location and thus show all items in situ.

 

Spiderfied markers of periodicals published in Carmarthen. Unexpanded clusters and single markers are also shown. Each marker contains the periodical’s title, place of publication and cover image (if available). This map is available at: http://tinyurl.com/ycnjsylv.

 

Carmarthen is an important town in the early history of printing in Wales. Some twenty-eight titles in the Welsh Journals and Welsh Newspapers Online digital collections were printed in the town, with twenty-six being first issued before 1900. These periodicals have the same place of publication (i.e. Carmarthen), and thus appear on the map at their shared coordinates. The markers are spiderfied, colour coded according to decade of publication and arranged in a chronologically ordered spiral, starting in the centre with the earliest publication.

In late 2017, a new geoshape datatype was implemented in Wikidata. As the name suggests, it is for storing geographic information in a manner that will produce shapes on a map. Geoshape data can be a single marker at a specific point, a line between two or more points, or a shape, known as a polygon, which is the area enclosed by a point-to-point line, traversing at least four points, with the first and last point being identical. A line or shape is created by structuring geocoordinates to represent the relationship between a series of points; somewhat like a dot-to-dot puzzle. Additional data about a place, such as the address, postcode, website or Wikipedia article, can be attached to a geoshape.

 

The National Library of Wales was the UK’s first Wikidata geoshape. The popup information box includes an image, link to the NLW Wikipedia article and SPARQL queries of the Library’s collections. Markers indicate features of interest and provide images from different vantage points.

 

Since a geoshape can contain multiple points, lines and polygons, it is possible to store data about a group of related locations like, for example, buildings with a similar use. This is a great way to visualise the booktrade locations in Wales that have been added to Wikidata.

Multipolygon geoshape data showing book trade locations in Carmarthen, past and present. These locations and those in other parts of Wales are stored as a single geoshape in Wikimedia Commons (see https://commons.wikimedia.org/wiki/Data:Book_trade_in_Wales.map).

Whilst the book trade locations geoshapes are relatively simple and, therefore, not burdensome to create manually, others, like, for instance, those of castles depicted in the Welsh Landscape Collection, are much more complex. The intricate geoshapes that are formed by sections of perimeter wall between towers or bastions and surrounding the bailey and keep of a castle can be slow and fiddly to make but, luckily, we can draw on linked data instead. A link is forged when an OpenStreetMap feature is tagged with a Wikidata ID and this enables a SPARQL query to retrieve OpenStreetMap data about Wikidata items in the results. Such linking can make a large amount of existing geoshape data accessible.

 

Geoshapes of buildings shown in the Welsh Landscape Collection prints shown in the historic counties of Wales. The buildings are OpenStreetMap features, tagged with the qid of a Wikidata item returned by a SPARQL query, imported as geoshape data and combined with historic county geoshapes from Wikimedia Commons.

 

Geoshapes of buildings shown in the Welsh Landscape Collection prints shown in the historic counties of Wales. The buildings are OpenStreetMap features, tagged with the qid of a Wikidata item returned by a SPARQL query, imported as geoshape data and combined with historic county geoshapes from Wikimedia Commons.

The results of a SPARQL query to retrieve images from the Welsh Landscape Collection can be visualised on a map with geoshapes to represent the building depicted. Shown above, the results for Tintern Abbey are spiderfied to expand a cluster of images with an identical geocoordinates from a single point within the geoshape. Previously, it was only possible to display one of these images on the map view results.

 

Simon Cobb, Wikidata Visiting Scholar at the National Library of Wales

Tags: , , , , , , , ,

Categories

Search

Archives

About this blog

A blog about the work and collections of the National Library of Wales.

Due to the more personal nature of blogs it is the Library's policy to publish postings in the original language only. An equal number of blog posts are published in both Welsh and English, but they are not the same postings. For a translation of the blog readers may wish to try facilities such as Google Translate.

About the blog