Skip to main content
InstantAtlas Server Logo
InstantAtlas™ Server 6
  Log In

12 March 2014

Greater Manchester Data Synchronisation Programme


We are currently involved in a really exciting piece of work, the Greater Manchester Data Synchronisation Programme (GMDSP), which is looking at the way that public bodies release their data as 5* open data.

Who is involved in this?

The programme is a collaboration between us (Trafford Council), Salford City Council, Manchester City Council, Future Everything, and two catapults - the Connected Digital Economy Catapult, and the Future Cities Catapult. There are plans to incorporate the other seven local authorities in Greater Manchester if phase 1 is successful. Other partner organisations in Greater Manchester are watching developments with interest, such as Greater Manchester Police, and Transport for Greater Manchester. Additionally, the potential impact of the programme has meant that there is national interest - with the Open Data Institute, the Department for Communities and Local Government and the Cabinet Office observing.

From Trafford's point of view, the project is being led by the team behind InfoTrafford, in the Partnerships and Performance Service.

So what is the programme about?

The programme is designed to help us (local authorities) overcome some of the barriers and challenges that we are facing in making our data available in properly open formats. The barriers are usually:

  • Technical - we may have difficulty extracting data from our systems, or transforming it so that it is in a format suitable for re-use.
  • Cultural - often, data is not released because no-one thinks it is of value, or because of misconceptions around data protection or licensing rules, or simply a fear of the unknown.
  • Capacity - many people and organisations do not prioritise the release of data as open data, because it may require significant time or resources to do so.
  • Awareness - the open data ecosystem, while gathering pace and growing, is still a niche area, not interwoven with public services.

How is the programme helping?

The programme is designed to establish networks, both within Local Authorities, and with wider partners and experts, almost as a support group. This has been well-used already, with discussions taking place around the selection of datasets, challenges, opportunities, and solutions.

As well as these networks, the programme has placed a Codefellow in each authority. The codefellows are experts in coding, and transforming and modelling data - critical for releasing the data in a synchronised, linked way. These Codefellows each have brought unique sets of skills to the programme, and are each using different coding styles and software. Trafford is working with Steven Flower, who has enormous experience of open data, and is primarily modelling the data using OpenRefine.

The Codefellows will be preparing toolkits and other resources to allow us to continue to release data in the future. The resources will be freely available, so that the programme is easily repeatable in other areas.

What progress has been made so far?

The Codefellows have been appointed, and placed into the Local Authorities. Six phase 1 datasets have been identified for release:

  • Gritting routes
  • Planning applications
  • Streetlights
  • Recycling centres
  • Allotments
  • Council tax bandings

The datasets were selected by the local authorities, in conjunction with Future Everything, based on a range of criteria.

In Trafford, we have handed all data to the Codefellow for modelling and transformation

A joint datastore has been commissioned and set up to hold the data, with various access routes.

What will come out of the programme?

The key outcomes from the programme centre around understanding and overcoming the challenges that are faced in identifying and releasing open data.

There will be, however, a collection of datasets that cover the geographical area of Manchester, Salford and Trafford, which will be able to be used, along with other linked datasets, such as those provided by the Department for Communities and Local Government. Two events are planned to focus exploration of these datasets by the wider open data community - a data dive on the 24th March 2014, and an innovation challenge on the 29th and 30th March, both organised by Future Everything.

Further down the line, we are hoping to make more and more datasets available in this way. We do not know what will happen off the back of the data, but at the very least, we, the public bodies involved, will be able to use our own data. This will give us a greater understanding of our communities and assets.

Enough of this - what are the technical details?

The joint data store is a quad store, which is essentially a triple store with an extra graph. All data is stored as a collection of triples, in a subject-predicate-object construct.

For more information, we would suggest you maybe start here. Swirrl are the company who are developing and hosting the quad store. They are also providing support on the principles of linked data, and hosting a fourth, 'technical', codefellow.

The three modelling and transformation tools being used are:

  • OpenRefine
  • Ruby
  • Python

Sounds good - how do I find out more?

We will be blogging from the local government side at InfoTrafford

You can also follow our updates on this, and other stuff on twitter

There is a programme website at GMDSP, though as at 12/03/2014 21:24, it is not live.

Future Everything has a project site here

You can find out about the two catapults here: Connected Digital Economy and Future Cities

Anything else, please feel free to contact us...