If your (public sector) organization considering launching an open data process, the question where all this open data would be coming from, might have been raised at some point. There are two ways to look at this. The one way is to ask where the actual data is originating from. Where is it produced, collected, who hands it over to those responsible for publishing them. The other way is to look at it from a process and content management perspective and ask where that open data – provided it exists and has been prepared for publishing – is coming from as a data source within your system and how it will reach the public (i.e. some form of client infrastructure, like a computer screen).  This second way to look at the question is interesting because it – correctly – assumes an IT-based solution for your organization in place that closely mirrors how your organization looks like in the real world. This means the data is available in some location within your organization already, and thus also located within your IT system, and now has to make its way to the location from where it is delivered to the public (an RSS feed for example) via an interface. Several things need to be clarified for this to happen: The delivery format, the update mechanism, file size, who signs the release of the data, how can it be ensured that the data is not tampered with, and how well does the system scale? Open Data only fulfills the criteria of open data advocates if it is raw data, unprocessed, not person-related and machine-readable. The challenge is for public institutions to disseminate the data in the required forma as structured datasets (such as tables, XML feeds, etc.) in a timely fashion, in some applications even in real-time. Especially large amounts of data require a proper back office mechanism to ensure that the provision of the open data sets a) does not overburden the organization or employees tasked with normally other work and b) can happened routinely. In order to create the ecosystem that open data can enable, the data also needs to have a real open API so that civil society or private sector entities can build applications on top of the data, from real time traffic data to crime statistics. Open Data as an application of e-government requires a complex technical process in place as well as an existing organizational infrastructure that is ideally completely electronic already. Media breaks will run counter to the idea of raw data, and real time data, and it would be illusory to demand that kind of advanced open government trickery from institutions that handle the rest of their work on paper still. The licensing and legal problems are a different challenge that also need to be sorted out prior, as well as the financial issues.

Related links: Open Knowledge Foundation, Open Data Network, Open Government Data

Related posts:

1. What Open Government Can Do For You
2. Open Government through open interfaces
3. The new face of Enterprise Content Management
4. From E-Government to Cloud-Government?