Accessing useful data about the status of our planet’s environment is often a struggle. From with the variety of sources to the diversity of formats and tools out there, journalists interested in using data in their environmental reporting often spend more time finding and downloading the information than actually analyzing it.
Over the past few years, we’ve seen a number of services providing near real-time data come online and in March 2016 Planet OS announced the launch of its first product, the Planet OS API – an new tool created to simplify access to the world’s weather and climate data; designed for all geospatial and Earth Observation data.
We sat down with Planet OS CEO Rainer Sternfeld to discuss how the company achieved this milestone and his vision for future of earth observation.
Why is the creation of Planet OS API important?
Where I come from, I didn’t start my career in data integration, I actually started with product development and machine design. In these fields there is a concept of “the 5 whys“ or what Elon Musk refers to as the “first principles” concept. It starts with a problem:
It is super hard to find and access Earth observation data.
Why? Not enough data has been put out in a machine-readable way.
Why? Well, because the interface for integrating data isn’t very friendly.
Why? The metadata is bad and protocols are too complicated across the formats.
Why? Because the creators of the formats are scientists, who are not trained to create consistent APIs since they have been working on solving their own problems, in silos at big organizations.
Why? The technology wasn’t mature enough to simplify the process and the demand for geospatial IoT has changed only recently.
The last “why” made us realize that APIs need to be solved first. Once we solve the APIs issue and provide reasonable access then everything else will follow because you enable more people to build applications and algorithms on top of this commonly accessible data.
With the announcement of Planet OS API, yes, we are putting out a ton of datasets, but we are signaling is that we are going after a problem that no one has solved—to create consistent APIs for Earth observation data. There is no one size fits all approach but the middle ground is very important.
Can you walk me through the key achievements that enabled you to launch the Planet OS API?
In order to understand how we’ve made this leap forward, we need to look back over the last few years. We began four years ago with Marinexplore.org focusing on ocean data. We developed a specialist community of about 8,000 users including many NOAA and NASA peers who were exploring, visualizing and accessing their public data through our system.
We came to the realization that many business communities other than the ocean community are also in need of big data infrastructure to support their environmental intelligence. So we renamed ourselves Planet OS to expand our services to include the full range of Earth observation data such as weather, climate, and geospatial IoT at large.
In April 2014, we were one of the few selected to be a part of the NOAA Big Data Project because of our experience with both data integration and data discovery. This step brought us into partnership with Amazon Web Services, ESRI, the Weather Company, and others such as IBM, Microsoft and the Open Cloud Consortium. We were one of the smallest company of the batch and are honored to be included.
That collaboration sparked a lot of interest from the private sector and we worked on a series of projects focused on developing an economically feasible means to access only the portions of data needed from large-scale datasets without writing code.
For example, we worked with a company in Brazil to assist an environmental research vessel that was conducting a survey of potential offshore drilling sites to understand if it was safe or not to drill. For this project we streamed their data via satellite connection on their Planet OS cloud-based account where data was refreshed every 3 hours. This meant that managers could look at a timely updated dashboard instead of an occasional Excel spreadsheet to determine where to send the expensive vessel. Now also managers and executives could suddenly participate in decision-making. This saved them a lot of time, and that translates to money.
Another key milestone came this February by signing a strategic partnership agreement with RWE – the 5th largest power company in Europe – to aggregate data for their wind farms. Like all wind farm operators, they too have a challenge on how to manage to their turbines efficiently – a handful of separate systems each with their own data streams are generally not integrated on a software level. Each turbine itself is producing an enormous diversity of data. Then there are weather stations, commercial weather forecast models, and marine vessels all streaming in data. In all, we integrate these streams of data into a secure Planet OS environment and aggregate them in a dashboard designed for operator workflows. The efficiencies generated through data integration and analyses are now putting more electricity on the grid, reducing operating costs and increasing safety.
The common denominator for all these projects was data integration services. Meaning we were able to set up a common cloud-based infrastructure for a diversity of data types ands formats, elegantly combining proprietary, pubic and commercial data on our customers’ Planet OS accounts. Previously, there was no single location where this information was available.
What does an operating system for the planet mean to you?
A useful analogy for it is the nervous system of our own body. If you pinch yourself and you don’t feel anything, you know that’s dangerous. If we don’t have a well-connected planetary nervous system; people won’t know how to respond well to changes in the environment.
Right now, we are seeing the mass proliferation of sensors quantifying many aspects of our planet health. The more sensors that exist the more sensitive we get and in many places around the world we are numb.
What role are sensors playing in Planet OS now?
Sensors are playing a vital role in Planet OS since they are the source of this vast volume of environmental data. In our case, we work with sensors from the sea floor to space. Data is coming from ships, buoys, high frequency radar stations, drones, satellites, and other geospatial IoT devices (like bus traffic).
But in the grand scheme, data in your area of interest is likely still sparse at a global scale, and a planetary nervous system is still in an early stage of development. Ground-based sensors put out by governments and companies add granularity when integrated with global scale open data. For us to be successful, it’s critical to be at the intersection of the digital and physical world. You can’t only stare at a computer screen and understand the planet without also being attuned to what is happening across the Earth’s dynamic living system.
NASA has more than a dozen Earth science spacecraft/instruments in orbit studying all aspects of the Earth system (oceans, land, atmosphere, biosphere, cryosphere), with several more planned for launch in the next few years. SOURCE: NASA JPL
As a journalist, the majority of my experience is with open data. Can you tell me a bit more about the role of commercial data in Planet OS?
Fundamentally, we intend to be the clearinghouse for the most important open and commercial datasets in the weather, climate, and environmental domain. By building Planet OS, we are aggregating data and supporting access so people can use this data with peace of mind. We believe that data is a commodity that should be accessible like electricity, wired into the system to the point where you can flip a switch and turn it on.
There were a number of commercial vendors in our community who were interested in listing their data to gage people’s interest. If someone wants to buy a dataset now, the customer would identify a point of interest and the company would manually clip, zip and ship that information to you typically on a hard drive or an FTP server. If you have to deal with each customer individually, that process is not going to scale.
So far, we’ve signed up 20 commercial data vendors (6 are providing metadata now) that are going to be listing their data in Planet OS. But before we integrate everything we are hoping to understand which datasets people really want to use. Access to commercial data in addition to open data is really important for us. The more open data is available in a friendly way, the more commercial data companies are going to produce unique data products or derivatives that fill in the gaps to address a specific need or area.
