Geospatial IoT: Market Highlights in Q1 ’16

iStock_000021333148_720p

​This is an inaugural issue in the series of Planet OS Market Highlights. While this particular post covers the first quarter of 2016, going forward we will be syndicating highlights for you on a monthly basis. ​To receive these market highlights via email, please sign up for the Planet OS newsletter.

The intersection of geospatial services and industrial IoT (IIoT) is heating up: as industrial organizations are transitioning from pure products business to services models, they become responsible for the life cycles of their assets under management. Meanwhile, sensor proliferation continues and unpredictable weather and climate increase our exposure to external risk, demanding reliable programmatic access to environmental data.

Data will dominate the energy market
Zarko Sumic, an analyst at Gartner, predicted that the largest energy company in the world will not own any grids or generation assets by 2020. Instead, it will focus on managing information about energy sources and consumers. This prediction is based on current trends of energy flow being determined by market forces. Sumic said that current regulation is the biggest barrier to this transition and that he does not believe technical barriers will be an issue as this change occurs.
Source

NOAA data production rate to double in two years
NOAA’s October launch of GOES-R next-generation satellite constellation will begin a new level of data production. NOAA estimates that this new system will produce some 100 megabits of data per second, or roughly 9 terabytes of data per day, by mid-2017. They will launch a second system in 2018, doubling its daily production of data.
Source

Six New Datasets, Developer Tools, and Our Own Slack Community

More datasets. Developer Tools. Slack Community

Planet OS has published six new public datasets to our open data catalog, including data from Australia’s Bureau of Meteorology, Pacific Islands Ocean Observing System (PacIOOS), Copernicus Marine Environment Monitoring Service (CMEMS) and NOAA.

The Planet OS API provides access to high-quality datasets from some of the world’s leading environmental data providers. As of today, Planet OS API users can now access the following datasets:

In addition to these datasets, we’ve released a new interactive console that provides detailed documentation about Planet OS API endpoints, parameters, and response structure. This console is a convenient tool for exploring the API, allowing you to quickly construct and send queries directly from your browser.

We’ve also updated our documentation and published a number of Javascript and Python examples that use the Planet OS API. There’s a new GitHub repository that contains a variety of Jupyter notebooks, as well as a javascript demo that allows you to select a point on a map and receive data values from the GFS forecast.

Planet OS Community on Slack

Finally, we’ve started a brand new Planet OS Slack community! Join us as we discuss datasets, feature requests, and future development plans. Request an invitation to become a member. Hope to see you there.

Chris Kalima
Director of Product Development

Simplifying Concurrency: Akka

I recently presented a talk about Akka and it’s usage at Planet OS at our Advanced Java and Scala meetup. Since it is hard to read code from slides and to give a better understanding of how the details work, this blog post contains code samples from a real application that uses Akka. Slides and video of my presentation are embedded in the end of this post.

All code examples are taken from a project where we implemented a dashboard for the operators of the world’s second largest offshore wind farm. More details about the project are in the slides (starting from slide 17). To keep the examples from getting too long, we’ll look more closely at the websocket serving example, which is relatively simple and short.

The server uses spray-can for the main http server and the spray-websocket extension for websocket protocol support.

To start off, we need to create the actor system and the server actor:

package com.planetos.example.ws_server
import akka.actor.{ActorSystem, Props}

object ServerProtocol {
    case object Start
}

object DataUpdateServer extends App  {
    val system = ActorSystem()
    val server = system.actorOf(Props[WebSocketServer], "ws-server")
    server ! ServerProtocol.Start
}

This is a very simple version of the startup sequence, where we:

  • create the actor system
  • ask it to instantiate our WebSocketServer class and create an actor based on that instance
  • send the server a message to start

This is very simple and works, as long as nothing goes wrong. For reactive systems it’s always very important to think about also the cases where things do go wrong and explicitly define what actions need to be taken.

Marinexplore.org is Shutting Down on May 1st

Marinexplore.org is shutting down

Nearly four years ago we set out on a mission to improve the global community’s understanding of our planet’s oceans. We believed that by making ocean data easy to find and access we could positively impact the world and promote informed, data-driven decisions with far reaching political, social and economic benefits.

Along the way we encountered detractors, but we also encountered a far greater number of supporters. People who shared our vision and were eager to lend their expertise and enthusiasm. We worked with startups building autonomous vessels, academic research teams, and national agencies. Each step of the way we continued to learn and evolve.

Through the efforts of everyone involved we built marinexplore.org, which provides data from over 40,000 in-situ platforms to a community of nearly 9,000 professionals. This is a tremendous achievement, but technology has only continued to accelerate during the past four years.

Using the Planet OS REST API with Javascript

One of our goals at Planet OS is to provide consistent and easy access to environmental data that’s scattered around the web in various formats. Before working at Planet OS I had never heard of OPeNDAP, eventhough it’s a well known standard, especially in the oceanographic community.

I’m a datavis designer and product developer working in Javascript, so I prefer data as json from a simple REST API, regardless of the original format that data is in. And that’s exactly what Planet OS provides, hiding the complexity of scientific protocols and file types and delivering the json I need. If you’re curious about how to use the Planet OS API and Javascript in your own applications, here are some tips to get you started.

point_api_demo

An API for the World’s Weather & Climate Data

One API. A World of Data.

Planet OS is excited to announce the launch of our first API designed specifically for geospatial and Earth observation data. The Planet OS API provides developers, researchers, and climate specialists with a new tool for working with Earth data from the world’s most respected providers. Using the Planet OS API, it is now possible to build scalable applications and data-driven solutions without developing custom interfaces for each individual dataset.

Weather and climate data is traditionally difficult to work with at scale. Data are typically buried in undocumented repositories that are hard to navigate, published in obscure scientific formats, and are often so large that downloading them can take hours. The Planet OS API allows users to access only the data they actually need, without having to download the entire dataset.

Global Forecast System (GFS)

Starting today API users can freely access three global forecast modelsNOAA WaveWatch III, NCEP Global Forecast System (GFS), and HYCOM Global Ocean Forecast. Through a mixture of automated discovery, user recommendation, and manual curation, we will continuously expand and update our catalog of available datasets. This includes not only open datasets, but commercial data as well. Planet OS is currently working with a select group of data vendors, including CustomWeather, The Climate Source, TCarta Marine, Weather Decision Technologies, and Freese-Notis Weather to bring their products to our platform.

RWE and Planet OS Sign Strategic Partnership Agreement to Introduce Big Data Infrastructure for Geospatial IoT at the German Power Company

Planet OS and RWE Partnership

RWE and Planet OS first joint project – RWE’s Gwynt Y Mor wind farm in Wales.

RWE (Nasdaq: RWEOY) and Planet OS signed a Strategic Partnership Agreement to build big data infrastructure for geospatial IoT on February 4th in Berkeley, California. The partnership’s first joint project is a next generation dashboard for RWE’s Gwynt Y Mor wind farm in Wales.

“As energy production is becoming increasingly decentralized and more of it will come from renewable sources, we need to start paying close attention to unpredictable weather patterns and other external risk factors the world is facing today,” said Mr. Peter Terium, Chief Executive of RWE Group during a visit to its Californian daughter company.

“Working with data technology developed by Planet OS can help us to monitor and analyze geospatial sensor data in one system, improving operational efficiency, safety and production output. This is an important step for RWE on its way to become one of the leading data-driven and technology-driven utilities in Europe,” continued Terium.

Better Color Palettes with Histogram Equalization in D3.js

Planet OS is handling a lot of climate and environmental data from sensors around the world for multiple projects. As a data visualization engineer, I’m always learning new tricks to makes sense of data. Visualizing data on maps can be challenging. Some colleagues smarter than myself asked me if we could have a color palette using histogram equalization. So I went to Wikipedia.

“Histogram equalization is a method in image processing of contrast adjustment using the image’s histogram.”

So with hints from this example, from this other one, and from this SO answer, I tried to understand it enough to make a version using D3 scales. It turned out to be very simple.

First, to prove that I have it right, I had as a goal to reproduce the example from the Wikipedia page.

Unequalized_Hawkes_Bay_NZ

Original image from the Wikipedia example

Equalized_Hawkes_Bay_NZ

Image contrasted using histogram equalization from Wikipedia

An Overview of Functional Programming and Persistent Data Structures: CTO Tom Faulhaber discusses “Okasaki for Dummies”

A few weeks ago, I had the pleasure of talking to the team at Intertrust Technologies about functional programming. Part of the fun part was the audience: programmers and non-programmers. I wanted to help both groups understand functional programming and why it’s a powerful alternative to traditional models. As a result, this talk is very high-level and filled with pictures. It should be a good introduction to this important concept regardless of your programming background.

To learn more about these ideas and how they are transforming the way we program, you can watch a video of my talk here:

“Measuring to Manage” – A New Food Security Visualization Initiative by Planet OS

Today we are delighted to present a new initiative to create and publish a series of data visualizations that highlight the most alarming environmental trends contributing to the global decrease in food security. The visualizations to be debuted today at the Zoological Society of London’s The future of food – the future of biodiversity? conference in London, United Kingdom will seek to leverage our big data platform for the closer analysis of the global food security crisis. Our partners at Intertrust Technologies will also present their work digging through the data to secure our global food supply.

Environmental trends to be highlighted include how climate change-induced ocean acidification is threatening marine organisms; how abnormal weather events, such as the California drought, are impacting regional food production; and how economic growth affects our natural ecosystems, such as forests. Reductions in global marine organism counts, curtailed regional food production, and reduced forest area are just some of the many trends threatening to exacerbate the global food security crisis.

blog ocean viz