Kurt's Weblog

Processing and visualizing {ship/sensor} data - the matched set

In case it wasn't clear, I'd like to re-iterate: These talks are not meant to be about ships. We want to you to think about YOUR data when watching these videos. If your data is AIS, then that's a lucky match, but we don't expect it. We want YOU to go forth, process lots of data and build amazing visualizations and analyitics systems that meet your needs (be they local or global). We think that AIS ship tracking makes a great example use case with many interesting stories that can be told with the data that give us chances to demonstrate techniques and strategies.

Brendan Kenny: "Google I/O 2013 - Google Maps + HTML5 + Spatial Data Visualization: A Love Story"

Francesc Campoy, Kurt Schwehr and Mano Marks: "Google I/O 2013 - All the Ships in the World: Visualizing Data with Google Cloud and Maps"

Jenifer Austin Folkes - Google I/O 2013 - Dive Into Underwater Street View

Amy Unruh and Kim Cameron: "Behind the Data Sensing Lab - Gathering, Processing and Analyzing Data at Scale using the Google Cloud Platform"

Massimo di Stefano - Ipython Notebook for GIS

Lightning Talk - Massimo di Stefano - Ipython Notebook from PyData on Vimeo.

UK Hydrographic Office 5 minute video

Thanks to GeoGarage for linking to this 5 minute video about the UK HO.

Cat nap

Cleaning out the email this morning, I ran into older images from our web cam. The rough life of our cat... we'd had him for 8 months at this point. Terrible quality with this webcam, but it was cheap and uses wifi.

All the ships in the world at Google I/O

We are still in the process of getting the talk videos out, but here is a start. I'll flush out this list as the related videos are published.

Google I/O 2013 - All the Ships in the World: Visualizing Data with Google Cloud and Maps

Google Cloud Platform - Google App Engine, Google Compute Engine, Google Cloud Storage, Google BigQuery, Google Cloud SQL and our many APIs (Maps, Terrain and many more).

Related Maps:

Related Cloud:

Google I/O 2013 - Behind Data Sensing Lab - Gathering, Processing, Analyzing Data using Google Cloud

I would like to thanks SpaceQuest / Analyze and NOAA SBNMS for providing such exciting data for the demonstrations!

SpaceQuest / Analyze

NOAA Stellwagen Bank National Marine Sanctuary (SBNMS)

Navigation Aids in Portsmouth, NH

Tug and barge going under the SF Bay Bridge

Our talks on ships at Google I/O are tomorrow.

Google API python client for fink on Mac OSX

To go with Google I/O and our Cloud Infrastructure talk "All the ships in the world," I've added oauth2client, uritemplate, gflags, and google-api-python-client to fink on Mac OSX 10.7/10.8. This lets you use OAuth2 to access Google REST APIs.

For example:

from apiclient.discovery import build
import apiclient.errors

from oauth2client.client import flow_from_clientsecrets
from oauth2client.file import Storage
from oauth2client.tools import run

storage = Storage('taskqueue_oauth.dat')
credentials = storage.get()
if not credentials or credentials.invalid:
  flow = flow_from_clientsecrets('client_secrets.json',
      scope='https://www.googleapis.com/auth/taskqueue',
      message='can not find client secrets')
  credentials = run(flow, storage)
http = credentials.authorize(httplib2.Http())
service = build('taskqueue', 'v1beta2', http=http)
tasks = service.tasks()

print tasks.list(project='s~my-appeng-project', taskqueue='myqueue')

You can now install bigquery and gsutil (Google Cloud Storage) tools from fink.

fink selfupdate
fink install bigquery-py27 gsutil-py27

There were a number of packages that I had to add to fink behind the scenes and I updated boto.

fink list -i oauth2client gsutil bigquery crcmod google-api-python boto socksipy google-apputil uritemplate

 i   bigquery-py27           2.0.12-1          BigQuery database access
 i   boto-py27               2.9.2-1           Python interface to Amazon Web Services
 i   crcmod-py27             1.7-1             Cyclic Redundancy Check
 i   google-api-python-clie  1.1-2             Access Google APIs via oauth2 and discovery
 i   google-apputils-py27    0.3.0-1           Google command line utility helpers
 i   gsutil-py27             3.29-2            Google cloud storage access
 i   oauth2client-py27       1.1-1             Client library for OAuth 2.0
 i   socksipy-py27           1.02-1            TCP interface to SOCKS4, SOCKS5 or HTTP proxy
 i   uritemplate-py27        0.5.2-1           RFC6570 up to level 4

/sw/bin/bq version
This is BigQuery CLI v2.0.12

/sw/bin/gsutil version
gsutil version 3.29

SF Fire Boat

The San Francisco fire department has their fire boat parked right in front of the Google SF offices. I had to wait for afternoon light, so I could capture a decent image.

Pre-Google I/O ships - Hornblower hybrid ferry

I don't know anything about the effeciency of these hybrids, but they have an interesting look with their wind and solar power setups. This image is from a trip several of us made to Angel Island and Alcatraz last month.