Musings on Business and Tech

Category — location

Mapping Transit Delays With Ushahidi

MTA Delays Crowdmap

I woke up at 5 this morning to the news that the two subway lines in my neighborhood were still not running, more than 36 hours after the “Boxing Day Blizzard” had begun in New York City. The question then was, well, what is running?

The awful MTA site wasn’t much help, especially with regard to the buses. It offered cryptic and non-specific messages like:

Due to ongoing snow related conditions, all MTA bus express services are running with system wide delays. There is no limited stop bus service in all boroughs.

There was no indication of how any one specific bus route was faring.

My neighborhood’s Yahoo group was abuzz with conversations about which trains and buses were and weren’t running, but the information was unorganized and freeform, because it was happening over email.

And while the MTA probably had a good internal grasp of which lines were having issues, they were not doing a good job of disseminating that information. Ideally their website should have been providing real-time geo-located incident reports so that any commuter could look at a map and quickly determine what the best route was to wherever they had to go. Even more ideal would have been, as my friend Michael McWatters suggested, a trip planner that could re-route you away from the suspended lines and to the freely-moving ones. But, at the very least, even just a little bit more detail would’ve been nice.

So this morning I thought to myself, this is exactly what crowdsourcing is good at. And I remembered hearing about a mapping tool called Ushahidi, which was put to good use during the crisis that followed the Haitian earthquake back in January. Indeed crisis mapping is a very powerful idea, and Ushahidi is leading the charge with their open source application that’s free to download and deploy, as well as with Crowdmap, their hosted version of Ushahidi that is also, surprisingly, free.

In the span of about an hour, I put up a site up using Crowdmap called I entered all the subway service changes from the MTA site, and told a few people about it. It got a little bit of Twitter buzz, but only one person submitted a report other than me. I think I was a little late to the game (I should’ve set it up on Sunday), but, it turns out, the tool also has a few shortcomings specific to this particular use case.

First, the tool was built for incidents whose geography is best described as a point (a latitude/longitude coordinate pair). But transit delays are best described as lines. When service is disrupted, there is a starting coordinate and ending coordinate for that event. Ushahidi had no good way of representing this, so I ended up just putting in two points for every incident. It was kind of a hack, and it also was misleading–the issue itself spanned an entire length of subway or bus line, not just the endpoints. I imagine that anyone who might’ve submitted a report would have run into the same issue, and I also suspect there are some incidents that are best described by a polygon instead of a point or a line.

Similar to the incidents being specified by a point in geographical space rather than a line, they are also represented by points in time rather than durations. Transit delays have a finite duration (even if the duration isn’t known up front). I would love if you could set incidents to expire after a certain amount of time (24 hours maybe?) rather than requiring an admin to go back in to the system and either edit or delete the incident. The overhead can be prohibitive.

Another issue is that it’s somewhat difficult to submit reports. You have to visit the website and submit a form. Actually that’s not really true–Ushahidi supports reporting via Twitter hashtags, SMS, or mobile apps (though there isn’t an iOS app yet). These are decent options, but you don’t really get the good geo-location data this way. It’s probably only a matter of time before the mobile options for Ushahidi reporting get really good, but for now it’s a bit clunky.

Despite these issues, though, it’s really interesting to see how far this kind of technology has come. It’s also interesting to think about how many different mature platforms Ushahidi is built on: there’s Linux, Apache, MySQL, PHP, the Google Maps API, the Twitter API, SMS, Email, RSS, probably many others. It’s pretty staggering when you think about it, and all I really had to do to set it up was press a “submit” button on a web page.

Even though the Haiti earthquake was a big moment in the spotlight for Ushahidi, I think we have yet to hear the last of them. They are building an amazing tool and I’m excited to see how it can evolve and continue to help communities deal with local crises and civic emergencies.

Related articles

December 28, 2010   1 Comment

The Next Phase is Not Web 3.0

O’Reilly Media’s Web 2.0 Summit, which took place over the last few days in San Francisco, got me thinking, why is the web still only in version 2.0?

Tim O’Reilly himself coined the phrase Web 2.0 back in 2004 for his first conference of the same name. It was defined by an evolution in front end technologies like AJAX and bubble letters, back-end technologies like web services and RSS feeds, and business models like crowdsourcing and software as a service.

So given that we’re 6 years in to Web 2.0, when will we get to Web 3.0? The answer is never. No one will ever start calling it Web 3.0. For one thing, it’s not catchy. Web 2.0 has a certain ring to it that Web 3.0 doesn’t. Also, I think it will be difficult for people to come to a consensus on when technologies have evolved enough to move to a new version number. Web 2.0 was coined by a single person. Web 3.0 would have to be more organic. We much more likely to describe future “versions” of the web in descriptive phrases rather than numbers.

Tim Berners-Lee has always been against this nomenclature anyway. His alternative to “Web 2.0″ was the “Read/Write Web,” because of the way in which users became empowered to contribute en masse to the data on the internet. And in 2006, when asked what Web 3.0 would be, he said that a component of it would be “The Semantic Web,” or “a web of data that can be processed directly and indirectly by machines.” In other words, a web in which the machines can glean meaning from the data, in addition to simply manipulating it.

But I would argue that we are already at the next evolution of the web, and yet it’s not about semantics. It’s about context. This new phase of the web has largely been catalyzed by two breakthroughs: advances in the power and reach of mobile computing, as well as what Mark Zuckerberg calls “the social graph.” Both of these lend not meaning but context to data, and that is a very powerful thing.

Mobile devices can contextualize data around locations, photos, video, and audio (among other things). And of course the social graph connects data to people. The “Internet of Things,” as it continues to grow, will increasingly connect data to objects (shall we call it the “object graph?”). Although context is a step in the direction of semantics, we are still a ways away from getting machines to the point where they can interpret meaning from this data.

Indeed the “web” isn’t even about machines anymore. What was once a network of machines connected by wires is now a network of people, places and things connected by context. There is a new network growing atop the old.

Perhaps the semantic web will come in version 4.0 (although we still won’t call it that). But I think the best characterization of the most recent evolution of the web is the “Contextual Web” (I am not the first to call it such). Twitter, Facebook, Foursquare, the iPhone, Android, and many other prominent technologies can fall under this term, and I think it best describes the current proliferation of mobile and social technology that is spawning so many new and interesting businesses.

Enhanced by Zemanta

November 18, 2010   2 Comments

What You Should Be Doing Instead of Checking In

I use Foursquare a lot. You could say I’m part of the passionate but niche group that checks in at least a couple of times per week (more like a couple of times per day).

The odd part of it is that I can’t really tell you why I do it. Is it for the badges and mayorships? Not really–for all the talk of “game mechanics,” these things are mostly pretty lame. Is it for the specials I can get from local retailers? No, there aren’t enough of those available yet. Is it because of the serendipitous encounters I can have with friends? No. Having two young children precludes that quite a bit.

So if I’m not doing it for any particular reason, maybe I should spend some of my check-in time doing something productive.

Enter CloudMade’s Mapzen POI Collector. This iPhone app exists for one purpose, and one purpose only: to add and update points of interest to the open-source geo database OpenStreetMap–the Wikipedia of geography.

I realized today that instead of always checking in everywhere I go, I could earn a lot more karma points (if not badges and mayorships) by entering and editing points of interest everywhere I go. (By the way, if you want to search points of interest, don’t use this app; use something like the Open Maps app.)

Why the karma? The data I contribute using the Mapzen app is open and licensed under the Creative Commons SA license, so it can be freely and easily used in myriad applications that are competing with closed platforms such as Foursquare and Yelp.

So from now on I’m going to try and do my part to make the world a better place–instead of checking in on Foursquare, I’m going to spend that time making OpenStreetMaps so good it’ll give Google, Foursquare and Facebook all a run for their money.

Related articles by Zemanta

Enhanced by Zemanta

September 12, 2010   1 Comment

More Google Coolness: In Stock Nearby?

Two months ago, I wrote on this blog:

I want an app that will show me all the stores near me that not only carry a particular product, but also let me sort by price and filter by availability…I realize this is not a trivial task, as it would entail mining tons of inventory data from an insane number of stores, many of which aren’t even tracking their inventory, and those that do are using a wide variety of systems. But here’s where the prediction comes in: this would be a perfect challenge for a company like Google to take on (I mean, hey, they mapped the freaking Earth, Moon and Mars already; no job is too big for them.)

Yesterday, Google announced “In Stock Nearby,” a new feature they added to Product Search for mobile.

We’re happy to announce that as of today, if you’re searching for a product that is sold by participating retailers, including Best Buy, Sears, Williams-Sonoma, Pottery Barn, or West Elm, you can just look for the blue dots in the search results to see if it’s available in a local store. If you see a blue dot, you can tap on the adjacent “In stock nearby” link, and you’ll be taken to the seller’s page where you’ll see whether the item is “In Stock” or has “Limited Availability” near you.

Official Google Mobile Blog: In stock nearby? Look for the blue dots.

Reblog this post [with Zemanta]

March 12, 2010   No Comments

Google Maps Get Biking Directions!

Great news for cyclists everywhere–but especially in New York City where bike lanes are crucial for the safety of urban bikers: Google Maps has finally added biking directions to their platform.

The LatLong blog has an interesting post about how the routing algorithm differs for bicycles than for cars, feet or public transit. The directions favor bike trails (Google has added 12,000 miles of them to its system), bike lanes, and recommended routes (human-curated). It also avoids both uphill and downhill slopes, busy roads and busy intersections.

Interestingly, when the routing algorithm considers all the different routes, it analyzes the slopes on the trip, calculates the effort and speed required to go up hills and factors these calculations into the route it recommends. “Sometimes the model will determine that it’s far more efficient to make you ride several extra blocks than to have to deal with a massive hill.”

Here’s a map of the bike lanes in lower Manhattan and Brooklyn. Great stuff Google!

Via Google LatLong: It’s time to bike.

Reblog this post [with Zemanta]

March 11, 2010   No Comments

Location Privacy Goes to Washington

Nice roundup yesterday from ReadWriteWeb on the recent congressional hearing regarding location based services and end user privacy. Key takeaways:

  • Most location-based services have privacy controls that are too limited, and privacy policies are not readily accessible
  • Some are recommending that the Electronic Communications and Privacy Act be updated to include protections for location information
  • “There is as yet no data on teens’ mobile social mapping or LBS use,” but texting is huge among kids so it is likely that mobile location sharing will grow in that age group

Location Privacy Goes to Washington.

Reblog this post [with Zemanta]

March 9, 2010   No Comments

Twitter Will Support Places, Not Just Coordinates

According to the Twitter API Announcements list, Twitter is rolling out a whole new set of geotagging features that will be hugely important in the location-based services space:

our goal is to provide a few more options to API developers (and the users they are servicing) through this contextual information.  people, we find, inherently want to talk about a “place”.  a place, for a lot of people, has a name and is not a latitude and longitude pair.  (37.78215, -122.40060), for example, doesn’t mean a lot to a lot of people — but, “San Francisco, CA, USA” does.  we’re also trying to help users who aren’t comfortable annotating their tweets with their exact coordinates, but, instead, are really happy to say what city, or even neighborhood, they are in.  annotating your place with a name does that too.

Presumably these features are a direct result of Twitter’s acquisition of Mixer Labs late last year. In any case, this goes a long way to solving Twitter’s location privacy issues, as users will be a lot more likely to turn on location sharing if they can share their general location rather than a pinpointed latitude and longitude.

Read Developer Preview: upcoming geo features (a.k.a “A place is not just a latitude and a longitude – it has a name”) – Twitter API Announcements | Google Groups.

Reblog this post [with Zemanta]

March 2, 2010   No Comments

Cell phones show human movement predictable 93% of the time

Aside from providing great value to the end-user, location data is also proving hugely valuable to researchers. A new study published in Science magazine analyzed mobile location data from frequent cell phone users and found that human movement is predictable at least 93% of the time.

The authors analyzed various aspects of the information related to the calls, as well as information that could be aggregated over multiple calls: number of distinct locations, historical probability that the location had been visited in the past, time spent at each tower, the order in which customers usually visited towers, and so on.

Customers that stuck to the same six-mile radius had predictability rates of 97 to 93 percent, and this fell off as the typical area of travel grew. But the predictability eventually stabilized, and remained at 93 percent even as the radius of travel rose to thousands of miles. Regardless of how widely they traveled, the researchers could adequately predict their locations, down to the specific tower, 93 percent of the time.

As the Ars article points out, these findings could be used for many different practical purposes: planning traffic flow in cities, tracking the spread of contagious disease, and anticipating cellular network usage patterns. Very interesting, indeed.

Read Cell phones show human movement predictable 93% of the time.

Reblog this post [with Zemanta]

March 1, 2010   No Comments

Please Rob Me

In case it wasn’t yet obvious, I’m quite interested in issues of location-sharing and privacy. Well it seems I’m not the only one. 22 year old Dutch computer science student Frank Groeneveld, along with a couple of friends, recently launched the site, which presents a cheeky take on the issue of location and privacy.

Essentially the site calls the Twitter and Foursquare APIs to publish people’s location updates from those services. And, true to the site’s name, it editorializes these updates by suggesting that, since the user is checking in on these services, they are clearly not home at the moment. Status updates on look like, “Jane Doe left home and checked in 10 minutes ago.”

While a would-be burglar would still need to know the user’s home address to take advantage of this information, I think Please Rob Me really hammers the point home: sharing your location publicly is not the same as sharing some photos on Facebook.

The solution to this, as I’ve said before, needs to be within these services themselves. There needs to be more granularity in the privacy settings, and defaults must be set such that users have to “opt-in” to public sharing.

Foursquare has posted a reasonable response to this brouhaha on their blog. I think for the most part Foursquare gets the privacy stuff right. And I give them credit for acting swiftly to publish their response.

Via Mashable, San Francisco Chronicle

Reblog this post [with Zemanta]

February 19, 2010   No Comments

Bing Maps Astounds

Check out this incredible talk from just last week at this year’s TED Conference. Microsoft’s Blaise Aguera y Arcas demos some of the new features in Bing Maps, including integrations with Photosynth and other augmented reality technologies.

Watching this video, I feel like I can see into the future to where all this technology is going. And it’s exciting indeed:

[ted id=766]

See Blaise Aguera y Arcas demos augmented-reality maps | Video on

Reblog this post [with Zemanta]

February 16, 2010   3 Comments