Category: Tools

My essential QGIS plugins

April 18, 2011 / / 0 Comments

Plugins for QGIS I use most often.

All of these can be installed automatically by QGIS. Click on Plugins>Fetch Python Plugins. Then search for the plugin, click on its name, and click Install Plugin. Few plugins require a restart.

  • MMQGIS – Great for working with CSV files; also merges layers (even if they have differing attributes); has various other useful functions, including converting string data to float data. Has Voronoi diagram function (takes a long time to process).
  • fTools – Replicates some of the most basic geographic tools in ArcGIS, like Clip, Dissolve, and Reproject. Can also add X/Y values to point attribute tables that are missing them (if you want latitude/longitude, you must reproject into a coordinate reference system first, like WGS84 [EPSG: 4326]). Unfortunately, there’s little information on what each fTools function does. Below are descriptions:
    • Extract Nodes – Create a point at each intersection of vertices.
    • Basic Statistics – Generate arithmetic statistics for fields, same as statistics function in ArcGIS. Great for quickly understanding the extent of values in a field (especially numeric values), like mean, max, min, standard deviation, and number of unique values.
    • Nearest Neighbour Analysis – More details here.
    • Geoprocessing Tools>Dissolve – Combine features based on a shared attribute. For example, all features with an identical STREET_TYPE be combined into a single feature. For example, all “Avenues” will become one feature and all “Boulevards” will become a second feature. Only works on polygon layers.
    • Descrição em português
  • Table Manager
  • Open Layers – Embed Google, Yahoo, Bing, and OpenStreetMap layers in your map. See my example.

Two new beta apps for Chicago geography

April 12, 2011 / / 0 Comments

I think 2011 is going to be the year of apps for me. I developed one for bikes and one for politics.

Bikes

I made the “Chicago Bike Data Portal” app on Saturday night. It gets your location via the HTML5 geolocation API and loads a list and map of all the reported bike crashes within half a mile. It works on some mobile phones, too (at least Android and iOS). Expect to see more data in the future, like ped and automobile crashes.

Politics

Then on Monday I made an app called, “Who is my Chicago Alderman?” where people can type in their address (or any address) and retrieve the alderman’s name, addresses, and phone numbers. I’ve already identified a problem where the app returns the wrong information for an address. I think this is a problem with the geocoder. Ever since CivicFootprint (a project of Center for Neighborhood Technology) disappeared, the city’s been missing an easy-to-use app for finding alderman. The city clerk’s website takes too long to use and is not as informative.

Other apps I built or worked on

TransportationCamp: Real-Time Pedestrian and Bike Location, Session Two

March 26, 2011 / / 0 Comments

Real-Time Pedestrian and Bike Location How can we get it? What can we do with it? How can it not be creepy?
By Eric Fischer.

My summary of the discussion

There are many existing data sources that are published or have APIs that could stand as reasonable proxies for tracking people who are walking, biking, or just ambling around the city – some of this information is given away (via Foursquare) by those who are traveling, and other information is collected in real time (buses and taxis) and after the trip (travel surveys and Flickr photos). I don’t think the group agreed on any good use for this data (knowing where people are in the city right now), nor did the group come up with ways to ensure this collection is not “creepy.”

Eric’s original question involved the location of people bicycling, but the discussion spent more time talking about pedestrians. However, some techniques in tracking and data gathering could be applied to both modes.

See final paragraph for links on “further reading” that I find relevant to this discussion.

Schedule board at TransportationCamp West on Saturday in San Francisco at Public Works SF, 161 Erie Street.

[Ideas and statements are credited where I could keep track of who said what, and if I could see your name badge.]

Eric, starting us off:
We have a lot of information about where motor vehicles (MV) are in cities.
A lot of experience of city is not about being in a MV, though.

How many bikers going through intersection that are NOT getting hurt.
Finding places where people walk and where people’ don’t.

Where do people go on foot and on bikes?
As far as I know this isn’t available

Foursquare has benefits (awards) so people are willing to give the data, but we don’t want another Please Rob Me.

In SF, there are flash mobs, sudden protests, Critical Mass

Data sources:
-buses – boarding and deboarding – you can get a flow map from this. Someone said that Seattle has this data open.
-CTPP (Census Transportation Planning Package)
-city ped count
-Eric: Where people get on/off taxis.

“CycleTracks” – sampling bias, people with iPhones
-70% of handheld devices are feature phones, not smart phones. So there’s another sampling bias.

Opt-in factor
How do you sample?

SF Planning Dept. had a little program or project ask people to plot on a map your three most common walking routes.
What is your favorite street, and where do you not like to walk?

Eric: My collection tool is Flickr. Geotags and timestamps.
flickr.com/walkingsf

Magdalena Palugh: Are there incentives for commuting by bike? There are incentives for people who vanpool.
If there is incentive, I would gladly give up my data.
Michael Schwartz (SFCTA, sp?) What is difference <> SFCTA/MTA?

-If part of this is to get at where the trouble spots are, could you have people contribute where the good/bad parts are? “This overpass really sucks.”

Tom: Can you get peds from aerial images?
-Yes, but there’re too many limitations, like shade, and tree cover. Also, aerial images may be taken at wrong time (for a while the image of Market/Castro was during festival).

Brandon Martin-Anderson: What strategies have you tried so far?
-aerial images
-Flickr/Picasa location
-Street View face blur (a lot false positives)
Anything you plot looks kind of the same.

People like to walk where other people are. For safety reasons. -Good point on real-time basis.
Eric: Not a lobbying group for peds.
Eric: Find interesting places to go.
Richard: We need exposure data.

Paris bike sharing report showed that “Cycling is faster on Wednesdays.”
Europeans more open to sharing their private details – possibly because of stricter regulation on what agencies can do with the collected data. (There was a little disagreement on this, I personally heard the opposite).

Andrew: Can we use something like Xbox Kinect to track these people?

National Bike/Ped Documentation Project – same format
Seattle – 4 different groups that do annual bike counts. UW bike planning studio.

Who pays for this?
-Transportation planners pay for this.
-Private development projects (from contractor).
-Universities, NSF, Google
-Community groups –

Further reading

People

Mike Fleisher – DS Solutions
Andrew – @ondrae – urbanmapping.com

Notes to self

Is Census question about commuting about time or distance of “most traveled” mode?
Splunk – data analysis tool
What is difference <> SFCTA/SFMTA?

Better bike crash map now available for Chicago

March 14, 2011 / / 0 Comments

I met Derek at a get together for “urban geeks” last Tuesday where he told me he was making a filterable/searchable version of my Chicago bike crash map using the Google Fusion Tables API. It essentially allows you to perform SQL-like queries to show different results on the map than one view. It’s possible to do this yourself if you open the bike crash map in the full Google Fusion Tables interface (do that now).

You can use it now!

New bike crash map, click through to view

Derek’s map has the benefit of great interface to drill down to the data you want. You can select a day, a surface condition, and the injury type. To download the data yourself, you’ll still have to access the full Fusion Tables interface.

Door lane photo and graphic by Gary Kavanagh in Santa Monica, California.

And since the data is the same as my original map, crash reports involving motor vehicle doors are not included. Here’s why doorings are excluded.

Reminder about open data and Obama’s Open Government Directive

March 8, 2011 / / 0 Comments

Quickly after taking office, President Obama issued a memorandum about open government and opening government data. Then came the Open Government Directive* which said:

To the extent practicable and subject to valid restrictions, agencies should publish information online in an open format that can be retrieved, downloaded, indexed, and searched by commonly used web search applications. An open format is one that is platform independent, machine readable, and made available to the public without restrictions that would impede the re-use of that information.

Essentially, the executive government (er, Obama Administration) adopts the presumption of openness, that distributing public data is the default position and action to take.

Don’t squat on the data. Don’t fret over how people will view or manipulate the data – this is not your concern. Don’t delay its release. If you do this, you are a frigid dataist and I will remember this.

Photo of visual note taking at an open data seminar by Karen Quinn.

*The Directive has a little more backbone than the original memorandum: “This memorandum requires executive departments and agencies to take the following steps toward the goal of creating a more open government.”

Thank you to Tech President.