Category: GIS

Obtaining Chicago Transit Authority geodata

October 12, 2010 / / 0 Comments

A reader asked where they could get Chicago Transit Authority (CTA) data I didn’t already have on the “Find GIS data” page. I only had shapefiles for train lines and stations. Now I’ve got bus routes and stops.

You can download General Transit Feed Specification (GTFS) data from the CTA’s Developer Center. It’s updated regularly when service changes.

Screenshot from ESRI ArcMap showing the unedited shapes.txt file loaded via Tools>Add XY Data. Shapes.txt is an 18 MB comma-delimited text file with thousands of points that can be grouped together with their shape_id.

The GTFS has major benefits over providing shapefiles to the public.

  1. It can be easily converted to the common shapefile format, or KML format.
  2. Google, the inventor of GTFS, has defined and documented it well; it is unencoded and plaintext. These attributes make it easy for programmers and hackers to manipulate it in many ways. (see also item 4)
  3. Google provides a service to the public on its website, an easy to use and robust transit planning service.
  4. The data is stored as plaintext CSV files.
  5. While an agency like CTA may have a geodata server on its intranet, it is less likely it has the addons that provide mapping and geodata services for the internet. A server like Web Mapping Service, or ArcIMS. These systems can be expensive to purchase and license. And we all know how the CTA seems to always be in a money crunch. While the CTA updates its GTFS data for publishing to Google Maps, the public can download it simultaneously to always have up-to-date information, providing the same geodata that ArcIMS or WMS would offer but for no additional cost.

I couldn’t have pulled off this conversion in 24 hours without the help of Steven Romalewski’s blog, Spatiality. He pointed me to the right ArcMap plugin in this post about converting the Metropolitan Transportation Authority’s GTFS data into shapefiles. I hope Steven doesn’t move to Chicago less my authority on GIS and transit be placed in check!

Make your own map of the CTA train routes and perform some kind of analysis – then share it with the rest of us!

Read more about my exercise in geodata conversion in the full post.
Continue reading

How to convert GTFS to GIS shapefiles and KML

October 12, 2010 / / 41 Comments

This tutorial will teach how you to convert any transit agency’s General Transit Feed Specification (GTFS) data into ESRI ArcGIS-compatible shapefiles (.shp), KML, or XML. This is simple to do because GTFS data is essentially a collection of CSV (comma separated values) text files (really, really large text files).

Note: I don’t know how to do the reverse, converting shapefiles or other geodata into GTFS data. I’m not sure if this is possible and I’m still investigating it. If you have tips, let me know.

Converting GTFS to GIS shapefiles

Instructions require the use of ArcGIS (Windows only) and a free plugin called ET GeoWizards GIS for any version of ArcGIS. I do not have instructions for Mac users at this time.

I wrote these instructions while converting the Chicago Transit Authority’s GTFS files into shapefiles based on a reader’s request. “Field names” are quoted and layer names are italicized.

  1. Download the GTFS data you want. Find data from agencies around the world (although not many from Europe) on GTFS Data Exchange.
  2. Import into ArcGIS the shapes.txt file using Tools>Add XY Data. Specify Y=lat and X=lon
  3. Using ET GeoWizards GIS tools, in the Convert tab, convert the points shapefile to polyline.
  4. Select the shapes layer in the wizard, then create a destination file. Click Next.
  5. Select the “shape_id” field
  6. Click the checkbox next to Order and select the field “shape_pt_sequence” and click Finish.
  7. Depending on the number of records (the CTA has 466,000 shapes), it may take a while.
  8. The new shapefile will be added to your Table of Contents and appear in your map.
  9. Import the trips.txt and routes.txt files. Inspect them for any NULL values in the “route_id” field. You will be using this field to join the routes and trips table. It may be a case that ArcGIS imported them incorrectly; the text files will show the correct data. If NULL values appear, follow steps 10 and 11 and continue. If not, follow steps 10 and 12 and continue. This happens because ArcGIS inspected some of the data and determined they were integers and ignored text. However, this is not the case.
  10. Export the text files as DBF files so that ArcGIS operates on them better. Then remove the text files from the Table of Contents.
  11. (Only if NULL values appear) Go into editing mode and fix the NULL values you noticed in step 9. You may have to make a new column with a more forgiving data type (string) and then copy the “route_id” column into the new column. Then continue to step 12.
  12. Join routes and trips based on the field “route_id” – export as trips_routes.dbf
  13. Add a new column to shapes.shp called “shape_id2”, with data type double 18, 11. This is so we can perform step 14. Use the field calculator to copy the values from “shape_id” (also known as ET_ID) to “shape_id2”
  14. Join routes_trips with shapes into routes_poly based on the field “shape_id” (and “shape_id2”)
  15. Dissolve routes_poly on “route_id.” Make sure all selections are cleared. Use statistics/summary fields: “route_long,” “route_url.” Save as routes_diss.shp
  16. Inspect the new shapefile to ensure it was created correctly. You may notice that some bus routes don’t have names. Since these routes are well documented on the CTA website, I’m not going to fill in their names.

Click on the screenshot to see various steps in the tutorials.

Converting GTFS to KML

After you have it in shapefile form, converting to KML is easy – follow these instructions for using QGIS. Or if you want to skip the shapefile-creation process (quite involved!), you can use KMLWriter, a Python script. Also, I think the latest version of ArcGIS has built-in KML exporting.

Converting GTFS to XML

If you want to convert the GTFS data (which are essentially comma-separated value – CSV – files) to XML, that’s easier and you can avoid using GIS programs.

  • First try Mr. Data Converter (very user friendly).
  • If that doesn’t work, try this website form on Creativyst. I tested it by converting the CTA’s smallest GTFS table, frequencies.txt, and it worked properly. However, it has a data size limit. (User friendly.)
  • Next try csv2xml, a command line tool. (Not user friendly.)
  • You can also use Microsoft Excel, but read these tips and caveats first. (I haven’t found a Microsoft application I like or think is user friendly.)

How to geocode a single address in QGIS

October 3, 2010 / / 0 Comments

Since the last time I wrote about how to use BatchGeocode.com to perform pseudo-geocoding tasks in QGIS, there have been considerable improvements in the multi-platform, free, and open source GIS software. Now, geocoding (turning addresses into coordinates) is more automatic, albeit difficult to setup. (Okay, this has been around June 2009 and I just found out about it in October 2010.)

Once you install all the components, you’ll never have to do this again.

This method can only geocode one address at a time, but it will geocode all of the addresses into a single shapefile.

  1. Download QGIS.
  2. Download and install Python SetupTools. This includes the easy_install function that will download a necessary Python script, simplejson. On Mac you will have to use the Terminal (Applications>Utilities). Email me if you run into problems.
  3. Install simplejson. In the command line (Terminal for Mac; in Windows press Start>Run>”cmd”>Enter), type “easy_install simplejson”.
  4. Download the GeoCode plugin by Alessandro Pasotti via QGIS>Plugins>Fetch Python Plugins. You may have to load additional repositories to see it.
  5. Install geopy. In the command line (like step 3), type “easy_install geopy”.
  6. Specify your project’s projection in File>Project Properties.
  7. Get a Google Maps API key and tell the GeoCode plugin about it (QGIS>Plugins>GeoCode>Settings). You will need a Google account. If you don’t have your own domain name, you can just enter “google.com” when it asks for your domain.
  8. Geocode your first address by clicking on Plugins>GeoCode>Geocode. Type the full address (e.g. 121 N LaSalle Street, Chicago, IL for City Hall).
  9. The geocoded address will then appear in your Layers list as its own shapefile. All addresses geocoded (or reverse geocoded) in this project will appear in the same layer (therefore same attribute table).

Once you install all the components, you’ll never have to do this again. Geocoding will be available each and every time you use QGIS in the future on that workstation.

Tips

  • When you’re done geocoding,  save your results as a shapefile (right click the layer and click “Save as shapefile”). Twice I’ve lost my results after saving the project and quitting QGIS. When I reopened the project, the results layer was still listed, but contained no data.
  • Add a “name” column to the GeoCoding Plugin Results layer’s attribute table (toggle editing first). You can then type in the name of the building or destination at the address you geocoded. Edit the layer’s properties to have that name appear as a label for the point.

A map I made with QGIS showing three geocoded points of interest in Chicago. Data from City of Chicago’s GIS team.

 

Trying out new GIS software

October 2, 2010 / / 0 Comments

I want to draw 50 and 120 feet buffers around the points of store entrances to show where bike parking should and shouldn’t be installed. I want to follow this example:

walgreens with bike parking buffers

Aerial photo of a Tucson, Arizona, Walgreens showing the location of existing bike parking and two buffers (50 and 120 feet) where proposed city rules would allow bike parking. I advocate for ratifying the 50 feet rule, which I’ve discussed on this blog and elsewhere many times.

I want to do this easily and accurately, so I will use GIS software to create a “buffer.” I use QGIS occasionally, but I want to try out other Mac-friendly applications. I’m getting my orthoimagery (geometrically corrected aerial photography) from the United States Geological Survey (USGS) using a web protocol called Web Map Server. I’m trying:

  • Cartographica, $495, with free trial license.
  • uDig, completely free software. UPDATE: I have had NO success getting any data to load from a WMS connection into uDig. I would like to understand why. Cartographica can obtain some of the WMS-stored data I want, although it messes up often.

I’m having success with neither – both are having issues downloading or maintaining a connection to the USGS orthoimagery. In one case, Cartographica trims the Bing Maps imagery to match the extent of my other objects (the buffer). In another case, it won’t even download the USGS imagery (and gives no indication that anything is happening). uDig hasn’t been able to download anything so far – I hope it’s asking for the current extent, instead of all data because it’s taking a looong time to do anything (so long that I just quit in the  middle of it).

This screenshot shows how to add new WMS connections to Cartographica.

UPDATE: I did it! I successfully used Cartographica (and the integrated Bing Maps) to create this drawing that shows the current (abysmal) bike parking at a Chicago Home Depot outside the 50 feet line.

Igniting the discussion on equity

February 20, 2010 / / 0 Comments

I want to have more conversations about transportation equity

My master’s project is all about it. You might have read me talking about it a little here two weeks ago. A then I shot off a post with some key quotes I’m using about the topic in my project.

The purpose of the map is to show the difference in distribution between 2008 and 2009.

This post, though, is all about the graphic above. A lengthy conversation has begun in the comments on the Flickr page. I want more people to get talking about why 2008 might look the way it does, and why 2009 looks the way it does. Perhaps you need a little background on 2009: I made sure to visit the most underserved Wards you see in 2008 and ensure they receive new bike racks in 2009.

A big question is why people in those areas aren’t asking for bike racks. Does no one there ride a bike to the store? Or maybe they do but don’t know how to request a bike rack or know the purpose of one? Maybe they got a bike stolen and need some tips on proper locking.

Those are all questions I want my project to answer – and I’m working hard 20 hours per week to answer them! But I want more questions. I want ideas that point me to look in new directions. If you don’t like my response, tell me.

Bike parking is almost always mentioned in nationwide bike plans as a necessary way to complete the urban bicycling network. Mia Birk, “famous” bicycle planner, and principal at Alta Planning and Design in Portland, Oregon, says that bike parking is part of “the tool kit for successful 
bicycle infrastructure in cities.” Another Portland entity is aware of equity: BikePortland.org.

What’s going on here? Photo by Eric Rogers.