Tag: geodata

How to make a map of places of worship in Cook County using OpenStreetMap data

August 7, 2015 / / 1 Comment

The screenshot shows the configuration you need to find and download places of worship in Cook County, Illinois, using the Overpass Turbo website.

If you’re looking to make a map of churches, mosques, synagogues and other places of worship, you’ll need data.

  • The Yellow Pages won’t help because you can’t download that.
  • And Google Maps doesn’t let you have a slice of their database, either.
  • There’s no open data* for this because churches don’t pay taxes, don’t have business licenses, and aren’t required to register in any way

This where OpenStreetMap (OSM) comes in. It’s a virtual planet that anyone can edit and anyone can have for free.

First we need to figure out what tag people use to identify these places. Sometimes on OSM there are multiple tags that identify the same kind of place. You should prefer the one that’s either more accurate (and mentioned as such in the wiki) or widespread.

The OSM taginfo website says that editors have added over 1.2 million places of worship to the planet using amenity=place_of_worship.

Now that we know which tag to look for, we need an app that will help us get those places, but only within our desired boundary. Open up Overpass Turbo, which is a website that helps construct calls to the Overpass API, which is one way to find and download data from OSM.

Tutorial

In the default Overpass Turbo query, there’s probably a tag in brackets that says [amenity=drinking_fountain]. Change that to say [amenity=place_of_worship] (without the quotes). Now change the viewport of the map to show only the area in which you want Overpass Turbo to look for these places of worship. In the query this argument is listed as ({{bbox}}).

The map has a search bar to find boundaries (cities, counties, principalities, neighborhoods, etc.) so type in “Cook County” and press Enter. The Cook County in Illinois, United States of America, will probably appear first. Select that one and the map will zoom to show the whole county in the viewport.

Now that we’ve set the tag to [amenity=place_of_worship] and moved the map to show Cook County we can click “Run”. In a few seconds you’ll see a circle over each place of worship.

It’s now simple to download: Click on the “Export” button and click “KML” to be able to load the data into Google Earth, “GeoJSON” to load it into a GIS app like QGIS, or “save GeoJSON to gist” to create an instant map within GitHub.

*You could probably get creative and ask a municipality for a list of certificates of occupancy or building permits that had marked “religious assembly” as the zoning use for the property.

Converting shapefiles to GeoJSON, and other format conversions

November 12, 2012 / / 0 Comments

To develop the Chicago Bike Map app, I had a problem I thought would be simple to solve: load train lines into a Leaflet-powered map. I had the train lines stored as a polyline shapefile but Leaflet can only read the GeoJSON format or a string of geographic coordinates representing lines.

I eventually found a solution (I can’t remember how) and I need to share it with you. The converter can do more than ESRI shapefiles to GeoJSON. It can reproject the data in the conversion. It can convert from several formats to several other formats.

The site is called MyGeodata Converter. You upload a ZIP file of geographic files – .shp and its companion files (.prj, .dbf, .shx), .kml, and .gpx. Let’s take the Chicago Transit Authority train lines shapefile straight from the City of Chicago’s open data portal. It downloads as a zipped collection of a shapefile and its buddies and we can take this file straight to the Converter and upload it. The Converter will unzip it and read the data; it will even identify the projection system (for Chicago-based geographic data, its common to use NAD83 Illinois StatePlane East FIPS 1201 Feet (SRID 102671, the same as SRID 3435).

The Converter will convert to one of the following formats, with same or new projection; accepts SQL statements to extract a subset of data:

  • ESRI shapefile
  • GML
  • KML, KMZ
  • GeoJSON
  • Microstation DGN
  • MapInfo File
  • GPX
  • CSV

How I created a map of Illinois Amtrak routes in TileMill in less than 30 minutes

October 22, 2012 / / 2 Comments

This interactive map was created for a Grid Chicago article to show the cities and Amtrak routes mentioned. Click and drag it around or hover your mouse on the red train station markers. 

Want to create a map like that and publish it on your own website? It’s easy. I’ll show you how to do it in less than 30 minutes. First, download the following files:

All shapefiles are from the United States Department of Transportation, Bureau of Transportation Statistics’s National Transportation Atlas 2012 edition except for Illinois places, which comes from the Census Bureau’s TIGER project.

At the end of this tutorial, you’ll have a good introduction on how to find geographic data, build a map with TileMill, style the map, and publish it for the public. Your map will not look like mine as this tutorial doesn’t describe how to add labels or use the hover/info feature.

Tutorial to make Amtrak Illinois map

  1. Unzip the four ZIP files you downloaded and move their contents into a folder, like /Documents/GIS/Amtrak Illinois/shapefiles. This is your project folder.
  2. Install TileMill and open it.
  3. Set up a project. In the Projects pane, click “New Project”. In the filename field, title it “amtrak_illinois”. Ensure that the checkbox next to “Default data” is checked – this shows a world map and helps you get your bearings (but it’s not absolutely necessary).
  4. Get familiar with TileMill’s layout. Your new project will open with the map on the left side and your Carto style code on the right side. There are four buttons aligning the left edge of your map. From top to bottom they are: Templates, Font list, Carto guide, and Layers.
  5. Add a layer. We’re going to add the four shapefile layers you downloaded. Click the “Layers” button and then click “Add layer”. In the ID field, type in “amtrak_routes”. For Datasource, browse to your project folder and find “amtrak.shp” – this file has the Amtrak route lines. Then click “Done”. Click “Save & Style”.
  6. Style that layer. When you click “Save & Style” after adding a layer, your attention will be called to the Carto style code on the right side of TileMill. A section of code with the “amtrak_routes” #selector will have been inserted with some default colors and styles. If you know CSS, you will be familiar with how to change the Amtrak routes line styles. Change the “line-color” to “#000”. After “line-color”, add a new line and insert “line-opacity: 0.5;”. This will add some transparency to the line. Press the “Save” button above the code.
  7. Add remaining layers. Repeat Step 5 and add 3 more layers: “amtrk_sta.shp” (ID field: “amtrak_stations”), “state.shp” (ID field: “states”), and “tl_2012_17_place.shp” (ID field: “illinois_cities”).
  8. Hide bus stations. The Amtrak stations layer shows bus and ferry stations as part of Amtrak’s Thruway connections. You probably don’t want to show these. In your Carto style code, rename the #selector from “#amtrak_stations” to “#amtrak_stations[STNTYPE=’RAIL’]”. That makes the following style code only apply to stations with the “rail” type. Since there’s no style definition for things that aren’t of that type, they won’t appear.

Screenshot of my map.

Prepare your map for uploading

TileMill has many exporting options. You can save it as MBTiles and publish the map for free using MapBox (TileMill’s parent), or you can export it as image files (but it won’t be interactive), or you can display the map using the Leaflet JavaScript map library (which I use for the Chicago Bike Map app). This tutorial will explain how to export MBTiles and upload to MapBox, the server I’m using to display the map at the top of this page.

  1. Change project settings. To upload to MapBox, you’ll have to export your project as MBTiles, a proprietary format. Click the “Export” button above your Carto style code and click “MBTiles”. You’ll be asked to provide a name, description, attribution, and version. Input appropriate text for all but version.
  2. Adjust the zoom levels. Adjust the number of zoom levels you want (the more you have the longer it takes to export and upload your project, and you might exceed MapBox’s free 50 MB account limit). My map has zoom levels 8-11.
  3. Adjust the bounds. You’ll then want to draw your bounds: how much of the map’s geographic extents you want to export. Zoom to a level where you can see the entire state of Illinois in your map. Hold down the Shift key and drag a box around the state, plus a buffer (so viewers don’t fall of your map when they pan to the edges).
  4. Export your map. Click Export and watch the progress! On a four-year-old MacBook it took less than one minute to export the project.
  5. Bring the export to your project folder. When export finishes, click the “Save” button and browse to your project folder. Click the file browser’s save button.
  6. Upload to MapBox. Login to MapBox’s website and click “Upload Layer”. Browse to your project folder, select the .mbtiles folder, and click “Upload file”. Upon a successful upload, your map will display.
  7. Embed it in your website. Click the “Share” button in the upper left corner of your map and copy the embed code. Paste this into the HTML source code of a webpage (or in a WordPress post) and save that (I’m not going to provide instructions on how to do that).

Now you know how to find geographic data, build a custom map using the TileMill application, begin to understand how to style it, and embed your map for the public on a website or blog.

N.B. I was originally going to use QGIS to build a map and then publish a static image before I realized that TileMill + MapBox (the website) can build a map but publish an interactive feature instead of a static image. I’m happy I went that route. However, I did use QGIS to verify the data and even create a new shapefile of just a few of the key train stations on the Lincoln Service (the centerpiece of my Grid Chicago article).

TransportationCamp: Real-Time Pedestrian and Bike Location, Session Two

March 26, 2011 / / 0 Comments

Real-Time Pedestrian and Bike Location How can we get it? What can we do with it? How can it not be creepy?
By Eric Fischer.

My summary of the discussion

There are many existing data sources that are published or have APIs that could stand as reasonable proxies for tracking people who are walking, biking, or just ambling around the city – some of this information is given away (via Foursquare) by those who are traveling, and other information is collected in real time (buses and taxis) and after the trip (travel surveys and Flickr photos). I don’t think the group agreed on any good use for this data (knowing where people are in the city right now), nor did the group come up with ways to ensure this collection is not “creepy.”

Eric’s original question involved the location of people bicycling, but the discussion spent more time talking about pedestrians. However, some techniques in tracking and data gathering could be applied to both modes.

See final paragraph for links on “further reading” that I find relevant to this discussion.

Schedule board at TransportationCamp West on Saturday in San Francisco at Public Works SF, 161 Erie Street.

[Ideas and statements are credited where I could keep track of who said what, and if I could see your name badge.]

Eric, starting us off:
We have a lot of information about where motor vehicles (MV) are in cities.
A lot of experience of city is not about being in a MV, though.

How many bikers going through intersection that are NOT getting hurt.
Finding places where people walk and where people’ don’t.

Where do people go on foot and on bikes?
As far as I know this isn’t available

Foursquare has benefits (awards) so people are willing to give the data, but we don’t want another Please Rob Me.

In SF, there are flash mobs, sudden protests, Critical Mass

Data sources:
-buses – boarding and deboarding – you can get a flow map from this. Someone said that Seattle has this data open.
-CTPP (Census Transportation Planning Package)
-city ped count
-Eric: Where people get on/off taxis.

“CycleTracks” – sampling bias, people with iPhones
-70% of handheld devices are feature phones, not smart phones. So there’s another sampling bias.

Opt-in factor
How do you sample?

SF Planning Dept. had a little program or project ask people to plot on a map your three most common walking routes.
What is your favorite street, and where do you not like to walk?

Eric: My collection tool is Flickr. Geotags and timestamps.
flickr.com/walkingsf

Magdalena Palugh: Are there incentives for commuting by bike? There are incentives for people who vanpool.
If there is incentive, I would gladly give up my data.
Michael Schwartz (SFCTA, sp?) What is difference <> SFCTA/MTA?

-If part of this is to get at where the trouble spots are, could you have people contribute where the good/bad parts are? “This overpass really sucks.”

Tom: Can you get peds from aerial images?
-Yes, but there’re too many limitations, like shade, and tree cover. Also, aerial images may be taken at wrong time (for a while the image of Market/Castro was during festival).

Brandon Martin-Anderson: What strategies have you tried so far?
-aerial images
-Flickr/Picasa location
-Street View face blur (a lot false positives)
Anything you plot looks kind of the same.

People like to walk where other people are. For safety reasons. -Good point on real-time basis.
Eric: Not a lobbying group for peds.
Eric: Find interesting places to go.
Richard: We need exposure data.

Paris bike sharing report showed that “Cycling is faster on Wednesdays.”
Europeans more open to sharing their private details – possibly because of stricter regulation on what agencies can do with the collected data. (There was a little disagreement on this, I personally heard the opposite).

Andrew: Can we use something like Xbox Kinect to track these people?

National Bike/Ped Documentation Project – same format
Seattle – 4 different groups that do annual bike counts. UW bike planning studio.

Who pays for this?
-Transportation planners pay for this.
-Private development projects (from contractor).
-Universities, NSF, Google
-Community groups –

Further reading

People

Mike Fleisher – DS Solutions
Andrew – @ondrae – urbanmapping.com

Notes to self

Is Census question about commuting about time or distance of “most traveled” mode?
Splunk – data analysis tool
What is difference <> SFCTA/SFMTA?

Obtaining Chicago Transit Authority geodata

October 12, 2010 / / 0 Comments

A reader asked where they could get Chicago Transit Authority (CTA) data I didn’t already have on the “Find GIS data” page. I only had shapefiles for train lines and stations. Now I’ve got bus routes and stops.

You can download General Transit Feed Specification (GTFS) data from the CTA’s Developer Center. It’s updated regularly when service changes.

Screenshot from ESRI ArcMap showing the unedited shapes.txt file loaded via Tools>Add XY Data. Shapes.txt is an 18 MB comma-delimited text file with thousands of points that can be grouped together with their shape_id.

The GTFS has major benefits over providing shapefiles to the public.

  1. It can be easily converted to the common shapefile format, or KML format.
  2. Google, the inventor of GTFS, has defined and documented it well; it is unencoded and plaintext. These attributes make it easy for programmers and hackers to manipulate it in many ways. (see also item 4)
  3. Google provides a service to the public on its website, an easy to use and robust transit planning service.
  4. The data is stored as plaintext CSV files.
  5. While an agency like CTA may have a geodata server on its intranet, it is less likely it has the addons that provide mapping and geodata services for the internet. A server like Web Mapping Service, or ArcIMS. These systems can be expensive to purchase and license. And we all know how the CTA seems to always be in a money crunch. While the CTA updates its GTFS data for publishing to Google Maps, the public can download it simultaneously to always have up-to-date information, providing the same geodata that ArcIMS or WMS would offer but for no additional cost.

I couldn’t have pulled off this conversion in 24 hours without the help of Steven Romalewski’s blog, Spatiality. He pointed me to the right ArcMap plugin in this post about converting the Metropolitan Transportation Authority’s GTFS data into shapefiles. I hope Steven doesn’t move to Chicago less my authority on GIS and transit be placed in check!

Make your own map of the CTA train routes and perform some kind of analysis – then share it with the rest of us!

Read more about my exercise in geodata conversion in the full post.
Continue reading