Tag: open source

Photo of the Adanet sitting on my desk

Say hello to Adanet, my new CO2 concentration monitor

December 16, 2022 / / 0 Comments

I acquired a homemade and open source Adanet carbon dioxide (CO2) monitor from a friend in Chicago and tested it on a short trip on the Brown Line ‘L’. The Adanet monitors the concentration of CO2 in the air, in parts per million, which is a proxy for how “fresh” the surrounding air is.

Monitoring CO2 concentration became a more common activity and point of discussion since the COVID-19 pandemic began. A key way to reduce risk of transmission is to have “fresher” air. I’ll establish that “fresher” air is replacing air that has people’s outgoing CO2 with air that has less CO2, namely outdoor air.

(Another reason to monitor CO2? Excessive CO2 can lead to a decline in cognitive ability and sleep quality.)

I conducted an unscientific test of the “freshness” of the air inside a single Brown Line car on my trip between the Western and Belmont stations. I took five readings, which was the most I could take given that the Adanet refreshes every three minutes.

Map showing gray markers indicating where readings were taken. The trip started at the Western Brown Line station and the last reading was taking just before the train pulled into the Belmont station.

The ambient global measurement of CO2 is 421 ppm, measured in May 2022.

On the transit trip, the lowest reading was 475 ppm, which was taken while the device was in my coat pocket before I boarded the train at the outdoor station.

The highest reading was 680 ppm, when the train car had the most people on it during my short trip.

I have been checking the Adanet since getting home two hours ago.

  • I left it in the hallway outside my apartment and a single reading was 556 ppm.
  • Inside my studio readings have been around 650±20 ppm.
  • The highest reading since I got home has been 830 ppm and this is because I partially burned a quesadilla, releasing additional carbon into the air (my standalone air filter also turned on automatically to deal with the reduction in air quality).
  • I opened the balcony door to let fresh air in and 15 minutes later the reading dropped to between 671 and 692 ppm (the more readings the better).

This is why we need more people editing OpenStreetMap

February 16, 2015 / / 1 Comment

Unmapped homes in the Irving Park community area

These homes were built after the City of Chicago’s building footprints dataset was created (2010?). Ian Dees imported the dataset in 2012. Many of the buildings that you can now see on Bing Maps have not been present on Bing’s satellite imagery since at least 2012.

1. OpenStreetMap is the world’s most complete free map, to which anyone can contribute their “ground truth” data (the location of wells and convenience stores, road names, and whether Lula Café at 2537 N Kedzie Boulevard in Logan Square has outdoor seating).

2. OpenStreetMap is used by thousands of non-profit and non-governmental organizations, corporations, apps, and people daily to locate themselves, locate others, get directions, and find places.

3. Nearly every map is out of date the moment it is published, including online, “current” maps like Google Maps, Bing Maps, their competitors, and OpenStreetMap.

4. Bing Maps provides its satellite imagery to OpenStreetMap editors – you and me – so that we can trace (copy) things on the planet to be things on the map. Google Maps doesn’t allow tracing (copying).

5. Bing updated its satellite imagery for Chicago (and probably a lot of other places) within the last six weeks…and there are hundreds of objects that aren’t yet mapped in OpenStreetMap. In Chicago most of these buildings are newly constructed houses.

Those hundreds of houses now need to be added to OpenStreetMap, with addresses, to complete the buildings collection in Chicago, and to expand the gazetteer (an address book) of places in Chicago.

I’m glad you want to help me do it! Here are two helpful things you can do:

  1. Start tracing the buildings yourself (here’s how new mappers can get started), or
  2. Leave notes at buildings which aren’t yet mapped so that map editors like myself know where to look to trace buildings.

Update: There’s a bonus third thing you can do, and that’s come to the next MaptimeCHI event on Thursday, February 26th, at the Chicago Community Trust (225 N Michigan, 22nd floor). RSVP for Anatomy of a Web Map. The Trust will also provide food and beverages. I’ll be there to teach new mappers and assist generally.

Adding notes is extremely helpful

You can contribute without editing by adding notes describing new things, or identifying problems with existing things. Click the “Add a note” button on OpenStreetMap.org.

Trying out uDig, a free, multi-platform GIS application

February 10, 2011 / / 0 Comments

ArcGIS is the standard in geographic information system applications. I don’t like that it’s expensive, unwieldy to install and update, and its user interface is stymying and slow*. I also use Mac OS X most of the time and ArcGIS is not available for Mac. It doesn’t have to be the standard.

I’ve tried my hand at Cartographica and QGIS. I really like QGIS because there’re many plugins, it’s open source, there’s a diverse community supporting it, and best of all, it’s free. I’ve written about Cartographica once – I’m not a fan right now.

My project

  • The data: Bicycle crashes in the City of Chicago as reported to IDOT for 2007-2009
  • Goal: Publish an interactive map of this data using Google Fusion Tables and its instant mapping feature.
  • Visualizing it: Added streets (prepared beforehand to exclude highways), water features, and city boundary (get that here)
  • Process: Combine bike crash data; reproject to WGS84 for Google; remove extraneous information; add latitude/longitude coordinates; export as CSV; upload to Google Fusion Tables; map it!
  • View the final product

Trying out uDig

In reaching my goal I had a task that I couldn’t figure out how to complete with QGIS: I needed to combine three shapefiles with identical table schemes into one shapefile – this one shapefile would eventually be published as one map. The join feature in fTools wasn’t working so I looked for a new solution, uDig, or “User-friendly Desktop Internet GIS.”

The solution was very easy. Highlight all the records in the attribute table of one shapefile, click Edit>Copy, then select the destination table and click Edit>Paste. The new records were added within a couple seconds. I could then bring this data back into QGIS to finish the process (outlined above under Project). I did use fTools later in the process to add lat/long coordinates to my single shapefile.

After adding more data to better visualize the crashes in Chicago, I noticed that uDig renders maps to look smoother and slightly prettier than QGIS or ArcGIS. See the screenshot below.

A screenshot of the three bicycle crash datasets (2007, 2008, 2009) with the visualization data added.

The end product: three years of police reported bicycle crashes in the City of Chicago on an interactive map powered by Google Fusion Tables, another product in Google’s arsenal of GIS for the poor man. View the final product.

*I haven’t used ArcGIS version 10 yet, which I see and read has an improved user interface; it’s unclear to me and other users if the program’s been updated to take advantage of multi-core processors. ESRI has a roundabout way of describing their support.

How to convert GTFS to GIS shapefiles and KML

October 12, 2010 / / 41 Comments

This tutorial will teach how you to convert any transit agency’s General Transit Feed Specification (GTFS) data into ESRI ArcGIS-compatible shapefiles (.shp), KML, or XML. This is simple to do because GTFS data is essentially a collection of CSV (comma separated values) text files (really, really large text files).

Note: I don’t know how to do the reverse, converting shapefiles or other geodata into GTFS data. I’m not sure if this is possible and I’m still investigating it. If you have tips, let me know.

Converting GTFS to GIS shapefiles

Instructions require the use of ArcGIS (Windows only) and a free plugin called ET GeoWizards GIS for any version of ArcGIS. I do not have instructions for Mac users at this time.

I wrote these instructions while converting the Chicago Transit Authority’s GTFS files into shapefiles based on a reader’s request. “Field names” are quoted and layer names are italicized.

  1. Download the GTFS data you want. Find data from agencies around the world (although not many from Europe) on GTFS Data Exchange.
  2. Import into ArcGIS the shapes.txt file using Tools>Add XY Data. Specify Y=lat and X=lon
  3. Using ET GeoWizards GIS tools, in the Convert tab, convert the points shapefile to polyline.
  4. Select the shapes layer in the wizard, then create a destination file. Click Next.
  5. Select the “shape_id” field
  6. Click the checkbox next to Order and select the field “shape_pt_sequence” and click Finish.
  7. Depending on the number of records (the CTA has 466,000 shapes), it may take a while.
  8. The new shapefile will be added to your Table of Contents and appear in your map.
  9. Import the trips.txt and routes.txt files. Inspect them for any NULL values in the “route_id” field. You will be using this field to join the routes and trips table. It may be a case that ArcGIS imported them incorrectly; the text files will show the correct data. If NULL values appear, follow steps 10 and 11 and continue. If not, follow steps 10 and 12 and continue. This happens because ArcGIS inspected some of the data and determined they were integers and ignored text. However, this is not the case.
  10. Export the text files as DBF files so that ArcGIS operates on them better. Then remove the text files from the Table of Contents.
  11. (Only if NULL values appear) Go into editing mode and fix the NULL values you noticed in step 9. You may have to make a new column with a more forgiving data type (string) and then copy the “route_id” column into the new column. Then continue to step 12.
  12. Join routes and trips based on the field “route_id” – export as trips_routes.dbf
  13. Add a new column to shapes.shp called “shape_id2”, with data type double 18, 11. This is so we can perform step 14. Use the field calculator to copy the values from “shape_id” (also known as ET_ID) to “shape_id2”
  14. Join routes_trips with shapes into routes_poly based on the field “shape_id” (and “shape_id2”)
  15. Dissolve routes_poly on “route_id.” Make sure all selections are cleared. Use statistics/summary fields: “route_long,” “route_url.” Save as routes_diss.shp
  16. Inspect the new shapefile to ensure it was created correctly. You may notice that some bus routes don’t have names. Since these routes are well documented on the CTA website, I’m not going to fill in their names.

Click on the screenshot to see various steps in the tutorials.

Converting GTFS to KML

After you have it in shapefile form, converting to KML is easy – follow these instructions for using QGIS. Or if you want to skip the shapefile-creation process (quite involved!), you can use KMLWriter, a Python script. Also, I think the latest version of ArcGIS has built-in KML exporting.

Converting GTFS to XML

If you want to convert the GTFS data (which are essentially comma-separated value – CSV – files) to XML, that’s easier and you can avoid using GIS programs.

  • First try Mr. Data Converter (very user friendly).
  • If that doesn’t work, try this website form on Creativyst. I tested it by converting the CTA’s smallest GTFS table, frequencies.txt, and it worked properly. However, it has a data size limit. (User friendly.)
  • Next try csv2xml, a command line tool. (Not user friendly.)
  • You can also use Microsoft Excel, but read these tips and caveats first. (I haven’t found a Microsoft application I like or think is user friendly.)

How to geocode a single address in QGIS

October 3, 2010 / / 0 Comments

Since the last time I wrote about how to use BatchGeocode.com to perform pseudo-geocoding tasks in QGIS, there have been considerable improvements in the multi-platform, free, and open source GIS software. Now, geocoding (turning addresses into coordinates) is more automatic, albeit difficult to setup. (Okay, this has been around June 2009 and I just found out about it in October 2010.)

Once you install all the components, you’ll never have to do this again.

This method can only geocode one address at a time, but it will geocode all of the addresses into a single shapefile.

  1. Download QGIS.
  2. Download and install Python SetupTools. This includes the easy_install function that will download a necessary Python script, simplejson. On Mac you will have to use the Terminal (Applications>Utilities). Email me if you run into problems.
  3. Install simplejson. In the command line (Terminal for Mac; in Windows press Start>Run>”cmd”>Enter), type “easy_install simplejson”.
  4. Download the GeoCode plugin by Alessandro Pasotti via QGIS>Plugins>Fetch Python Plugins. You may have to load additional repositories to see it.
  5. Install geopy. In the command line (like step 3), type “easy_install geopy”.
  6. Specify your project’s projection in File>Project Properties.
  7. Get a Google Maps API key and tell the GeoCode plugin about it (QGIS>Plugins>GeoCode>Settings). You will need a Google account. If you don’t have your own domain name, you can just enter “google.com” when it asks for your domain.
  8. Geocode your first address by clicking on Plugins>GeoCode>Geocode. Type the full address (e.g. 121 N LaSalle Street, Chicago, IL for City Hall).
  9. The geocoded address will then appear in your Layers list as its own shapefile. All addresses geocoded (or reverse geocoded) in this project will appear in the same layer (therefore same attribute table).

Once you install all the components, you’ll never have to do this again. Geocoding will be available each and every time you use QGIS in the future on that workstation.

Tips

  • When you’re done geocoding,  save your results as a shapefile (right click the layer and click “Save as shapefile”). Twice I’ve lost my results after saving the project and quitting QGIS. When I reopened the project, the results layer was still listed, but contained no data.
  • Add a “name” column to the GeoCoding Plugin Results layer’s attribute table (toggle editing first). You can then type in the name of the building or destination at the address you geocoded. Edit the layer’s properties to have that name appear as a label for the point.

A map I made with QGIS showing three geocoded points of interest in Chicago. Data from City of Chicago’s GIS team.