Category: Information

Download GIS data from ArcGIS servers using a simple command

October 21, 2017 / / 0 Comments

A lot of geospatial data (GIS) in the world is stored on ArcGIS MapServers, a part of the Esri “stack” of products that municipalities and other governments use to manage and publish GIS data.

A lot of people want that data, and not every government has enabled the ArcGIS data hub features that make it easier to get. For an example data hub, check out Cook County, Illinois.

If you have ArcGIS software on your Windows computer, then it can be pretty easy to plug in the map server URL and manipulate and extract the data.

For the rest of us who don’t have a very expensive license for that software, you can use a “command line” tool (written in Python) on any computer to download any layer of GIS data hosted on the ArcGIS MapServer and automatically convert it to GeoJSON.

Tutorial

You’ll need to install the Python package pyesridump, from the OpenAddresses GitHub repository, created by Ian Dees and other contributors.

Installing pyesridump is easy if you have pip installed, using the command pip install esridump.

The next thing you’ll need is the URL to a layer in a MapServer, and these are not easy to find.

Finding data to download

I can almost guarantee the county where you live has one. Before you continue, check to see if your county (or other jurisdiction) has the “open data portal” add-on to their ArcGIS stack.

A handful of suggestions:

  • Here are links to the open data portals enabled by Esri for Lake County, Illinois, and Broomfield County, Colorado.
  • A reader suggested looking for MapServers for jurisdictions around the world by looking through Esri’s portal of open data called ArcGIS Hub.
  • Use a search engine to search for the URL pattern that’s common to ArcGIS servers: paste “inurl:arcgis/rest/services” into a search engine. (You can also limit this search to a particular domain name, say, “chicago.gov”, by adding “site:chicago.gov” to the search query, making the full search query “inurl:arcgis/rest/services site:chicago.gov”.

Once you locate a dataset you want, you can find the MapServer URL under About>Data Source on the right side of the page.

I normally find them by looking at the HTML source code of a MapServer I already know about.

For this example I’ll use one of the GIS layers in the Cook County, Illinois, election service MapServer – here’s the layer for the Cook County commissioners districts.

Fetch the data

Once you have the URL the command is simple:

esri2geojson http://cookviewer1.cookcountyil.gov/ArcGIS/rest/services/cookElectnSrvc/MapServer/11 cookcounty_commissioners.geojson

  • The first term, esri2geojson tells your computer which program to load.
  • The second term is the URL of the MapServer URL.
  • The third term is the filename and location where you want to store the file. I prefer running the command “inside” the folder where I want the file to be stored. You can also specify a full path of the file. On a Mac this would look like ~/Users/username/Documents/GIS/projectname/cookcounty_commissioners.geojson

After you enter the command into your computer’s terminal, press enter. esri2geojson will report back once, after it finds and understands the MapServer URL you gave it. When it’s done, the command will “close” and your computer’s terminal will wait for the next command.

Do you have questions, or need some help? Leave a comment below.

Fun with stats: Building permits by street name and number edition

September 17, 2017 / / 0 Comments
John Hancock Center

The John Hancock Center. Photo by Kevin Dickert.

 

On which street are the most building permits issued?

Michigan Avenue!

But where on Michigan Avenue are the most building permits issued?

Take a guess!

First, can you answer: Are most building permits issued to North Michigan Avenue (between Madison Street, 0 north/south, and Oak Street, 1000 north), or South Michigan Avenue (between Madison Street, 0 north/south, and um, somewhere south of 130th Street, 13000 south)?

Here’s the answer…

Even though South Michigan Avenue is at least 13x longer than North Michigan Avenue, South Michigan Avenue has 39 percent fewer building permits!

From 2006 to yesterday (Saturday), there were 7,828 building permits issued to projects on North Michigan Avenue and 4,714 building permits issued on South Michigan Avenue.

The most common address on North Michigan Avenue to receive building permits was 875 N Michigan Avenue. It’s also the most common address to receive building permits on all Chicago streets.

What’s there? The John Hancock Center (tower)!

The average building address number on North Michigan Avenue is 540.6. That means that building permits on North Michigan Avenue concentrate around Grand Avenue, which is near the city’s biggest Marriott hotel, and is where the Under Armor flagship store is.

The next most common street – after South Michigan Avenue – is North Clark Street, which extends from Madison Street (0 north/south) to the northern edge of the city at Howard Street, which is 7600 north, about 7.6 times longer than North Michigan Avenue.

S. Clark Street Signs

Businesses in the 400 block of South Clark Street, as of when the photo was taken in November 2008. I believe the hotel is still there. This is the busiest block of South Clark Street, for building permits. Photo by Bruce Laker.

South Clark Street doesn’t register in the top 10 or even the top 100. It comes it at number 162, with 772 building permits. This is surprising to me because South Clark Street runs from Madison Street (0 north/south) in downtown and goes to 2200 south, and has a lot of downtown office buildings.

South LaSalle Street (3,613 building permits), South Wabash Avenue (2,916), and South Dearborn (1,611) are all in the top 50. The data could be wrong somehow.

It’s impossible to track the many investment programs in disinvested Chicago neighborhoods

September 14, 2017 / / 0 Comments

An article in the Chicago Tribune that announces Chase bank’s increase in the amount of grants it is offering in Chicago goes on to detail myriad existing grant, loan, and donation programs from public and private sources for neighborhoods that have few jobs, few resources, no privilege, and lots of quality of life problems.

But not all of the programs. There are more, but I don’t even know how many more, nor do I know all of their names. I just know that I’ve read about them before.

The article is where I learned that Benefit Chicago – a $100 million investment fund  operated by the Chicago Community Trust, MacArthur Foundation, and Calvert Foundation, but hasn’t finished raising all the money – has started giving out loans and grants to Chicago recipients, including Garfield Produce Company.

Calvert Foundation has a brokerage (I think that’s the best name for it) through which regular Chicagoans can invest $20 minimum and earn 1.0% interest on that investment after 1 year. Longer periods net higher returns.

Anyway, back to my point…

If I were a business owner in Chicago, and I wanted financial assistance to expand my business – say, buy more kitchen equipment to be able to produce more food – where would I start looking?

Is there a list somewhere? Will my alder know? Is there a group in my neighborhood that can help me track down a funder? Is this more complicated than getting a VC to fund a “Bodega killer“?

One of the things I’ve tried to do with the tens of thousands of maps on Chicago Cityscape is highlight when a business or property owner could be eligible for financial assistance based purely on their geography.

Map of areas where you, as a business or property owner, can get funding assistance from publicly-funded programs.

These geographers where government funding is available are marked with a green icon of a dollar bill that links to a Resources page I adapted from a pamphlet the city’s planning department used to produce. These include:

  • TIF (tax increment financing) districts, including whether the district participates in the Small Business Improvement Fund
  • MMRP (micro market recovery program)
  • Enterprise Zone (a state of Illinois program)
  • Industrial Growth Zone (expedited approval processes + environmental remediation money)
  • Special Service Area (SSA; business improvement district)
  • Chicago landmark and National Register of Historic Places districts
  • Planned Manufacturing Districts (PMD), although I forget what assistance is available here
  • Neighborhood Opportunity Fund zones (an interesting policy that charges developers for additional density and grants that money to small business owners on the South and West Sides)

Not every area within the above categories is in a disinvested neighborhood because not every program was designed for that. 

Green dollar bill signs on Chicago Cityscape

Once you know this, I guess you can target your research. But there’s still a lot more to do. To start: Where the heck is Chase investing? Where the heck is Benefit Chicago investing? They don’t publish maps, as far as I can tell.

Actually, thinking about this more, as I reach nearly 400 words in this blog post, I’ve got another idea: Show up at Rahm’s new Small Business Center at City Hall and ask them.

A map of maps

August 3, 2017 / / 0 Comments

The map of maps.

Over on my website Chicago Cityscape I’ve assembled a map of maps: There are 20,432 maps in 36 layers. You might say there are 36 maps, and each of those maps has an arbitrary number of boundaries within. I say there are 20,000+ maps because there’s a unique webpage for each of them that can tell you even more information about that map.

This post is to throw out some analysis of these maps, in addition to the simple counts above.

The data comes from the City of Chicago, Cook County, and the U.S. Census Bureau. Some layers have come from bespoke sources, including the entrances of CTA and Metra stations drawn by Yonah Freemark and me for Transit Explorer. The sections of the Chicago River were divided and sliced by the Metropolitan Planning Council. The neighborhood and business organizations layers were drawn by me, by interpreting textual descriptions of the organizations’ boundaries, or by visually copying an organization’s own map.

There are 6,879 unique words longer than 2 characters, in the metadata of this map of maps. The most common word is “annexation”, which makes sense, given that the layer with the most maps shows the 10,668 Cook County annexation actions since 1830 – the first known plat was incorporated in the City of Chicago.

The GeoJSON file, an open source, human readable GIS format, comes out to 30 MB, and it make break your browser when you try to display this layer.

The next group of words are also generic, like “planned” and “development”, related to the Planned Development kind of zoning process in Chicago – called Planned Unit Development in other jurisdictions.

After that, some names of municipalities that traded back and forth between unincorporated Cook County and incorporated municipalities are on the list.

Working down the list, however, it gets really boring and I’m going to stop. I bet if you’re a smarter data science person you can find more interesting patterns in the words, but I’ve also increased the number of generic words (like planned development) by adding these as keywords to each map’s “full text search” index, to ensure that they would respond to a variety of search phrases from users.

Inclusionary zoning calculator will tell you how many units a developer can afford to make “affordable”

July 28, 2017 / / 0 Comments

An “inclusionary zoning” calculator can help you determine how much affordable housing your town should require that developers build in their new construction residential buildings.

I learned about Grounded Solutions Network’s Inclusionary Housing Calculator at the second-ever YIMBYtown conference in Oakland, California, two weeks ago.

YIMBY (yes in my back yard) is a movement to reduce barriers to building more housing in order to be able to house everyone at a level they can afford. It’s a movement for other things, and it means a lot of different things to a lot of different people but the end result is that more housing needs to be built.

An interested person inputs a lot of values relevant to their local housing market into the IHC and it will calculate the cost of construction per unit and the rental income from those units, and then will figure the profit margin for the developer. What makes this “inclusionary” is that one also needs to enter the desired portion of units that are set aside as “affordable” (to people making a certain income) and subsidized by the developer’s rental income.

I put the IHC through a real world exercise by inputting as much data as I knew about a rejected proposal in Pilsen.

The first proposal from Property Markets Group had 500 units, and 16 percent of them were set aside (news on this and their subsequent proposals) [I cannot find the source of the “16 percent on-site” factor]. Chicago’s Affordable Requirements Ordinance, or ARO, requires that 10 percent of the units are affordable, and that 25 percent of those 10 percent must be built on site. The other 75 percent can be built on site, or the developer can pay an in-lieu fee per unit.

Needless to say, 16 percent on-site is much, much higher than 25 percent of 10 percent. A neighborhood organization, the Pilsen Land Use Committee, however, requires 21 percent in the area, and the city council member, Danny Solis, 25th Ward, adheres to.

PMG said they couldn’t go that high, and that’s what I wanted to test.

According to this Inclusionary Housing Calculator, could the developer make enough profit (considered as 10 percent) if the building had 21 percent of units as affordable?

In this exercise, the answer was “no, PMG could not make a profit if they had to set aside 21 percent of the units as affordable.”

But the calculator showed that they could earn a 12 percent profit if 16 percent of the units were affordable. 

Some of the inputs are actual, like the sale price of the land (found in the Illinois Department of Revenue’s transactions database), but I had to make up some inputs, including the apartments’ bedroom mix, and the future rental prices of those apartments.

Further reading

  • It’s tough for people to move into one of these set-aside apartments in Chicago (DNAinfo Chicago, July 28, 2017)
  • Inclusionary zoning cannot create enough affordable units (City Observatory, February 11, 2016)
  • Other housing cost calculators like this one (City Observatory, July 26, 2016)