The Art of Data Visualization: Activity Mapping Lecture

Juan Francisco Saldarriaga presented two recent center projects in his talk ‘Activity Mapping’ during the Art of Data Visualization conference held at Columbia University on April 6th. The talk took place at the Davis Auditorium (room 412 Schapiro CEPSR) at 10:50 AM.

Here's a video of the talk:

Here’s a description of the talk: Foursquare check-ins? Citibike rides? Open data can tell us a lot about our cities and how we use them: what we think of them, how we feel about them and how we live in them. In this talk we present two research projects that use this data to explore and understand how people live in New York. We analyze check-in data from Foursquare and Facebook to examine how social media activity relates to socio-economic factors and what this kind of data can tell us about how people feel about the modern city. We also analyze Citibike ride data visualizing the imbalance problems the system faces. All of this, while also exploring multiple ways of representing spatial data.

Read More

News Archive
Title Initiative Category Date
{{news.node.full_title}} {{news.node.initiative}} {{news.node.news_category}} {{news.node.posted_ts * 1000 | date: 'MMM d, yyyy'}}
{{ news.node.intro_text}}
The Banking Divide For Taxi Access: Evidence From New York City
In this project we use multiple datasets to explore taxicab fare payments by neighborhood and examine how access to taxicab services is associated with use of conventional banking services.
  • Browse by Initiative
 

Since 2008 yellow taxis have been able to process fare payments with credit cards, and credits cards are a growing share of total fare payments. However, the use of credit cards to pay for taxi fares varies widely across neighborhoods, and there are strong correlations between cash payments for taxi fares, cash payments for transit fares and the presence of unbanked or underbanked populations. In this paper we use multiple datasets to explore taxicab fare payments by neighborhood and examine how access to taxicab services is associated with use of conventional banking services.

Green Cab Origins
Green Cab Origins

Taxicabs are a critical aspect of the public transit system in New York City. The yellow cabs that are ubiquitous in Manhattan are as iconic as the city’s subway system, and in recent years green taxicabs were introduced by the city to improve taxi service in areas outside of the central business districts and airports. Approximately 500,000 taxi trips are taken daily, carrying about 800,000 passengers, and not including other livery firms such as Uber, Lyft or Carmel. Since 2008 yellow taxis have been able to process fare payments with credit cards, and credits cards are a growing share of total fare payments. However, the use of credit cards to pay for taxi fares varies widely across neighborhoods, and there are strong correlations between cash payments for taxi fares, cash payments for transit fares and the presence of unbanked or underbanked populations.

These issues are of concern for policymakers as approximately ten percent of households in the city are unbanked, and in some neighborhoods the share of unbanked households is over 50 percent. In this paper we use multiple datasets to explore taxicab fare payments by neighborhood and examine how access to taxicab services is associated with use of conventional banking services. There is a clear spatial dimension to the propensity of riders to pay cash, and we find that both immigrant status and being ‘unbanked’ are strong predictors of cash transactions for taxicabs. These results have implications for local regulations of the for-hire vehicle industry, particularly in the context of the rapid growth of services that require credit cards. Without some type of cash-based payment option taxi services will isolate certain neighborhoods. At the very least, existing and new providers of transit services must consider access to mainstream financial products as part of their equity analyses.

Overall there are observable differences for cash payments by taxi type, location, trip origin and trip destination. It is impossible to know what characteristics differ between a typical yellow cab passenger and a typical green cab passenger, but something leads green cab passengers to use cash far more often than yellow cab passengers. The results shown on the maps suggest that there is a spatial factor in play.

In all maps there are stark lines that demarcate where riders predominately use cash (shown in yellow) and where they use credit (shown in blue). The areas marked with yellow are the places where cash is king. With the exception of a credit card hotspot surrounding Columbia University in Morningside Heights Manhattan payment types divide cleanly along income lines, where wealthy neighborhoods flanking Central Park (the empty white rectangle in the middle of the map surrounded by blue to the south and yellow to the north) on the Upper West Side and Upper East Side pay for taxi trips mostly with credit cards and poorer neighborhoods to the north in Spanish and Central Harlem are dominated by cash. One interesting aspect is that the socio-demographic characteristics of neighborhoods seemingly play a large role in determining payment type. It is likely that the cash or credit choice is a function of access to a bank account, for which these spatial data are a good proxy. Another takeaway is that much of the city still does not produce a lot of taxi trips and there is not enough data to present primary payment types at all.

Project Team
Name Project Role
{{person.name}} {{person.role}}
 
Launch Preview of Conflict Urbanism Aleppo on November 17th, 2015
Nov 18, 2015 — Dare Brawley

On November 17th, 2015 the Center for Spatial Research, in collaboration with the Tow Center for Digital Journalism at Columbia University, hosted a launch preview of Conflict Urbanism: Aleppo.

Laura Kurgan and Madeeha Merchant were joined by collaborator Jamon Van Der Hoek in presenting the research and development that has led to the interactive map of the city of Aleppo, Syria.

The event also featured presentations by Josh Lyons of Human Rights Watch, Tyler Radford of Humanitarian Open Street Maps, and Timothy Wallace of The New York Times about how their work might ineract with a project like Conflict Urbanism: Aleppo. 

Acknowledging that conflict zones are information rich and analytically poor, we hope to begin an interdisciplinary discussion about the potential of detecting the effects of urban conflict through satellite imagery analysis.

Focusing on the current catastrophe in Syria, the Conflict Urbanism: Aleppo project began with an attempt to link eyes in the sky with algorithms and ears on the ground. Towards these ends, we have created an open-source web platform that allows users to navigate maps and satellite images of the city of Aleppo, at the neighborhood scale, across multiple data sets. Working with data from Human Rights Watch, UNOSAT, and the Violations Documentation Center, the platform combines our algorithmically-derived damage identification with their expertise.

We discussed what the correlation of human rights data with satellite imagery analysis tells us about the conflict in Syria, and Aleppo in particular. What possibilities do machine learning and remote sensing algorithms promise for damage detection?  Can and should we use crowd-sourcing and citizen science to better train our algorithms?  We invite advocates and researchers from human rights organizations, humanitarian and development agencies, the academy, and the news media to join us in exploring potential uses of the platform and our tools.

News Archive
Title Initiative Category Date
{{news.node.full_title}} {{news.node.initiative}} {{news.node.news_category}} {{news.node.posted_ts * 1000 | date: 'MMM d, yyyy'}}
{{ news.node.intro_text}}
 
Mapping for the Urban Humanities: A Summer Institute
A summer intensive course on digital mapping for faculty.
   
About

Mapping for the Urban Humanities is an intensive workshop on digital mapping designed for humanities faculty and advanced graduate students at Columbia University and offered through the Center for Spatial Research at the Columbia University Graduate School of Architecture, Planning, and Preservation with support from the Office of the Dean of Humanities. In 2016 and 2017 the course was taught collaboratively by Dare Brawley, Leah Meisterlin, and Eric Glass. In 2018 the course will be taught by Michelle McSweeney and Buck Fivel.

The syllabus for the course is available via GitHub here.

This hands-on workshop is intended to broaden and transform the disciplinary locations within which data analysis takes place. This workshop introduces participating faculty to open-source mapping software, QGIS, to methods of data collection and creation, and to approaches and concepts in critical spatial analysis. With support from the course’s three instructors, participating faculty will incorporate newly-acquired spatial analysis skills into course assignments and syllabi. The ultimate aim of the summer intensive is to equip faculty with tools to transform their humanities courses into places where students learn spatial data analytical skills and apply them to humanistic questions. 

The course condenses topics from a semester long introductory GIS course into a two-week hands-on intensive and one-week practicum. Skills based tutorials draw on diverse datasets relevant to investigations in the urban humanities, including: CEISEN’s Gridded Population of the World, UN Population Division national population estimates, the National Historic Geographic Information System’s historic census records, historical rail lines from the University of Nebraska’s Digital History Project, Reference USA records for music industry businesses, American Community Survey demographic data, New York State Board of Education records of schools, and scanned historical maps from Columbia University’s map collection. Course tutorials and lectures have been developed concurrently and will expose faculty to critical approaches in GIS, introductory spatial analysis methods, and modes of data creation.

Structure and Rhythm

Over the course of the first two weeks, participants learn critical methods in digital mapping and data collection through the use of open-source software (QGIS). The course meets for 4.5 hours each day in an intensive workshop and lab format. The third week focuses on how to incorporate newly acquired skills into course assignments and syllabi and on bolstering relevant skills development, using the syllabi submitted with the application to the course as a starting point. During the third week participation is optional and is scheduled as individual skill and syllabus workshop sessions. 

Departments Represented in 2016

Anthropology, Architecture, Art History and Archaeology, Classics, Heyman Center for the Humanities, History, Institute for Comparative Literature and Society, Jewish Studies, Journalism, Language Resource Center, Latin American and Iberian Cultures, Middle Eastern, South Asian, and African Studies

The courses that faculty have workshopped during the course cut across a wide spectrum of topics including titles such as: "Bombay/Mumbai and its Urban Imaginaries," "Democratizing Architecture," "Reading the multilingual city: Linguistic landscapes and urban multilingualism," "The Greek city-state in world-history 1000 BCE-400 CE," "Geopolitics." 

Syllabi developed through the course are available below. This will continue to be updated as these courses are introduced into the curriculum.

Departments Represented in 2017

Architecture, Classics, East Asian Languages and Cultures, Economics, English and Comparative Literature, History, Real Estate Development, Urban Planning, Teachers College. 

Titles of courses and research projects that participants workshopped during the course included: "The Great Syrian Revolt of 1925: A History of Biocultural Diversity and International Politics in the Post-Ottoman Era," "Harlem Stories: Archives and Digital Tools and Our Wadleigh: The Complex Struggle for Educational Justice in Harlem," "Mapping Jewish Life in Eighteenth-Century Amsterdam," "Mapping Decline and Fall of the Roman Empire," "Thessaloniki Down the Ages: a City and its Many Voices," "Architecture of Colonial Modernity," "Foreigners in the 15th – 18th Century European City."

TUTORIAL
TUTORIAL
 
Mapping for Architecture, Urbanism and the Humanities
An introduction to mapping theory and geographic information systems tools.
   
About

This course provides an introduction to mapping theory and geographic information systems tools. Through the use of open-source GIS software (qGIS) and open data (OpenStreetMap) students will learn how to critically use mapping tools and geographic data for spatial analysis and representation. In this course, students will work through a series of web tutorials and hands-on in-class exercises to gain a better understanding of how these tools and data can be leveraged to analyze, represent and study past or present urban phenomena. In addition to using existing data, students will also be able to create or bring their own sets of data and questions from other courses and will be able to work with these in our class.

No prior experience in mapping, design or data analysis is required for this course. The course is open students in GSAPP and the Graduate School of Arts and Sciences and others by permission.

The full syllabus is available here. 

General Topics
  • Basic mapping concepts and techniques
  • Data types
  • Census data
  • Metadata
  • Data creation
  • Geocoding
  • Georeferencing
  • Vector and raster data
  • Webmapping and crowdsourced data
Spring 2016

The Spring of 2016 iteration of this course had twenty one students from seventeen different departments across the University. 

Fall 2016

This course will be offered again in the Fall 2016 semester with course number A4122.

 
Conflict Urbanism Aleppo
The first in a series of Mellon Foundation supported seminars on Conflict Urbanism.
   
About

This is the first in a series of multidisciplinary Mellon seminars on the topic of Conflict Urbanism, as part of a multi-university initiative in Architecture, Urbanism and the Humanities. This year, we will focus on Aleppo in Syria. Conflict Urbanism is a term that designates that cities are not only destroyed but also built through conflict.

This spring, the seminar focus will be on the city of Aleppo and the places now inhabited by refugees from the city. We will begin with an art historical, and historical survey of one of the most continuously inhabited cities in the world dating back to 10 000 BC. We will examine the urban artifacts of colonialization, the diverse religious and cultural monuments, and the trade routes that have formed the city as it existed until 2011. Then we will look beyond the recent dramatic eruption of state violence and civil war to its antecedents and contexts in rural-urban migration, largely driven by other factors, including poverty and drought. We will examine the time-based destruction of Aleppo at macro and micro levels, analyzing conflict patterns in its social and urban structure. We will then expand what we call "Aleppo" to include the tens of thousands of former residents now sheltered in other cities and in refugee camps in Turkey and Jordan. In order to re-imagine the city in it’s post-conflict state our work will be informed by an analysis of the problems encountered in the rebuilding of cities as diverse as Beirut, Kabul, and Sarajevo, which are in various states of rebuilding post-conflict and offer a range of results that need interrogation.

Methods

Our work will be by necessity multidisciplinary and include the history, architecture, politics, media, culture, literature and contemporary manifestations of particular cities as they relate to the topic of Conflict Urbanism. Our work will also be multi-media. To study cities today, we have an abundance of data and databases accessible to focus our research. Students will have access to a platform developed by the Center for Spatial Research for their work. It provides students a user friendly framework for navigating, analyzing, and interpreting a series of maps and imagery, characterizing changes resulting from social conflict or natural disasters during the conflict years in Aleppo. The platform can be used in one of two ways: to facilitate navigating and understanding the conflict from a distance, in order to and write about a topic; or, to add to the platform, in a multi-media format on a specific topic or issue, with regard to a specific neighborhood, monument, institution or network.

Final Projects

The final projects which emerged from the course were diverse in focus and approach. Students worked in groups (often composed of members from multiple schools or departments) and drew on diverse archives about the city including: data about the conflict, reporting on social media, official and ‘unofficial’ planning documents in the lead up to the war, records of cultural and religious artifacts in order to produce rich multimedia projects. 

The results of the student work have been collected and published online here

Course Files
 
Project
Close Up at a Distance, Mapping Technology Politics
person role
Author(s): 
Laura Kurgan
Publication date: 
Tuesday, April 23, 2013
Publication name, page number: 
Zone Books, 2013
Description (optional): 
The past two decades have seen revolutionary shifts in our ability to navigate, inhabit, and define the spatial realm. The data flows that condition much of our lives now regularly include Global Positioning System (GPS) readings and satellite images of a quality once reserved for a few militaries and intelligence agencies, and powerful geographic information system (GIS) software is now commonplace. These new technologies have raised fundamental questions about the intersection between physical space and its representation, virtual space and its realization. In Close Up at a Distance, Laura Kurgan offers a theoretical account of these new digital technologies of location and a series of practical experiments in making maps and images with spatial data. Neither simply useful tools nor objects of wonder or anxiety, the technologies of GPS, GIS, and satellite imagery become, in this book, the subject and the medium of a critical exploration. Close Up at a Distance records situations of intense conflict and struggle, on the one hand, and fundamental transformations in our ways of seeing and of experiencing space, on the other. Kurgan maps and theorizes mass graves, incarceration patterns, disappearing forests, and currency flows in a series of cases that range from Kuwait (1991) to Kosovo (1999), New York (2001) to Indonesia (2010). Using digital spatial hardware and software designed for military and governmental use in reconnaissance, secrecy, monitoring, ballistics, the census, and national security, Kurgan engages and confronts the politics and complexities of these technologies and their uses. At the intersection of art, architecture, activism, and geography, she uncovers, in her essays and projects, the opacities inherent in the recording of information and data and reimagines the spaces they have opened up.
Intro text (homepage): 
Close Up at a Distance records situations of intense conflict and struggle, on the one hand, and fundamental transformations in our ways of seeing and of experiencing space, on the other.
Lead image: 
Author C4SR: 
Author Last Names for table: 
Kurgan
Publication short title (carousel): 
Close Up at a Distance, Mapping Technology Politics
Is Website?: 
no
dashboard_sort_date: 
Tuesday, April 23, 2013
(currently rendering default node template)
CitiBike Rebalancing Study
An investigation into ways to rebalance CitiBike stations throughout New York City.
  • Browse by Initiative
   
 

As has been recently documented by the press, one of the major challenges that Citi Bike is facing is the rebalancing of their stations. As origins and destinations of Citi Bike trips are not necessarily symmetrical during the day, Citi Bike has been forced to constantly move bikes around the city, taking them from full stations and delivering them to empty ones. This problem is both financially expensive and frustrating for Citi Bike users: many people complain about either not finding bikes at their stations of origin or not finding empty spots when they arrive at their final destinations.

To study this problem we have created a series of visualizations which should serve as a starting point for further analysis.

First, we visualized the average activity for weekdays in October 2013.

Citi Bike Hourly Activity

As the above image shows, the activity hotspots remain pretty constant throughout the day, specially between 10am and midnight, with most of the activity centered around Union Square. In addition, we also see how both Grand Central and Penn Station become strong hotspots during peak hours. Of interest, though, is the sudden shift that occurs around 5am, with the activity hotspots switching from the East Village/Lower East Side area, to Grand Central and Penn Station. This is probably due to the fact that during most of the night, compared to other areas, the stations in the East Village/Lower East Side continue to have high activity, but during most of the day, and specially during peak hours, they are not as active as the stations around Union Square or Grand Central and Penn Station.

Citi Bike Hourly Balance

Next, we visualized overall patterns of origins and destinations. As the above image shows, the big hotspots of imbalance are mostly located around the East Village, Lower East Side, Midtown East and West and Union Square. However, the variation of these hotspots throughout the day is pretty extensive and it's very difficult to detect smooth transitions apart from peak hours. Of note are a couple of big "jumps" between origins and destinations, one of them around 1-2pm on the East Village/Lower East Side and another one around 5am also in the same area.

 

We also created a series of imbalance matrices (by hour of day) for every single station on the system. Again, using the same data as the animations above, this first matrix (Citi Bike Hourly Balance) clearly shows how the big imbalances happen (as expected) mainly between 6am and 10am (morning peak hour) and between 4pm and 8pm (evening peak hour). However, there are some stations whose imbalance starts and ends earlier, like 8th Ave. & 31st Street, W 33rd Street & 7th Ave. and W 41st Street & 8th Ave. (more origins than destinations starting around 2pm). In addition, this matrix also shows that not all of the stations suffer from big imbalances during peak hours. Indeed, stations like E 31st Street & 3rd Ave or E 32nd Street & Park Ave. barely have any imbalances during peak hours. You can download a high-res version of this matrix here.

Imbalance matrix normalized by hourly activity

Furthermore, as not all of the stations have the same level of activity, we produced two more matrices, both showing station imbalance, but this time comparing it to the overall hourly activity for each station. The first one (Imabalance matrix normalized by hourly activity) shows the imbalance as a percentage of the activity for that hour. Hence, the great imbalances appearing late at night, when there are fewer trips and there's a higher chance of having all of them as origins or destinations. However, it is still interesting to see that there are higher imbalances during the morning peak hour than during the evening one, as a percentage of the overall activity.

Activity and imbalance matrix

The second matrix (Activity and imbalance matrix) shows the imbalance as colors and the overall activity as brightness, so we can see how in the hours between the peak times there's still a lot of activity but it is mostly well balanced. In addition, we can see how late at night (imbalanced as it may be) there's still very little activity. Finally, we can also see some outlier stations with a lot of activity and still pretty imbalanced: for example, in the morning 8th avenue and 31st street, 17th street and Broadway, Lafayette and 8th street, and Pershing Square (north); and in the evening 8th avenue and 31st street, 41st street and 8th avenue and again Pershing Square (north). You can download both of these matrices at high res here and here.

Imbalance Hotspots - A.M. Peak Hour

Finally, we have created hotspot maps for both the AM and the PM peak hours. As you can see from the maps below, Citi Bike activity closely matches what we would expect to see in New York: the AM peak hour map shows people leaving residential neighborhoods (Lower East Side, East Village, Chelsea and Hells Kitchen) and arriving at Midtown East and the Financial District, and the PM peak hour map shows the reverse. To note, however, is the fact that these two maps are not completely symmetrical, meaning that there are certain trips that happen in the morning which do not have their counterpart in the evening, and vice versa. Also, there are some stations that while being inside imbalance hotspots do not show that large of an imbalance. These stations have been outlined on the maps and should be further studied. You can view high-res versions of these maps here: AM and PM.

Imbalance Hotspots - P.M. Peak Hour