Array of Things Releases APIs for Chicago Data, Enabling Applications

Array of Things Releases APIs for Chicago Data, Enabling Applications

Earlier this year, we released the first data from Array of Things Chicago in the form of bulk CSV downloads through our beta file browser site. While a bit unwieldy and updated just once per day, we knew that researchers would want data in the richest possible detail to study the climate, air quality, and other measurements currently available from the over one hundred nodes installed around the city.

But scientists are just one of the audiences for AoT data, and we are happy to announce that AoT Chicago data can now also be accessed via API, making it easier for developers to start using near real time data in applications.

Through the API and its documentation, users can access both proven and raw data from the Chicago nodes. These measurements include calibrated temperature, humidity, pressure data, as well as raw air quality data, including for various gases and particulate matter, that are still under evaluation. Measurements extracted from image processing, such as pedestrian and vehicle counts, will be available in the future.

Unlike the bulk downloads, which provide data in daily, weekly, or monthly installments back to early 2017, the API data is focused only on recent data, going back one week. However, the data are updated more frequently, within five minutes of collection.

“The older data is mainly for people who are really more interested in larger analysis on more powerful machines,” said Vince Forgione, lead engineer for the Urban Center for Computation and Data, which runs the Array of Things project. “The API is definitely more geared for application developers and other people who want near real-time information.”

To help those users, Forgione and colleague Jesse Bracho also created a number of functions to do analysis within the API. Whether within direct requests or by using client libraries for Python, Javascript, and R, programmers can isolate specific observations by node or by sensor type and aggregate or filter data before retrieving it. The API also sports an implementation for GraphQL, an increasingly popular API language and framework.

“These libraries enable you to just drill down and grab exactly what you want through a really expressive set of parameters,” Forgione said.

On the horizon are changes to the Plenario data portal, where users can work with AoT data alongside other public data sets, and possibly an additional way to access AoT data, through web sockets. Ultimately, the goal is to make data both thorough and accessible for a variety of audiences, from scientists to app developers to the general public.

For more on AoT data engineering, visit the UrbanCCD Github page.

Array of Things Expands with Partner Projects Around U.S.

Array of Things Expands with Partner Projects Around U.S.

This summer, we’re excited to announce the first class of Array of Things partner projects, a group of collaborations with universities, governments, and industry in cities across the United States. In recent months, teams in Palo Alto, Seattle, Portland, Denver, Detroit, Chapel Hill, and Syracuse have received a small number of AoT nodes that they will deploy and test, collecting data on issues of local importance.

Five years, 100 nodes, and more to come

Five years, 100 nodes, and more to come

In 2013, we started with an idea: could we use new technology to provide much more detailed measurements of cities? At Argonne National Laboratory and the University of Chicago, we designed a sensor platform that could withstand the elements in urban environments, and asked dozens of scientists at other universities and research laboratories for the types of data they would find useful. In 2015, we proposed an experiment — to put hundreds of these devices into Chicago — the National Science Foundation provided funding, and the Array of Things project was born.

The project was and is envisioned as a new kind of community technology — an urban sensing project designed for communities, educators, students, policymakers, and residents to better understand cities. Originally, we expected to install a network of environmental sensors in Chicago alone. But once we got started we began to hear from other research groups and cities expressing interest in using our technology in their cities.

At the same time, further interactions with science communities refined the specifications for exactly what we would measure, and this included the concept of using computer vision to “measure” things like vehicle or pedestrian flow, or flooding. Today we have partners and associated deployments of AoT nodes not only in Chicago, but also in cities around the country, and soon, globally.

This spring, five years after we first imagined an urban-scale measurement instrument, Array of Things is hitting many significant milestones. Here’s what we’ve been up to:

The first one hundred nodes in Chicago

Last week, electricians from the Chicago Department of Transportation (CDOT) installed the Array of Thing’s 100th node at the intersection of Western Avenue and Addison Street. You can find nodes installed all across the city: near Soldier Field, on The 606, in neighborhoods from Little Village to Garfield Park to Edgewater. Currently, there’s a node within two kilometers of 80 percent of the Chicago population. And we still have many more to install — the plan is to deploy hundreds of additional nodes in the coming months.

It is an enormous effort to deploy an experimental technology on this scale, and our path to 100 nodes led to important insights regarding urban-scale scientific instrumentation and key partnerships with the City of Chicago’s Department of Innovation and Technology and CDOT. Unlike traditional sensor networks that use very rugged, pre-programmed microprocessors, AoT nodes are remotely programmable with fully functional, and more delicate, computers. AoT nodes thus include custom electronics to increase reliability, for example by putting the delicate computers into hibernation if the temperature is too high or if moisture is detected.

This resilience proved essential in the first test deployments of late 2016 when moisture build-up in some of the units caused them to go “off the air” for hours or days at a time over their first winter. Without the resilience features, electronics in the units would have simply burned out. Instead, they just went offline until the moisture evaporated, then rebooted and continued to take measurements.

Updates to the housing of the nodes addressed the condensation issues, as tested through this past winter, and the next big test will be extreme heat in the coming summer months. We anticipate the possibility of brief outages on sunny August afternoons, but our resilience electronics will protect the systems from burning out permanently. These experiments and continuous improvements underscore the nature of the project — Array of Things is a science project, first and foremost.

Array of Things data is now available

From the beginning, our mission with Array of Things was to make data collected by the sensor nodes openly, publicly, and freely available. But before that could happen, we wanted to make sure we had enough nodes to provide meaningful geographic coverage and, moreover, that the data could be scientifically evaluated, particularly given that many of the sensors use new technologies that have not previously been installed at such scale. We’re happy to announce that the first batch of data is now available for use by the public.

At our beta site, users can download temperature, humidity, and barometric pressure data from nodes across our growing network. These readings provide hyper-local looks at environmental data over the last several months. Currently, you can download this data from individual nodes as a (very large) csv file, but we are working on integrating the data into Plenario, a database system that supports both the City’s OpenGrid data portal and our own browser-based tool for finding and visualizing data.

The currently available data come from just a subset of the sensors installed in AoT nodes; we are also collecting data on other conditions, such as vibration, light and sound levels, and magnetic field, which should be available in the coming weeks. Additionally, we are measuring key air pollutants using a new generation of sensor technology. For these measurements, calibration and testing is necessary before we can start releasing the their data. To fully evaluate these sensors, we have deployed test nodes with air quality sensors at an Illinois EPA testing facility on the South Side, so that we can test the accuracy of these readings against the industry standard measurements.

Later this year, we are also hoping to begin to report on measurements that are derived from image processing. We have been experimenting with algorithms for detecting flooding, as well as for reporting pedestrian and vehicle counts at intervals of several minutes, creating a measure of pedestrian and vehicle flows over time. All of these measurements are in line with an extensive set of privacy policies and procedures that are documented at our website.

We’ll announce new data releases on the Array of Things website and social media, and welcome community feedback on the initial release. In the coming months, we will be seeking input from and collaboration with residents at community events to review AoT data and better understand how it can be presented in a way that provides the most value. These events will be announced on our website and social media soon.

Lane of Things starts its third year, expands.

The Array of Things project was conceived not just as a science project, but as a project that we hoped would inspire Chicago’s youth about the potential for using science and technology for positive impact on their neighborhoods. Lane of Things, our high school educational program based at Lane Tech College Prep, is currently in its third year of operations.

Executed in partnership with the School of the Art Institute of Chicago and run by Douglas Pancoast, Robb Drinkwater, Kate Kusiak Galvin, Satya Mark Basu, Jeff Solin, and Dan Law, Lane of Things is an eight-week curriculum that uses similar technologies to AoT as a platform to help students learn about programming, data science, digital fabrication, and additional computer science concepts. This year, the students installed their sensors in and around Wrigley Field. You can read more about the project from the Chicago Tribune.

With support from Motorola Solutions Foundation, Lane of Things will also be expanding to more schools in Chicago this year. We’re formalizing the curriculum and holding a professional development workshop for Chicago Public Schools teachers this summer, so that students across the city can receive hands-on experience with sensor technology and science through building their own version of the Array of Things project.

It’s been an exciting year for Array of Things, and we’re just getting started. For the latest news, visit our website and follow us on Twitter.

- Charlie Catlett, Director, Urban Center for Computation and Data

AoT Update - Community Partnerships

AoT Update - Community Partnerships

There were many motivations for Chicago’s Array of Things (AoT) project. While it is primarily an experimental research platform to measure the city and provide researchers with a testbed for new “smart city” concepts, AoT also aspires to be a research platform for Chicago educators, students, and residents.

In April, we concluded the second iteration of Lane of Things, an 8-week high school curriculum sponsored by the Motorola Solutions Foundation and developed by a team that included Kate Kusiak Galvin (UrbanCCD), Jeff Solin and Dan Law from Lane Technical High School’s Computer Science Department, and Douglas Pancoast, Satya Mark Basu, and Robb Drinkwater from the School of the Art Institute of Chicago.

As was the case last year, over 150 students worked in teams of three to learn about science, measurement, design and problem solving, data analytics, teamwork, and in the process, acquire hands-on experience with the concept of “Internet of Things” (or “IoT”)—an underlying enabler of the Array of Things.

For this second year of the program, IoT device platform makers Particle joined the team, providing the students with programmable, wireless networked microprocessors that served as the internal brains for their sensor “motes.” Students programmed these devices—called Photons—using Particle’s web portal. They learned how to make different internet services interact; for instance, programming the Photons to send data to online spreadsheets, streamlining the process of collecting and analyzing data.

Over the course of eight weeks, the student teams learned important skills, such as:

1.     Formulating an hypothesis or question to be answered through experimentation.  Although the curriculum revolves around sensors and IoT technologies, these are means rather than ends. To conduct a real scientific project, student teams first conceived of an hypothesis or a question. A good example is the project from Group 403. These students were interested in learning whether there is “a correlation between temperature, humidity, carbon monoxide, hydrogen, and UV levels in a greenhouse. The greenhouse gets the most sunlight out of any room in Lane Tech, so we wondered if there was any correlation with UV from the sun and the various gas levels in the room.”

2.     Developing an experiment.  Here the students designed a device that would take measurements relevant to their hypothesis or question. Group 708 designed a device that would let them measure “whether or not the time of day influences the amount of people that enter/exit the attendance office.” Typically, such an experiment would involve an observer with a clipboard, but the students used a motion sensor, placed in the doorway to the attendance office.

3.     System design and problem solving.  Once student groups decided on what kind of measurement to do, and what kind of sensor would be needed, they learned how to build the electronics that would use the sensor to gather data, as well as an enclosure and mounting system to position their devices for optimal measurement. Group 678 needed to enclose and mount their device in the dance studio in order “to record the level of sound in terms of volume and how loud a room gets, the temperature and humidity, and [use] a motion sensor to get an estimate of how many people walk by, get close, or interact with the photon. The three can all show correlation to how active the classroom is.”

4.   Data analytics and web applications. While taking measurements for two weeks, project teams stored their data in online databases and spreadsheets, allowing them to graph and analyze the data. To do this, students learned how to expose variables to Particle’s cloud, then program online databases and spreadsheets to pull that data. Group 709 built a system to measure water temperature and clarity in Lane Tech’s aquaponics laboratory in order to provide “early warning” of system failure. They used graphs to analyze the data and concluded that their “data did not reveal any tampering or major failures in the aquaponic system (which is good) and we are confident that the mote would have detected a major problem if it had occurred. We are considering leaving the mote up over the summer when power to the aquaponic system is most likely to be accidentally shut off.”

5.   Teamwork. Unexpected challenges often bring out the best in teams. Group 402 discovered a hardware issue that delayed their installation, and had to pull together under a deadline to resolve it. They describe it quite well: “The day before we were supposed to deploy, we came into class and found that the pins connecting the wires on our sound sensor were broken off. So, we spent the whole period re-soldering the sensor and wiring it to the breadboard. We missed at least 3-4 days of data pulling, but it all worked out in the end.”

What’s Next for Lane of Things?

The LoT team is already working on packaging the curriculum and developing a workshop to enable faculty from other high schools to bring the program to their schools.  And of course, the team is eagerly preparing for next year’s program, which will include teaching the students how to use the Array of Things application programming interfaces to incorporate data from the Array of Things! If you are a teacher or school representative interested in participating in future versions of Lane of Things, please contact us at urbanccd.education@uchicago.edu.

- Charlie Catlett, Director, Urban Center for Computation and Data