Researchers are designing a data-exchange platform that can act as a city's dashboard

We live in a sea of information. Every day, 2.5 billion gigabytes of data are produced by our smartphones, sensors and satellites, among other sources. Much of this data is useless, but what if the disparate bits of meaningful data incidentally produced during the day-to-day operations of a city could be brought together and analyzed as an integrated whole to make the lives of urban dwellers flow more smoothly?

Singapore's Smart Nation initiative aims to have the separate elements of a city working in unison to improve the lives of its citizens. A big part of the challenge is designing systems that are capable of handling huge amounts of data in vastly different, often unstructured, forms and assembling it to extract usable information on everything from traffic congestion to land usage. The A*STAR Urban Systems Initiative (USI) is perfecting the art of big-data consolidation in the cityscape.

A data marketplace

Launched by the A*STAR Institute for Infocomm Research (I2R) in 2012, USI consists of five programs: Complex Systems, Urban Logistics, Integrated City Planning and Sense & Sense-abilities. But the initiative's backbone is formed by the A*STAR Data Analytics Exchange Platform, or A*DAX.

When Ng See-Kiong took over leadership of USI in November 2012, he realized that being able to combine data sets from multiple data owners would be the key to its success. "The city really is a complex system of systems," says Ng. "To understand transportation, for example, you can't just look at transportation; you also need to look at demography, job distribution, weather and other issues." This is where A*DAX comes into play—an open-standards, cross-domain platform for data linking, sharing and analytics across Singapore's public and private sectors.

Simply making the data freely accessible was not an option since a lot of it was proprietary and subject to privacy considerations. A*DAX's solution was to provide a platform capable of bringing data owners together and allowing them to exchange and integrate information on a 'peer-to-peer' basis. "The platform itself is a controlled environment with strict user access controls—if you are the data owner or service provider you can give access to other users," explains Ng. In layman's terms, "you can think of it kind of like a data marketplace, where people come together and agree to exchange information in a safe and controlled environment."

Once the data buyers and sellers had agreed to do business, the platform designers then needed to ensure that the different types of data—structured, semi-structured and unstructured—could be combined and extracted at the same time. In response, Ng's team created the A*DAX Fusion middleware, which allows a user to issue a single 'query' to any combination of multiple data sets of different data types based on a framework familiar to web service designers called RESTful APIs (representational state transfer application program interfaces).

User friendly

Having access to reams of data is one thing; being able to interpret and make use of it, is another. In a pivotal exchange between the team and some early adopters, the A*DAX team quickly learned that they also needed to develop the platform's analytic capabilities.

"We found that the users didn't want more data—they wanted more answers," Ng recalls. Generally, this meant helping users to find data, even when they couldn't articulate exactly what they were looking for. "The user typically has lots of data, knows they want to make use of it but doesn't know exactly what they can use it for," says Ng. "Oftentimes we would have to put ourselves in the shoes of the user and try to create some visual analytics routines to produce insights that they might want to see from the data. This would act as inspiration for users to tell us what they wanted. It's an iterative process—we have to enter this discovery journey together."

The team eventually honed in on the strategy. "We decided not to give away the raw data and instead provide people with a way to query it for less specific information, but more valuable insights such as aggregates or predictions." This had the added benefit of preserving a higher level of data security, while ensuring the data was still 'usable'—a very difficult balance to strike.

One of the greatest joys for a researcher is seeing an idea they have conceptualized come to fruition in the real world. For Ng, A*DAX first came to life in the Jurong Lake District (JLD), Singapore's first smart city 'testbed'. A shared sensor and camera network was set up in the JLD, with A*DAX as the core data management system. Participating developers from different sectors have been able to use A*DAX's capabilities to enhance their own applications and trials within JLD. Examples include a web-based service that integrates environmental information to help urban planners monitor and measure microclimates, as well as an automated system that is capable of detecting people sneaking a cigarette into non-smoking zones.

Predicting the unexpected

A*DAX is designed to make predictions based on existing data, such as from public transport smart cards, taxi global positioning systems, and mobile phone locations. But not all human behavior is well documented, or easily foreseen. "In real life, there are many situations where there's no historical path for you to base your predictions on," says Ng. "This is where complex systems come in—to use modeling and simulations to help people prepare for the unexpected."

Christopher Monterola from the A*STAR Institute of High Performance Computing heads up the USI's Complex Systems arm. "We provide the modeling and simulation framework and perspective on how to understand the interactions and mechanisms of the various constituents that make up urban systems—it's a natural fit for the USI."

Specifically, the team contributes mechanism-based models to complement the data, visualization and analytics from the other USI programs. Monterola and his team have previously designed sophisticated 'agent-based' mathematical models that predict diverse urban-mobility parameters, from land use patterns for urban planning, through to train delays and traffic congestion—essential capabilities for a 'smart city'. Combining the team's predictive models with A*DAX's data capabilities opens exciting possibilities for urban planners. These include being able to compare and contrast overlapping land use and travel infrastructure in different city designs, as well as handling traffic congestion due to a sudden surge of commuters in a given locality, and managing the 'pulse' of people in a city—their spending patterns, energy consumption and travel preferences, says Monterola.

Moving forward

Employing A*DAX as a 'city dashboard' is a logical move in a tech-savvy country like Singapore. But, A*DAX has the potential to be applied in other sectors as well, such as finance. Ng and the team are exploring 'smart finance' applications, specifically the use of A*DAX to bring together financial models and natural cartography for the calculation of more accurate insurance risk models. "We're also hearing a lot about the Internet of things, industry 4.0, autonomous vehicles, the sharing economy—a lot of these will require data analytics exchange platforms similar to A*DAX," says Ng.

Now at the end of their initial five-year mandate, the USI team is set to continue making Singapore a 'smarter' nation. "We're definitely not closing shop—we've only just started!" says Ng.