The UN Global Pulse shows how data gleaned from social media interactions can come in handy for today’s policymakers.
IF companies are able to harness a whole lot of benefits from big data, why not governments too?
This is the idea behind the United Nations (UN)’s Global Pulse initiative, which was established in 2009. The initiative explores how digital data and real time analytics tools can be used to help policymakers efficiently track and deal with various socio-economic issues such as poverty, hunger and the spread of disease.
“While long-term developments (within a nation) still rely largely on trackers such as household surveys and national census data, authorities need to have access to reliable, real time information on a big picture level that would aid them in responding quickly to any problems that arise,” says I-Sah Hsieh, global manager for international development at analytics software development company, SAS Institute (SAS).
SAS has been working alongside the UN by providing the software, technical infrastructure and analytics support required for UN Global Pulse.
“In the past, this (analysis of big data) simply wasn’t possible due to the nature of conventional data collection and analysis methods which are both laborious and time consuming,” he says.
However, Hsieh says things have changed now, thanks to recent developments in the digital world. With the aid of analytics software, it is now possible to tap into vast amounts of information across cyberspace (i.e. what many refer to today as Big Data) and to actually make sense of it.
“Using SAS Social Social Media Analytics and SAS Text Miner... enables researchers to accurately and efficiently track all the chatter that was taking place in the online space as it happens,” he says.
“Once captured, researchers can then analyse the documents and materials obtained to quantify feelings, moods, concerns and strategies. They can also use this data to correlate with officially available statistics to see how it matches up.”
Scrutinising online chatter
One of the projects that the UN Global Pulse has conducted so far is to study how online conversations can be used in conjunction with official national statistics to gain a better understanding of unemployment trends. This project was carried out in the United States (US) and Ireland over a period of three months.
“The main finding was that it was possible to use online data that is publicly available to provide accurate predictive indications of a pending spike in unemployment,” says Hsieh.
“In Ireland, there was an increased chatter about food, then housing, and then downgrading transportation that preceded a spike in unemployment. In the United States, negative chatter about healthcare expenses increased, followed by negative employment chatter, and then increased use of public transportation preceded unemployment spikes.”
Since the project’s completion, he says government agencies from various countries have shown great interest in incorporating the best practices from the project into their existing real-time reporting environments.
Promising possibilities
According to Hsieh, the US and Ireland were chosen for the project because they were both English speaking nations, thus simplifying the analysis between the countries. At the same time, however, both maintained very distinct cultures and also had large amounts of official labour statistics on hand.
Since the success of its pioneer project, the UN Global Pulse has been researching wider applications for such real-time online data feedback mechanisms.
It also carried out another project with SAS in Indonesia to track the movement of food, fuel and job indicators.
Other applications it has considered include using anonymous telecommunications call records to monitor mass migration patterns and identifying signs of financial hardship from mobile account reload patterns; for instance, when people who used to reload US$5 (RM15) each time now only reload in US$1 (RM3) increments.
“There are also health related projects they are considering which are similar to Google’s study on flu trends, but perhaps using Twitter or Facebook instead,” says Hsieh, adding that social media trends have proven to be better indicators than web search patterns alone.
In terms of privacy issues, Hsieh says, “The UN sets for itself the strictest standards in protecting the privacy of global citizens in all its studies and research undertakings. It only uses publicly available data and its efforts are focused on reporting about significant trends and issues from a general macro level.”
When asked about the possibility of online information being unreliable or even false in nature, he responds, “In this case, it didn’t matter if the chatter was factual or just rumours. The fact that people were talking about something more than usual is what was relevant for analysis.”
Hsieh adds that some quarters had also questioned whether relying on online chatter would skew the results of the analysis, as those who conversed online the most tended to be of a younger age and came from wealthier backgrounds.
“It’s important to note,” he says, “that we account for that by only looking at how trends in the data correlate with the question we’re seeking to answer,” implying that such demographics did not have a significant bearing on the results.
Room to improve
On the whole, Hsieh admits that there are still areas to be ironed out in terms of how real-time online data analysis can be conducted.
“The processes used in mining data online for hidden trends and knowing which data sources are best suited for which types of problems will continue to improve with real world practice,” he says.
Nevertheless, Hsieh sees a bright future for the use of such analytics tools when placed in the hands of the UN and similarly, other national policymakers.
“Never before has the UN had the opportunity to have a conversation with all the global citizens as they draft and create new policies pertaining to complex socioeconomic issues. With SAS, they now have the ability to pose a question, listen to and analyse responses from the citizens of the entire world in order to help everyone make better informed decisions,” he says.
Related story:
Data dynamics
