title.South Africa Security Tracker
date. Spring 2014
location. South Africa and New York City
Role: Design Researcher and Strategist on a Graduate Team of Four
Problem: The Council on Foreign Relations needed an online platform to track service delivery protests complement their research. Service Delivery protests are protests in response to the lack of adequacy in public services). Tracking service delivery protests manually is a time-consuming and costly process. What would be the most appropriate method for consolidating information on protests from different media sources while considering reliability of sources, relevance of data, and external validity of data?
Process: We explored the problem through field research and testing numerous online media-monitoring platforms.
Outcome: We developed a methodology and technical specifications for a platform that uses online news reports to automate the process of tracking service delivery protests. The result was a minimum viable product that included aggregated news sources through CartoDB and AlchemyAPI, and two methods of tracking protests (manually in automatically) for CFR to start using, with an analysis of pros and cons of each method.
To both evaluate the methodology and supplement the media research component, we traveled to South Africa to conduct interviews and field research from March 14 to 28. The field interviews included two distinct parts. The first part of the fieldwork was to engage with expert analysts of South African media to determine whether the South African media was a reliable resource for information on urban township unrest. We also conducted interviews with consumers of the South African media. The interviews shed light on whether the project’s findings concur with events and general feelings and findings on the ground.
Our interviews and analyis after testing multiple methodologies and platforms revealed that: tracking mentions of service delivery protests in the online news media in South Africa is valuable in itself, as a measure of social engagement, but it has numerous limitations. First, news mentions do not necessarily correlate with actual service delivery protests. In fact, the definition of service deliver protest was not always clear and used interchangeably across different reports and news sources. Therefore, it is important for CFR to maintain local partnerships with existing organizations that track individual protests, instead of just media mentions, in order to supplement its analysis. Second, selection bias is hard to avoid with a methodology that utilizes news reports. We have attempted to mitigate this concern by adding as many local news sources from across South Africa to our research as possible. However, technological limitations and language barriers have limited the scope of this strategy. Third, an automated methodology has inherent limitations due to the fact that computer algorithms, while improving rapidly, are still far less capable than humans at understanding context.
After a thorough analysis of existing media-monitoring platforms, we determined that the best course of action was to develop a new platform that integrated a number of existing online tools. This modular approach makes the platform more agile, has low recurring costs, and can be developed relatively inexpensively. Building upon the methodology CFR used to build the Nigeria Security Tracker, we created a roadmap to design an online platform that automates the process of tracking and aggregating data on these protests and subsequently visualizing mentions of service delivery protests, both geographically and over time.
Our research found that the automated and manual methodology both yield remarkably similar results in some areas, particularly regarding such as dimensions like the proportion of articles about service delivery protest that mention violence over time. The automated methodology is also fairly accurate at identifying general geographic trends at the municipal level, and categories of protest. A statistical analysis showed that there was no significant difference in the proportions of mentions of categories of protests between the two methodologies. However, the automated methodology falls short when trying to ascertain more granular geographic information, identify individual protests, or determine whether the article is about a protest, protests in general, or if service delivery protest is just a minor component of the article.
With our product of two methods of tracking service delivery protests using appropriate technology, pulling from a diverse set of media reports and news sources, and further recommendations supported by our field research, CFR was able to begin using the online tools to track mentions of service delivery protests. Despite some limitations, the organization was able to test out the product with greater hope that the tool will be a source of real-time data on protests that will be useful for policy makers, think tanks, journalists, academics and the general public.
To read more information, click here.
.