Using Public Policy for Social Change - Part 9

Finding Evaluation Evidence for Policy Guidelines and Evaluations

Using Research in Policy Cycle

The policymaking cycle relies heavily on research and evidence at all its stages. In-depth analysis helps policymakers understanding the complexities of social problems, their underlying factors and outcomes, enabling accurate issue identification and informed agenda setting. Through research, we can generate policy ideas, analyzing various policy options, and guiding decisions on which policies to implement. Research is crucial for effective policy implementation, particularly in evaluating the processes involved. Certainly, program and policy evaluation represent a critical area of research, enabling us to determine the effectiveness of implemented policies and whether they achieve their intended impacts and outcomes. If data, research outcomes, and scientific evidence play a crucial role in the policymaking process, how is this information communicated to those involved? This occurs through various means. Many policymakers have received formal education and training that includes a strong focus on research. Additionally, individuals can certainly access scientific journals and reports that present research findings. 

How does scientific information get to people involved with policymaking?

• Formal education and training
• Scientific journals
• Media
• Legislative aids and other staff
• Lobbyists
• Social action and advocacy groups
• Expert testimony
• Government research
• Many other mechanisms

Experts are regularly asked to lend their expertise, including testifying before legislators and regulatory agencies. In many countries, there are several governmental agencies and departments that prioritize science and research. Yet, we recognize that public policy is not always driven by scientific findings and research. Beyond scientific information, a wide range of influences contribute to the policy process. What is the reason for this? While it's a complicated issue,  it largely has to do with three categories of barriers: those related to the research process itself, communication hurdles, and political obstacles. 

First and foremost, what barriers exist within the research process? Because strong studies with good internal validity are costly and take a long time to conduct, there is a substantial amount of research available that includes limitations and caveats.  There are also several important protections concerning research on and data collection from humans, which is certainly a positive thing. There are significant ethical concerns when it comes to researching human subjects, which can impede progress and limit the kinds of studies that can be conducted. Due to these reasons, implementing randomized control trials in the policy sector is difficult. Consequently, this makes it challenging to determine causal relationships, as we understand that association does not equate to causation. Yet, all too often, we must rely on association-based research to guide the policymaking process. Additionally, policymakers often find that the research process takes significantly longer than they would like. It can take a considerable amount of time, sometimes years, to witness the impact of policies and programs on outcomes. Moreover, some outcomes are difficult to assess, even within a short timeframe. As a result, it often takes considerable time for research findings to be published in journals or other types of publications. 

Additionally, several communication-related barriers hinder the use of research and evidence in the policy process. This also highlights that scientific research tends to be technical and highly specialized, which can make it difficult for laypeople, including policymakers, to grasp. Moreover, scientists often contribute to the challenge because their fields are filled with jargon and specialized terminology. Truthfully, researchers and scientists are not always the best at communicating their findings to general audiences. There are timing challenges too. Scientific insights might lag behind policymakers' needs due to delays in availability or communication. Lastly, media coverage of studies can be misleading. These situations arise because of valid reasons, journalists seek to communicate sophisticated concepts in a straightforward manner and at an elevated level. However, researchers possess specialized knowledge and technical expertise, often accompanied by numerous caveats regarding their findings, as no study is without its limitations.

Lastly, one of the most substantial barriers is political factors. Political factors frequently misrepresent or overlook research findings that conflict with their defined problems or policy objectives. This behavior is not limited to any single ideology or political party; it encompasses a broad spectrum of individuals, including politicians, their staff, lobbyists, advocacy organizations, trade associations, government bureaucrats, and even researchers themselves. At times, the misrepresentation of research results or the intentional data favoritism is a deliberate act. However, many individuals are frequently unaware of their own biases and how these biases influence their search for and use of research and evidence. 

Ultimately, the key point is that the policy process, While the policy process is rational in some respects, it is also influenced by a multitude of factors, with scientific information being just one among many. All policy is influenced by values, encompassing those related to political ideology, ethics, moral reasoning, and cultural norms and biases. While we all understand the crucial and undeniable impact of politics and values on the policymaking process, thorough research and analysis are essential at every stage of the policymaking process. The objective should be to develop evidence-based policy whenever feasible, as this approach is crucial for maintaining the core pillars of the public sector strong. It is essential for a government to support rather than hinder the economy, to utilize public resources efficiently, and to create and implement effective policies and programs. Additionally, it plays a critical role in addressing and reducing societal inequalities and inequities. Let me clarify that being objective— which is why we rely on reliable data and evidence— is not the same as being neutral. The generation and application of objective data and research evidence in the policy process can certainly guide conclusions and recommendations at every stage, including problem definition, agenda setting, policy formulation, decision-making, evaluations, and more. The aim is to utilize research to maintain objectivity regarding problems, their causes, and potential policy solutions. However, this does not imply that the data cannot also inform opinions and recommendations.

Systematic Literature Reviews

As policymakers, it is crucial to assess the effectiveness of different policies and to comprehend why some policies succeed while others do not achieve their intended goals.  Systematic literature reviews (SLRs) are an essential resource for comprehending the current landscape of evaluation research, supporting evidence-based decision-making, and enhancing scientific knowledge in public policy. Thus, it is crucial for policy professionals to develop the skills necessary to search for, identify, and summarize SLRs effectively. 

A Systematic Literature Review (SLR) is is a complete and comprehensive review of the existing research evidence on a particular topic, typically requiring a more structured and rigorous methodology than other types of literature reviews, such as scoping or argumentative reviews. SLRs characteristically frame the research discoveries within evaluations of the study designs' validity. Thus, SLRs are crucial for collecting and evaluating evidence in a reliable and unbiased manner. 

What is the importance of a Systematic Literature Review, and what are the methods for conducting one? Researchers conduct systematic literature reviews to draw comprehensive conclusions regarding the current state of evidence in both science and policy practice. In a systematic literature review, researchers gather all published studies relevant to a specific topic or clearly defined research question, following predetermined inclusion criteria. They then assess the quality of the research designs and findings critically, both overall and by the quality of each study design, in order to identify weaknesses, inconsistencies, and gaps in the evaluation research. 

Figure 1 illustrates the typical stages involved in performing a systematic literature review.

The initial stage of a systematic literature review entails formulating a clear question and defining the focus of the review. The questions should be sufficiently focused to ensure that a systematic literature review is feasible. The next stage is protocol development, where researchers establish the inclusion and exclusion criteria that will direct the selection of relevant studies. Criteria often include: 

• Intervention and Settings
• Peer-Reviewed vs. Grey Literature
• Language
• Study Designs

Researchers generally use a variety of strategies to locate potential studies and then apply the criteria to the studies they have identified. Furthermore, they develop and implement a coding form to review the intervention studies. Sample details that can be recorded for studies include:

• Dates 
• Setting 
• Population(s) under study 
• Statistical analysis approach

The significance of evaluating systematic literature reviews, in the policymaking process, it is crucial to identify and analyze high-quality evaluation studies of policies and programs to gain insights into what the overall body of evaluation research indicates about a particular policy strategy. "Systematic Literature Reviews (SLRs) can be valuable in answering questions about the efficacy of an intervention or the effects of exposure to a specific program or initiative." When presented in policy briefs, Systematic Literature Reviews (SLRs) offer essential information to policymakers, decision-makers, and other stakeholders who often lack the time or resources to thoroughly review and evaluate the evidence pertaining to a specific concern or policy issue. Moreover, program evaluation findings contribute to every stage of the policymaking cycle.

Finding systematic literature reviews, identifying systematic literature reviews doesn’t need to be an overwhelming task. Google Scholar can serve as a useful resource in your search for systematic literature reviews that you can include in your upcoming memo or policy brief. To improve your search, you can begin your policy topic of interest with the phrase “systematic literature review.” Some examples include:

• Google Scholar “systematic literature review” “early childhood education” 

• Google Scholar “systematic literature review” “early childhood education policy”

Finding Policy Resources for Issues You Care About

• Whole Child Policy Policy Toolkit
• OECD: Organisation for Economic Cooperation and Development Social Policies and Data
• Housing Policy Toolkit
• UNICEF Social Policy Resources
• Policy Link Resource

Public Policy Evaluation: A Real Story in Detroit, Michigan, United States

Post-Great Recession, Detroit struggled with a surge in tax foreclosures. As incomes declined, city officials were slow to adjust property taxes to reflect the decreasing property values. One of the many unfortunate repercussions of this crisis was that, alongside the foreclosure of numerous owner-occupied homes, many renter households were also evicted from properties that had been subject to tax foreclosure. As the owners of those homes, their landlords had neglected to pay the owed property taxes. These properties would subsequently be auctioned off at the Wayne County tax foreclosure sale. Often purchased in large quantities by investors who failed to maintain them, these properties were quickly abandoned, leading to greater instability and decline in Detroit's housing environment. In this context, the city, along with the United Community Housing Coalition and the Rocket Community Fund, initiated a program known as Make It Home. Under this program, tax foreclosed homes that housed renter families would be taken off the tax foreclosure list, allowing the tenants living there to purchase the properties at a significantly reduced price, often around $1,000. The Rocket Community Fund will take care of the remaining back taxes. The goal of the program was to keep homes in productive use while providing greater housing stability for these renting families. However, there were significant concerns about whether this program could succeed, as this low-income population might struggle to maintain stable homeownership due to the costs associated with owning a home. How would their situation compare to that of households with similar incomes that chose to remain renters? Poverty Solutions allocated funding for certain individuals, Faculty members at the University of Michigan will investigate these questions to contribute to the program's design. 

They conducted interviews with program participants and two different control groups. One group comprised individuals who were eligible for the program but chose different rental housing options, while another group included those who obtained homeownership through other avenues. To gain insight into the experiences of individuals who participated in the program. The findings were mixed. It was revealed that participants in the program experienced greater housing stability compared to those in the other two control groups. Nonetheless, their stability remained at serious risk. They encountered substantial housing repair expenses, and many quickly fell behind on their property taxes. This is precisely where the importance of high-quality evaluation is highlighted. The following year, the program continued to function as before, but with an added modification informed by the evaluation results. Households received up to $8,000 to cover emergency home repairs, enhancing their chances of achieving stability. They assessed this program as well and discovered that, although homeowners reported an improvement in perceived housing stability, many were still struggling to keep up with their property taxes. The following year, counseling services were introduced to help participants apply for property tax relief. After each new phase, they continued to assess the program's outcomes, resulting in adjustments and further evaluations, creating an ongoing cycle of improvement.

Source: coursera

Using Public Policy for Social Change - Part 8

Program Evaluation Examples Part 1

Let's start by analyzing the research design used to evaluate the Creating Moves to Opportunity Program in Seattle, Washington. We start by identifying the issue at hand, which in this instance is the difficulty of publicly subsidized housing that this program seeks to address. The United States invests over $45 billion each year in initiatives aimed at providing affordable subsidized housing, which includes the Housing Choice Voucher Program. This is sometimes called the Section 8 Program. Each year, over two million families, totaling around nine million Americans, benefit from vouchers that provide reduced rent for apartments and homes throughout the United States. Tenants pay a portion of the rent, while the government covers the rest by making direct payments to the landlords. Although housing vouchers allow families to rent anywhere in the city, the majority tend to remain in high-poverty, low-opportunity neighborhoods when they receive a new housing voucher. The Creating Moves to Opportunity Program in Seattle was designed to improve the relocation of families with housing vouchers to higher opportunity neighborhoods by addressing specific barriers to their mobility.

The Creating Moves to Opportunity Program consists of three essential elements (Intervention Components).

1. First, it provides search assistance to families with vouchers, which includes education about higher opportunity neighborhoods throughout the city, support in completing rental applications, and services to help locate housing.
2. The second component focuses on landlord engagement. Because not all landlords are willing to accept families with housing vouchers, this part of the intervention seeks to recruit more landlords throughout the city who are open to working with voucher clients, while also simplifying the approval process for their participation in the program and assisting with relocations to diverse neighborhoods. Furthermore, this intervention established a new insurance fund to cover property damage, addressing landlords' concerns that tenants might cause more harm than the security deposit could compensate for. This insurance fund aims to alleviate landlords' concerns, whether they are justified or not, regarding property damage.
3. Third, the program offers short-term cash assistance to families to support their relocation to different areas of the city. As is well known, moving within a city can be expensive, so the Creating Moves to Opportunity Program provides grants to help individuals accumulate the necessary funds to cover their moving costs. This includes costs like renting a moving truck, setting up new cable services, and assisting with application fees and security deposits. The average support provided to families through the program is around $1,100. The Creating Moves to Opportunity Program is currently being evaluated using a Randomized Controlled Trial (RTC) design. In this design, there is no pretest observation, but this is acceptable since the intervention and the families participating in the research study are all new to public housing vouchers. 

The counterfactual refers to the control or comparison group. Families recently approved for the Housing Voucher Program in Seattle were randomly assigned to participate in the Creating Moves to Opportunity Program, and its three components compared to receiving standard services. This includes providing information about the process and encouraging families to search for housing across the entire city, but without the additional support and resources offered by the new intervention. The researchers plan to monitor the families involved over an extended period. At present, they are conducting an impact evaluation that focuses on specific short- and near-term outcomes. This includes the types of neighborhoods where individuals live after being approved for the Housing Voucher Program, distinguishing between high - and low-opportunity neighborhoods, as well as whether they make use of their vouchers. Next, we will look at adults' feedback regarding their satisfaction with their neighborhood. 

Let’s explore some initial findings that compare outcomes between the treatment and control groups. In the intervention or treatment group, 54% lived in high-opportunity areas, 88% used the housing voucher for a new lease or rental agreement, and 68% expressed a high level of satisfaction with their neighborhood. In the control group, only 14% ended up living in a high-opportunity area after receiving a housing voucher. However, this was not because of a lack of voucher usage or participation in the program; 84% utilized their housing vouchers for new leases or rental agreements, yet only a small percentage took advantage of the opportunity to relocate to a different neighborhood. In the control group, just 33% reported a high level of satisfaction with their neighborhood. The preliminary results from the Creating Moves to Opportunity study in Seattle suggest that this program has substantially increased the number of families living in higher opportunity areas within the city and has also improved family satisfaction with their neighborhoods. It is widely recognized that individuals typically prefer to remain in familiar neighborhoods or communities. The findings presented here are quite fascinating, and many cities are eagerly anticipating further updates from the evaluation team. 

Program Evaluation Examples Part 2

Let’s take a look at an example of a time series design. Keep in mind that a time series research design analyzes data on trends and outcomes of interest over various time periods. Analyzing shifts in the slope or intercept of the trend line at the intervention or policy change indicated by x. The primary threat to the internal validity of this design is the influence of history. An alternative event may have taken place concurrently with the policy change, which could account for some or all of the changes observed after the intervention. 

This graph depicts one of several potential scenarios within the time-series design. In this particular case, we see a flat trend, with the slope of the line remaining consistent prior to the implementation of the intervention. However, after the intervention or policy change, the line's slope begins to decline. Following the intervention or policy change, the slope of the line starts to decrease. For example, an evaluation of the impact of a fare increase in a metropolitan subway system might reveal that an increase in prices, following a long period of stable fares, led to a subsequent decline in ridership. In this case, the intervention was the rise in subway fares, and after this change, ridership could decrease

Now, let's turn our attention to evaluating a significant change in alcohol policy and regulation that took place in Russia in 2006. The primary question driving this evaluation was: given the established connection between alcohol abuse and suicide, did the 2006 policy change impact the national suicide rate? The 2006 policy involved several components, including new regulations on both the volume of sales and the quality of products available in the market. The 2006 policy implemented new regulations requiring alcohol production and distribution facilities to register with the government and undergo enhanced monitoring. Both types of regulations operated similarly to a tax, resulting in a decrease in the number of alcohol producers and distributors, as well as higher prices for consumers.

Researchers utilized a time-series research approach to track monthly suicide rates in the country, comparing data from both before and after the major policy changes. Additionally, they conducted this analysis separately for males and females. Their primary findings indicated significant seasonal variations in suicide rates. It is essential to take seasonal variations and the outcomes under investigation into account in a time series analysis. The results also showed that monthly suicide counts for both males and females decreased consistently during the entire stable period. There was a steady decline in suicide rates over the years of the time series, which started before the intervention was implemented. Nevertheless, statistical analysis of the time-series data revealed an immediate and sustained 9% reduction in male suicides.

No similar outcome was observed for females. Here is a clear graph displaying the results. The time series demonstrates the seasonal or monthly fluctuations in suicide deaths each year. The graph also depicts a downward trend for males, which appears to accelerate following the 2006 policy change. We can visually inspect the figure to identify the main findings. However, the actual analysis was quite complex, involving advanced statistical techniques such as regression methods to determine whether the trend line altered after the intervention. Whether you choose to be a detailed policy evaluator or not, having a basic understanding of this critical aspect of the policy-making cycle will be vital for your work.

Implementation Evaluation: A Real Story

For instance, the federal program established in Michigan provides additional nutrition and food support for pregnant individuals and children under five. This initiative, known as the Women, Infants, and Children (WIC) Nutrition Benefit, has seen many eligible individuals fail to enroll. Evaluators recognize that accessing the program can be challenging; some people are unaware of their eligibility, while others do not know how or where to start the application process. To address this, the evaluators implemented a new strategy by sending text messages to potential clients who likely qualify but are not enrolled. The evaluators aimed to determine whether (a) informing them of their likely eligibility for benefits and (b) providing a simple, one-touch method to initiate the application would enhance their access to the program. 

To address the question, the evaluator conducted a Randomized Control Trial. They compared WIC enrollment rates between a test group that received text messages and a control group that did not. Ultimately, the test group experienced a 91% increase in enrollment compared to the control group. This project made all the challenges faced during statistics at the Ford School completely worthwhile.

It’s not uncommon to discover insights during an evaluation that suggest the need for a new agenda item. For instance, a recent evaluation of a food assistance program revealed that many low-income individuals and families facing hunger did not qualify for benefits simply because their assets exceeded the $5,000 limit, which included the value of their vehicle.

The evaluation revealed that the evaluator needed to shift the focus from preventing fraud and abuse to aligning with the economic realities of today. It highlighted that individuals with $6,000 to $10,000 in assets, who are struggling with hunger, should be eligible for food assistance. Consequently, the department aimed to increase the asset limit, exclude car values from the calculation, and allow most applicants to self-report their net worth. This update was in line with the governor's initiative to address food insecurity and poverty in Michigan. Additionally, it streamlined the application process, saved time, and reduced paperwork for state caseworkers.

Today, the evaluator continues to assess the implementation of all their support programs to identify who is being served and who is being overlooked, in order to develop the next set of agenda items.

Real-World examples of Experimental Design: Voter Information & Political Selection in India

An example of experimental design can be observed in India. In general, India is known as the most populous democracy in the world, with over 800 million registered voters. This environment is marked by a significant prevalence of criminal charges against both political candidates and elected officials. For instance, 34% of current members of India’s national parliament face serious criminal charges, including offenses such as assault, kidnapping, attempted murder, and even murder.

Furthermore, recent high-quality empirical research conducted by the study's authors demonstrates a causal relationship between the presence of legislators facing multiple or serious criminal charges and subsequent declines in economic growth and insufficient public service delivery. With this in mind, a paper authored by an assistant professor and his colleagues at the Ford School of Public Policy at the University of Michigan aims to address several important research questions.

The first research question focuses on the extent to which voters in India are aware of candidates' criminal charges and whether they feel indifferent to or may even view such criminality positively in their elected officials. There is a general notion that voters might prefer officials with criminal backgrounds, as it could imply they possess the capability to hold office, irrespective of the legality of their actions. Conversely, it is also possible that limited access to information about candidates' criminal records is preventing many voters from rejecting those with serious criminal histories or multiple charges.

The second research question investigates whether a gentle approach to providing information—specifically through mobile phone messaging and voice/text communication—can help close any existing information gap and subsequently influence voting behavior. The focus is on the Northern Indian state of Uttar Pradesh, which is highlighted in red on the map. Here are some additional details about Uttar Pradesh (UP).

First and foremost, Uttar Pradesh is the most populous state in India, with a current population exceeding 200 million residents. It is also one of the poorest states in the country, with a substantial rural population that makes up nearly 80% of its inhabitants. This situation suggests that many individuals may encounter difficulties in accessing publicly available information about candidates' criminal records on websites.

On the other hand, researchers noted a high prevalence of mobile phone usage in these areas. At the time of the study, more than 85% of households owned at least one mobile phone, suggesting that mobile messaging could be an effective means of communicating information about candidates' criminal charges. Uttar Pradesh is not an exception in having a substantial number of political candidates and elected officials with criminal charges. In fact, during the 2017 State Assembly elections, it was discovered that 25% of the winning candidates faced serious criminal charges. If the researchers concentrate more intensely on the particular region within Uttar Pradesh where their experiments were conducted, they can provide several relevant statistics. Notably, 80% of electoral contests involved at least one major party or incumbent candidate facing some form of criminal charge, while 26% of these contests included at least one candidate charged with murder or attempted murder. Additionally, when analyzing candidates with criminal charges, it was discovered that they were involved in an average of 2.2 criminal cases each. What was the focus of the intervention, and what methods did the researcher employ to implement it?

To begin, it is important to note that the 2007 State Assembly elections in Uttar Pradesh were conducted over seven distinct phases. The elections were organized by geographic regions, with election days scheduled in phases over several weeks, and the researchers operated within Phase 4. Two days before election day, the researcher sent voice and text messages to mobile phone subscribers, informing them about any criminal charges against major party and incumbent candidates, along with the number of cases associated with those charges. It is important to highlight that a significant challenge in initiating this project was securing a partnership with a willing telecom company. As expected, there were concerns regarding potential backlash from political parties or individual politicians due to their involvement in disseminating information about candidates' criminal charges. 

Now, as the researcher examines the details of the experimental design, they should take into account their experimental sample and the precise locations of their work. Within the phase 4 area, there were roughly 3,800 villages and about 5,000 polling stations—locations where voters would go to cast their ballots on election day. These villages were spread across 38 assembly constituencies, which are regions that elect one representative to serve in the state assembly. The researchers were operating within 38 out of a total of 403 constituencies. To ensure they were focusing on relatively average villages, they restricted their sample to those with either one or two polling stations. Furthermore, these villages neither had excessively large nor small populations, and they did not exhibit extreme variations in Vodafone Idea's mobile phone subscriber coverage. To provide context, the villages in the sample had an average of approximately 1,200 registered voters. To determine the average impact of the messaging treatment, the researcher needed to randomly assign villages to either the treatment group or the control group. These villages included those where mobile phone subscribers received the researcher’s voice and text messages containing information about candidate criminality, as well as the control group, where no such messages were sent. The researcher allocated approximately 1/3 of the villages, roughly 3,800, to the treatment group and 2/3, also roughly 3,800, to the control group. 

By means of  random assignment, the researcher can assess the average impact of the messaging treatment on voting outcomes by comparing the average results at polling stations in treatment villages with those in control villages. Significantly, the researcher ultimately sent messages to more than 370,000 individuals in the treatment villages. Essentially, the implementation of the messaging treatment allowed the researcher to evaluate the intervention's effects using multiple datasets. At the outset, a key consideration for the researcher was gaining access to information about candidates, their criminal cases, and the related charges. 

Fortunately, the Indian government requires all candidates seeking office to disclose this information, which is then made public. Consequently, the researcher was able to obtain this data. Secondly, the researcher needed various datasets related to the villages. The first dataset concerned the number of mobile phone subscribers in each village, which was sourced from Vodafone Idea. The second dataset included population data for each village, which the researcher obtained from public census data. Additionally, the researcher required shapefiles—essentially digital maps for each village—which were acquired from the company ML Info Map India. The researcher needed those map files to combine them with polling station location data. This enabled the researcher to ascertain whether each polling station was located in a treatment village that received messages or in a control village that did not. 

To evaluate the effects of the treatment on voting, the researcher needed polling station-level voting results, which were publicly accessible from the government of Madhya Pradesh. By merging all these datasets, the researcher could first carry out the experiment and then assess the impact of their intervention. The researchers are presenting a figure that summarizes the key voting outcomes from their mobile information provision experiment, categorizing the candidates into five groups based on the severity of their criminal charges. On the left are the candidates with the least severe charges, specifically those with no pending criminal charges. Next are the candidates whose most serious charge is non-violent. Following that are candidates with violent charges that do not involve murder. The fourth group includes those facing attempted murder charges, and finally, the rightmost group consists of candidates charged with murder-related offenses. 

This analysis explores the average effects on voting outcomes for candidates in villages that received the messaging treatment compared to those that did not. Being situated above the red line indicates that candidates of a certain type experience positive voting outcomes in treatment villages, while being below the red line suggests that these candidates encounter negative impacts on their voting results in treatment villages.

The researcher has identified a distinct negative correlation between the voting impacts of the messaging treatment and the severity of charges faced by candidates. Notably, upon closer examination, the leftmost portion of the graph shows a positive impact on votes for candidates with no criminal charges at all.

We can conclude that candidates without criminal charges experienced an average increase of 2.8% in votes in areas that received the messaging treatment compared to those that did not. In contrast, candidates facing murder charges experienced a substantial average decrease of 11.1% in votes in the same comparison. This analysis reveals a negative correlation between the voting impacts of our messaging treatment and the severity of candidates’ charges. Beyond the specifics of these research findings, it is crucial to consider their implications for policymaking and policy decisions.

The primary implications the researcher aims to highlight are as follows:

They observed that voters did respond to this information. This contradicts a scenario in which most voters are already aware of the candidates' criminal charges and remain indifferent or view them positively. This indicates a potential positive outcome from conducting these information provision campaigns.

Secondly, this relates to the nature of the information provision. They found that their approach was relatively low-profile, utilizing a few text messages and voice recordings sent out prior to the elections. The positive results we achieved are encouraging for two main reasons. First, this method reduces the risk of political parties intervening during the information dissemination process. For example, if we were to conduct door-to-door outreach or hold public meetings in a village center, there would be a significantly greater chance of political parties becoming aware of these activities and disrupting the sharing of information. Second, from a budgetary standpoint, this approach is more cost-effective for reaching a target audience compared to methods like in-person outreach or purchasing advertising space in newspapers or on radio. Therefore, if our aim is to maximize outreach within a limited budget, these findings are quite promising.

Finally, when considering not only the specific context of this experiment but also the broader implications for the political and logistical feasibility of future campaigns—whether on a large scale within a single electoral cycle or across multiple cycles—several positive insights emerge from this work. Firstly, it is noteworthy that we encountered no negative repercussions from political parties or politicians, either for ourselves as authors, our team, or Vodafone Idea. This is encouraging because, in future scenarios where one might aim to scale up a similar intervention in subsequent election cycles, it provides a clear example to telecom companies that may be hesitant. They can refer to this instance where there were no adverse effects for the telecom involved.

However, it is crucial to acknowledge that while operated in thousands of villages, this was still a relatively small portion of the overall election landscape. If such efforts were to be conducted across multiple elections over time, it becomes more probable that political parties would take notice and potentially attempt to interfere with the information dissemination or undermine its effectiveness. Despite this, the current indications are encouraging regarding the potential benefits of this intervention in enhancing the quality of elected officials in India, particularly with regard to criminal charges.

Source: coursera