Gunhild Berg and Bilal Zia, April 2013
Our latest impact note provides one of the still rare examples of where an encouragement design was successful in getting sufficient take-up to measure the effects of a program, and shows how an aggregate policy can be evaluated.
Financial education is important, yet there is a considerable knowledge gap in how best to deliver it. The literature on careful evaluations of financial literacy is small but growing, and has moved away from classroom based interventions to more innovative delivery mechanisms such as videos and DVDs. Yet, the scope and reach of even the best produced DVDs is limited on the supply side, and attracting viewership can be significantly challenging on the demand side.
Entertainment media offers not only broader outreach since nearly every household nowadays has a TV, but also a captive audience. Furthermore, as emotional connections are established between a show and its audience, the program provides a potentially powerful platform for communicating messages and influencing behavior. There is considerable evidence, especially in the health and education fields, on the success of media campaigns in improving social behavior.
We evaluate financial education through a popular television soap opera in South Africa, “Scandal!” The intervention entailed a 2 month long storyline featuring one of the main characters of the soap borrowing excessively through hire-purchase, gambling, and falling into a debt trap; and eventually seeking help to find her way out. We study the effectiveness of these messages through three quantitative surveys and three qualitative focus groups, with the former providing insight on the mechanism of measured impacts.
The measurement challenge
The methodological challenge associated with such an evaluation is non-trivial. Critically, it is impossible to externally restrict viewership since the soap opera is nationally televised with no variation in TV signal strength, at least in urban areas where this study focuses.
We employ a symmetric encouragement where a randomly selected treatment group was encouraged through financial incentives to view Scandal, and another set of individuals – the control group – was similarly encouraged through identical financial incentives to view a comparable soap opera (“Muvhango”) that overlaps with Scandal in television primetime. Hence, we were able to create an artificial separation between treatment and control groups, and our analysis shows that adherence to these allocations was very high – more than 95%.
What do we find?
- The analysis finds individuals assigned to watch Scandal had significantly higher financial knowledge on issues highlighted in the soap opera storyline, in particular messages delivered by the leading character – a 4.5 percentage point increase over the control group.
- We find an increase in the likelihood of borrowing through formal channels – 22 percent compared to 13 percent in the control group, and a greater likelihood to borrow for productive purposes such as investments in household durable goods and vehicles. Direct measures of hire-purchase use also show a significant decline – 15 percent compared to 19 percent in the control group. Further, we find a 5 percentage point lower propensity to gamble among the treatment group.
- One aspect of the soap storyline that did not sustain traction beyond immediate effects was a public call to action for the South African National Debt Mediation Association (NDMA). The soap featured an NDMA counselor who helped the main character build a financial plan to enable her to save enough and repay her debts, yet the longer term analysis shows no recall of the NDMA or even memory that a debt counselor appeared in the soap storyline.
Mechanism of impact
In order to understand the mechanism of impacts, we conducted three focus group discussions three months after our final quantitative survey. In these meetings, the focus group leaders were unable to elicit recall of NDMA from the participants and only a few women seemed to remember, and they associated their memory more with the fact that the NDMA character was female rather than her representation of the NDMA. In contrast, all participants readily recalled the main soap character and aspects of the financial storyline delivered by her. Upon further inquiry, they attributed better recall to the fact that the main character was a mainstay in the soap opera, and was popular and likeable.
Upon reflection, it is clear that the NDMA advisor was external to the main soap and only appeared in 2-3 episodes, never to reappear again. Hence, the audience did not get the opportunity to establish or maintain a connection or following with this character.
- Our findings show an important role for entertainment media as an accessible and important tool for policymakers to deliver carefully designed educational messages that resonate with the audience and can influence financial knowledge and behavior.
- Our analysis suggest that emotional connections and familiarity with media personalities certainly play a role in motivating knowledge and behavior change among viewers, and that harnessing such potential can be an important channel for achieving development impact.
- Finally, our methodology illustrates the benefits of complementing quantitative analysis with qualitative work to better understand mechanisms behind measured impacts.
For further reading see:
Berg, Gunhild, and Bilal Zia. 2013. “Harnessing Emotional Connections to Improve Financial Decisions – Evaluating the Impact of Financial Education through Mainstream Media,” World Bank Policy Research Working Paper no. 6407, April 2013.