Hostname: page-component-669899f699-b58lm Total loading time: 0 Render date: 2025-05-01T16:35:35.377Z Has data issue: false hasContentIssue false
  • English
  • Français

Real, but Limited: A Meta-Analytic Assessment of Framing Effects in the Political Domain

Published online by Cambridge University Press:  27 August 2020

Eran Amsalem Open the ORCID record for Eran Amsalem [Opens in a new window]  and
Alon Zoizner
Eran Amsalem*
Affiliation:
Department of Communication, Hebrew University of Jerusalem, Israel
Alon Zoizner
Affiliation:
Department of Political Science, Hebrew University of Jerusalem, Israel
*
*Corresponding author. E-mail: eran.amsalem@mail.huji.ac.il
Get access Rights & Permissions [Opens in a new window]

Abstract

In the past three decades, scholars have frequently used the concept of framing effects to assess the competence of citizens' political judgments and how susceptible they are to elite influence. Yet prior framing studies have reached mixed conclusions, and few have provided systematic cumulative evidence. This study evaluates the overall efficacy of different types of framing effects in the political domain by systematically meta-analyzing this large and diverse literature. A combined analysis of 138 experiments reveals that when examined across contexts, framing exerts medium-sized effects on citizens' political attitudes and emotions. However, framing effects on behavior are negligible, and small effects are also found in more realistic studies employing frame competition. These findings suggest that although elites can influence citizens by framing issues, their capacity to do so is constrained. Overall, citizens appear to be more competent than some scholars envision them to be.

Keywords

framing effectsmeta-analysiscitizen competencepolitical attitudesmedia effects
Type
Article
Information
British Journal of Political Science , Volume 52 , Issue 1 , January 2022 , pp. 221 - 237
Copyright
Copyright © The Author(s) 2020. Published by Cambridge University Press

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable

References

Aarøe, L (2011) Investigating frame strength: the case of episodic and thematic frames. Political Communication 28(2), 207–226.CrossRefGoogle Scholar
Achen, CH and Bartels, LM (2017) Democracy for Realists: Why Elections Do Not Produce Responsive Government. Princeton, NJ: Princeton University Press.Google Scholar
Ajzen, I (1991) The theory of planned behavior. Organizational Behavior and Human Decision Processes 50(2), 179–211.10.1016/0749-5978(91)90020-TCrossRefGoogle Scholar
Amsalem, E and Zoizner, A (2020) “Replication Data for: Real, but Limited: A Meta-Analytic Assessment of Framing Effects in the Political Domain”, https://doi.org/10.7910/DVN/YJIZBJ, Harvard Dataverse, V1, UNF:6:a16wtad7iEyCjj9UfKIZBw== [fileUNF]CrossRefGoogle Scholar
Arceneaux, K (2012) Cognitive biases and the strength of political arguments. American Journal of Political Science 56(2), 271–285.CrossRefGoogle Scholar
Benoit, WL, Leshner, GM and Chattopadhyay, S (2007) A meta-analysis of political advertising. Human Communication 10(4), 507–521.Google Scholar
Bolsen, T, Druckman, JN and Cook, FL (2014) Communication and collective actions: a survey experiment on motivating energy conservation in the US. Journal of Experimental Political Science 1(1), 24–38.10.1017/xps.2014.2CrossRefGoogle Scholar
Bonett, DG (2010) Varying coefficient meta-analytic methods for alpha reliability. Psychological Methods 15(4), 368–385.CrossRefGoogle ScholarPubMed
Borenstein, M et al. (2009) Introduction to Meta-Analysis. Chichester: Wiley.CrossRefGoogle Scholar
Brader, T and Marcus, GE (2013) Emotions and political psychology. In Huddy, L, Sears, D and Levy, J (eds), Oxford Handbook of Political Psychology, 2nd Edn. New York: Oxford University Press, pp. 165–204.Google Scholar
Brader, T, Valentino, NA and Suhay, E (2008) What triggers public opposition to immigration? Anxiety, group cues, and immigration threat. American Journal of Political Science 52(4), 959–978.CrossRefGoogle Scholar
Brugman, BC and Burgers, C (2018) Political framing across disciplines: evidence from 21st-century experiments. Research & Politics 5(2), 1–7.CrossRefGoogle Scholar
Cacciatore, MA, Scheufele, DA and Iyengar, S (2016) The end of framing as we know it… and the future of media effects. Mass Communication and Society 19(1), 7–23.CrossRefGoogle Scholar
Chong, D and Druckman, JN (2007a) Framing public opinion in competitive democracies. American Political Science Review 101(4), 637–655.CrossRefGoogle Scholar
Chong, D and Druckman, JN (2007b) Framing theory. Annual Review of Political Science 10, 103–126.CrossRefGoogle Scholar
Cohen, J (1988) Statistical Power Analysis for the Behavioral Sciences. Hillsdale, NJ: Lawrence Erlbaum.Google Scholar
Druckman, JN (2001a) The implications of framing effects for citizen competence. Political Behavior 23(3), 225–256.CrossRefGoogle Scholar
Druckman, JN (2001b) On the limits of framing effects: who can frame? The Journal of Politics 63(4), 1041–1066.CrossRefGoogle Scholar
Druckman, JN and Kam, CD (2011) Students as experimental participants. In Druckman, JN, Green, DP, Kuklinski, JH and Lupia, A (eds), Cambridge Handbook of Experimental Political Science. Cambridge, MA: Cambridge University Press, pp. 41–57.CrossRefGoogle Scholar
Druckman, JN and Leeper, TJ (2012) Learning more from political communication experiments: pretreatment and its effects. American Journal of Political Science 56(4), 875–896.CrossRefGoogle Scholar
Druckman, JN and McDermott, R (2008) Emotion and the framing of risky choice. Political Behavior 30(3), 297–321.10.1007/s11109-008-9056-yCrossRefGoogle Scholar
Duval, S and Tweedie, R (2000) Trim and fill: a simple funnel-plot–based method of testing and adjusting for publication bias in meta-analysis. Biometrics 56(2), 455–463.10.1111/j.0006-341X.2000.00455.xCrossRefGoogle ScholarPubMed
Fishbein, M and Ajzen, I (1975) Beliefs, Attitude, Intention and Behavior: An Introduction to Theory and Research. Reading, MA: Addison-Wesley.Google Scholar
Gerber, AS et al. (2018) The comparative effectiveness on turnout of positively versus negatively framed descriptive norms in mobilization campaigns. American Politics Research 46(6), 996–1011.CrossRefGoogle Scholar
Hale, JL, Householder, BJ and Greene, KL (2002) The theory of reasoned action. In Dillard, JP and Pfau, M (eds), The Persuasion Handbook: Developments in Theory and Practice. Thousand Oaks, CA: Sage, pp. 259–286.CrossRefGoogle Scholar
Hedges, LV, Tipton, E and Johnson, MC (2010) Robust variance estimation in meta-regression with dependent effect size estimates. Research Synthesis Methods 1(1), 39–65.CrossRefGoogle ScholarPubMed
Hopkins, DJ (2018) The exaggerated life of death panels? The limited but real influence of elite rhetoric in the 2009–2010 health care debate. Political Behavior 40(3), 681–709.CrossRefGoogle Scholar
Howe, LC and Krosnick, JA (2017) Attitude strength. Annual Review of Psychology 68, 327–351.CrossRefGoogle ScholarPubMed
Hunter, JE and Schmidt, FL (2004) Methods of Meta-Analysis: Correcting Error and Bias in Research Findings. Thousand Oaks, CA: Sage.CrossRefGoogle Scholar
Ioannidis, JPA and Lau, J (2001) Evolution of treatment effects over time: empirical insight from recursive cumulative meta-analyses. Proceedings of the National Academy of Sciences 98(3), 831–836.10.1073/pnas.98.3.831CrossRefGoogle Scholar
Iyengar, S (1991) Is Anyone Responsible? Chicago, IL: University of Chicago Press.10.7208/chicago/9780226388533.001.0001CrossRefGoogle Scholar
Kalla, JL and Broockman, DE (2018) The minimal persuasive effects of campaign contact in general elections: evidence from 49 field experiments. American Political Science Review 112(1), 148–166.CrossRefGoogle Scholar
Koch, T and Peter, C (2017) Effects of equivalence framing on the perceived truth of political messages and the trustworthiness of politicians. Public Opinion Quarterly 81(4), 847–865.CrossRefGoogle Scholar
Kühberger, A (1998) The influence of framing on risky decisions: a meta-analysis. Organizational Behavior and Human Decision Processes 75(1), 23–55.CrossRefGoogle ScholarPubMed
Kuklinski, JH and Quirk, PJ (2001) Conceptual foundations of citizen competence. Political Behavior 23(3), 285–311.CrossRefGoogle Scholar
Kurdi, B et al. (2019) Relationship between the implicit association test and intergroup behavior: a meta-analysis. American Psychologist 74(5), 569–586.CrossRefGoogle ScholarPubMed
Lecheler, S and De Vreese, CH (2012) News framing and public opinion: a mediation analysis of framing effects on political attitudes. Journalism & Mass Communication Quarterly 89(2), 185–204.CrossRefGoogle Scholar
Lecheler, S, de Vreese, CH and Slothuus, R (2009) Issue importance as a moderator of framing effects. Communication Research 36(3), 400–425.CrossRefGoogle Scholar
Lecheler, S, Schuck, ART and de Vreese, CH (2013) Dealing with feelings: positive and negative discrete emotions as mediators of news framing effects. Communications 38(2), 189–209.CrossRefGoogle Scholar
Leeper, TJ (2017) How does treatment self-selection affect inferences about political communication? Journal of Experimental Political Science 4(1), 21–33.CrossRefGoogle Scholar
Leeper, TJ and Slothuus, R (2015) Can Citizens Be Framed? How Information More than Emphasis Changes Opinions. Working paper. Department of Political Science, Aarhus University.Google Scholar
Levine, AS and Kline, R (2017) A New approach for evaluating climate change communication. Climatic Change 142(1), 301–309.10.1007/s10584-017-1952-xCrossRefGoogle Scholar
Malhotra, N and Margalit, Y (2010) Short-term communication effects or longstanding dispositions? The public's response to the financial crisis of 2008. The Journal of Politics 72(3), 852–867.10.1017/S0022381610000216CrossRefGoogle Scholar
Matthes, J (2009) What's in a frame? A content analysis of media framing studies in the world's leading communication journals, 1990–2005. Journalism & Mass Communication Quarterly 86(2), 349–367.10.1177/107769900908600206CrossRefGoogle Scholar
Nabi, RL (2002) Discrete emotions and persuasion. In Dillard, JP and Pfau, M (eds), The Persuasion Handbook: Developments in Theory and Practice. Thousand Oaks, CA: Sage, pp. 289–308.10.4135/9781412976046.n15CrossRefGoogle Scholar
Nabi, RL et al. (2019) Can emotions capture the elusive gain-loss framing effect? A meta-analysis. Communication Research. doi: 10.1177/0093650219861256.Google Scholar
Nelson, TE, Clawson, RA and Oxley, ZM (1997) Media framing of a civil liberties conflict and its effect on tolerance. American Political Science Review 91(3), 567–583.CrossRefGoogle Scholar
Newman, BJ et al. (2015) Easing the heavy hand: humanitarian concern, empathy, and opinion on immigration. British Journal of Political Science 45(3), 583–607.CrossRefGoogle Scholar
O'Keefe, DJ (2013) The relative persuasiveness of different message types does not vary as a function of the persuasive outcome assessed: evidence from 29 meta-analyses of 2,062 effect sizes for 13 message variations. Annals of the International Communication Association 37(1), 221–249.10.1080/23808985.2013.11679151CrossRefGoogle Scholar
O'Keefe, DJ and Jensen, JD (2007) The relative persuasiveness of gain-framed loss-framed messages for encouraging disease prevention behaviors: a meta-analytic review. Journal of Health Communication 12(7), 623–644.10.1080/10810730701615198CrossRefGoogle ScholarPubMed
Panagopoulos, C (2010) Affect, social pressure and prosocial motivation: field experimental evidence of the mobilizing effects of pride, shame and publicizing voting behavior. Political Behavior 32(3), 369–386.10.1007/s11109-010-9114-0CrossRefGoogle Scholar
Powell, TE et al. (2015) A clearer picture: the contribution of visuals and text to framing effects. Journal of Communication 65(6), 997–1017.CrossRefGoogle Scholar
Quattrone, GA and Tversky, A (1988) Contrasting rational and psychological analyses of political choice. American Political Science Review 82(3), 719–736.CrossRefGoogle Scholar
Quick, BL et al. (2014) Prospect theory, discrete emotions, and freedom threats: an extension of psychological reactance theory. Journal of Communication 65(1), 40–61.CrossRefGoogle Scholar
Scheufele, DA (1999) Framing as a theory of media effects. Journal of Communication 49(1), 103–122.10.1111/j.1460-2466.1999.tb02784.xCrossRefGoogle Scholar
Scheufele, DA and Iyengar, S (2017) The state of framing research: a call for new directions. In Kenski, K and Hall Jamieson, K (eds), The Oxford Handbook of Political Communication Theories. New York: Oxford University Press, pp. 619–632.Google Scholar
Schmitt-Beck, R (2012) Comparing effects of political communication. In Esser, F and Hanitzsch, T (eds), The Handbook of Comparative Communication Research. New York: Routledge, pp. 400–411.Google Scholar
Schuldt, JP, Roh, S and Schwarz, N (2015) Questionnaire design effects in climate change surveys: implications for the partisan divide. The ANNALS of the American Academy of Political and Social Science 658(1), 67–85.CrossRefGoogle Scholar
Sears, DO (1986) College sophomores in the laboratory: influences of a narrow data base on social psychology's view of human nature. Journal of Personality and Social Psychology 51(3), 515–530.10.1037/0022-3514.51.3.515CrossRefGoogle Scholar
Simonsohn, U, Nelson, LD and Simmons, JP (2014) P-curve: a key to the file-drawer. Journal of Experimental Psychology: General 143(2), 534–547.CrossRefGoogle Scholar
Sniderman, PM and Theriault, SM (2004) The structure of political argument and the logic of issue framing. In Saris, WE and Sniderman, PM (eds), Studies in Public Opinion. Princeton, NJ: Princeton University Press, pp. 133–165.CrossRefGoogle Scholar
Soroka, SN (2014) Negativity in Democratic Politics: Causes and Consequences. New York: Cambridge University Press.CrossRefGoogle Scholar
Zaller, JR (1992) The Nature and Origins of Mass Opinion. New York: Cambridge University Press.CrossRefGoogle Scholar
Zoizner, A (2018) The consequences of strategic news coverage for democracy: a meta-analysis. Communication Research. doi: 10.1177/0093650218808691.Google Scholar
Supplementary material: File

Amsalem and Zoizner supplementary material

Online Appendix

Download Amsalem and Zoizner supplementary material(File)
File 3.3 MB
Supplementary material: Link

Amsalem and Zoizner Dataset

Dataset

https://doi.org/10.7910/DVN/YJIZBJ
Link