Workshop on Advanced Research Methods (WARM)
Sharing innovations and innovative uses of advanced research methods, Thursdays 8:30-9:30 PM (US ET)
Organizing Committee: Weihua An, Shawn Bauldry, Tim Liao, Yan Long, Ian Lundberg, Xi Song, Jun Xu
2024 Schedule
(Register for selected meetings. Limited spots, first come first serve.)
Correspondence Audits: Design Issues and Practical Examples
Time: February 22, 8:30-9:30 PM (US ET)
Speaker: S. Michael Gaddis, NWEA and formerly UCLA
Abstract: During the past decade, field experiments in the social and behavioral sciences have gained in popularity as the internet has made implementing experiments easier, cheaper, and faster. However, although researchers may have a conceptual knowledge of how experiments work, the actual experience of implementing a field experiment for the first time is often frustrating and time consuming. The initial learning curve may be steep but the rewards are plentiful as experiments produce highly valued original data, lend themselves to causal analysis in ways that traditional survey data cannot, and become easier to implement as a researcher’s experience level increases. This talk will introduce social scientists to the basics of a particular type of field experiment -- the correspondence audit. Attendees will learn when and why correspondence audits are an appropriate method, how to navigate ethical issues and IRB, and we will walk through a number of design issues that first time users often struggle with. Dr. Gaddis will provide practical examples from his own and others' work to illuminate some of the pitfalls of this method and help the audience gain confidence in embarking on their own field experiments.
2. Sensitivity Analysis for Quantifying the Robustness of Causal Inference
Time: March 21, 8:30-9:30 PM (US ET)
Speaker: Ken Frank, Michigan State University
Abstract: Social scientists seeking to inform policy or public action must carefully consider how to identify effects and express inferences because actions based on invalid inferences will not yield the intended results. Recognizing the complexities and uncertainties of social science, we seek to inform inevitable debates about causal inferences by quantifying the conditions necessary to change an inference. Specifically, we review existing sensitivity analyses within the omitted variables and potential outcomes frameworks. We then present the Impact Threshold for a Confounding Variable (ITCV) based on omitted variables in the linear model and the Robustness of Inference to Replacement (RIR) based on the potential outcomes framework. We extend each approach to include benchmarks and to fully account for sampling variability represented by standard errors as well as bias. We exhort social scientists wishing to inform policy and practice to quantify the robustness of their inferences after utilizing the best available data and methods to draw an initial causal inference. [Paper] [Slides]
3. Multigenerational Social Mobility: A Demographic Approach
Time: August 22, 8:30-9:30 PM (US ET). Note you may need to re-register for this event because of the re-scheduling.
Speaker: Xi Song, University of Pennsylvania
Abstract: Most social mobility studies take a two-generation perspective, in which intergenerational relationships are represented by the association between parents’ and offspring’s socioeconomic status. This approach, although widely adopted in the literature, has serious limitations when more than two generations of families are considered. In particular, it ignores the role of families’ demographic behaviors in moderating mobility outcomes and the joint role of mobility and demography in shaping long-run family and population processes. This article provides a demographic approach to the study of multigenerational social mobility, incorporating demographic mechanisms of births, deaths, and mating into statistical models of social mobility. Compared with previous mobility models for estimating the probability of offspring’s mobility conditional on parent’s social class, the proposed joint demography-mobility model treats the number of offspring in various social classes as the outcome of interest. This new approach shows the extent to which demographic processes may amplify or dampen the effects of family socioeconomic positions because of the direction and strength of the interaction between mobility and differentials in demographic behaviors. The author illustrates various demographic methods for studying multigenerational mobility with empirical examples using the IPUMS linked historical U.S. census representative samples (1850–1930), the Panel Study of Income Dynamics (1968–2015), and simulation data that show other possible scenarios resulting from demography-mobility interactions. [Paper]
4. Generative AI and Social Research
Time: September 19, 8:30-9:30 PM (US ET)
Speaker: Thomas Davidson, Rutgers University and Daniel Karell, Yale University
Abstract: How can generative artificial intelligence (GAI) be used for sociological research? In this talk, we explore applications to the study of text and images across multiple domains, including computational, qualitative, and experimental research. Drawing upon recent research and stylized experiments with DALL-E and GPT-4, we illustrate the potential applications of text-to-text, image-to-text, and text-to-image models for sociological research. These illustrations emphasize how GAI can make advanced computational methods more efficient, flexible, and accessible, but that there are also several challenges, including interpretability, transparency, and reliability. There are also important considerations regarding bias and bias mitigation efforts and the trade-offs between proprietary models and open-source alternatives. We end the talk by discussing a recent project examining how using GAI tools to learn about the world affects people’s knowledge. Across two experiments, we find that people who read AI-generated summaries of historical events more accurately recall factual information about the events compared to people who read human-written summaries of the same event. The AI historical summaries have a positive effect regardless of whether subjects know they were generated by AI, as well as whether or not the summaries exhibit political biases. Furthermore, while synthetic summaries improve knowledge for participants across all levels of educational attainment, the largest improvements occur at the lowest levels. These findings show how AI-generated texts can enhance learning and reduce, rather than amplify, educational inequalities. [A Related Paper] [Recording]
5. Covariance Regression Models for Studying Treatment Effect Heterogeneity
Time: October 17, 8:30-9:30 PM (US ET)
Speaker: Deirdre Bloome, Harvard University
Abstract: Causal analyses typically focus on average treatment effects. Yet for substantive research on topics like inequality, interest extends to treatments’ distributional consequences. When individuals differ in their responses to treatment, three types of inequality may result. Treatment may shape inequalities between subgroups defined by pretreatment covariates, it may induce more inequality in one subgroup than another, or it may polarize people across multiple dimensions of well-being. We introduce a model, called a covariance regression, that captures all three types of inequality via the means, variances, and correlations between multiple outcomes. The model can test for heterogeneous treatment effects, quantify the heterogeneity, and explain its structure using covariates. Finding that a treatment creates inequalities could drive theoretical refinement and inform policy decisions (targeting groups where payoffs will be most predictable). We illustrate the utility of covariance regressions by analyzing the effects of sharing information about income inequality on redistributive preferences. [Paper]
6. Transnational Fieldwork
Time: November 21, 8:30-9:30 PM (US ET)
Speaker: Yan Long, UC Berkeley
Abstract: TBA