Workshop on Advanced Research Methods (WARM)
Sharing innovations and innovative uses of advanced research methods, Thursdays 8:30-9:30 PM (US ET)
Organizing Committee: Weihua An, Shawn Bauldry, Tim Liao, Yan Long, Ian Lundberg, Xi Song, Jun Xu
2024 Schedule
(Register for selected meetings. Limited spots, first come first serve.)
Correspondence Audits: Design Issues and Practical Examples
Time: February 22, 8:30-9:30 PM (US ET)
Speaker: S. Michael Gaddis, NWEA and formerly UCLA
Abstract: During the past decade, field experiments in the social and behavioral sciences have gained in popularity as the internet has made implementing experiments easier, cheaper, and faster. However, although researchers may have a conceptual knowledge of how experiments work, the actual experience of implementing a field experiment for the first time is often frustrating and time consuming. The initial learning curve may be steep but the rewards are plentiful as experiments produce highly valued original data, lend themselves to causal analysis in ways that traditional survey data cannot, and become easier to implement as a researcher’s experience level increases. This talk will introduce social scientists to the basics of a particular type of field experiment -- the correspondence audit. Attendees will learn when and why correspondence audits are an appropriate method, how to navigate ethical issues and IRB, and we will walk through a number of design issues that first time users often struggle with. Dr. Gaddis will provide practical examples from his own and others' work to illuminate some of the pitfalls of this method and help the audience gain confidence in embarking on their own field experiments.
2. Sensitivity Analysis for Quantifying the Robustness of Causal Inference
Time: March 21, 8:30-9:30 PM (US ET)
Speaker: Ken Frank, Michigan State University
Abstract: Social scientists seeking to inform policy or public action must carefully consider how to identify effects and express inferences because actions based on invalid inferences will not yield the intended results. Recognizing the complexities and uncertainties of social science, we seek to inform inevitable debates about causal inferences by quantifying the conditions necessary to change an inference. Specifically, we review existing sensitivity analyses within the omitted variables and potential outcomes frameworks. We then present the Impact Threshold for a Confounding Variable (ITCV) based on omitted variables in the linear model and the Robustness of Inference to Replacement (RIR) based on the potential outcomes framework. We extend each approach to include benchmarks and to fully account for sampling variability represented by standard errors as well as bias. We exhort social scientists wishing to inform policy and practice to quantify the robustness of their inferences after utilizing the best available data and methods to draw an initial causal inference. [Paper] [Slides]
3. Multigenerational Social Mobility: A Demographic Approach
Time: April 25, 11 AM-12 PM (US ET). Note this is on Thursday morning.
Speaker: Xi Song, University of Pennsylvania
Abstract: Most social mobility studies take a two-generation perspective, in which intergenerational relationships are represented by the association between parents’ and offspring’s socioeconomic status. This approach, although widely adopted in the literature, has serious limitations when more than two generations of families are considered. In particular, it ignores the role of families’ demographic behaviors in moderating mobility outcomes and the joint role of mobility and demography in shaping long-run family and population processes. This article provides a demographic approach to the study of multigenerational social mobility, incorporating demographic mechanisms of births, deaths, and mating into statistical models of social mobility. Compared with previous mobility models for estimating the probability of offspring’s mobility conditional on parent’s social class, the proposed joint demography-mobility model treats the number of offspring in various social classes as the outcome of interest. This new approach shows the extent to which demographic processes may amplify or dampen the effects of family socioeconomic positions because of the direction and strength of the interaction between mobility and differentials in demographic behaviors. The author illustrates various demographic methods for studying multigenerational mobility with empirical examples using the IPUMS linked historical U.S. census representative samples (1850–1930), the Panel Study of Income Dynamics (1968–2015), and simulation data that show other possible scenarios resulting from demography-mobility interactions. [Paper]
4. Generative AI and Social Research
Time: September 19, 8:30-9:30 PM (US ET)
Speaker: Daniel Karell, Yale University
Abstract: TBA
5. Covariance Regression Models for Studying Treatment Effect Heterogeneity
Time: October 17, 8:30-9:30 PM (US ET)
Speaker: Deirdre Bloome, Harvard University
Abstract: Causal analyses typically focus on average treatment effects. Yet for substantive research on topics like inequality, interest extends to treatments’ distributional consequences. When individuals differ in their responses to treatment, three types of inequality may result. Treatment may shape inequalities between subgroups defined by pretreatment covariates, it may induce more inequality in one subgroup than another, or it may polarize people across multiple dimensions of well-being. We introduce a model, called a covariance regression, that captures all three types of inequality via the means, variances, and correlations between multiple outcomes. The model can test for heterogeneous treatment effects, quantify the heterogeneity, and explain its structure using covariates. Finding that a treatment creates inequalities could drive theoretical refinement and inform policy decisions (targeting groups where payoffs will be most predictable). We illustrate the utility of covariance regressions by analyzing the effects of sharing information about income inequality on redistributive preferences. [Paper]
6. Transnational Fieldwork
Time: November 21, 8:30-9:30 PM (US ET)
Speaker: Yan Long, UC Berkeley
Abstract: TBA