First, we fit mixed effects logistic regression models regressing binary choice (choice of the gamble = 1, guaranteed alternative = 0) on a variety of variables, and second, we fit a hierarchical Bayesian model of the underlying nonlinear processes.
Mixed effects logistic regression models
We fit mixed effects logistic regression models to the risky choice data using the “lme4” package in R (R version 3.5.0; “lme4” version 1.1–21)26. For this analysis, we discuss three types of recent events: previous outcome (the value in dollars of the outcome on the previous trial), previous decision (gamble or guaranteed, coded as 1 for gamble and 0 for guaranteed), and mean expected value (EV) of the previous choice options (the mean of the safe and risky options on the previous trial; see Methods). All mixed effects logistic regression models included binary choice as the binomial outcome variable and both a constant and current choice options (the dollar values of the risky gain, risky loss, and guaranteed alternative) as predictor variables (e.g. choicet ~ β0 + β1 × risky gaint + β2 × risky losst + β3 × guaranteedt + β4 × outcomet-1). The constant was modeled as a random effect and the choice options were modeled as fixed effects. Each model varied only by the type of recent events included as additional predictor variables (for reliable convergence all were modeled as fixed effects). See Supplementary Material and Table S3 for lme4 code for all models.
Across all mixed effects models discussed in this analysis, current choice options had consistent, significant effects on binary choice, such that larger potential gains led to more risk-taking and larger potential losses or guaranteed alternatives led to less risk-taking. Hereafter, we only discuss the additional effects of recent events on risk-taking, controlling for effects of current options. See Supplementary Material and Table S3 for full regression results, including the effects of current choice options.
To first examine which types of recent events had the strongest effects on subsequent risk-taking, we performed three separate regressions (models 1–3), each featuring one of the following: the previous outcome (model 1), the previous decision (model 2), or the mean EV of the previous choice option to capture the average magnitude of the previous trial (model 3). Because the number of parameters in each of these models was identical, to compare models, we used log-likelihood values, or the degree to which each model produced choice likelihoods that reflected the actual choices on each trial. The best-fitting model had the highest (least negative) log-likelihood value.
We found that previous outcomes (model 1) had a significant, negative influence on binary choice, such that as outcomes on the previous trial increased in magnitude, risk-taking on the current trial decreased (β = -0.03 (0.003), p < 2 × 10−16). This model (model 1, log-likelihood = -10,638.5) outperformed models that, instead of the previous outcome amount, featured previous decisions (model 2, log-likelihood = -10,682.7) or the mean EV of the previous choice options (model 3, log-likelihood = -10,669.2). When all three regressors (previous outcomes, decisions, and mean EV) were pit directly against each other by including them in the same model, model 4, previous outcomes predicted risk-taking on the current trial beyond previous decision and mean EV of the previous choice options (previous outcome: β = -0.03(0.004), p = 4.08 × 10−15; previous decision: β = -0.04(0.02), p = 0.06; mean EV of previous choice options: β = -0.002(0.006), p = 0.78).
To illustrate the effect size of past outcomes on a given choice, assuming that the participant was indifferent between the current choice options (i.e. the value of the gamble and guaranteed alternative currently under consideration were equal), the estimates from model 1 indicate that the probability of choosing the risky option following a gain of +$20 was 35%, while that following a loss of -$20 was 65% (Fig. 2a), a 30% difference in the probability of risk-taking.
Having established that the effect of previous events on current binary choices was most clearly accounted for by previous outcomes, we subsequently sought to establish the characteristics of that effect over time, valence, and outcome type in models 5–8. First, we examined how far the effect of previous outcome extended over time. In model 5, we regressed binary choice onto the outcome one trial back (t-1), two trials back (t-2), and three trials back (t-3). We found that the negative effect of previous outcomes declined over time, such that it was significant at one and two trials back, but not three trials back (outcomet-1: β = -0.03(0.003), p < 2 × 10−16; outcomet-2: β = -0.008(0.003), p = 0.01; outcomet-3: β = -0.004(0.003), p = 0.19; Fig. 2b).
Next, in model 6, we tested whether the previous outcome effect was due to the outcome amount or simply the outcome valence, as previous outcomes included gain, zero, and loss amounts. Regressing binary choice onto regressors for previous outcome amount and for previous outcome valence (modeled as +1 for gains, 0 for zero, and -1 for losses) in model 6, we found that the amount of the previous outcome predicted risk-taking on the current trial beyond the valence of the previous outcome (previous outcome amount: β = -0.04(0.005), p < 5.95 × 10−14; previous outcome valence: β = 0.08(0.04), p = 0.08).
Finally, we examined whether the previous outcome effect differed as a function of outcome type (gain, loss, or guaranteed). Differences on the basis of outcome type could arise from differences between gains (risky and safe) and losses due to loss aversion27,28, or differences between risky outcomes (gain and loss) and guaranteed outcomes due to possible expectation-based learning processes (for example, processes reflecting prediction errors29,30,31) as well as more complex interactions between valence and uncertainty. In model 7, we thus regressed binary choice onto three separate regressors, one each for previous risky gain outcome amount, previous risky loss outcome amount, and previous guaranteed outcome amount. All three regressors were unsurprisingly significant, replicating the overall effect of previous outcomes on binary choice (previous gain: β = -0.03(0.004), p < 2 × 10−16; previous loss: β = -0.03(0.008), p = 0.0003; previous guaranteed: β = -0.02(0.009), p = 0.005). Furthermore, these three coefficients were not significantly different from one another as tested in two ways. First, model 7 (with separate coefficients for previous risky gain, risky loss, and guaranteed outcomes) did not perform significantly better than model 1 (with a single previous outcome amount regressor; because model 1 is a nested version of model 7, we used the likelihood ratio test; LRT statistic = 1.17, df = 2, p = 0.56). Second, estimates for the effects of previous gain, loss, and guaranteed outcomes in model 7 were not significantly different from one another in any of the three possible pairwise comparisons (Wald tests, all three p’s > 0.31; Fig. 2c). Together, these findings demonstrate that the effect of previous outcomes on the current binary choice did not differ by outcome type.
For an alternative approach to the analysis of outcome type, in model 8, we regressed binary choice on two regressors that represented outcome amount separately by valence only (with regressors for previous gain amount collapsing across risky and guaranteed outcome amounts, and previous loss amount), and included a separate term for previous risky versus guaranteed choices (coded +1/−1, respectively, as in the other models) which was interacted with previous gain amount. Model 8 similarly identified significant effects of previous gain and loss amounts (previous gain amount: β = -0.03(0.005), p = 3.22 × 10−8; previous loss amount: β = -0.04(0.009), p = 5.49 × 10−5) that were not significantly different from each other (Wald test, p = 0.44). Model 8 also replicated model 4’s finding of a weak effect of previous choices (β = -0.05(0.02), p = 0.06) when accounting for previous outcomes. Finally, the interaction of previous gain outcome amount with previous risky versus guaranteed choices was not significant (β = -0.0009(0.005), p = 0.86). Model 8’s results are consistent with models 6 and 7 in identifying that the effect of previous outcomes does not significantly vary by the valence of the outcome or by the type of previous outcome, but is instead best described as an effect of the previous outcome amount, whatever its type or valence.
Thus far, we established that the effect of previous outcome on the current binary choice was short-lasting (Model 5), driven by outcome amount (Model 6), and did not differ by outcome type or valence (Models 7 and 8).
All of the above effects are value-independent in that recent events directly shift the probability of choosing the risky option. However, it is unclear whether recent events may additionally influence valuation of the current choice options. To test whether previous outcomes influence how individuals assess the value of the options on the current trial, in model 9 we regressed binary choice onto previous outcome and included three interaction terms between previous outcome and each of the current choice options (risky gain, risky loss, and guaranteed). While the value-independent effect of previous outcomes on the current binary choice remained (β = -0.46(0.21), p = 0.03), we additionally found that the outcome on the previous trial increased the weight put on the potential losses (outcomet-1 × risky losst: β = 0.1(0.03), p = 0.002), but not potential gains (outcomet-1 × risky gaint: β = 0.01(0.04), p = 0.8) or potential guaranteed alternatives (outcomet-1 × guaranteedt: β = -0.06(0.08), p = 0.5). To illustrate the effect size of past outcomes on the valuation of subsequent potential losses, the weight placed on potential losses after an outcome of + $20 was 0.46, or 142% of the weight on losses after an outcome of -$20, which was 0.33 (Fig. 2d).
Hierarchical Bayesian estimation
Mixed effects logistic regressions allow us to detect individual- and group-level differences for linear processes, but we know that some risky decision-making processes such as risk aversion are not linear2,3,4,5,21,32.
To address this, for the second analysis we used hierarchical Bayesian estimation, an approach that allowed the fitting of all of the decision-making data at once while simultaneously estimating both individual- and group-level parameters of nonlinear models (like prospect theory) previously shown to fit these data well2,3,4,5,10,21,32,33,34. We used Markov-Chain Monte Carlo (MCMC) sampling techniques in a hierarchical Bayesian framework (using “rstan” version 2.17.3)35 to fit a modified version of a 4-parameter prospect theory-inspired model (Prospect Theory Plus, PT + )10. PT + captured four distinct decision-making processes: risk aversion (ρ), loss aversion (λ), choice consistency (μ), and decision bias (db), assuming a linear probability weighting function. Using this Bayesian framework, we estimated both the group-level distribution and individual values for each of the four PT + parameters. We also modeled the change in each of these four parameters over time by altering PT + to include four additional updating parameters (four δθ terms, i.e. δρ, δλ, δμ, δdb). The updating parameters controlled how much previous decision outcomes shifted each of the four original PT + parameters on the subsequent trials. Positive (negative) values of δθ would indicate that θ increased (decreased) as previous outcomes increased, while a zero value of δθ would indicate no net adjustment of θ by previous outcomes. Because PT + consists of value-dependent parameters (ρ, λ), a value-independent parameter (db), and an intermediate parameter linking value and action (μ), we were able to simultaneously detect the effects of context on linear and non-linear action- and valuation-related decision-making processes. See Methods for the complete modeling procedure.
MCMC estimation procedures produce “chains” of sampled parameter values, in proportion to their likelihood. Using Stan (“rstan” version 2.17.3)35, we ran twenty chains of 10,000 samples each, discarding the first 5,000 samples of each chain as a burn-in period, resulting in 100,000 samples. Priors were selected to be as uninformative as possible and were normal, uniform or Cauchy distributions, described in more detail in Table S1. Each of the twenty chains converged on similar distributions of parameter values (mean Rhat for group-level mean parameters = 1.002, range = 1.0003–1.0034; ideal = 1). The total number of effective samples for each of the group-level mean parameters were ρ = 10,782, λ = 100,000, μ = 100,000, db = 12,783, δρ = 6,292, δλ = 13,003, δμ = 12,594, and δdb = 6,592.
First, we examined mean values and 95% confidence intervals for each of the baseline group-level parameter estimates for risk aversion, loss aversion, choice consistency and decision bias. In our sample, participants were risk averse for gains and risk seeking for losses (ρ = 0.65, 95% CI = [0.58 0.73]), were mildly loss averse (λ = 1.57, 95% CI = [1.44 1.71]), were consistent in their choices for the risky option (μ = 22.2, 95% CI = [18.9 26.0]), and had a bias to gamble (db = -0.58, 95% CI = [-0.68 -0.47]), consistent with others’ findings2,3,4,5,10,21,32.
Next, we tested whether each of the four parameters updated as a function of previous outcomes. Examining the mean values and 95% CIs for each of the four group-level mean updating parameters, we found that large positive previous outcomes increased loss aversion (δλ = 0.013, 95% CI = [0.003 0.02]), consistent with our finding from the linear mixed effects logistic regression analyses that large previous outcomes increased the weight put on potential losses (see above, model 9). We also found that large positive previous outcomes increased consistency across binary choices (δμ = 0.08, 95% CI = [0.05 0.10]) and reduced the bias to gamble (δdb = 0.03, 95% CI = [0.02 0.05]). We found no effect of previous outcomes on risk aversion (δρ = 0.005, 95% CI = [−0.01 0.02]). The variation identified in these confidence intervals means that for identical previous outcomes, different people will react differently. More importantly, the finding of any effect of recent outcomes also means that if otherwise initially-similar people have different experiences, they will subsequently make systematically different risky choices. Figure 3 illustrates this effect by plotting loss aversion over time for one individual experiencing three different histories of outcomes. Risky choice behavior is not just about the individual decision-maker and their choice preferences, but also their past experiences.