Assessing Uncertainty in Multicriteria Evaluation of Centralized and Decentralized Dual Water Supply Strategies | Journal of Water Resources Planning and Management | Vol 148, No 12

AbstractAs much of our urban water supply infrastructure reaches the end of its useful life, water managers are using the opportunity to explore alternative strategies that enable them also to utilize alternative local water sources (e.g., untreated raw water, reclaimed water, graywater, and stormwater). However, evaluating alternative strategies is challenging because water managers are required to balance the needs of multiple stakeholders, consider all the costs and benefits, reduce risks, and, most importantly, ensure public health and protect the environment. Here, the benefits and trade-offs of dual supply of raw and treated water were assessed with consideration of centralized and decentralized options. The study resulted in a complex decision process involving 45 performance indicators and 17 stakeholders. Uncertainty and global sensitivity analyses were applied to the study’s decision models to assess the reliability of the results given long-term uncertainty, improve transparency in the decision process, and elucidate the benefits and trade-offs of alternatives for dual supply of raw and treated water. The key drivers of model variance were regulatory and political, including access to alternative nonpotable water sources in the future, the ability to sell potable water to adjacent utilities, and the possibility of cost savings by avoiding the need for water rights conversion. Although technical considerations are important, the results reveal the importance of addressing regulatory uncertainty related to dual water supply systems and decentralized water systems. The decentralized alternatives were particularly sensitive to uncertainty in regulatory risks. Resolving these issues highlights the importance of including a wide range of local, regional, and regulatory stakeholders in the decision process.IntroductionWater infrastructure planners seeking to diversify the water supply and match water quality with the intended use, promoting fit-for-purpose water, frequently consider dual water supply systems. Dual water supply systems supply more than one water source (e.g., drinking water, reclaimed water, and raw water) to different end uses (e.g., potable, irrigation, and toilet flushing) for fit-for-purpose use. Potential benefits of dual water supply systems include increasing water security, conserving high quality water sources, increasing environmental flows, decreasing surface water pollution, reducing costs and energy use, improving drinking water quality, and facilitating cooperation with local agriculture (Kang and Lansey 2012; Grigg et al. 2013; Bischel et al. 2012; Crook 2005; Cole et al. 2018a; Smith 2009). Despite several implementations of dual water supply systems (Grigg et al. 2013; Wilkins-Wells et al. 2003), there still is uncertainty regarding adoption and long-term performance. Uncertainties can include limited information on the costs and benefits, regulatory barriers, consideration of centralized versus decentralized strategies, uncertainty in long-term performance, and conflicting stakeholder priorities (NASEM 2016; Scholten et al. 2015). Research is needed to enhance understanding of these uncertainties and the role of these considerations in decision making.Several studies have evaluated dual water supply alternatives that incorporate financial, social, and environmental criteria or objectives. Evaluation methods have included cost–benefit analysis (Chen and Wang 2009), multiobjective optimization (Liner and de Monsabert 2011; Kang and Lansey 2012; Newman et al. 2014), and multicriteria decision analysis (MCDA) to incorporate triple bottom line (TBL) criteria and stakeholder priorities (Bichai et al. 2014; Cole et al. 2018a). Other than Kang and Lansey’s (2012) study, in which they tested their model for sensitivity to changes in topography, demand, and service area, studies that assess uncertainty and sensitivity of model inputs remain limited, particularly with respect to broader consideration of design considerations and stakeholder priorities.An additional opportunity provided by dual water supply strategies is the opportunity to consider decentralized systems. Decentralized strategies for water systems have been recognized as critical to enable integrated urban water management and the One Water approach (Daigger et al. 2019). Decentralization of the water supply allows for a strategy that optimizes the trade-offs between economies of scale found in water treatment (WT) and the diseconomies of scale found in distribution (Kim and Clark 1988; Chung et al. 2008; Woods et al. 2013). Woods et al. (2013) considered centralized and decentralized water reclamation facilities with the dual supply of recycled and treated water. Their results suggested that higher peripheral demand, limited capacity in existing systems, topography, and lower discount rates favored decentralized systems. Newman et al. (2014) considered multiple-scale urban water systems with different water sources and levels of decentralization. Their findings indicated that economies of scale in water treatment outweighed the diseconomies of scale in pipe networks for most water sources. However, Cole et al. (2018a) considered centralized and decentralized water treatment for the dual supply of raw and treated water and found that the centralized alternatives outperformed the decentralized alternatives. These studies suggest that the performance of dual water supply systems with decentralization is context-specific and driven by a number of uncertain system characteristics. Water system performance under long-term uncertainty and understanding of the decision parameters driving uncertainty is essential for reducing decision risk and gaining the support needed for adoption (Manikkuwahandi et al. 2019). This is especially true when considering dual water supply systems in which the decision parameters and long-term performance are not well known.Cole et al. (2018a) extended the previous research on dual water supply by considering the benefits and trade-offs of centralized and decentralized water treatment strategies, varying scales of dual distribution, and the use of existing irrigation ditches throughout the service area using a hybrid TBL-MCDA methodology and an inclusive, participatory approach. The research presented by Cole et al. (2018a) provides a case-specific application of the methodology to inform decisions on water infrastructure. The present research applies uncertainty and global sensitivity analyses models to evaluate the reliability of the results, identify key input factors driving uncertainty in the alternatives’ financial, social, and environmental performance and elucidates the benefits and trade-offs of alternative strategies for the dual supply of raw and treated water. The present results expand upon the results of Cole et al. (2018a) to identify the input factors that had the most effect on the uncertainty of centralized and decentralized alternatives to assess conditions in which these solutions may be more favorable.ApproachCole et al. (2018a) evaluated the benefits and trade-offs of four alternative strategies for the dual supply of raw and treated water in Fort Collins, Colorado. Using integrated urban water management (IUWM) principles, they attempted to balance financial, social, and environmental performance and support a flexible transdisciplinary approach to decision-making involving multidisciplinary experts and stakeholders. Four alternatives for the dual supply of raw and treated water were evaluated, along with maintaining the existing conventional water supply system (Conventional): 1.Central Dual continued drinking water treatment at the existing conventional central water treatment facility, used the existing distribution system to distribute raw water for fire and irrigation demand, and installed a new potable distribution system to distribute drinking water for indoor use.2.Neighborhood conveyed raw water to neighborhoods using the existing distribution system. Water for indoor demand was treated at new satellite neighborhood water treatment facilities (ultrafiltration membrane systems) and then distributed via new neighborhood potable distribution systems.3.Point-of-Entry (POE) conveyed raw water to the service connection using the existing distribution system. Water for indoor demand was treated on-site at point-of-entry water treatment systems (activated carbon and kinetic degradation fluxion media with ultraviolet disinfection).4.Separated Irrigation used the existing central water treatment facility and distribution system to treat and distribute drinking water for indoor and fire-protection use. The network of irrigation ditches throughout the city would distribute raw water to neighborhood separated irrigation systems for landscape irrigation.In the TBL-MCDA method used by Cole et al. (2018a), the project team and stakeholders defined 11 main criteria and a total of 45 quantitative and qualitative performance indicators to evaluate these main criteria from the financial, social, and environmental bottom lines. The stakeholder group included representatives from nine city departments comprising the Nature in the City Group (NICG) (Cole et al. 2018b). This group had direct engagement with residents, community and business partners, the mayor, and city councilmembers through participation in projects focused on improving the community’s access to nature. The 17 stakeholders then weighted the relative importance of the main criteria for each bottom line. This resulted in a separate MCDA model for each bottom line. Uncertainty and sensitivity analyses were applied to these models using the following approach (Fig. 1):Steps 1–3: Define Uncertain Input Factors and Generate Monte Carlo SamplesPerformance Indicator InputsThe uncertain input factors used to calculate the alternatives’ performance in the performance indicators for the TBL-MCDA models were identified (Table 1; Fig. 1, Task 1). Probability distributions were defined for each (Fig. 1, Task 2; Table S1). The inputs to the quantitative performance indicators were defined as uniform or triangular probability density functions based on utility data and/or literature review (Table 1; Table S1; Cole et al. 2018a). The inputs used in the assessment of qualitative indicators used a mix of probability mass and probability density functions.Table 1. Main criteria, performance indicators, and uncertain input factorsTable 1. Main criteria, performance indicators, and uncertain input factorsMain criteriaFinancial performance indicatorsSocial performance indicatorsEnvironmental performance indicators1. Impact of new infrastructure1.1 Cost of new infrastructure (Cds, Ct, Cbfp, Csis, Cpoe, Di)1.1 Disruption from constructionc1.1 GHG emissions construction (Cds, Ct, Cbfp, Di, Cpoe, Csis)a1.2 Net replacement costs (Cds, Ct, Csis, Cpoe, Ltm, Lds, Cwtf, RLcwt, Lcwt, Lnwt, Lpoe, i, Di, Do)1.2 Increase in temporary employment (Cds, Ct, Cbfp, Di, Cpoe, Csis)a1.2 Temporary stormwater pollutiona2. Energy use2.1 Annual energy costs (CEwt, CEvar, PEpot, PEnon, NEwt, POEEwt, Di, Do)2.1 Health impacts associated with GHG emissions from energy production (Di, Do, CEvar, CEwt, Newt, PEpot, POEEwt, PEnon)a2.1 Annual GHG emission (Di, Do, CEvar, CEwt, Newt, PEpot, POEEwt)3. Routine maintenance (excluding energy)3.1 Annual O&M costs for water treatment (Di, Do, ECwtom, ACwtom, Nwtom, POEwtom)3.1 Disruption to community from maintenance3.1 GHG emission maintenance (GHGdds, GHGnwt)3.2 Annual O&M distribution costs (DSpotom, TMpotom, SISom)3.2 Chemical consumables (Di, Do, Clc, Al2SO43, CaOH2, F, CO2, Cln)4. Staffing4.1 Full-time employees (Brate, FS, FTEpoewt, Bno, Di, Do)4.1 Employment (Bno, Brate, FS, Di, Do, FTEpoewt)4.1 Employee transport GHG emissions (Bno, Brate, FS, FTEpoewt, Di, Do)a4.2 Training costs for new tech. (Tdd, Twt, Tno) (ordinal)4.2 Increased earning potential (Tdd, Twt, Tno) (ordinal)5. Consumer water quality5.1 DBP exposure health care costsa,c5.1 Water age in potable distribution systemc5.1 Water quality receiving water bodies (ordinal)5.2 Cross-connection costs (RFpdd, RFsdd, RFp, RFfi) (ordinal)5.2 Health risks from cross-connections (RFfi, RFpdd, RFp, RFsdd) (ordinal)5.3 Source water contamination event costs (Sn, Spoe) (ordinal)5.3 Adaptability to source water contamination event (Sn, Spoe) (ordinal)6. Environmental flows6.1 Avoided transaction costsb (ATC) (ordinal)6.1 Enhancement of natural areas and benefits to canal companies (ATC) (ordinal)6.1 Benefits to species and natural systems from increasing ISFs (ATC) (ordinal)7. Supply risk7.1 Cost of alternative supplies (LSms, LAS, LSs, LSb, LSpop) (ordinal)7.1 Resiliency of infrastructure to limited supply (LAS, LSs, LSb, LSpop, LSms) (ordinal)7.1 Variable supply effects on water corridors (LAS) (ordinal)7.2 Risk of obsolete infrastructure (ROc, ROn, ROpoe, ROsis) (ordinal)8. Rate risk8.1 Confidence in O&M projections (Rdd, Rn, Rsis) (ordinal)8.1 Affordability (ECwtom, Di, Do, ACtwom, Nwtom, POEwtom, DSpotom, TMpotom, SISom8.1 Changes in water demand (ECwtom, Di, Do, Nwtom, POEwtom, ACwtom, DSpotom, TMpotom, SISom)a9. Alternative source opportunity9.1 Savings from use of alternative nonpotable sourcesa (LAS)9.1 Being an innovative community and improving natural areas (LAS)9.1 Improvements to water corridor ecosystems (LAS)10. Revenue opportunity10.1 Wholesale water revenuea (Rev)10.1 Improve regional water security in adjacent communities (Rev)a10.1 Environmental benefits from decreasing need for new WTF to meet growth (Rev)a11. Regulatory of political risk11.1 New regulation costs (ordinal)11.1 Public acceptance (PAdd, PAnwt) (ordinal)11.1 Negative environmental impacts from mitigation for new reg. requirements (RRcwt, RRraw, RRnwt, RRpoe)11.2 Public relations costs (ordinal)For the qualitative indicators, the inputs were the factors evaluated to compare the alternatives. For example, the resiliency of infrastructure to limited supply performance indicator (Table 1, Social 7.1) considered two scenarios: seasonal shortages in supply, and a short-term service disruption. Five binary factors were considered when assessing alternative performance: (1) whether the alternative has access to multiple raw water sources (LSms); (2) whether the alternative has a dual distribution system that would allow for the use of alternative nonpotable sources in the future (LAS); (3) whether the alternative increases finished water storage (LSs); (4) whether the alternative has access to backup water treatment systems (LSb); and (5) whether the alternative reduces the number of customers that would be affected in the event of a service disruption (LSpop). The alternative with the most points received the best rating. Not all the alternatives had uncertainty of these input factors. For example, the city has access to two raw water sources. However, the nonpotable irrigation systems in Separated Irrigation have access only to river sources, which may result in a shortage later in the summer.For some performance indicators, it was apparent the uncertain input factors would have no impact on the alternatives’ relative performance and would not change the MCDA results (Table 1). For example, public relations costs (Table 1, Financial 11.2) assumed that the further the alternatives deviated from the conventional system customers were used to, the higher were the public relations costs for that alternative. In this case, alternatives closer to the conventional system always will rank better, with no change to MCDA results.Stakeholder InputsThe correlations between the 11 main criteria weights given by the 17 stakeholders for each bottom line were evaluated using the Pearson product-moment correlation coefficient (Sheskin 2011, Test 28). The analysis indicated that 24%, 9%, and 44% of the r0.05 values exceeded the critical value in the financial, social, and environmental decision models, respectively (Tables S2–S4). The results suggest some correlation between stakeholders’ criteria weights. Therefore, rather than use a separate input for each criteria weight, a dummy variable was created to represent the criteria weight vectors for preferences in each financial, social, and environmental category in each MCDA simulation to avoid nonorthogonal inputs and generation of unrealistic combination of criteria weights. Then a uniform probability mass function was used to represent the probability distribution in each TBL category for the 17 stakeholders.A Monte Carlo sample of the uncertain input factors used in the financial, social, and environmental MCDA models was generated for each model using Sobol low-discrepancy quasi-random sequences, from the R Random Toolbox Package version 1.17.1, with a modification to allow probability mass functions for the discrete inputs (Fig. 1, Task 3; Chalabi et al. 2018). Low-discrepancy sampling sequences allow for more-uniform sampling of the entire sample space and reduce the number of samples needed for the analysis (Saltelli et al. 2004; Iooss and Lemaitre 2014).Steps 4–5: TBL-MCDA Model SimulationsThe generated samples were run through the separate financial, social, and environmental MCDA models from Cole et al. (2018a) using the weighted sum method (WSM), from the R MCDA Package version 0.0.18 (Bigaret et al. 2017) and the Preference Ranking Organization Method for Enrichment of Evaluation II (PROMTHEE II) (Brans and Mareschal 2005). The simplest preference function was used with PROMETHEE II, due to the large number of performance indicators and difficulty defining appropriate threshold values for the qualitative indicators (Cole et al. 2018a).The WSM assumption of additive utility requires application of cardinal properties to ordinal data and the conversion of incommensurate criteria to a common rating scale when a mix of qualitative and quantitative criteria is used (Triantaphyllou 2000). This can exaggerate the differences in alternative performance and allow for high performance of one indicator to compensate for low performance of another, which might not be a true representation of alternative performance (Langemeyer et al. 2016; Mare et al. 2015). The Monte Carlo simulation results did reveal similar results for the best-performing alternatives for both methods, as in other studies that compared MCDA methods (Raju et al. 2000; Hajkowicz and Higgins 2008). However, the sensitivity results for the WSM models did show compensability effects in which there were fewer sensitive input factors and they tended to be the input factors that resulted in an extreme performance rating of a 1 or 5 (results not shown).Outranking techniques are better suited than the WSM when considering a mix of cardinal and ordinal data (Langemeyer et al. 2016). Rather than assuming additive utility to calculate the overall value of an alternative, the outranking method’s goal is to determine if there is enough information to indicate that one alternative is at least as good as another. This makes it better suited to decisions that involve incomplete performance value information, which is common when considering long-term social and environmental performance. For these reasons, and because of the compensability effects observed with the WSM simulations, only the results for the PROMETHEE II method are presented and discussed in the “Results” section (Fig. 1, Tasks 4 and 5).Step 6: Sensitivity AnalysisSensitivity and uncertainty analyses are used to quantify model parameter, structure, and predictive uncertainties and to determine how uncertainty in the model output can be allocated to variation in the model inputs (Saltelli et al. 2008). Most MCDA analyses in the water resources planning and management field use one-at-a-time or local sensitivity analysis, and focus primarily on the sensitivity of the criteria weights (Scholten et al. 2015; Hyde et al. 2004). However, long infrastructure lifetimes and the potential for significant interaction effects between inputs makes a global sensitivity analysis more appropriate when evaluating water supply infrastructure alternatives. Global sensitivity analysis characterizes the entire decision space by simultaneously altering model parameters over their plausible ranges (Saltelli et al. 2008).The sensitivity analysis was conducted using the Sobol method, with is a variance-based global sensitivity analysis technique. Variance-based methods decompose the variance of the model output into fractions that can be attributed to individual inputs. These methods are model-independent, consider the plausible ranges and probability distributions of input factors, and can determine the main and interaction effects between input factors on model output variance individually and in combination (Saltelli et al. 2004, 2008). The Sobol method was selected because of its robustness in dealing with problem with high-dimensional parameter hypercubes (Tang et al. 2007). The Sobol method decomposes the variance of the model output [V(Y)] into terms of increasing dimensionality for the case in which all model inputs are orthogonal (Saltelli et al. 2004) (1) V(Y)=∑iVi+∑i∑j>iVij+⋯+V1,2,…,kwhere Vi = conditional variance which is the first-order effect of input Xi on V(Y; Vij = second-order interaction effects of inputs Xi and Xj on V(Y); and k = number of input factors. The vector of input factors (X) in this study included the inputs used to calculate the effects of decision alternatives on the performance indicators (Table 1) and the dummy criteria weight variable in each bottom line.The first-order (main effects) sensitivity index represents the main effect contribution of each input to the output variance (Saltelli et al. 2008) and is represented by the normalized conditional variance Vi (Saltelli et al. 2004) (2) The total effects indexes are used to determine the interaction effects that are not captured by the main effects (Saltelli et al. 2008). For the calculation of the total effects sensitivity index, Homma and Saltelli (1996) introduced the conditional variance, V(E(Y|X−i), which represents the impact on output variance due to everything except Xi. The difference between the output variance and the new term represents the total of all terms in the variance decomposition that include Xi. The total effects sensitivity index can be calculated using (Saltelli et al. 2004) (3) STi=V(Y)−V(E(Y|X−i)V(Y)=E(V(Y|X−i))V(Y)The decomposition of variance in the Sobol’s method is based on the assumption that all inputs are orthogonal, and therefore STi≥Si and STi−Si quantify the interaction effects for factor Xi. However, this does not hold true for correlated inputs in which the input variance also affects other input factors, in which case the sum of the conditional variances can be higher than the output variance, and the assumptions based on orthogonal inputs mentioned previously no longer are true (Saltelli et al. 2004).The R Sensitivity Analysis Package version 1.15.0 (Pujol et al. 2017) provides several methods for the Monte Carlo estimation of Sobol indexes with independent inputs. The sobolEff method, based on Janon and Gilquin (2017), was selected because it provided the most-consistent results and is good for large first-order indexes (Janon and Gilquin 2017). All first-order indexes or all total effect indexes were estimated at a cost of N(p+1) model simulations (Janon and Gilquin 2017). A base sample size (N) of 1,100 was used in the analysis. Bootstrapping was used to obtain the 95% confidence intervals of the sensitivity indexes due to the large number of uncertain inputs (Archer et al. 1997). This technique resamples, with replacement, the Monte Carlo samples B times (eliminating the need for further model evaluations), then at each stage and for each input the sensitivity indexes are calculated to estimate the sampling distribution of the indexes (Archer et al. 1997). A B value of 10,000 was used in the analysis.Factor prioritization identifies key inputs for future analysis (Fig. 1, Task 6.1) by identifying model inputs that, when fixed, result in the greatest reduction to output variance (Saltelli et al. 2008). It assumes that inputs are fixed one at a time, and therefore it is concerned only with first-order effects and not with higher-order (interaction) effects (Saltelli et al. 2004). The total effects sensitivity indexes also were calculated to provide overall understanding of the inputs driving the output variance, the amount that is due to main effects versus interaction effects, and for factor fixing (Fig. 1, Task 6.3). Factor fixing allows for model simplification by identifying noninfluential inputs (STi∼0) that can be fixed to a nominal value for future analysis (Saltelli et al. 2004).TBL Weighting AnalysisThe main benefit of the TBL-MCDA approach taken by Cole et al. (2018a) was to elucidate the TBL benefits and trade-offs between the alternatives considered. Although financial, social, and environmental performance are not interchangeable, meaning that high performance in one cannot compensate for low performance in another, it may be useful for decision makers to simulate stakeholder weights for each bottom line to determine the range of values in which the overall top alternative may change when considering total performance. An exploratory TBL analysis was conducted to assess variability of results based on the importance stakeholders place on financial, social and environmental performance. The analysis of the weights that stakeholders place on the importance of financial (WFIN), social (WSOC), and environmental (WENV) performance of the top ranked alternative if the bottom lines were aggregated into a single total bottom line was determined using (4) Overall Performancei=WFINPFINi+WSOCPSOCi+WENVPENViwhere i = alternative WFIN+WSOC+WENV=1(0≤WFIN≤1,0≤WSOC≤1,0≤WENV≤1)and PFINi, PSOCi, and PENVi = mean performance from uncertainty results.ResultsUncertainty Analysis ResultsThe uncertainty analysis supported Cole et al.’s (2018a) findings that the centralized water treatment alternatives generally outperform the decentralized alternatives in Fort Collins (Fig. 2). The utility’s ability to use gravity-fed distribution, spare capacity in the existing water treatment facility, and multiple raw water sources with different seasonal variations in water quality all favor these alternatives (Cole et al. 2018a). Although decentralized water treatment systems can offer energy and pipeline cost savings in areas where pumping is required (Kang and Lansey 2012), the gravity feed in this case study eliminated those savings. There was overlap in financial performance between the existing conventional system, Central Dual, and Separated Irrigation. Central Dual offered the most social benefits, whereas Separated Irrigation offered the most environmental benefits (Fig. 2).In addition to uncertainty in the performance rating of each alternative in each TBL category, the uncertainty of alternative ranking also was explored (Fig. 3). This provides insight into the level of confidence that a decision maker can have regarding the uncertainty of the ranking. The uncertainty analysis showed that Conventional, Central Dual, and Separated Irrigation each were the top-ranked alternative about one-third of the time for the financial bottom line (Fig. 3, Financial). For the social bottom line, Central Dual was the top-ranked alternative about 60% of the time, with the remainder split between Conventional and Separated Irrigation (Fig. 3, Social). For the environmental bottom line, Separated Irrigation was the top-ranked alternative about 90% of the time, followed by Central Dual (Fig. 3, Environmental). The decentralized alternatives (Neighborhood and Point-of-Entry) had the lowest financial and social performance and were ranked in the bottom three for environmental performance.Overall, the uncertainty analysis results showed that Separated Irrigation, followed by Central Dual, had the best performance, but there still was some overlap between Conventional, Central Dual, and Separated Irrigation, which presents decision risk. Although the probability that Separated Irrigation is the top-ranked alternative for environmental performance was very high (0.90) (Fig. 3, Environmental), there was large uncertainty in the rating for environmental performance (Fig. 2). This demonstrates that an alternative can outperform other alternatives in a bottom-line category with high frequency, even when there is notable uncertainty in performance based on input factors. It is important to consider uncertainty in performance based on input factors as well as ranking in the decision-making process.TBL Weighting Analysis ResultsTo assess the impact of stakeholder weighting of TBL categories on ranking of alternatives, importance of financial, social, and environmental performance was varied for the top-ranked alternative (Separated Irrigation or Central Dual) (Fig. 4). Central Dual was favored over Separated Irrigation only when financial performance was weighted less than 0.4 and social performance was weighted greater than 0.6. Central Dual was favored as the social weight increased and the environmental weight decreased. Separated Irrigation was favored across more weight preferences. However, stakeholder preferences, particularly for the importance of social and environmental performance, can change the top-ranking alternative.Sensitivity Analysis ResultsSensitivity analysis was conducted to provide insight into the input factors driving this uncertainty. Additionally, sensitivity analysis can identify the input factors that had the most effect on the uncertainty of centralized and decentralized alternatives, which could help identify conditions in which these solutions may be more favorable. The total effects sensitivity indexes provide visibility into the input factors driving the uncertainty in the TBL-MCDA model results. The sum of the total effects sensitivity indexes for all the alternatives in the financial, social, and environmental models was greater than 1, indicating higher-order interaction effects between the input factors (Fig. 5, patterned bars). The total effects sensitivity indexes were used to identify input factors that did not contribute to the output variance of any alternative and can be fixed to their nominal values in the future. The analysis showed that 29%, 26%, and 48% of the financial, social, and environmental input factors, respectively, can be fixed to simplify models for future analysis.The main effects indexes (Fig. 5, solid bars) were used to identify the input factors in which stakeholders should focus future research efforts. Reducing uncertainty in these input factors would have the most effect on reducing the uncertainty in alternative performance. The results also elucidate the input factors driving uncertainty between centralized versus decentralized dual water supply systems. Input factors were characterized as either regulatory, collaborative, technical, or public health and acceptance (Fig. 5) to explore characteristics of input factors identified as being most sensitive to uncertainty.Centralized Alternatives ResultsReducing uncertainty regarding stakeholder weights, the likelihood the utility can use alternative nonpotable sources in the future, water rights, and the ability to generate additional revenue through spare capacity in the water treatment facility [Fig. 5, Weights, likelihood of using other nonpotable sources (LAS), avoided transaction costs associated with changing beneficial uses (ATC), and Rev] would have the most impact on reducing uncertainty in the performance of the centralized alternatives (Fig. 5, Conventional, Central Dual, and Separated Irrigation). Input factors with sensitivity indexes greater than 0.2 for centralized alternatives (Fig. 6) were weights, likelihood of using other nonpotable sources, and avoided transaction costs associated with changing beneficial uses. The main effects sensitivity indexes for the stakeholder weights, characterized as a collaborative input factor, contributed between 16% and 48% of the total variance for Conventional, Central Dual, and Separated Irrigation performance (Fig. 5, Weights). The results’ sensitivity to disagreement among stakeholders is not surprising given its focus in other studies (Hyde and Maier 2006; Scholten et al. 2015) and its impact on all criteria. Stakeholder input was particularly important for the Separated Irrigation alternative because it accounts for 48% (social) and 46% (environmental) of its variance. This was expected, because Separated Irrigation always outperformed the other alternatives in Criterion 6 (Table 1, Environmental flows) making it sensitive to whether stakeholders rate the importance of this criterion high.Focusing on securing the right to use alternative nonpotable sources (e.g., recycled water, gray water, stormwater) in the future (Fig. 5, LAS) and a more-in-depth water rights analysis (Fig. 5, ATC) would have the largest impact on minimizing financial uncertainty. LAS financial main effects contributed about 20% of the total variance for all three top alternatives, and ATC contributed another 15% for Separated Irrigation. Reducing uncertainty about whether the utility will be able to use alternative supplies also would reduce uncertainty in the social and environmental bottom lines, for which the LAS input factor’s main effects contributed between 24% and 35% of the total variance for Conventional, Central Dual, and Separated Irrigation.The likelihood that the utility can use alternative nonpotable water sources in the future (Fig. 5, LAS), characterized as a regulatory input factor, gives the centralized alternatives with dual distribution systems (Central Dual and Separated Irrigation) advantages over the existing single distribution system (Conventional), because a large part of the infrastructure needed already is in place. Regulations and water rights can limit a utility’s right to put these alternative water sources to beneficial use. Dual distribution systems represent large capital investment and an opportunity to use recycled water, stormwater, and gray water, in addition to the raw water sources considered in this study, for nonpotable uses (e.g., irrigation, firefighting, and toilet flushing). This allows for additional benefits to offset the initial capital investment, such as conservation of high-quality water sources and an alternative use for reclaimed water to reduce surface water pollution and avoid costs from increased regulations on wastewater discharges (Bischel et al. 2012; Grigg et al. 2013).Separated Irrigation allows the utility to use their water rights designated for irrigation use without the additional costs of changing the beneficial use of those rights (ATC). ATC is characterized as a collaborative input factor. For the utility to use these existing water rights for the other alternatives, they would incur the additional costs associated with changing this beneficial use to municipal use. Avoiding these transaction costs gives Separated Irrigation a unique advantage over the other alternatives. However, these savings are difficult to assess because they depend on the success of the water rights conversion process, making the financial performance of Separated Irrigation sensitive to this input factor (Fig. 5, ATC).Central Dual and Separated Irrigation both free up capacity at the water treatment facility. This creates an opportunity to generate revenue by selling treated water to adjacent utilities that may not have capacity to meet growing demand (Fig. 5, Rev), and thus is characterized as a collaborative input factor. Wholesale revenue could offset alternative implementation costs and provide social and environmental benefits to the region by reducing the need to augment existing regional water treatment facilities to meet demands from a growing population. However, these benefits are dependent of demand from adjacent utilities and the ability to negotiate a collaborative agreement with regional water utilities.Decentralized Alternatives ResultsThe performance of the decentralized alternatives (Neighborhood and Point-of-Entry) was sensitive to stakeholder weights, regulatory risk (RRnwt and RRpoe), and centralized water treatment operating and maintenance (O&M) cost inputs (Figs. 5 and 6). Regulatory risks are driven by uncertainty regarding future regulations requiring augmentation of the water treatment facilities and the increased complexity of making changes across many distributed systems.Neighborhood and Point-of-Entry performance were sensitive to the energy use and operating and maintenance (O&M) cost inputs associated with the three water treatment technologies considered in the alternatives. In particular, Neighborhood had high sensitivity to O&M costs (Fig. 5, ECwtom and NWtom; Fig. 6) due to its high energy use for distribution (Cole et al. 2015). Arden et al. (2020) also showed the importance of energy consumption for water treatment in environmental performance of decentralized water systems. Energy use at the central water treatment facility (Fig. 5, CEwt) had interaction effects for Point-of-Entry because there is overlap in energy use performance between Conventional and Point-of-Entry. Both alternatives are gravity-fed, and conventional water treatment is less energy-intensive than Point-of-Entry; however Point-of-Entry treats a lower volume of water (Cole et al. 2015). Overlap also was found in the performance between the O&M costs of the central and neighborhood water treatment facilities (Fig. 5, ECwtom and Nwtom); the difference in the average annual O&M costs was only 1% (Cole et al. 2015).DiscussionThe results of this analysis are case-specific, and every location will have different local conditions, drivers, constraints, evaluation criteria, and stakeholder priorities. However, this study provides more insight into the benefits and trade-offs of dual water supply systems and the critical uncertainties limiting adoption of dual water supply strategies.The Separated Irrigation alternative had the best overall performance (Fig. 2) and potentially benefits both urban and agricultural interests. Separated irrigation systems, in which raw water historically used for agricultural irrigation is used for landscape irrigation, are common in the western US (Smith 2009). These systems can benefit agricultural producers by compensating for non-pass-through costs associated with urbanization around canals (Wilkins-Wells et al. 2003). They also provide value to the urban community by improving the urban green spaces supported by these waterways, which was not considered in previous studies evaluating separate irrigation systems (Wilkins-Wells et al. 2003; Smith 2009). Urban green spaces contribute to the livability of a city by improving air and water quality, reducing urban heat island effects, enhancing physical and psychological wellbeing, and providing opportunities for social connection and recreation (Mukherjee and Takara 2018; Gopalakrishnan et al. 2018).Key Input FactorsThe key input factors driving the variance in the alternatives’ TBL performance can be categorized as regulatory, collaborative, and technical (Figs. 5 and 6). The critical uncertainties in the analysis were regulatory and collaborative rather than technical, highlighting the need to expand the analysis beyond traditional engineering concerns to address these issues, considering integrated urban water management and One Water management (Daigger et al. 2019).Regulatory risk associated with the ability to use alternative sources (Figs. 5 and 6, LAS) and regulatory risk associated with decentralized systems (Figs. 5 and 6, RRnwt and RRpoe) were identified as sensitive input factors. Others have identified the need for regulatory frameworks that support decentralizes systems as an important barrier to overcome (Luthy et al. 2019; Lackey et al. 2020; Daigger et al. 2019). The ability to use alternative sources in the future is important when considering dual water supply systems because it increases flexibility in the future should regulations or water rights change. The regulatory and legal framework around the use of new technologies and strategies that allow for use of alternative sources is changing rapidly (NASEM 2016). Some states recently have passed legislation regarding the use of alternative sources, making these strategies more available, but there still is uncertainty about legal rights and future regulatory changes (NASEM 2016).Stakeholder perspectives and priorities (Fig. 5, Weights), an input factor characterized as collaborative, had a large impact on the financial, social, and environmental output variance, making them integral to the planning process. Efforts to improve consensus among stakeholders would reduce output variance and clarify the relative performance of the alternatives. It also is important to ensure that stakeholder perspectives are preserved in the decision models, because the results showed a correlation between stakeholder weights. Otherwise, there is a risk of preference models that do not represent actual stakeholder priorities. An informed decision process requires investigating these risks in more depth and expanding the stakeholder group to include regional utilities, canal companies, regulators, and the public so that all the benefits, trade-offs, and risks are made transparent to decision makers and stakeholders. Collaboration among this larger group of stakeholders also is critical for creating the collaborative and regulatory agreements necessary for the successful implementation of a dual water supply strategy.The technical input factors that had the most influence on the uncertainty of the results were unexpected. For example, it was hypothesized that indoor and outdoor demand would be the most influential input factors because they were used in the calculation of several performance indicators. Although these had total sensitivity indexes ranging from 0.01 to 0.04, other technical input factors, such as water treatment O&M costs and energy use, had a larger impact on output uncertainty. The Neighborhood and Conventional alternatives were most sensitive to indoor and outdoor demand, and approximately 50% of the sensitivity was due to higher order interaction effects.The Neighborhood decentralized alternative did not perform well relative to the central water treatment alternatives. However, it was comparable in performance for water treatment energy use and O&M costs, which may make it a more favorable alternative in areas without a gravity-fed water distribution system. The O&M costs and water treatment energy use were comparable to those of the central water treatment facility. However, the need to add pumping for distribution gave the Neighborhood alternative the lowest energy rating. This likely would change in areas with non-gravity-fed distribution systems.The alternatives considered in this study had differences that decreased the importance of uncertainty in technical input factors to the alternatives’ relative performance. For example, there was little uncertainty in relative performance due to distribution system metrics, such as water age in the distribution system, because the main drivers of alternative performance were not uncertain in this analysis. Water age decreased with the decentralization of water treatment and moving fire supply to the nonpotable distribution system. These input factors were important in comparing alternative performance, but were not influential in uncertainty associated with alternatives.Making Final DecisionsThe purpose of considering alternative urban water supply strategies is to improve the public’s quality of life, making public engagement essential. Water managers need to understand the public’s acceptance of alternative strategies and have a clear understanding of their priorities. The benefits and trade-offs of existing systems and the alternatives under consideration should be transparent to the public, to allow an informed decision about which benefits they are willing to fund. Uncertainty and sensitivity analysis provide an approach to assess the risk associated with varying alternatives. It is important to consider uncertainty of performance based on input factors as well as uncertainty in the ranking.The approach used in the analysis maintained separation between the bottom lines, rather than aggregating results to a single bottom line, with the view that financial, social, and environmental costs and benefits are not interchangeable. Therefore, how do decision makers ultimately make their final decision when no alternative outperforms all the others for all bottom lines? Historically, financial performance has been the driver, but sustainable urban water management solutions are better informed through a triple-bottom-line approach (Cole et al. 2018b).The TBL weighting analysis results showed that Separated Irrigation is the top overall alternative. However, caution should be used when using only an aggregated bottom line for the final decision, because it allows Separated Irrigation’s high environmental performance to compensate for its lower social performance. This information, along with an assessment of the value stakeholders place on the social and environmental benefits, can help water managers make a more informed decision, but should be considered in conjunction with the benefits and trade-offs elucidated in the TBL-MCDA analysis. Arden et al. (2020) further contributed to use of indicators to explore trade-offs and inform decisions on treatment processes for decentralized water systems.Future analysis may be needed to evaluate structural and stakeholder uncertainty. Structural uncertainty, which is uncertainty resulting from the decision models’ structure, can be addressed by evaluating the models with different architectures (Myrgiotis et al. 2018). A variance-based sensitivity analysis for nonorthogonal input factors can be used to measure stakeholder preferences’ impact on specific criteria (Saltelli 2004). The decision problem and stakeholder priorities will evolve in the future. As more information is obtained and stakeholders have a better understanding of the benefits and trade-offs, drivers may change, other alternatives may be considered, and more constraints identified. Adopting new infrastructure strategies involves an iterative decision process, long implementation timelines, and an adaptive process.ConclusionsIn this paper, a recent study of the benefits and trade-offs of four alternative centralized and decentralized strategies for the dual supply of raw and treated water (Cole et al. 2018a) was extended using uncertainty and sensitivity analyses to address decision risk, improve transparency in the decision process, and provide insight into the benefits and trade-offs of alternative strategies. Uncertainty analysis improves confidence in decision results by allowing decision makers to consider alternative performance over the full range of plausible input factor values and identify the most robust alternative over the entire range. The key drivers of model variance were regulatory and political in nature, which was not predicted originally by the project team and was revealed only through the sensitivity analysis.The uncertainty analysis results found that the Separated Irrigation alternative had the best overall performance when the alternatives were considered across the range of plausible uncertainty. However, the sensitivity analysis identified a few key drivers that should be addressed to reduce decision risk in the future. These include uncertainties that are more regulatory and political in nature rather than technical. Stakeholder priorities had the most influence, followed by the utility’s access to alternative nonpotable water sources in the future, the ability to sell potable water to adjacent utilities, and the possibility of cost savings by avoiding the need for water rights conversion. Resolving these issues depends on collaboration between the stakeholders from the initial study, state regulators, regional utilities, and local irrigation districts and canal companies.Analysis revealed that when uncertainty was considered, the centralized alternatives outperformed the decentralized alternatives considered here (Point-of-Entry and Neigborhood-scale systems). Risk of regulatory uncertainty was identified as sensitive. In addition, the Neigborhood alternative was sensitive to O&M uncertainty. However, the decentralized alternatives should not be ruled out in other locations. The Neighborhood alternative offers many benefits, comparable water treatment energy use and O&M costs to conventional systems, and modular solutions for areas that may not have capacity to accommodate future growth.The participatory decision process used in the study resulted in a large number of qualitative and indirect quantitative performance indicators that require more information to characterize alternatives’ performance (Cole et al. 2018a). Sensitivity analysis provided valuable insight for prioritizing which input factors on which to focus in the future and which input factors would not have much impact on performance. The results of any strategic decision-making process regarding urban water infrastructure will depend on local conditions and stakeholder priorities. Uncertainty and sensitivity analysis help provide confidence in the results and visibility into key uncertain input factors that require more attention in future analysis.References Archer, G. E. B., A. Saltelli, and I. M. Sobol. 1997. “Sensitivity measures, ANOVA-like techniques and the use of bootstrap.” J. Stat. Comput. Simul. 58 (Apr): 99–120. https://doi.org/10.1080/00949659708811825. Arden, S., B. Morelli, M. Schoen, S. Cashman, M. Jahne, X. Ma, and J. Garland. 2020. “Human health, economic and environmental assessment of onsite non-potable water reuse systems for a large, mixed-use urban building.” Sustainability 12 (Sep): 5459. https://doi.org/10.3390/su12135459. Bichai, F., H. Ryan, C. Fitzgerald, K. Williams, A. Abdelmoteleb, R. Brotchie, and R. Komatsu. 2014. “Understanding the role of alternative water supply in an urban water security strategy: An analytical framework for decision-making.” Urban Water J. 12 (3): 175–189. https://doi.org/10.1080/1573062X.2014.895844. Bigaret, S., R. E. Hodgett, P. Meyer, T. Mironova, and A. Olteanu. 2017. “Supporting the multi-criteria decision aiding process: R and the MCDA package.” EURO J. Decis. Processes 5 (1–4): 169–194. Bischel, H. N., G. L. Simon, T. M. Frisby, and R. G. Luthy. 2012. “Management experiences and trends for water reuse implementation in Northern California.” Environ. Sci. Technol. 46 (1): 180–188. https://doi.org/10.1021/es202725e. Brans, J. P., and B. Mareschal. 2005. “Ch. 5 PROMETHEE methods.” In Multiple criteria decision analysis: State of the art surveys. Berlin: Springer. Chen, R., and X. C. Wang. 2009. “Cost-benefit evaluation of a decentralized water system for wastewater reuse and environmental protection.” Water Sci. Technol. 59 (8): 1515–1522. https://doi.org/10.2166/wst.2009.156. Chung, G., K. Lansey, P. Blowers, P. Brooks, W. Ela, S. Stewart, and P. Wilson. 2008. “A general water supply planning model: Evaluation of decentralized treatment.” Environ. Modell. Software 23 (7): 893–905. https://doi.org/10.1016/j.envsoft.2007.10.002. Cole, J., D. Fourness, S. Sharvelle, N. Grigg, and L. A. Roesner. 2015. Fort Collins Utilities dual water systems study. Final Rep. Fort Collins, CO: City of Fort Collins Utilities. Cole, J., S. Sharvelle, D. Fourness, N. Grigg, L. A. Roesner, and J. Hakkas. 2018a. “Evaluation of centralized and decentralized strategies for dual water supply: Case study.” J. Water Resour. Plann. Manage. 144 (1): 05017017. https://doi.org/10.1061/(ASCE)WR.1943-5452.0000856. Cole, J., S. Sharvelle, N. S. Grigg, G. Pivo, and J. Haukaas. 2018b. “Collaborative, risk-informed, triple bottom line, multi-criteria decision analysis planning framework for integrated urban water management.” Water 10 (12): 1722. https://doi.org/10.3390/w10121722. Crook, J. 2005. “St. Petersburg, Florida, dual water system: A case study.” In Water conservation, reuse and recycling. Tunis, Tunisia: National Academies Press. Daigger, G. T., S. Sharvelle, M. Arabi, and N. G. Love. 2019. “Progress and promise transitioning to the One Water/resource recovery integrated urban water management systems.” J. Environ. Eng. 145 (10): 04019061. https://doi.org/10.1061/(ASCE)EE.1943-7870.0001552. Gopalakrishnan, V., S. Hirabayashi, G. Ziv, and B. R. Bakshi. 2018. “Air quality and human health impacts of grasslands and shrublands in the United States.” Atmos. Environ. 182 (23): 193–199. https://doi.org/10.1016/j.atmosenv.2018.03.039. Grigg, N. S., P. D. Rogers, and S. Edmiston. 2013. Dual water systems: Characterization and performance for distribution of reclaimed water. Water Research Foundation Rep. No. 4333. Washington, DC: USEPA. Hajkowicz, S., and A. Higgins. 2008. “A comparison of multiple criteria analysis techniques for water resource management.” Eur. J. Oper. Res. 2008 (184): 255–265. Iooss, B., and P. Lemaitre. 2014. “A review on global sensitivity analysis methods.” In Uncertainty management in simulation-optimization of complex systems: Algorithms and applications. Berlin: Springer. Kim, H. Y., and R. M. Clark. 1988. “Economies of scale and scope in water supply.” Reg. Sci. Urban Econ. 18 (4): 479–502. Lackey, K., S. Sharkey, S. Sharvelle, P. Kehoe, and T. Chang. 2020. “Decentralized water reuse: Implementing and regulating onsite nonpotable water systems.” J. Sustainable Water Built Environ. 6 (1): 02519001. https://doi.org/10.1061/JSWBAY.0000891. Langemeyer, J., E. Gomez-Gaggethen, D. Haase, S. Scheuer, and T. Elmqvist. 2016. “Bridging the gap between ecoystem service assessments and land-use planning through multi-criteria decision analysis (MCDA).” Environ. Sci. Policy 62: 45–56. Manikkuwahandi, T. D. S., G. M. Hornberger, and H. Baroud. 2019. “Decision analysis of expansion of Mahaweli multipurpose reservoir system in Sri Lanka.” J. Water Resour. Plann. Manage. 145 (9): 05019013. https://doi.org/10.1061/(ASCE)WR.1943-5452.0001094. Mare, G. D., M. F. Granata, and A. Nestico. 2015. “Weak and strong compensation for the prioritization of public investments: Multidimensional analysis for pools.” Sustainability 7 (12): 16022–16038. https://doi.org/10.3390/su71215798. Mukherjee, M., and K. Takara. 2018. “Urban green space as a countermeasure to increasing urban risk and the UGS-3CC resilience framework.” Int. J. Disaster Risk Reduct. 28 (1): 854–861. https://doi.org/10.1016/j.ijdrr.2018.01.027. Myrgiotis, V., R. M. Rees, C. F. E. Topp, and M. Williams. 2018. “A systematic approach to identifying key parameters and processes in agroecosystem models.” Ecol. Modell. 368 (23): 344–356. https://doi.org/10.1016/j.ecolmodel.2017.12.009. NASEM (National Academies of Sciences, Engineering, and Medicine). 2016. Using graywater and stormwater to enhance local water supplies: An assessment of risks, costs, and benefits. Washington, DC: The National Academies Press. Newman, J. P., G. C. Dandy, and H. R. Maier. 2014. “Multiobjective optimization of cluster-scale urban water systems investigating alternative water sources and level of decentralization.” Water Resour. Res. 50 (10): 7915–7938. https://doi.org/10.1002/2013WR015233. Raju, K. S., L. Duckstein, and C. Arondel. 2000. “Multicriterion analysis for sustainable water resources planning: A case study in Spain.” Water Resour. Manage. 2000 (14): 435–456. https://doi.org/10.1023/A:1011120513259. Saltelli, A., M. Ratto, T. Andres, F. Campolongo, J. Cariboni, D. Gatelli, M. Saisana, and S. Tarantola. 2008. Global sensitivity analysis—The primer. New York: Wiley. Saltelli, A., S. Tarantola, F. Campolongo, and M. Ratto. 2004. Sensitivity analysis in practice—A guide to assessing scientific models. New York: Wiley. Scholten, L., N. Schuwirth, P. Reichert, and J. Lienert. 2015. “Tackling uncertainty in multi-criteria decision analysis—An application to water supply infrastructure planning.” Eur. J. Oper. Res. 242 (1): 243–260. https://doi.org/10.1016/j.ejor.2014.09.044. Sheskin, D. J. 2011. Handbook of parametric and nonparametric statistical procedures. Boca Raton, FL: Taylor & Francis. Tang, Y., P. Reed, R. Wagener, and K. van Werkhoven. 2007. “Comparing sensitivity analysis methods to advance lumped watershed model identification and evaluation.” Hydrol. Earth Syst. Sci. 11: 793–817. Triantaphyllou, E. 2000. Multi-criteria decision making methods: A comparative study. Amsterdam, Netherlands: Kluwer Academic. Wilkins-Wells, J., et al. 2003. The benefits and costs of pressurized dual water systems in Colorado and the potential role of canal companies and irrigation districts in providing the pressurized irrigation water supply portion of dual systems. Fort Collins, CO: Colorado State Univ. Woods, G. J., D. Kang, D. R. Quintanar, E. F. Curley, S. E. Davis, K. E. Lansey, and R. G. Arnold. 2013. “Centralized versus decentralized wastewater reclamation in the Houghton Area of Tucson, Arizona.” J. Water Resour. Plann. Manage. 139 (3): 313–324. https://doi.org/10.1061/(ASCE)WR.1943-5452.0000249.



Source link