© D. Spitale, Published by EDP Sciences 2017

1 Introduction

Biological monitoring, or biomonitoring, is the use of biota to track changes in the environment (Friberg et al., 2011). In freshwater habitats, biomonitoring is a key tool used by managers and environmental agencies to support their decisions. In European Union, the Water Framework Directive (WFD 2000/60/EC) is the main normative in the field of water monitoring and conservation. The WFD introduced profound innovations, changing the perspective of monitoring from a mere detection of pollution to a determination of the ecological status comparing the observed status against the reference condition. Even though this approach is ecologically sound, as any measurement of the ecological status, it is associated with an unknown source of uncertainty that would hamper a correct decision-making process by water managers (Friberg, 2014).

The benthic macroinvertebrates are indicators used throughout the world for evaluating the river quality for their capacity to assess multiple anthropogenic pressures. Member States of EU employ different sampling methods to collect macroinvertebrates in rivers (e.g. fixed time kick sampling, Surber sampler on fixed or not area), and they record abundance either using real estimates or classes (Bennett et al., 2011). Although assessed in the early stages of the intercalibration process among Member States, the use of fixed number of individuals to calculate macroinvertebrate index was not widely adopted (Clarke et al., 2006a). However, the effect of using different sampling efforts (=different number of individuals) has long been recognised as a serious pitfall in comparing samples (Gotelli and Colwell, 2001).

In Italy, the ecological status of rivers is assessed using diatoms, macrophytes, fish and macroinvertebrates, each one with specific methods and procedures (ISPRA, 2014). Concerning macroinvertebrates, the sampling method relies on the multihabitat approach (Hering et al., 2003), and the ecological status is calculated with the STAR_ICMi index as ratio with the reference condition (Buffagni et al., 2006; Bennett et al., 2011). The STAR_ICMi (for an historical account of the index see Bo et al., 2016) is a multimetric index composed of 6 metrics thought to represent different aspects of the macroinvertebrate community. Even though the performance of this index has been extensively tested across Europe especially for harmonisation purpose (e.g. Bennett et al., 2011), a wide knowledge gap still exists concerning the accuracy of the metrics (but see Clarke et al., 2006b; Clarke, 2013; Ramos-Merchante and Prenda, 2017).

All the assessments of the ecological status of water bodies using macroinvertebrate sampling are subjected to uncertainty and errors due to a range of factors including sample processing (subsampling, unfixed number of individuals), misidentification of organisms, different trained staff and so on (Clarke et al., 2006a). The sources of uncertainty, which could generate variability in the STAR_ICMi index, are diverse and not simple to disentangle. Restricting the analysis to the fine scale, namely that one considered for the bioassessment, three main groups of causes could be recognised: ecological, errors and sampling-related causes (Clarke et al., 2006a, 2006b). Quantification of these uncertainties is necessary to assign a confidence level in the classification of water bodies and to prescribe the correct management options in the case of deterioration or amelioration. Wrong classification of the ecological status has relevant consequences in terms of expensive remedial measures but also as missed economic benefits.

In Italy, like in many MS of EU, macroinvertebrate indices are calculated examining a variable number of individuals (Bennett et al., 2011). However, the use of unfixed abundance hinders the correct comparison of samples because of the inconsistent sampling effort. We urgently need additional studies addressing the effect of this unfixed abundance on bioassessment. The ecological status of many rivers could be wrongly assigned only as a result of this source of error. Moreover, the lack of a measure of uncertainty in the STAR_ICMi index is a serious drawback, which should be overcome (Friberg, 2014). An estimate of uncertainty can help the Environmental Agencies in interpreting situations in which the ecological status of rivers falls near boundary classes. If on one hand a careful attention to these topics seems desirable, it should be admitted that the application of the STAR_ICMi index is more demanding in terms of time, in respect to early indices like the IBE (the Italian version of the Extended Biotic Index, Ghetti, 1997). Therefore, it seems useful to devise alternative methods ensuring rigorous assessments in shorter time.

Aim of the study was threefold. (1) To test if and to what extent subsampling has an effect on the STAR_ICMi index and classification of the water bodies; (2) to suggest a method able to provide a measure of index precision, even on a single sample basis; and (3) to assess the effects of less time-consuming alternatives to the complete census of individuals in the samples.

2 Methods

2.1 The multihabitat approach

The multihabitat approach has been devised to provide a standardised assessment of the macroinvertebrates community in a water body. The method requires two steps: (1) visual quantification of the different abiotic substrates within a representative river reach, and (2) proportional assignments of samples according to the area occupied by substrates. For example, if in a reach the relative abundance of substrates is 80% megalithal (stones > 40 cm) and 20% mesolithal (20–40 cm), the multihabitat sampling requires to collect 8 subsamples on megalithal and 2 on mesolithal. Subsamples are taken disturbing an area of 0.1 m² and collecting macroinvertebrates with an hand net (mesh size 500 micron). In standard biomonitoring assessment, these subsamples are then integrated into a single sample representative of the river reach (ISPRA, 2014).

The 8 rivers used as example were localised in Trentino Alto Adige (Italy), in two hydro-ecoregions, Southern pre-Alps and Dolomites and Eastern Inner Alps. In contrast to what suggested in the official sampling protocol (ISPRA, 2014), samples were immediately fixed with ethanol (final concentration ∼70%) and processed in laboratory. No attempts were made to sort the samples in the field. All the individuals were counted and identified at the taxonomic resolution of genus and family.

2.2 The STAR_ICMi index

The STAR_ICMi is a multimetric index which combines 6 metrics (Tab. 1): ASPT (Average Score Per Taxon), Log10_Sel_EPDT+1 (logarithm of the sum of selected Ephemeroptera, Plecoptera, Diptera and Trichoptera taxa), 1-GOLD (where GOLD is the relative abundance of Gastropoda, Oligochaeta and Diptera), number of families, number of families in the EPT orders (=Ephemeroptera, Plecoptera and Trichoptera) and the Shannon–Wiener diversity index (H′). The selected indices include different parameters of benthic communities, such as taxa sensitivity, abundance and diversity. To calculate the index, each metric is combined according to a specific weight, and the final score is expressed as the ratio between the observed and the reference condition (Tab. 1) (ISPRA, 2014).

2.3 Data analyses

2.3.1 Rarefaction

In order to investigate the effect of subsampling on the STAR_ICMi index, I used rarefaction technique. Rarefaction consists in extracting, at random and without replacement, some individuals from the original community (Gotelli and Colwell, 2001). In this way, it is possible to study the index performance when a different number of individuals are counted and identified. The rarefaction curve was constructed by extracting the samples of different size (100 individuals, 200, 300, etc.), and the procedure was repeated 1000 times for each sample size. Thus, the rarefied community was composed of 1000 extractions of size 100 individuals, 1000 of 200 individuals, 1000 of 300 and so on until the total number of individuals (1600 in the example showed). On each one of these rarefied communities, the STAR_ICMi index was calculated to test its performance. Rarefaction was performed on all the 8 rivers, but results showed a similar common trend. Therefore, I presented only a single representative example. All the analyses were conducted using R (R Development Core Team, 2016).

2.3.2 Bootstrap

The index uncertainty was estimated bootstrapping the observed abundance of taxa (Dixon, 1993; Manly, 2006). The bootstrap distribution of the STAR_ICMi index was calculated by resampling the individuals of the original sample. For example, if the sample assemblage consists of 4 species for a total abundance of 100 individuals (80 individuals of sp1, 15 of sp2, 5 of sp3), the bootstrap algorithm extracts, at random and with replacement, 100 individuals. Because the extraction is random and with replacement, the occurrence and the abundance of the species change on each run of bootstrap. Repeating the procedure many times, it is possible to generate a family of bootstrap assemblages on which to calculate the STAR_ICMi index. The resampling procedure was repeated 10,000 times to generate the sampling distribution of the index. The confidence interval was calculated from the bootstrap distribution of the STAR_ICMi index using the percentile method (95%). The procedure accounts only for the stochastic sampling effect, and it represents only a fraction of the entire variability represented by other sources of variation, such as local heterogeneity (e.g. Laini et al., 2014; Ramos-Merchante and Prenda, 2017). In other words, the procedure attempts to estimate the index precision, not the accuracy (Dixon, 1993). Elsewhere, this bootstrap sampling distribution would be called a “null distribution”, as the variability accounted for by the model is assumed to arise with no particular ecological process or mechanism: it represents a sampling variability generated by random selection from the species assemblage according to each species' relative abundance (Gotelli and Graves, 1996).

2.4 Different input data

To evaluate possible means to save time in the enumeration of all the individuals in the sample, different data transformations were tested. The four methods, from the most to the less time demanding were: (1) counting all the individuals of taxa with abundance less than 10 and estimating the most abundant (>10 ind.) subsampling the original sample (Buffagni and Erba, 2007). However, subsampling introduces an additional source of uncertainty in the STAR_ICMi index that should be avoided. A possible way to predict the unknown abundance is to use a model of species abundance distribution (Magurran, 2004 and references therein). Among the many species of abundance distribution models, the geometric series was selected as one of the oldest, and most known models. When the species abundance distribution follows a geometric series, the relationship between the log abundance and the species ordered by rank is linear (Fattorini, 2005). Therefore, the most abundant taxa are estimated in two steps, (i) calculating the linear regression between the log abundance and rank using only the low abundant taxa and then (ii) forecasting the abundance of the most abundant taxa assuming their correct rank. In Appendix 1, a spreadsheet was provided to perform this calculation. See also Fattorini (2005) to an explanation on how to fit a geometric series using linear regression. (2) The second method thought to be less time demanding, is the use of rank, instead of the real abundance of taxa. Rank could be more quickly determined using subsamples. (3) For similar reasons, I tested the effect on the STAR_ICMi index of using classes of abundance (10 equal classes). The advantage of using classes is that they are attributed with more confidence than rank. (4) Finally, I evaluated the effect of using only the presence of taxa. Even though this last method is not consistent with the WFD, which requires quantitative assessment (European Union, 2000), this last option was tested for the sake of completeness.

3 Results

3.1 Subsampling effects

The effect of subsampling on the STAR_ICMi index was substantial in all the 8 rivers considered (Fig. 1 reported only one representative example, Arnò High). Half of the subindices were sensible to the number of individuals (logEPTD, number of families and EPT), whereas ASPT, GOLD and Shannon index were poorly affected. Even though the subindices have different weight in the STAR_ICMi (Tab. 1), the index was clearly dependent by the number of individuals. In the example, the ecological status inferred from the index changed from POOR (when only 100 individuals were counted), to MODERATE (between 200 and 800 individuals) and GOOD above. The final STAR_ICMi index with all the individuals counted was 0.826 in the example.

Fig. 1 The effect on the STAR_ICMi index, and relative subindices, of the incomplete counting of individuals in a sample. Error bars are standard deviation.

3.2 Index uncertainty

The precision of the STAR_ICMi index was different in the 8 rivers and ranges from 1.2% to 5.9% (coefficient of variation). Although quite precise that variation increased the risk of misclassification when the index was close to a boundary class. For example, in Pescara river, the index was 0.724 (0.71 was the boundary class between MODERATE and GOOD), and in the 16.5% of cases, the ecological status would fall into the lower class. The 95% confidence interval for the river was 0.699–0.747. The STAR_ICMi index distribution was not always symmetric and centred on the observed value as shown in the example (Fig. 2). In some rivers, the index distribution was skewed, as deducible observing the asymmetrical error bars in Figure 3.

Fig. 2 Application of the bootstrap approach to estimate the confidence interval of the STAR_ICMi index in the Pescara river. The observed index was 0.724, and the 95% confidence interval was 0.699–0.747. The boundary between MODERATE and GOOD class is 0.71. In this example, the river would be classified as GOOD (0.724>0.71), but the confidence interval suggested that the probability of wrong classification was not negligible (in the 16.5% of cases the ecological status would fall into the MODERATE class).

Fig. 3 Comparison of the effect of different input data on the STAR_ICMi index in 8 rivers. Observed is the real score counting all the individuals; the error bars on the observed value are calculated using the bootstrap approach and represent 95% confidence intervals. Estimate is the index value when the abundance of the most abundant taxa was estimated using the geometric series (see Appendix 1 for examples); rank, class and presence is the index value when the real input data were reduced to rank, classes and presence respectively.

3.3 Different input data

The method that estimates the most abundant taxa produced the less biased results (Fig. 3). The STAR_ICMi index, calculated on the estimated abundance, was close to the observed value and most of the times within the confidence interval calculated by bootstrap (Tab. 2). Using rank, classes and presence as input data produced results that were substantially different from the real ecological status calculated by counting all the individuals.

4 Discussion

The risk of misclassification can have serious ecological and economic consequences, especially when the water body is put in the moderate rather than in the good class. It has been demonstrated that the risk of misclassification is not negligible, and a measure of index uncertainty is essential for managers to take informed decisions. The approach used here to calculate index uncertainty is general, and as such could be applied also for other indices and taxonomic groups beyond the macroinvertebrates and the STAR_ICMi index.

The STAR_ICMi is a multimetric index affected by subsampling because three out of six subindices are sensible to the number of individuals counted. The three subindices affected by number are calculated as sums (logEPTD, number of families, EPT), whereas the three that are not have ratios in their equations (ASPT, GOLD, Shannon). The subindex logEPTD strictly depends on taxa abundance as it is calculated by summing the individuals belonging to several EPTD families (Tab. 1). As the number of individuals is added on, the number of taxa increases, and similarly does the probability to include the EPTD families. Similar arguments hold true for the subindices number of families and EPT. In contrast, the ASPT subindex reached an asymptote with about 400 individuals. The ASPT is calculated considering the score of taxa, and more precisely it is computed as the ratio between the sums of scores and the number of taxa (Armitage et al., 1983). Thus, as individuals are added on, both the numerator and denominator increase, and the ratio changes little. Analogous explanations apply for the GOLD (Pinto et al., 2004) and Shannon index (Shannon and Weaver, 1949). The cumulative sum of these effects determines the strong sensibility of the STAR_ICMi index to the number of individuals enumerated. Interestingly, the STAR_ICMi index did not show clear signs of levelling off in any of the 8 rivers examined. There is no general agreement on the minimum number of individuals needed to get the true ecological status. For example, Doberstein et al. (2000) showed that subsamples containing 100–300 individuals produced highly inaccurate results, while Ostermiller and Hawkins (2004) suggested to consider samples with more than 350 individuals. It is unlikely that more studies on the minimum sampling effort will produce general recommendations, as precision changes according to the type of stream and the indices used (Lorenz et al., 2004). The most appropriate suggestion seems to define a target number, and to rarefy the communities to that common sampling effort. Only in this way, samples with different number of individuals can be correctly compared (Gotelli and Colwell, 2001). According also to other authors (e.g. Clarke et al., 2006b), 700 individuals could be a reasonable compromise between effort and precision. Hence, the best strategy would be to census at least, or more than 700 individuals, and when more, to rarefy the assemblage to the target.

Three main groups of causes could be recognised as sources of uncertainty in the STAR_ICMi index: (1) imputable to the ecological variability, (2) because of technical errors, and (3) due to sampling-related causes. To the first group belongs all the ecological processes which lead to the heterogeneous distribution of the macroinvertebrate assemblages in terms of species and density, such as habitat preferences, competition, predation, dispersal, behavioural and so on (Taylor, 1984). Even though the multihabitat method requires a standard approach with fixed number of subsamples, fixed area and fixed proportion of substrates on a given stretch of river, the procedure will never provide two identical samples. These differences eventually lead to different results of the STAR_ICMi index, and sometimes to different ecological status. This variability can be accounted for adopting stratified sampling (Laini et al., 2014), or hierarchical sampling (e.g. Li et al., 2001). Similar methods can be adopted also to estimate uncertainty due to errors (2), like identification of taxa, different trained staff and so on (Clarke et al., 2006a). However, another source of variability exists, and this is related to the sampling process (3). Unless it would be possible to identify and count the entire benthic community, we need some kinds of sampling to estimate the abundance of each taxon. As a result, the data used to calculate the index contain sampling error. If we have a jar with different coloured balls (the species), and we extract (the kick sample) several balls, results will not be the same in consecutive draws even replacing the balls into the jar. Uncertainty calculated by bootstrap represents the part of variability in the STAR_ICMi index that would be obtainable repeating the identical sampling survey. This form of variability is rarely included in studies of index uncertainty, but it is always present (see Dolph et al., 2010 for a quantification of sampling variability with bootstrap). Importantly, it has been shown that this variability was not negligible, especially when the ecological status was near a boundary class. Therefore, it is suggested to present the STAR_ICMi index with the uncertainty at least due to the sampling variability (3) when other estimates (1, 2) are unavailable (as is usually the case in routine biomonitoring).

One of the critical aspects of routine application of the multihabitat method and STAR_ICMi application is that it is extremely time consuming (Bo et al., 2016). Low budget and limited human resources put at risk the biomonitoring of many watercourses, which were previously surveyed with less demanding methods and procedures. Indeed, early works considered pro and cons of this new approach providing evidences about the size of sample, the minimum number of individuals, and the time requested by the different options (Vlek, 2004). However, still lacking was a work testing the effect of less time-consuming alternatives than the whole census of all the individuals in the sample. The most promising method, namely that one producing results close to those observed by processing the whole sample, was to estimate the most abundant taxa using the rank-abundance model. The limit of this approach is that it assumes the geometric distribution in all the samples, which might not be the case in all the assemblages (Magurran, 2004).

5 Conclusions

Three main conclusions can be drawn from this study. First, because the STAR_ICMi index is affected by subsampling, it is suggested to sort at least 700 individuals, and when more to calculate the index rarefying the community to the target (700). In this way, sampling effort is consistent among surveys, and comparison is unbiased. As a medium-term perspective, the STAR_ICMi index might be modified in order to become less dependent on sampling effort. Second, index precision should always be presented along with the index score to quantify uncertainty in assigning the quality classes. Although the bootstrap approach does not account for all the sources of index variability, it is a valuable resource for the competent authorities to take informed decisions. Third, borrowing one of the oldest and most universal laws of ecology, I showed how to use a rank-abundance model for application purpose. The geometric series is an easy model, simple to compute, and for this it could be added to the practitioner's toolbox.

Supplementary Material

Appendix 1.

Excel spreadsheet with two examples showing how to estimate the number of individuals of the most abundant taxa.

References

All Tables

All Figures

	Fig. 1 The effect on the STAR_ICMi index, and relative subindices, of the incomplete counting of individuals in a sample. Error bars are standard deviation.
In the text

Fig. 2 Application of the bootstrap approach to estimate the confidence interval of the STAR_ICMi index in the Pescara river. The observed index was 0.724, and the 95% confidence interval was 0.699–0.747. The boundary between MODERATE and GOOD class is 0.71. In this example, the river would be classified as GOOD (0.724>0.71), but the confidence interval suggested that the probability of wrong classification was not negligible (in the 16.5% of cases the ecological status would fall into the MODERATE class).

In the text

Fig. 3 Comparison of the effect of different input data on the STAR_ICMi index in 8 rivers. Observed is the real score counting all the individuals; the error bars on the observed value are calculated using the bootstrap approach and represent 95% confidence intervals. Estimate is the index value when the abundance of the most abundant taxa was estimated using the geometric series (see Appendix 1 for examples); rank, class and presence is the index value when the real input data were reduced to rank, classes and presence respectively.

In the text