ISO 11132:2021 pdf download – Sensory analysis — Methodology — Guidelines for the measurement of the performance of a quantitative descriptive sensory panel.
4.1.3? Ongoing? monitoring? via? routine? product? profiling Another approach consists of monitoring profile data that was already collected. To review ongoing profile data generated by a panel, it can be appropriate to use data that originated from quite different profiling experiments using different product types, product numbers, etc. The procedure is the same as that shown in Figure 1. However, as there are no predefined differences, it is recommended that attributes for which the products are significantly discriminated by the panel as a whole for a given profile be used as the key measures to check the performance of individual sensory assessors. Attributes that result in no significant difference cannot be reliably used to check consistency since the lack of agreement within and between sensory assessors probably means that the products are very similar for those characteristics. In this case, over a given period, it will be necessary to check on a set of products more different that the panel is indeed capable of highlighting difference in these characteristics. 4.2 Indicators of panel or individual assessor performance For one assessment, the following indicators can be determined: — discrimination of the panel, measured as the ability of the panel to exhibit significant differences among products; — discrimination of an assessor, measured as the ability of the assessor to exhibit significant differences among products; — agreement of an assessor, measured as the degree of alignment between the assessor’s average product scores and the ones of the panel; — agreement of the panel, measured as the degree of alignment between the assessors’ average product scores. For replicate assessments: — repeatability of an assessor, measured as the degree of homogeneity between replicated assessments of the same product;
— repeatability of a panel, measured as the average degree of homogeneity between replicated assessments of the same product for each assessor. 4.3 Statistical analyses A single, consistent approach to statistical analysis of the results is described in this document. However, some indicators of panel performance can be assessed by more than one measure. For instance, error mean square and error standard deviation (SD) (its square root) both express variability in the evaluation of a product. The measures used should be those that are usual in the field of application. Other relevant measures of agreement between assessors in the use of the scale for an attribute are the interaction of assessor and product and the coefficient of correlation between an assessor’s scores and the panel means. An assessor may have no bias but may use the scale in a different way. A correlation close to 1, a regression slope close to 1 and a regression intercept close to 0 indicate good agreement between an assessor and the rest of the panel. When each assessor evaluates a small number of samples (fewer than six), the correlation coefficient should be interpreted with caution, as it can be high (up to 0,7) by chance alone.