Flow Cytometric Analyses of Lymphocyte Markers in Immune Oncology: A Comprehensive Guidance for Validation Practice According to Laws and Standards

Many anticancer therapies such as antibody-based therapies, cellular therapeutics (e.g., genetically modified cells, regulators of cytokine signaling, and signal transduction), and other biologically tailored interventions strongly influence the immune system and require tools for research, diagnosis, and monitoring. In flow cytometry, in vitro diagnostic (IVD) test kits that have been compiled and validated by the manufacturer are not available for all requirements. Laboratories are therefore usually dependent on modifying commercially available assays or, most often, developing them to meet clinical needs. However, both variants must then undergo full validation to fulfill the IVD regulatory requirements. Flow cytometric immunophenotyping is a multiparametric analysis of parameters, some of which have to be repeatedly adjusted; that must be considered when developing specific antibody panels. Careful adjustments of general rules are required to meet legal and regulatory requirements in the analysis of these assays. Here, we describe the relevant regulatory framework for flow cytometry-based assays and describe methods for the introduction of new antibody combinations into routine work including development of performance specifications, validation, and statistical methodology for design and analysis of the experiments. The aim is to increase reliability, efficiency, and auditability after the introduction of in-house-developed flow cytometry assays.


Data
The data are from an experiment investigating repeatability and variance of a factor (e. g. operator).
The data refer to a 3 (operators) x 5 (replicates) design using 3 samples.
Sample factor repl 1 repl 2 repl 3 repl 4 repl 5 1 1 9.9 9.8 11.7 11.9 9.7 1 2 9.6 9 8.9 8.9 8.5 1 3 11.9 11.3 12.9 11. The description shows dependency of standard deviations on concentration, the relative standard deviation (%CV), however, does not depend on concentration. In that case, natural-log-transformed values (ln) can be used for analysis. The advantage of using ln-transformed values is that the standard deviation (e. g. 0.1) obtained for these transformed values can directly be read as %CV for original scaled values (e. g. 10%). This approximation holds for all %CVs <30%.

Estimation of variance components by sample, based on statistical software
The advantage of using statistical software is the availability of confidence intervals. Moreover, the analysis of precision experiments with 2 or more factors is possible, and unbalanced data (datasets with missing values) can be analysed.
The disadvantage, however, is, that typically a huge number of options is available. The results between different software as well as between versions of the same software might slightly differ which makes it difficult to validate the analyses. It is recommended to gain experience with known data sets before using such software. The software version to be used should be prospectively defined in the experimental protocol.

Preparation of data for analysis
For statistical analysis with statistical software (R, Analyse-It® or other), the data should be provided in long format.

Analysis with software R
The analysis with software R requires programming skills in this software.
Within R, one should install package VCA, which was developed for analysis of hierarchical precision experiments according CLSI guideline EP05-A3.
The following code can be used (for pooled analysis, ignore the results for total and sample): #load Package VCA library(VCA) path<-file.choose(new=FALSE) study<-read.csv(path, header=TRUE, sep=",", dec=".") str(study) fit<-anovaVCA(ln_value~sample/factor,study) inf <-VCAinference(fit, VarVC=TRUE) as.matrix(inf, what="SD", digits=6) Note: The code for precision analysis is just 1 line (fit…). The both last lines refer to estimation of CI. Note that results for CI could differ between software packages. Note that other methods for estimation of variance components are available (e. g. REML [remlVCA]-restricted maximum likelihood instead of ANOVA, which is advantageous in case of missing values).
Whereas the package VCA (see https://cran.r-project.org/web/packages/VCA/index.html for details) was especially programmed for analysis of precision experiments, one might use other packages which are programmed for the underlying statistical models (so called mixed models).

Analysis using Software Analyse-It®
The results obtained with software Analyse-It (a commercial Excel-Add-On, www.analyze-it.com) are presented. The confidence intervals for between-factor results are really large, so these results underlie a large uncertainty. This is due to the relative small sample size for the experiment per sample (15 measurements).
Using the sample as a naïve factor in a 2-factor analysis, one achieves the pooled results with smaller confidence intervals.

Discussion
A simple 1-factor experiment can be analysed using spreadsheet-software like MS-Excel.
However, the calculation of uncertainty of the results as expressed by 95%-confidence intervals requires statistical software. Given the homogeneity of the variances, the pooling of the results over the samples improves the reliability, however, the sample size chosen in the example (3 samples x 3 factor-item x 5 replicates) is still small. Using more samples, factor items or replicates might be appropriate to achieve larger sample sizes. In the CLSI-guideline EP05-A3 about 75 .. 80 samples are used for an hierarchical experiment with 2 factors, thus a 5 samples x 3 factor-item x 5 replicates -experiment ( 75 samples) is more efficient.