Cystic fibrosis (CF) is a relatively common orphan disease caused by loss-of-function mutations in the gene encoding CFTR (cystic fibrosis transmembrane conductance regulator), a tightly regulated anion channel (Riordan, 2008). CFTR mediates secretion across many epithelia in the body and is required for efficient mucociliary clearance of inhaled pathogens from the lungs (Ratjen et al., 2015). CF modulators such as lumacaftor (VX-809) that partially correct the misfolding and/or potentiate the activity of mutant CFTR channels are available and more are in the pipeline, however, they are expensive and their clinical benefit varies between individuals (Boyle et al., 2014; Hanrahan et al., 2017). With precision medicine one could potentially test multiple drugs on cells from a patient to identify the one that is most efficacious for that individual.

Although CF seems ideally suited for applying the precision medicine approach (Amaral, 2015; Martiniano et al., 2016; Burgener and Moss, 2018; Cholon and Gentzsch, 2018), some practical issues remain to be addressed. Foremost among these is whether differences in functional correction measured using primary cultured cells from different patients are statistically significant. This is obviously essential if prospective testing in vitro is to be useful for making prescribing decisions. The variable clinical benefit provided by Orkambi^® is well known (Boyle et al., 2014), however, it remains unclear if similar variability exists at the level of epithelial cells and persists in cell culture. If cells from different patients vary in their response to drugs and the responses correlate with clinical benefit, a practical question arises as to how many assays would be needed to conclude that one corrector is more effective than another for that individual. The causes of variability are not yet understood therefore it is not possible to develop a statistical model capable of predicting responses to a drug. Nevertheless, methods for collecting and analyzing data are needed if efficacy in cell-based assays is to be useful for precision medicine.

Assays of CFTR function that utilize different cell types have been developed and could potentially be used to test the drug responsiveness of individual patients. Rescue of the mutant CFTR can be assayed directly by measuring Cl⁻ transport across tissue samples or primary cell cultures using electrophysiology (Van Goor et al., 2011; Brewington et al., 2018). CFTR function can also be assayed indirectly by measuring net fluid transport across intestinal organoids prepared from rectal biopsies (Dekkers et al., 2013, 2016) or airway epithelial spheroids prepared from cells that are obtained by brushing or curettage of the nasal or bronchial mucosa (Guimbellot et al., 2017; McCarthy et al., 2018). Induced pluripotent stem cells (iPSCs) may be useful if they can be made to differentiate fully and recapitulate variable drug responses between patients having the same CFTR genotype, which may depend on both genetic and epigenetic factors. Correlations have recently been reported when assaying CFTR function in the nasal epithelium in vivo and in cultured cells from patients with different CFTR mutations (Pranke et al., 2017). Similar responses have also been observed in nasal and bronchial epithelial cultures (Pranke et al., 2017; Brewington et al., 2018). Importantly, for seven patients homozygous for F508del-CFTR there was a correlation between mean functional rescue in nasal cell cultures and the clinical response to Orkambi measured as % FEV1 (% predicted forced expiratory volume in 1 s) (Pranke et al., 2017).

In this paper we start by quantifying variability in functional correction amongst patients having the same genotype (F508del-CFTR/F508del-CFTR) using well differentiated bronchial epithelial cell cultures. Then we explore the reproducibility of correction by repeatedly testing samples from large pools of cells from two individuals, analogous to sampling the airway in vivo. We observe a design effect caused by group sampling that needs to be considered when testing the statistical significance of differences in correction.

Cystic fibrosis bronchial epithelial cells from all 20 patients were more strongly stimulated by forskolin + genistein after 24 h pretreatment lumacaftor (Figure 1D). The average stimulation after treatment with lumacaftor was 11.4% that measured in non-CF cells expressing wild-type CFTR (pooled data from 11 non-CF donors). This is similar to a previous report in which cells from seven patients that had been cultured in medium supplemented with Ultroser G and assayed acutely using a different potentiator (Van Goor et al., 2011).

The I_sc responses were ∼15-fold more variable after drug pretreatment than after pretreatment with vehicle. The variation was quantified using a mixed, two-way analysis of variance (ANOVA) of all data in both groups (i.e., baseline I_sc and maximum I_sc after stimulation by forskolin + genistein). Most (51.0%) of the variability was associated with the effect of drug pretreatment while 19.3% could be attributed to patients and 11.9% was explained by interaction between these factors; i.e., patient-to-patient differences in the response to lumacaftor (parameter of interest for precision medicine). Despite the small sample size, the null hypothesis that cells from different patients respond similarly to lumacaftor could be rejected with P = 0.0001. This is consistent with a previous study using a linear random effects model in which most variability in the response to lumacaftor was due to inter-patient rather than intra-patient variability (Pranke et al., 2017). Our power calculations to estimate the number of replicates needed for lumacaftor responses to reach statistical significance for a given effect size (i.e., increase of 20, 50, or 100%) yielded variable results when based on triplicate assays (Figure 1E) suggesting a larger sample size is needed to reliably estimate the standard deviation. Nevertheless, the results provide support for precision medicine in CF in that they show significant variability in drug responses between airway epithelial cells cultured from different individuals.

To assess the reproducibility of F508del-CFTR correction we compared the stimulation by forskolin + genistein on 3 successive weeks using different ALI cultures. The cells were prepared from different patients, cryopreserved and then thawed at 1 week intervals and cultured under identical conditions for 1 month for assays. Cells from one patient responded similarly to forskolin + genistein after pretreatment with lumacaftor in all three sets of experiments. However, ΔI_sc was more variable when cells from the other two patients (Figure 1F).

To explore this variability further we simulated in vivo sampling by dividing a pool of 170 million HBE cells from one patient into aliquots of 2 million. Each aliquot was considered a group sample from the airway surface epithelium to mimic repeated collection of small samples in vivo by bronchial brushing, but under well controlled conditions. Multiple ALI cultures were prepared using 23 randomly chosen aliquots (2–18 cultures per aliquot) and the ΔI_sc response to forskolin + genistein was measured using equal numbers of vehicle- and lumacaftor-treated replicates from each aliquot. Cells that had been treated with lumacaftor were always more responsive to forskolin + genistein than DMSO controls as expected (Figure 2A). One-way random effects ANOVA of ΔI_sc revealed significant variation between the groups (P < 0.0001), despite coming from the same original pool of cells.

Since prospective testing for the response to a CF drug will likely involve analyzing small cell samples in vitro, we examined the statistical consequences of such group sampling on correction. We calculated the intraclass correlation coefficient (ICC, also called ρ rho), which is the ratio of the variance between groups to the sum of the variances between and within groups. This was done when correction by lumacaftor was assayed repeatedly using 135 cultures from one patient (BCF00572). On average, each group consisted of η_o = 5.86 replicates and had ρ = 0.377, yielding a design effect [1 + ρ(η_o−1)] = 2.83. ICC was somewhat higher for vehicle-treated cells (ρ = 0.4466), in agreement with the lower variability of DMSO controls in Figure 2A.

To determine if the design effect is observed generally when assaying F508del correction we analyzed cells from a second patient (BCF00710). We selected 19 aliquots of cells (2 million cells each) at random from a pool of 280 million cells, cultured them at the ALI for 1 month, and assayed their stimulation by forskolin + genistein with and without lumacaftor pretreatment as before (Figure 2B). In this experiment there were η_o = 4.34 replicates per lumacaftor treatment group and equation (2) yielded ρ = 0.2816 and a design effect of 1.94. Plotting the residuals from a one-way random effects ANOVA of all data from lumacaftor-treated cultures from patient BCF00572 gave a bell-shaped frequency distribution that showed positive kurtosis but only moderate deviation from normality, supporting the validity of ANOVA (Figure 2C). In summary, the results from both patients indicate that variability of drug responses measured using group samples underestimates the true variation in the original population. Correcting for the design effect is necessary when comparing drug responses in vitro for precision medicine; two- to three-fold more replicates than estimated by regular power calculations are needed to draw conclusions based on two-tailed t-tests.

With this in mind we asked “How many replicates are needed to conclude that corrector B is more effective than corrector A using cells from one individual that have been collected and cultured in vitro?” To answer this we assumed a typical stimulation by forskolin + genistein after pretreatment with corrector A (lumacaftor; ΔI_sc = 3.3 μA/cm², s.d. = 0.962 μA/cm²; the mean from 20 patients studied under our conditions). This response was compared to that generated by hypothetical “corrector B,” which was assumed to have the same variance as lumacaftor. We set α = 0.05 (probability of Type I error or false positive when testing the null hypothesis H_o that correctors A and B have the same efficacy) and β = 0.2 (probability of a Type II error or false negative, corresponding to a statistical power of 1-β = 0.8; see Figure 2D). The number of replicates needed was then calculated as a function of power when corrector B would provide a real improvement in F508del-CFTR correction that is 20, 30, 50, or 100% higher than corrector A. Replicates were assumed to be independent samples, then the predicted number of replicates was then multiplied by 2.83 to correct for the effect of group sampling (Figure 2E).

The results indicate that if hypothetical corrector B has twice the efficacy of corrector A (i.e., increases F508del-CFTR rescue by 100%), 12 replicates from a group sample would be sufficient to conclude there is a significant difference between correctors. However, more replicates are needed if the real change in efficacy is less dramatic, e.g., 153 when corrector B increases F508del-CFTR function by only 20%.

The present results demonstrate statistically significant variation in responses to lumacaftor when assayed using cultured cells from different patients. Variable corrector responses have been observed previously (Van Goor et al., 2011; Eckford et al., 2014) and were analyzed for homozygous F508del CFTR patients in one study (Pranke et al., 2017), however, the implications for comparing efficacies of different drugs have not been explored. We also found considerable variability between cell samples from the same patient and explored its consequences when distinguishing between correctors. Prospective drug testing in vitro will likely begin with the collection of a small sample of epithelial cells from the patient. We used bronchial epithelial cells, however, similar results would be expected when small numbers of epithelial cells are harvested from the nasal or bronchial mucosa or by rectal biopsy. Since group samples have reduced standard deviation compared to the original epithelial cell population, they are expected to cause underestimation of the sample size needed for statistical testing, although this estimate can be corrected by determining the design effect.

Air-liquid interface cultures are considered to be the “gold standard” for testing CFTR modulators and have been accepted by the FDA when evaluating new drug applications (Durmowicz et al., 2018). Using this model to assay samples from a single large population of airway cells we estimated the design effect to range between 2 and 3. The sources of variability between group samples and replicates within groups are unknown and may reflect heterogeneity in the cell isolate and/or subtle differences during differentiation in prolonged culture. Cells were handled identically according to detailed standard operating procedures, nevertheless we cannot exclude slight variations in cell viability after thawing that could affect seeding density, or volume when the cells were fed with fresh medium. Cells were treated with corrector for 24 h in our study, however, lumacaftor is hydrophobic and longer exposures (48–72 h) are expected to increase its uptake and might reduce variability. The efficacy of Orkambi appears to be limited mainly by the modest efficacy of lumacaftor, therefore we have focussed on it in this study and pretreated cells only with lumacaftor. Simultaneous exposure to another drug such as ivacaftor could increase variability since variances add, although it will be important to test prolonged exposure to ivacaftor at clinically-relevant, low nanomolar free concentrations to avoid adverse effects on CFTR functional rescue by lumacaftor (Matthes et al., 2016). When multiple corrector drugs become available for CF they may be compared in vitro for precision medicine on the assumption that more functional expression in vitro will correlate with better symptoms in vivo. Such a correlation is observed between mean responses of multiple homozygous F508del-CFTR patients to Orkambi in vivo and nasal epithelial cultures from the same patients treated with ivorcaftor + lumacaftor (Pranke et al., 2017). Those results are also consistent with the ability of ivorcaftor alone to increase Cl⁻ conductance in cells expressing G551D-CFTR but not F508del-CFTR (Van Goor et al., 2009) and to improve lung function in patients with G551D (Accurso et al., 2010) but not in F508del homozygotes (Flume et al., 2012). Why correction varies between patients with the same CFTR genotype remains uncertain. Variation in genetic background (i.e., genes other than CFTR) may contribute since genome wide association studies have identified several genes that can modify CF disease severity (Cutting et al., 1990; Wright et al., 2011; Sun et al., 2012; Blackman et al., 2013; Corvol et al., 2015). Epigenetic variation caused by exposure to environmental factors could also contribute to variable corrector responses in vitro. Measuring correction in vitro probably excludes some off-target drug effects, however, the underlying genetic and epigenetic factors that affect F508del-CFTR expression, folding and trafficking may persist.

We were interested to compare the variability of correction in cells from patients and in F508del-CFTR homozygous mice (van Doorninck et al., 1995; Wilke et al., 2011). The genomes and epigenomes of CF mice are expected to be more similar than humans because they are inbred and housed under identical conditions. We examined correction in the ileum because the CF phenotype is stronger in the intestine than in the lung in mice. In preliminary experiments we isolated small pieces of ileum from CF mice (Cftr^tm1Eur) (van Doorninck et al., 1995) in quadruplicate and measured the forskolin-stimulated ΔI_sc after exposing tissues to DMSO or lumacaftor for 4 h ex vivo as described previously (Robert et al., 2010). Mixed, two-way ANOVA showed that the variability of F508del-CFTR correction was less in inbred mice than in patients. The interaction term (i.e., variability in drug response between individuals) accounted for ∼6.5% of the variance in mouse intestinal assays vs. 11.9% in HBE cultures from patients but was still significant (P < 0.0001). We believe this variation reflects the health of the mice or the condition of tissues after dissection, but cannot exclude that genetic and/or epigenetic variations persist in the mice despite inbreeding and the same environment.

The application of personalized medicine in CF has a bright future, however, the predictive value of in vitro assays remains to be established when there are alternative drugs and multiple patients carrying the same mutation. The present results indicate that patient-to-patient differences in F508del-CFTR correction can be assayed using airway cell cultures, however, a design effect due to group sampling needs to be compensated by increasing the number of replicates. Hopefully, CF correctors and corrector combinations will continue improving until they become so effective for all patients that precision medicine is no longer necessary (Hanrahan et al., 2017).

JH, EM, and DT: conceptualization. EM, JG, CM, JS, JL, and JH: investigation, and validation. JH: writing—original draft. JH, EM, JG, CM, and DT: writing—review and editing. JH and DT: funding acquisition. JG, CM, JL, and EM: resources.

CF Translational Research Centre seminar series is supported by Vertex Pharmaceuticals Inc. JH is a mentor for the Vertex CF Research Innovation Award program. Vertex had no input into this study. The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.