We selected randomly 1,000 Caribbean Hispanics as part of an original genotyped cohort of 3,138 individuals: genotyped data can be downloaded at dbGaP Study Accession: phs000496.v1.p1. 719 individuals were derived from Estudio Familiar Investigar Genetica de Alzheimer (EFIGA), a study of familial LOAD; and 281 individuals from the multiethnic longitudinal cohort, Washington Heights, Inwood, Columbia Aging Project (WHICAP). The information on study design, recruitment and GWAS methods for the EFIGA and WHICAP study was previously described in Tosto et al. (2015).

Genotyped data underwent quality control using PLINK (v1.90b4.9 64-bit) (Purcell et al., 2007). Briefly, we excluded SNPs with missing rate ≥ 5% followed by exclusion of SNPs with MAF ≤ 1%. We then removed SNPs with P-value < 1e-6 for Hardy-Weinberg Equilibrium. Samples with missing call rate ≥ 5% were excluded from analysis.

Prior to imputation, we estimated global ancestry using the ADMIXTURE (v.1.3.0) software (Alexander et al., 2009; Zhou et al., 2011). We conducted supervised admixture analyses using three reference populations: African Yoruba (YRI) and non-Hispanic white of European Ancestry (CEU) from the HAPMAP project as representative of African and European ancestral populations; and eight Surui, 21 Maya, 14 Karitiana, 14 Pima and seven Colombian individuals from the Human Genome Diversity Project (HGDP) were used to represent native American ancestry (Li et al., 2008). We used ∼80,000 autosomal SNPs that were: (I) genotyped in all three datasets (Caribbean Hispanics, 1000G and HGDP); (II) common (i.e., MAF > 5 %); and III) in linkage equilibrium. Supervised admixture analyses with the three reference populations (YRI, CEU, and Native Americans) revealed that European lineage accounted for most of the ancestral origins (59%), followed by African (33%) and native American ancestry (8%). We then selected only individuals with at least 1% of all three ancestral populations.

HRC reference panel contained over 39M SNPs from 27,165 individuals who participated in 17 different studies (Table 1). The data were downloaded from the Wellcome Trust Sanger Institute (WTSI).

1000G phase 3 reference panel contained over 81M SNPs from 2,504 individuals¹. It includes 26 ethnic groups, with most variants rare, approximately 64 million had MAF < 0.5%; approximately 12 million had a MAF between 0.5 and 5%; and approximately eight million have MAF > 5%. In order to perform imputation with MaCH-Admix, 1000G Phase 3 pre-formatted data were downloaded from ftp://yunlianon:anon@rc-ns-ftp.its.unc.edu/ALL.phase3_v5.shapeit2_mvncall_integrated.noSingleton.tgz that contained over 47M SNPs.

The subsequent analyses were restricted to autosomal chromosomes, only.

We compared SHAPEIT2 (Delaneau et al., 2013) and Eagle2 (Loh et al., 2016) by phasing and then imputing (see next section) a single chromosome (Chromosome 21), using both reference panels. We refer to SHAPEIT2 as SHAPEIT when used in tandem with IMPUTE2 for the remainder of paper.

Imputation was carried out using two bioinformatics tools: IMPUTE2 (Howie et al., 2009) and MaCH-Admix (Liu et al., 2013). For both, imputation quality ranged from 0 to 1, with 0 indicating complete uncertainty in imputed genotypes, and 1 indicating no uncertainty in imputed genotypes.

IMPUTE2 uses “Info” parameter to report imputation quality that measures relative statistical information about SNP allele frequency from imputed data. It reflects the information in imputed genotypes relative to the information if only the allele frequency were known. “Info” metric is used to filter poorly imputed SNPs from IMPUTE2 and is reported for all imputed SNPs. In addition, IMPUTE2 uses an internal metric known as R², reported for genotyped SNPs only: it measures squared correlation between genotyped SNPs and the same SNPs that have been first masked internally and then imputed. MaCH-Admix uses Rsq to report imputation quality. The R² metric is also known as variance ratio, calculated as proportion of empirically observed variance (based on the imputation) to the expected binomial variance p(1-p), where p is the minor allele frequency. A threshold of 0.30 is recommended to filter out poorly imputed SNPs.

Despite quality measures from IMPUTE2 and MaCH-Admix being highly correlated (Marchini and Howie, 2010), we calculated a r2hat score to generate a single common metric to assess imputation quality across the software (Hancock et al., 2012) (v109)².

We compared performance of MaCH-Admix and SHAPEIT-IMPUTE2 by: (a) Reporting raw SNP counts based on quality (MaCH-Admix “Rsq” and IMPUTE2 “Info”); (b) Comparing r2hat for overlapping imputed SNPs from both tools; (c) Conducting a Wilcoxon Signed-Rank Test (R v3.4.2) on r2hat value of overlapping SNPs.

We compared performance of Eagle2 and SHAPEIT2 phasing tools in tandem with IMPUTE2 as imputation tools across reference panels by: (a) Comparing their respective IMPUTE2 R²: (b) Conducting a Wilcoxon Signed-Rank Test on R² value; (c) Reporting raw counts of imputed SNPs based on IMPUTE2 “Info” metric and stratified by MAF bins (e.g., common, rare, ultra-rare).

In all comparisons, the MAFs are estimated from imputed data according to the reference panel employed. We retained monomorphic SNPs in our analyses for several reasons. A monomorphic SNP in one study might not be monomorphic in other cohorts. This has profound affects, for example, when performing meta-analysis across different studies. In addition, monomorphic SNPs provide information about MAF across studies. Without the information it is difficult to tell, for instance, if a SNP is monomorphic or failed quality control in that study.

To further test the quality of imputation -without relying on software’s internal metrics (i.e., “Info” and R²) - we calculated genotyped concordance between imputed and WES data using the VCF-compare tool (v0.1.14-12-gcdb80b8) (Danecek et al., 2011). First, we converted posterior probabilities obtained from imputation into genotype data using the PLINK software (v1.90b4.9) by applying a threshold of 0.9 (Supplementary Section S1), such that SNPs that failed on this criterion were left uncalled. For example, an imputed SNP with P(G = 0,1,2) = (0.01,0.9,0.09) would be called as a ‘1’ (heterozygous), whereas an imputed SNP with P(G = 0,1,2) = (0.2, 0.6, 0.2) would be left uncalled. We restricted the comparison to overlapping SNPs between HRC, 1000G reference panels and whole-exome sequencing (WES) data for Chromosome 14 only, on SNPs with 0% missingness (plink –missing flag) in WES data. We also assessed variants’ agreement according to different MAF bins for “high-quality” (“Info” ≥ 0.8) SNPs. The output resulted in number of variant “mismatches,” i.e., the count of allele not matching between imputed and sequenced variants per individual. Work-flow for VCF-compare can be found in Supplementary Figure S1. To measure interrater reliability we computed Cohen’s kappa coefficient (McHugh, 2012) for both the reference panels against WES data. Kappa coefficient ≤ 0 indicates no agreement, 0.01–0.20 as none to slight, 0.21–0.40 as fair, 0.41–0.60 as moderate, 0.61–0.80 as substantial, and 0.81–1.00 as almost perfect agreement. Work-flow for Cohen’s kappa coefficient calculation can be found in Supplementary Figure S2.

To assess how ancestry affected imputation quality, we conducted a Poisson regression using R. We used percentage of global ancestry (European (CEU), Native (NAT) and African (YRI) as predictors, and total number of mismatches as the outcome; analyses were restricted to “high-quality” SNPs, only.

To evaluate imputation performance of a specific rare variant, we examined a founder mutation, p.Gly206Ala (G206A - rs63750082) in the PSEN1 gene (PSEN1-G206A) (Athan et al., 2001; Lee et al., 2015). The PSEN1-G206A mutation is a rare variant observed primarily in Puerto Ricans with familial early onset Alzheimer’s disease (EOAD), but it is rare in Puerto Ricans and other populations with late-onset Alzheimer’s disease (LOAD) (Arnold et al., 2013). The mutation was present in the 1000G phase 3 reference panel with an allele frequency of 0.001, but was absent in the HRC reference panel. To verify whether individuals who were found to carry the PSEN1-G206A mutation based on 1000G-imputation, they were genotyped using the KASP genotyping technology by LGC genomics³, which uses allele-specific PCR for SNP calling. Agreement between imputed and genotype data for the PSEN1-G206A mutation was then assessed. We also tested the effect on imputation quality based on different IMPUTE2-parameters settings, more specifically by modifying the chunk size (i.e., 1 MB vs. 5 MB).

To select the optimal tool for phasing, we compared SHAPEIT2 with Eagle2 using Chromosome 21 with 13,066 genotyped SNPs by performing subsequent imputation with IMPUTE2 on phased outputs, and using both reference panels. We found SHAPEIT2 better than Eagle2 when evaluated based on mean R² and “Info” metric using either the reference panels. For instance, using the 1000G, we observed higher mean R² for data phased with SHAPEIT2 as compared to Eagle2 (0.92 vs. 0.91; Wilcoxon p-value < 0.001). Similarly, when HRC panel was employed, mean R² of 0.89 was observed for SHAPEIT2 against 0.85 for Eagle2 (Wilcoxon Signed-Rank test p-value < 0.001).

SNP count comparison details can be found in Supplementary Tables S1, S2. Regardless of the reference panel employed, we observed higher percentage of “high-quality” rare and ultra-rare SNPs for SHAPEIT-IMPUTE2 than Eagle2-IMPUTE2. For instance, 1000G-imputation retrieved 51.02% of “high-quality” rare SNPs using SHAPEIT-IMPUTE2 vs. 48.38% with Eagle2-IMPUTE2. Detailed comparisons for different MAF bins and quality threshold can be found in Supplementary Section S2. Nevertheless, we found Eagle2 faster than SHAPEIT2 when computation times were compared; for instance, with HRC Eagle2 was ∼6 times faster than SHAPEIT2 (Supplementary Table S3). We therefore imputed the remaining chromosomes on phased output from SHAPEIT2. Comparison of phasing tools by assessing switch error rate was beyond the scope of this paper due to limited resources, for e.g., availability of phased reference panel for an admixed population.

We found that SHAPEIT-IMPUTE2 performed better than MaCH-Admix. For Chromosome 21, we imputed 1,104,648 and 646,594 SNPs for SHAPEIT-IMPUTE2 and MaCH-Admix, respectively, 549,091 SNPs were overlapping. For SHAPEIT-IMPUTE2 we observed 446,591 bi-allelic SNPs with “Info” ≥ 0.40, in contrast with 598,943 SNPs with Rsq ≥ 0.30 from MaCH-Admix (Supplementary Table S4). SNP counts for different MAF bins based on platform-specific quality index can be found in Supplementary Table S5. When the two outputs were compared in terms of r2hat, SHAPEIT-IMPUTE2 showed a higheraverage r2hat of 0.62 against 0.36 from MaCH-Admix (Wilcoxon Signed-Rank test p-value < 0.001). Also, MaCH-Admix was 109 times slower than IMPUTE2 (Supplementary Table S6), thus, comparison between different panels using MaCH-Admix were excluded due to limited resources. For the remaining of this manuscript, we focused on imputation employing SHAPEIT-IMPUTE2, only.

Using SHAPEIT-IMPUTE2, we imputed 81,240,392 and 38,532,090 SNPs across all autosomal chromosomes with 1000G and HRC reference panels, respectively (Table 2).

Overall, we observed slightly higher mean R² with 1000G than with HRC panel (0.94 vs. 0.92; Wilcoxon p-value < 0.001). Nevertheless, when the analyses were restricted to only “good-” and “high-quality” SNPs, HRC consistently performed better: 60.82% of HRC-imputed SNPs were “good-quality” and 48.87% were “high-quality” (Wilcoxon Signed-Rank test p-value < 0.001). On the contrary, 40.32% of 1000G imputed SNPs were “good-quality” and 30.11% were “high-quality.”

Further, we evaluated performance for uncommon, rare and ultra-rare SNPs. For “good-” and “high-quality” SNPs, HRC outperformed 1000G. For example, HRC panel produced 62.85% of “high-quality” rare SNPs, whereas 1000G had 53.83% (Table 3). When average imputation “Info” quality was evaluated, HRC-imputation again performed better than with 1000G (Wilcoxon p-value < 0.001) (Figure 1).

Next, we restricted our analyses to overlapping SNPs across the two reference panels only, based on their chromosome and position mapping, reference and non-reference alleles. For “good-”and “high-quality” SNPs, imputation in both panels performed similarly (Table 2). When restricted to uncommon, rare and ultra-rare SNPs, we observed higher percentage of “good-” and “high-quality” SNPs for HRC panel as compared to 1000G reference panel (Table 3). For example, 7.44% of HRC-imputed ultra-rare SNPs were “good-quality” vs. 4.95% with the 1000G. 1.69% of HRC-imputed ultra-rare SNPs were “high-quality” vs. 0.75% with the 1000G. Further, Wilcoxon test on “Info” value of “high-quality” ultra-rare SNPs (2,972) again showed better performances when HRC was employed vs. 1000G (P-value < 0.001). Complete list of counts and percentages across reference panels, MAF bins and quality score can be found in Table 3.

SNP rs63750082 is absent from HRC panel therefore no imputation was achieved. Using 1000G reference panel, 12 individuals were imputed as G206A carriers. SNP rs63750082 was imputed with an IMPUTE2 “Info” score of 0.48 using 1MB as chromosomal region parameter. When we increased the chunk size to 5MB, IMPUTE-Info score drastically improved to 0.94 (Figure 2). Those patients labeled as mutation-carriers according to imputation were then genotyped: all 12 were confirmed to be G206A carriers, therefore achieving a perfect imputation prediction (100% agreement) for that specific SNP.

Out of the 1,000 individuals included in our study, 262 had whole exome sequencing (WES) data available (Raghavan et al., 2018). We had 14,157 overlapping SNPs in WES, HRC and 1000G reference panels with 0% missingness in WES data on Chromosome 14; SNPs imputed with each reference panel were compared against WES data separately. When concordance was evaluated, HRC panel performed slightly poorer, despite showing higher number of “high-quality” variants as compared to 1000G (Table 4). Using 1000G, we observed 3,542 rare and 35 ultra-rare “high-quality” SNPs; across 262 samples, we counted 1,245 {[(1,245/(3,542 × 262)] × 100 = 0.13%} and 10 (0.10%) mismatches for rare and ultra-rare, respectively. Using HRC, we retrieved 3,759 rare and 93 ultra-rare “high-quality” variants; we observed 2,439 (0.24%) and 32 (0.13%) mismatches for rare and ultra-rare variants, respectively. Details about pipeline can be found in Supplementary Section S3.

Next, we computed Cohen’s kappa coefficient (K) for 14,157 imputed SNPs common in WES and the two reference panels. For both HRC and 1000G-imputation, we observed Kappa (K) of ∼0.99 for both rare and ultra-rare “high-quality” variants (Table 4). Details about pipeline can be found in Supplementary Section S4.

We evaluated the effect of individual ancestral component separately on SNP mismatches for Chromosome 14 on 262 individuals. For both reference panels we found that higher African ancestry (YRI) was associated with higher number of mismatches (Supplementary Table S7). For instance, with 1000G reference panel, for rare variants (“Info” ≥ 0.80), we observed an estimate of 1.46 (P-value < 0.001) for YRI component (indicating that for each unit increase in YRI ancestry, it results in 1.46 additional mismatches). Details on confidence intervals and robust standard errors can be found in Supplementary Table S7 and Supplementary Section S5). We did not observe significant effect of ancestry on “high-quality” ultra-rare variants in both panels.