Theoretical frameworks for TDI typically differentiate between cognitive (attribute-based), affective (feeling-based), and conative (behavioral intention) components (Baloglu and McCleary, 1999). The proliferation of digital photography and social media has positioned user-generated photographs and reviews as pivotal in image formation, both for individual tourists and destination management organizations (Marine-Roig and Clave, 2015; Stepchenkova and Zhan, 2013). Computational studies have utilized visual analysis to classify the esthetics of tourism photographs (Zhang et al., 2022), deduce perceived destination attributes from Flickr images (Yang et al., 2014), and analyze textual sentiment in review corpora (Bi et al., 2020). These studies illustrate that perception is quantifiable; yet, they infrequently link perception to subsequent recommendation tasks.

The domain of vision–language representation learning has progressed rapidly. CLIP (Radford et al., 2021) demonstrated that contrastive pre-training on image–text pairs at a web scale produces representations that are transferable to a variety of downstream tasks. BLIP-2 (Li et al., 2023) introduced a lightweight querying transformer to connect frozen image encoders with large language models. DINOv2 (Oquab et al., 2024) revealed that self-supervised pre-training with curated data generates universal visual features that outperform supervised alternatives across numerous tasks. SigLIP (Zhai et al., 2023) enhanced contrastive learning with sigmoid losses (Hassan et al., 2024). These advancements provide a robust foundation for encoding tourism visual content; however, their direct application to TDI necessitates domain-specific adaptation concerning perception attributes absent in generic pre-training distributions.

Point-of-interest (POI) recommendation has been extensively examined through collaborative filtering (Rendle et al., 2009), visually-aware ranking (He and McAuley, 2016), and graph neural network methodologies (He et al., 2020; Wang et al., 2019). Context-aware and spatiotemporal extensions integrate time, mobility, and sequential behavior (Liu et al., 2013; Lian et al., 2020). Tourism-specific research has analyzed hybrid content-collaborative methods (Quercia et al., 2014; Lim et al., 2018) and addressed cold-start challenges stemming from sparse coverage of less-visited attractions (Zhang and Chen, 2020; Cui et al., 2020). Explainability in recommender systems has emerged as a distinct research trajectory (Zhang and Chen, 2020; Yaqoob et al., 2023), although the evaluation of faithfulness assessing whether the cited evidence genuinely influences recommendations remains insufficiently explored.

Concerns regarding reproducibility have led to calls for standardized, openly licensed benchmark datasets within the recommender systems community (Zangerle and Bauer, 2022; Beel et al., 2016). Tourism datasets, in particular, are often derived from proprietary platform APIs or scraped under ambiguous terms, thereby limiting auditability and replication (Zhang et al., 2023). Our research directly addresses this issue by constructing a benchmark exclusively from public sources with documented licensing, preprocessing, and version control.

All datasets utilized in this study must adhere to four criteria: (1) the dataset must be publicly accessible without the need for proprietary access agreements; (2) the license or terms of service must allow for academic research use; (3) the specific extraction version and access date must be documented; and (4) the dataset must not contain any records that necessitate consent revocation under applicable privacy regulations. Data scraped from commercial tourism platforms (e.g., TripAdvisor and Booking.com) and Kaggle mirrors of uncertain provenance are explicitly excluded.

Table 1 provides a summary of the public sources utilized. Collectively, these sources offer the visual, textual, geospatial, interaction, and entity-linking modalities required for the six tasks formalized in Section 4.

The point-of-interest (POI) entity table is developed by incorporating the Overture Maps Places release (version 2024-04-16.0, extracted 2024-05-01) and standardizing records with Wikidata entity identifiers where applicable. Duplicate entries across sources are resolved through coordinate-radius matching with a 50-m threshold, followed by string similarity filtering using the Jaro–Winkler method with a threshold of 0.92 (yielding a de-duplication yield of 8.3% of raw records across all cities). Wikivoyage and Wikipedia guide texts are linked to POI entities via Wikidata QIDs, offering multilingual textual descriptions. Yelp business records are aligned with entity table entries through coordinate matching and fuzzy category alignment. SNAP check-in records are spatially joined to the entity table within a 100-m radius to derive implicit user–POI interaction matrices. Geographic context features attached to each entity include: (i) decimal-degree WGS84 coordinates; (ii) a city-level region identifier; (iii) a three-level Overture Maps category hierarchy (e.g., eat_and_drink > restaurant > italian_restaurant); (iv) a Wikivoyage region tag used as a soft spatial prior during graph construction.

Images associated with each POI entity are sourced from GLDv2 landmark identifiers, YFCC100M geotag-radius lookups, and Wikimedia Commons category pages. All images undergo a three-stage filtering process: (1) license verification against an approved list of Creative Commons and equivalent terms; (2) face detection and blurring using a publicly available OpenCV DNN detector to mitigate inadvertent personal data exposure; and (3) near-duplicate removal via perceptual hashing with a pHash Hamming distance of eight or less. Text associated with each entity combines Wikivoyage sentences, Wikipedia lead paragraphs, and Yelp review aggregates (tip summaries, not raw reviews, to limit re-identification risk). The dataset construction pipeline, including all filter thresholds and random seeds, is released with the article.

Three complementary split strategies are constructed. In-city split: a random 70/10/20 train/validation/test division within each city, preserving user–POI interaction distribution. Cold-start split: POIs with fewer than five training interactions are withheld from training and evaluated separately, stratified by the interaction count ( { 0,1 - 5,6 - 20,21 - 50 , > 50 } ) . Cross-city split: models are trained on all POIs and user interactions from Phoenix and Pittsburgh (training cities) and evaluated zero-shot on six unseen cities: Las Vegas, Chicago, Seattle, Boston, Austin, and Denver (test cities).

Table 2 reports the exact record counts per city and split after all filtering. Across all eight cities, the benchmark contains 40,542 unique POI entities, 283,946 images (after license and privacy filtering), 80,306 unique users, and 1,418,622 check-in interactions. Image source distribution across the benchmark is reported in Figure 1.

A comprehensive POI entity table is constructed utilizing data from Overture Maps Places and Wikidata (refer to Section 3). Each entity record encompasses coordinates, a normalized name, a Wikidata QID (when available), a tourism category derived from Overture and Wikivoyage hierarchies, and a media license manifest. This manifest is instrumental in guiding the downstream image filter to ensure that no unlicensed images are incorporated into the training or evaluation processes.

Images associated with each POI are encoded using two complementary backbones: a CLIP ViT-B/32 encoder (Radford et al., 2021) for semantic image–language grounding and a DINOv2 ViT-S/14 encoder (Oquab et al., 2024) for capturing robust low-level and mid-level visual features. A Places365-style scene head (Zhou et al., 2018) appends scene-category logits, while a GLDv2-trained landmark head appends landmark probability scores. The final image embedding for POI i is achieved through the L2-normalized concatenation of these four representations, followed by a learned projection to a dimension of d = 512 .

The text associated with each POI, including Wikivoyage sentences, Wikipedia lead paragraphs, Yelp tip aggregates, and tag strings from YFCC100M, is encoded using a multilingual pre-trained transformer [mDeBERTa-v3-base (He et al., 2023)]. Sentence-level embeddings are mean-pooled within each text type, and type-specific embeddings are integrated via a learned gating network into a unified d -dimensional text embedding.

Image and text embeddings are aligned with a contrastive InfoNCE loss (van den Oord et al., 2018): L align = − 1 N ∑ i = 1 N log exp v i ⊤ t i / τ ∑ j = 1 N exp v i ⊤ t j / τ , (2) where τ is a learned temperature. Hard negatives are sampled from geographically nearby or visually similar POIs to discourage trivial separation and encourage within-domain discrimination (Robinson et al., 2021).

The perception bottleneck is designed to learn a set of K = 7 interpretable perception attributes, corresponding to theoretically grounded TDI dimensions (Baloglu and McCleary, 1999): (1) landmarkness, (2) naturalness, (3) cultural/historical salience, (4) esthetic appeal, (5) crowding proxy, (6) sentiment affect, and (7) activity affordance. Weak supervision is obtained from several sources: (i) Places365 scene logits for naturalness and activity affordance, (ii) GLDv2 landmark scores for landmarkness, (iii) Wikivoyage category tags for cultural/historical salience, and (iv) Yelp sentiment aggregates for affect. The bottleneck head comprises a two-layer MLP that generates K sigmoid-activated scores from the fused multimodal embedding. These scores serve as both interpretable outputs and regularizers, ensuring that the latent POI embedding retains perception-relevant structure.

Relationship to POI evaluation categories. The seven perception attributes are distinct from the six POI evaluation categories (landmark, natural scenery, cultural/historical, beach/resort, museum/gallery, and urban attractions) used in the classification experiment. Perception attributes are continuous scores that any POI can exhibit to varying degrees, whereas POI categories are discrete labels used to organize the evaluation benchmark. Table 3 maps each attribute to its primary POI category association and weak supervision signal.

A heterogeneous graph is constructed with users and POIs as nodes, incorporating four types of edges: user–POI interactions, POI–POI category similarity, image–POI association, and geospatial proximity (Euclidean distance ≤ 1 km, weighted by inverse distance). A LightGCN-style (He et al., 2020) propagation scheme is utilized to aggregate neighborhood information over two layers (justified by the sensitivity analysis in Figure 2) without non-linear activation, leading to propagated user and item embeddings. The ranking score is computed as in Equation 3: s u , i = e u * + α   e u 0 ⊤ e i * + β   p i , (3) where e * denotes the aggregated embedding after graph propagation, e ( 0 ) is the initial user embedding (retained for training stability), p i ∈ R d is the perception embedding from Module M5 projected to dimension d , and α , β ∈ R are jointly learned scalars. The additive injection of β p i into the item embedding is deliberate: unlike concatenation, which would opaquely entangle perception with graph-propagated features, additive injection preserves the interpretability of the individual perception attribute scores while allowing the ranker to modulate their influence via β . The ablation study (Section 7; Figure 5) shows that setting β = 0 (w/o Perc) causes a 13.6% decrease in point in absolute Recall@10.

In accordance with the maximal marginal relevance (MMR) framework (Carbonell and Goldstein, 1998), a re-ranking procedure is utilized to select items from the top-50 candidates, aiming to optimize a convex combination of relevance and intra-list diversity, as regulated by λ 4 . Diversity is quantified as the mean pairwise cosine distance within the perception embedding space, thereby encouraging the re-ranker to favor destinations with complementary perceptual profiles.

For each recommended POI, the explanation module provides: (i) the top-three perception attributes ranked by magnitude, (ii) the public image from the POI’s image set that exhibits the highest alignment similarity (subject to licensing constraints), and (iii) the review sentence from the Yelp tip corpus that is most semantically similar. The faithfulness of the explanation is assessed through a deletion protocol: the top-cited image is removed from the visual input, and the resultant decrease in score is measured and compared to the mean decrease in score from 100 random-image deletions. An explanation is deemed faithful if the deletion of top-cited evidence leads to a significantly larger decrease in score than random deletions (paired t -test; p < 0.05 ).

This section presents the algorithmic summary of the MM-DIPR training procedure in Algorithm 1.

Algorithm 1

MM-DIPR training procedure.

Require: Public POI table, public images, public texts, public check-in data, license manifest

To validate the perception bottleneck’s seven attributes beyond weak supervision, we conducted a controlled annotation study on a stratified random sample of 2,000 images (approximately 333 images per POI category). Three trained annotators independently scored each image on all seven attributes using a five-point Likert scale (1 = not at all; 5 = strongly). Scores were binarized at the per-attribute median score (computed on the pilot set of 100 images) to yield binary present/absent labels before computing agreement statistics.

Annotation guidelines were developed iteratively through pilot rounds on 100 images that were not included in the final sample, achieving consensus on operationalized definitions for ambiguous attributes (e.g., crowding proxy operationalized as the presence of ≥ 5 identifiable people or queue structures visible in the image; activity affordance operationalized as the range of observable physical activities that the scene supports). Disagreements were adjudicated by majority vote, and the majority-vote label was used as the consensus (gold) label for all downstream comparisons.

Inter-annotator agreement calculation: With three annotators, Cohen’s κ was computed as the mean of the three pairwise κ values (annotators A vs. B, A vs. C, and B vs. C) for each attribute, following the mean-pairwise approach of Artstein and Poesio (2008). We report the mean pairwise κ rather than Fleiss’s κ to maintain comparability with the two-annotator literature; reported values are averages over the three pairs.

Results (Figure 6) show that all seven attributes achieve substantial or near-perfect inter-annotator agreement ( κ ∈ [ 0.68 , 0.83 ] ) , exceeding the widely adopted threshold of 0.61 for substantial agreement (Landis and Koch, 1977). Crowding proxy ( κ = 0.68 ) and sentiment affect ( κ = 0.71 ) show the lowest agreement, consistent with their reliance on subjective interpretation; these are flagged as lower-confidence attributes.

F1 measures were defined. Two F1 scores are compared. Weak-supervision F1 is the macro-averaged F1 of the weak-label predictions (derived from Places365, GLDv2, Wikivoyage, and Yelp proxies, as described in Section 5) against the consensus annotation labels. Human-annotation F1 is the macro-averaged F1 of each individual annotator’s labels against the majority-vote consensus, averaged across the three annotators and then across attributes. Comparison of the weak-supervision F1 to human-annotation F1 reveals a maximum gap of 4.1% points (crowding proxy: 0.812 vs. 0.853), confirming that the public-data proxies provide reliable operational approximations of the intended constructs.

The experiments utilize the Yelp Open Dataset (Yelp Inc., 2023) for personalized POI recommendation (Tasks 3–5) and the GLDv2+Wikimedia split for image–text retrieval (Task 2). Perception classification (Task 1) utilizes Places365 scene labels and a manually annotated subset of 2,000 images, incorporating the seven perception attributes detailed in Section 5. The TREC Contextual Suggestion 2016 (Dean-Hall et al., 2016) offers additional evaluation for recommendations through graded relevance judgments. The SNAP Gowalla dataset (Cho et al., 2011) provides supplementary check-in baselines.

Split sizes: For the in-city split, the two training cities (Phoenix and Pittsburgh) together yield 8,079 POIs, 317,711 check-in interactions, 19,060 unique users, and 64,628 image–text pairs, with a 70/10/20 train/validation/test partition (train: 5,655 POIs and 222,398 interactions; validation: 808 POIs and 31,771 interactions; test: 1,616 POIs, 63,542 interactions). For the cold-start split, 1,613 POIs (20% of training-city POIs, those with ≤ 5 interactions) are withheld from training and used exclusively for cold-start evaluation. For the cross-city split, all six test cities contribute 28,663 POIs, 1,100,911 interactions, and 61,246 users; no interactions or user histories from these cities appear during training.

User deduplication: Users are identified by their hashed Yelp pseudonym. A user whose hashed identifier appears in more than one city is counted only once in the global user pool (80,306 total unique users), but their city-specific interaction records are retained separately. No user from a test city is present in the training split.

The following seven baselines are used for the core comparison (Table 4):

Popularity/category heuristics: The rank by global check-in count and tourism-category prior, a necessary tourism-specific baseline.

BPR (Rendle et al., 2009): Matrix factorization with Bayesian personalized ranking loss; interaction signals only.

VBPR (He and McAuley, 2016): Extends BPR with visual item factors from a pre-trained CNN; strong visual recommendation baseline.

Text-only: BPR-style ranker using mDeBERTa text embeddings as static item factors; no image or graph component.

Image-only. BPR-style ranker with CLIP ViT-B/32 image embeddings only.

LightGCN (He et al., 2020): Graph-based collaborative filtering; no multimodal content.

Late fusion: Combines text-only and image-only ranking scores via learned linear interpolation after independent training.

MM-DIPR (ours): The full proposed framework.

Section 7.6 and Figure 7 additionally compare MM-DIPR against four stronger baselines: GeoMF (Lian et al., 2020) (geography-aware matrix factorization), STGCN-POI (Han et al., 2020) (spatiotemporal graph convolution network for POI recommendation), SASRec (Kang and McAuley, 2018) (self-attentive sequential recommender), and KGAT (Wang et al., 2019) (knowledge-graph attention network), yielding 11 total baselines (seven core + four extended). BERT4Rec (Sun et al., 2019) appears in Appendix A (Table A1) for completeness (it uses the same sequential self-attention paradigm as SASRec, but with a masked-language-model training objective), but it is not reported as a separate baseline in the main results to avoid redundancy with SASRec; its performance is 0.168 ± 0.003 Recall@10, between SASRec and KGAT.

Eight ablated variants isolate the contribution of each module: (i) without the perception bottleneck (w/o Perc), (ii) without the graph ranker (w/o Graph), (iii) without the geo/context branch (w/o Geo), (iv) without the text encoder (w/o Text), (v) without the visual encoder (w/o Visual), (vi) with shuffled image–text pairs as a sanity check (Shuffled), (vii) without diversity re-ranking (w/o Div), and (viii) without geographic edges in the heterogeneous graph (w/o GeoEdges), which removes geospatial proximity edges while retaining all other edge types (user–POI interactions, POI–POI category similarity, and image–POI association), directly testing the model-level contribution of explicit spatial relations in the graph structure (results are reported in Section 7.7).

Perception metrics include macro-F1, AUROC, and calibration error (ECE). For retrieval, the metrics are Recall@5, MAP@10, and MRR. Recommendation metrics encompass Recall@ K , Precision@ K , NDCG@ K , and MAP@ K for K ∈ { 5,10,20 } . Diversity is assessed through intra-list diversity (ILD: the mean pairwise cosine distance among top- K recommendation embeddings), catalog coverage: the fraction of all POIs appearing in at least one recommendation list, and long-tail exposure: the fraction of recommendations drawn from POIs in the bottom 20% by interaction count. Explanation is evaluated using the faithfulness deletion score.

Statistical significance is determined using paired bootstrap tests with 10,000 resamples and a significance level of α = 0.01 over users/queries. The paired bootstrap resamples users (with replacement), recomputes the per-method metric on each resample, and counts the proportion of resamples in which MM-DIPR does not exceed the baseline; this proportion is the one-sided p -value. All tests are conducted at the user level, not the interaction level. All reported confidence intervals are 95% bootstrap confidence intervals computed over 10,000 resamples. The effect sizes are reported as Cohen’s d for continuous comparisons and as percentage change relative to the best baseline for recommendation metrics.

Each model is run with five independent random seeds (seed ∈ { 0,1,2,3,4 } ). The results in Table 4 report the mean; standard deviations across runs are reported in Appendix A (Table A1). Standard deviations for MM-DIPR are ≤ 0.003 across all metrics, confirming stability.

The MM-DIPR model is implemented using PyTorch 2.2. Visual encoders are initialized from public HuggingFace checkpoints and fine-tuned with a learning rate that is 10 × lower than that of the projection layers. The graph ranker is trained using the AdamW optimizer with a learning rate of 5 × 1 0 − 4 , a weight decay of 1 0 − 4 , and a batch size of 2,048. The temperature parameter τ is initialized to 0.07 and is learned jointly. All experiments are conducted on a single NVIDIA A100 40 GB GPU. Hyper-parameters λ 1 - 4 are selected through grid search based on the validation NDCG@10; the full search space is reported in Appendix A (Table A2), and the joint sensitivity of λ 1 and λ 2 is visualized in Figure 2. All baselines are tuned under the same validation protocol (grid search on validation NDCG@10; no test-set access during hyperparameter selection).

Figure 2 reports a joint sensitivity grid over λ 1 ∈ { 0.01 , 0.05 , 0.10 , 0.20 , 0.50 } and λ 2 ∈ { 0.01 , 0.05 , 0.10 , 0.20 , 0.50 } (validation NDCG@10) and individual sensitivity curves for graph propagation depth ( L ∈ { 1,2,3,4 } ) and projection dimension ( d ∈ { 128,256,512,1024 } ) . The selected configuration ( λ 1 = 0.10 , λ 2 = 0.10 , L = 2 , and d = 512 ) is marked by a star. Performance is robust within a broad region around the optimum, supporting the generalizability of the results.

Figure 8 presents the Recall@10 and NDCG@10 metrics for all baseline models and MM-DIPR on the Yelp benchmark. MM-DIPR achieves a Recall@10 of 0.213 and an NDCG@10 of 0.152, indicating statistically significant improvements over the strongest baseline (late fusion, 0.174/0.117) by 22.4% and 29.9%, respectively ( p < 0.01 , Cohen’s d = 1.42 , and d = 1.67 , respectively). These enhancements are consistent across K ∈ { 5,20 } and on the TREC contextual suggestion split, demonstrating that the improvements are not artifacts of a specific evaluation depth.

Table 4 provides the full numerical results at K ∈ { 5,10,20 } .

Figure 5 illustrates the results of the ablation study. The removal of the perception bottleneck leads to the most significant decline for a single module (Recall@10: 0.213 → 0.184 , − 13.6 % ), indicating that the perception layer plays a crucial role beyond the mere fusion of raw multimodal embeddings. The elimination of the graph ranker leads to a similarly substantial decrease ( − 16.4 % ) , whereas the removal of the geo/context branch, text encoder, or visual encoder leads to moderate yet notable degradations. The shuffled image–text variant, serving as a sanity check, deteriorates to a performance level close to the popularity baseline, thereby affirming the complexity of the alignment objective.

Figure 9 presents the Recall@10 metric, categorized by the number of interactions per POI. For POIs with no prior training interactions, known as strict cold-start scenarios, interaction-only methods such as BPR and LightGCN inherently lead to zero recall. In contrast, content-based methods maintain some level of signal. The MM-DIPR method achieves a Recall@10 of 0.061 for cold-start POIs, establishing a meaningful baseline facilitated by visual–perceptual representations. Furthermore, MM-DIPR surpasses late fusion by 9.7% points within the 1–5 interaction category. The relative benefit of MM-DIPR decreases as the number of interactions increases, indicating that the perception layer is particularly beneficial when collaborative signals are limited.

The term cross-city transfer in this article refers specifically to POI-level zero-shot transfer: the model is trained on all POIs and user interactions from Phoenix and Pittsburgh and is then evaluated on POIs from six unseen cities without any fine-tuning on those cities’ data. We clarify the following aspects explicitly in response to the reviewer’s concerns.

What is zero-shot? The zero-shot condition applies to cities and their POIs: no POI entity, image, text description, or graph structure from the six test cities (Las Vegas, Chicago, Seattle, Boston, Austin, and Denver) is used during training or model selection. The model relies entirely on transferable multimodal and spatial representations learned from the two training cities.

Test-user histories and user overlap: All test-city users are unseen during training (confirmed by global user deduplication, Section 6). Their interaction histories within test cities are available at inference time for ranking: the evaluation follows a leave-last-out protocol within each test user’s history (final interaction = positive test item; prior interactions = observable history). Users with fewer than three interactions in a test city are excluded from that city’s evaluation set. Users do not overlap across training and test cities.

What is not zero-shot? Test-city images and textual descriptions are encoded by the trained model at inference time (modality encoders generalize across cities by design). No graph propagation involves test-city nodes during training; the item embedding ( e * i + β p i ) is computed from content features only, which explains why content-rich cities (Las Vegas and Boston) benefit most from the transfer.

Labeling: This evaluation is accurately described as cross-city transfer (trained on city A + B and evaluated on cities C–H) rather than user cold-start or full zero-shot learning. The six-city results should be treated as an exploratory analysis of geographic generalization; no causal claims about spatial factors are made.

For the experimental protocol of this evaluation, see Section 7.4 above. Figure 10 presents the NDCG@10 results for six previously unexamined test cities. The MM-DIPR model consistently surpasses all baseline models across these cities, with the most significant improvements observed in Las Vegas and Boston, where it achieves increases of 63% and 61% over BPR, respectively. The comparatively modest enhancements in Austin and Denver align with the lower density of POIs and the availability of images in these locations, indicating that perception-aware representations are most advantageous in image-rich destination contexts; a quantified analysis of the POI-density–performance relationship is provided in the spatial analysis (Section 7.7).

Figure 7 reports Recall@10 and NDCG@10 for all 11 baselines (seven core plus four extended). MM-DIPR (Recall@10 = 0.213) outperforms the best spatiotemporal model GeoMF (0.158) by 34.8%, the best sequential model SASRec (0.171) by 24.6%, and the knowledge-graph model KGAT (0.163) by 30.7% ( p < 0.01 in all cases). The consistent advantage over geographically-aware (GeoMF) and graph-structured (KGAT and LightGCN) baselines confirms that the gains are attributable to the perception bottleneck and cross-modal alignment, not to any single architectural component.

To establish the geoinformatics foundation of the framework, we compute Moran’s I (Moran, 1950) for each of the seven perception attributes across all 40,542 benchmark POIs. The spatial weights matrix W is row-standardized and uses an inverse-distance kernel: w i j = d i j − 1 if d i j ≤ 1  km , else 0, where d i j is the Euclidean distance between the POIs i and j in decimal degrees projected to meters. Significance is assessed by a permutation test with 999 random permutations of the attribute scores; the reported p -values are the proportion of permuted Moran’s I values exceeding the observed statistic (one-sided and upper tail).

Figure 11 presents the Moran’s I scatter plot for landmarkness (panel a), per-attribute Moran’s I values (panel b), a city × model NDCG@10 heatmap (panel c), and a POI-density vs. NDCG@10 scatter (panel d). All seven attributes exhibit statistically significant positive autocorrelation ( I ∈ [ 0.289 , 0.421 ] , all p < 0.05 ), confirming that perception is not spatially random. Landmarkness shows the strongest autocorrelation ( I = 0.421 ) , consistent with the geographic clustering of tourist attractions; crowding proxy shows the weakest autocorrelation ( I = 0.289 ) , reflecting diffuse distribution of high-foot-traffic environments.

The POI-density–performance scatter (panel d) reveals a positive correlation ( r = 0.78 ) between city-level POI count and MM-DIPR’s NDCG@10, confirming that geospatial data quality is a significant performance determinant. This correlation is based on six data points and should be interpreted as exploratory; it indicates a plausible relationship between data coverage and model performance but does not establish a causal mechanism. This finding has direct implications for usage in data-sparse regions.

Geographic-edges ablation: To directly assess the model contribution of the geospatial proximity edges in the heterogeneous graph (rather than merely post hoc spatial analysis), we compare MM-DIPR against the w/o GeoEdges ablation variant introduced in Section 6. Removing the geographic edges from the graph reduces Recall@10 from 0.213 to 0.198 ( − 7.0 % , p < 0.01 ) and NDCG@10 from 0.152 to 0.141 ( − 7.2 % , p < 0.01 ). This confirms that the spatial proximity structure encoded in the graph provides meaningful complementary signal beyond POI-category and image-based edges, grounding the Moran’s I analysis in a model-level causal test.

The perception bottleneck achieves a macro-F1 score of 0.894 on the six-category perception classification task (Figure 12). The highest per-category F1 score is observed for natural scenery (0.934), while the lowest is for historic sites (0.868), where visual ambiguity with Urban Landmarks constitutes the primary source of confusion. In the context of image–text retrieval (Figure 13), the MM-DIPR encoder surpasses CLIP ViT-B/32 by 9.3% points on Recall@5 and the CLIP + DINOv2 ensemble by 3.7% points on MAP@10. This demonstrates that domain-adapted contrastive training with perception-aware hard negatives enhances retrieval performance beyond that of off-the-shelf foundation models.

In a comparative analysis of image–text retrieval across various encoder variants, the MM-DIPR encoder demonstrates superior performance, as evidenced by its highest scores in Recall@5, MAP@10, and MRR. The error bars indicate 95% bootstrap confidence intervals (GLDv2 + Wikimedia test split). Figure 14 presents radar plots of the mean perception attribute scores categorized by POIs. The profiles exhibit theoretically consistent patterns: nature and parks POIs achieve the highest scores in naturalness and esthetic appeal, historic sites excel in cultural/historical salience and landmarkness, and beaches/resorts attain the highest scores in sentiment affect. These cluster-level patterns substantiate the perception bottleneck’s capacity to discern TDI-relevant structures through weak supervision, even in the absence of explicit perception-label training data.

Figure 15 demonstrates the impact of varying the MMR re-ranking weight λ 4 on NDCG@10, intra-list diversity (ILD), catalog coverage, and long-tail exposure. At λ 4 = 0.3 , which was determined through validation, MM-DIPR achieves an ILD of 0.520 and a catalog coverage of 0.392, with only a 4.6% decrease in NDCG@10 compared to the configuration without re-ranking. Long-tail exposure increases monotonically with λ 4 , aligning with the re-ranker’s propensity to promote less-frequented destinations with complementary perceptual profiles, an outcome directly pertinent to sustainable tourism policy.

The evaluation of deletion faithfulness reveals that the removal of the top-cited image leads to an average score reduction of 0.048 (SD = 0.019), in contrast to 0.011 (SD = 0.014) for random-image deletion ( t ( 999 ) = 28.4 , p < 0.001 , Cohen’s d = 2.34 ). Regarding textual evidence, the removal of the top-cited review sentence leads to a score reduction of 0.031 compared to 0.009 for random deletion ( p < 0.001 ) . These findings confirm that the evidence cited by Module M8 is causally relevant to the recommendation score, offering–stronger faithfulness assurances than attention–weight visualizations.

Four systematic failure patterns have been identified. Over-touristed landmark bias: High-popularity landmarks (e.g., Eiffel Tower analogs in the data) are over-represented in Wikimedia images, inflating their landmarkness scores and biasing recommendations towards them. Although partially mitigated by the MMR re-ranker, this remains a residual concern. Nighttime and seasonal variability: The perception bottleneck’s scene head, pre-trained on daytime scenes, assigns inconsistent naturalness and esthetic appeal scores to night or winter images of the same POI. Low-resource cities: Cross-city transfer performance degrades more significantly for cities with sparse overture POI coverage, indicating that entity quality is a critical determinant of performance. Multilingual review noise: Non-English Yelp reviews outside the top languages are handled noisily by the mDeBERTa tokenizer, potentially diluting textual representations for non-Anglophone destinations.

Geographic bias analysis: The underperformance in Austin (NDCG@10 = 0.121) and Denver (NDCG@10 = 0.132) relative to the benchmark average (0.133) is quantifiably linked to lower POI entity density (3,789 and 3,102 POIs, respectively, vs. the benchmark average of 5,072 POIs for test cities). The POI-density–NDCG@10 correlation ( r = 0.78 , Figure 11, panel d) confirms this relationship. We propose that future geoinformatics-oriented extensions of this framework could apply spatial Kriging or Gaussian process regression to impute perception scores in data-sparse regions, leveraging the demonstrated spatial autocorrelation structure (Moran’s I > 0 for all attributes) to propagate perception estimates from neighboring data-rich POIs.

The MM-DIPR framework operationalizes the cognitive–affective TDI model (Baloglu and McCleary, 1999) as a computable bottleneck that links perception theory to ranking models. The perception attribute profiles (Figure 14) provide empirical validation for the multidimensionality postulate, demonstrating that different destination categories exhibit distinct perceptual signatures consistent with theoretical expectations, which can be extracted from public UGC without the need for survey data. This serves as a bridge between the qualitative tradition of TDI research and the quantitative tradition of recommender systems.

For destination management organizations, the perception scores and explanation panels serve as a monitoring tool: variations in aggregate scores of landmarkness, naturalness, or esthetic appeal over time may indicate shifts in UGC-mediated destination image, facilitating proactive marketing responses. The diversity re-ranker’s capability to highlight long-tail destinations with high perceptual complementarity provides a technically sound mechanism for promoting lesser-known attractions within sustainable tourism strategies (Butler, 1980).

Three limitations require explicit acknowledgment. First, the benchmark is primarily derived from North American Yelp sub-graphs, which limits its generalizability to destinations in the Global South, where Overture and Wikidata coverage is less comprehensive. Second, while the controlled annotation study (Section 5.10) validates the perception attributes on a 2,000-image subset and achieves substantial inter-annotator agreement, the remaining 98% of benchmark images rely on weak supervision proxies; future work should extend human annotation to a larger, more geographically diverse image pool. Third, recommendation evaluation relies on implicit feedback (check-ins), which conflates visitation with satisfaction and may favor popular but average destinations over niche but highly satisfying ones.

Four directions are prioritized. Controlled user studies with explicit preference elicitation would complement the implicit evaluation. Multilingual perception modeling using cross-lingual text encoders could enhance coverage for non-Anglophone destinations. Real-time integration of trip constraints (transport, opening hours, and seasonal closures) would increase practical utility. Finally, causal analysis of the relationship between algorithmically promoted perception attributes and actual visitor flows using natural experiments around platform algorithm changes would advance the tourism policy implications.