AUTHOR=Bassit Amina , Hahn Florian , Rezgui Zohra , Shahreza Hatef Otroshi , Veldhuis Raymond , Peter Andreas 

TITLE=Template recovery attack on encrypted face recognition systems with unprotected decision using synthetic faces

JOURNAL=Frontiers in Imaging

VOLUME=Volume 4 - 2025

YEAR=2025

URL=https://www.frontiersin.org/journals/imaging/articles/10.3389/fimag.2025.1476377

DOI=10.3389/fimag.2025.1476377

ISSN=2813-3315

ABSTRACT=IntroductionHomomorphic encryption (HE) enables privacy-preserving face recognition by allowing encrypted facial embeddings to be compared without decryption. While efficient, these systems often reveal comparison scores in plaintext, introducing a security risk. Revealing these scores can potentially allow adversaries to reconstruct sensitive facial embeddings and infer demographic attributes, thus compromising user privacy.MethodsThis work proposes a training-less face template recovery attack leveraging the Lagrange multiplier optimization method. The attack requires only a small set of randomly generated synthetic facial images and their associated comparison scores with a target template. The method assumes attackers use spoofed synthetic faces and lack direct access to the face recognition system, aligning with real-world threat models.ResultsExperimental evaluation demonstrates the feasibility and effectiveness of the proposed attack. It shows that between 50 and 192 comparison scores and synthetic images are sufficient to recover the target face template with 100% success under strict system thresholds. The recovered templates closely resemble the original and retain identifiable soft biometric traits.DiscussionThe findings reveal a critical vulnerability in face recognition systems employing inner product similarity measures under homomorphic encryption. Even without system access or training data, attackers can exploit leaked comparison scores to compromise facial privacy. The study underscores the need to reassess how score leakage is handled in encrypted recognition systems and explore stronger protection mechanisms against template reconstruction.