Research on the robustness of the open-world test-time training model

Shu Pi^*Jiatian Pi

• Chongqing Normal University, Chongqing, China

Generalizing deep learning models to low latency unknown target domain distribution has motivated research into test-time training/adaptation (TTT/TTA). However, using TTT/TTA in an open-world environment is challenging mainly because it's difficult to distinguish between strong out-of-distribution(OOD) samples and regular weak OOD samples. In response to this challenge, several approaches to OWTTT have emerged. Despite its powerful functionality, it has a problem, that's the test-time poisoning attacks, which are substantially different from previous poisoning attacks that occur during the training time of ML models (i.e. adversaries cannot intervene in the training process). In this paper, we design a test-time poisoning method and tested on the open-world test-time training(OWTTT) model. Specifically, considering that the model gradients will change during testing, we design a single-step query attack data poisoning method to dynamically update perturbations and input them into the open-world test-time training model. The experimental results show that the OWTTT method is vulnerable to attacks. Our results demonstrate that OWTTT algorithms lacking a rigorous security assessment are unsuitable for deployment in real-life scenarios. As such, we advocate for the integration of defenses against test-time poisoning attacks into the design of open-world test-time training methods.