selected publications [full list]

1. ECCV Flatness-aware Sequential Learning Generates Resilient Backdoors

   Hoang V Pham, The-Anh Ta, Anh Tran, and Khoa D Doan

   In European Conference on Computer Vision 2024

   Abs Bib PDF Code Slides Submission History

   Recently, backdoor attacks have become an emerging threat to the security of machine learning models. From the adversary's perspective, the implanted backdoors should be resistant to defensive algorithms, but some recently proposed fine-tuning defenses can remove these backdoors with notable efficacy. This is mainly due to the catastrophic forgetting (CF) property of deep neural networks. This paper counters CF of backdoors by leveraging continual learning (CL) techniques. We begin by investigating the connectivity between a backdoored and fine-tuned model in the loss landscape. Our analysis confirms that fine-tuning defenses, especially the more advanced ones, can easily push a poisoned model out of the backdoor regions, making it forget all about the backdoors. Based on this finding, we re-formulate backdoor training through the lens of CL and propose a novel framework, named Sequential Backdoor Learning (SBL), that can generate resilient backdoors. This framework separates the backdoor poisoning process into two tasks: the first task learns a backdoored model, while the second task, based on the CL principles, moves it to a backdoored region resistant to fine-tuning. We additionally propose to seek flatter backdoor regions via a sharpness-aware minimizer in the framework, further strengthening the durability of the implanted backdoor. Finally, we demonstrate the effectiveness of our method through extensive empirical experiments on several benchmark datasets in the backdoor domain. The source code is available at https://github.com/mail-research/SBL-resilient-backdoors

   @inproceedings{pham2024SBL,
     title = {Flatness-aware Sequential Learning Generates Resilient Backdoors},
     author = {Pham, Hoang V and Ta, The-Anh and Tran, Anh and Doan, Khoa D},
     booktitle = {European Conference on Computer Vision},
     year = {2024},
     code = {https://github.com/mail-research/SBL-resilient-backdoors},
     bibtex_show = {true},
     abbr = {ECCV},
     pdf = {https://www.arxiv.org/abs/2407.14738},
     slides = {https://docs.google.com/presentation/d/1YEyQDSBardXdHCv-qBmxZnBwnXMAtRKZfWsawknowFs/edit?usp=sharing},
     submissions = {CVPR'24 -- ECCV'24},
     selected = {true}
   }

   CVPR'24 -- ECCV'24

2. ECCV Data Poisoning Quantization Backdoor Attack

   Tran Huynh, Anh Tran, Khoa D Doan, and Tung Pham

   In European Conference on Computer Vision 2024

   Abs Bib PDF Submission History

   Deep learning (DL) models are often large and require a lot of computing power. Hence, model quantization is frequently used to reduce their size and complexity, making them more suitable for deployment on edge devices or achieving real-time performance. It has been previously shown that standard quantization frameworks can be exploited to activate the backdoor in a DL model. This means that an attacker could create a hijacked model that appears normal and free from backdoors (even when examined by state-of-the-art defenses), but when it is quantized, the backdoor is activated, and the attacker can control the model’s output. Existing backdoor attack methods on quantization models require full access to the victim model, which might not hold in practice. In this work, we focus on designing a novel quantization backdoor based on data poisoning, which requires zero knowledge of the target model. The key component is a trigger pattern generator, which is trained together with a surrogate model in an alternating manner. The attack’s effectiveness is tested on multiple benchmark datasets, including CIFAR10, CelebA, and ImageNet10, as well as state-of-the-art backdoor defenses.

   @inproceedings{huynh2024quantizedbd,
     title = {Data Poisoning Quantization Backdoor Attack},
     author = {Huynh, Tran and Tran, Anh and Doan, Khoa D and Pham, Tung},
     booktitle = {European Conference on Computer Vision},
     year = {2024},
     bibtex_show = {true},
     abbr = {ECCV},
     pdf = {https://www.ecva.net/papers/eccv_2024/papers_ECCV/papers/11142.pdf},
     submissions = {CVPR'24 -- ECCV'24},
     selected = {true}
   }

   CVPR'24 -- ECCV'24

3. ICPR Composite Concept Extraction through Backdooring

   Banibrata Ghosh, Haripriya Harikumar, Khoa D Doan, Svetha Venkatesh, and Santu Rana

   In 27th International Conference on Pattern Recognition 2024

   Abs Bib PDF Submission History

   Learning composite concepts, such as "red car", from individual examples—like a white car representing the concept of "car" and a red strawberry representing the concept of "red"—is inherently challenging. This paper introduces a novel method called Composite Concept Extractor (CoCE), which leverages techniques from traditional backdoor attacks to learn these composite concepts in a zero-shot setting, requiring only examples of individual concepts. By repurposing the trigger-based model backdooring mechanism, we create a strategic distortion in the manifold of the target object (e.g., "car") induced by example objects with the target property (e.g., "red") from objects "red strawberry", ensuring the distortion selectively affects the target objects with the target property. Contrastive learning is then employed to further refine this distortion and a method is formulated for detecting objects that are influenced by the distortion. Extensive experiments with in-depth analysis across different datasets demonstrates the utility and applicability of our proposed approach.

   @inproceedings{ghosh2024conceptbd,
     title = {Composite Concept Extraction through Backdooring},
     author = {Ghosh, Banibrata and Harikumar, Haripriya and Doan, Khoa D and Venkatesh, Svetha and Rana, Santu},
     booktitle = {27th International Conference on Pattern Recognition},
     year = {2024},
     bibtex_show = {true},
     abbr = {ICPR},
     pdf = {https://arxiv.org/pdf/2406.13411},
     submissions = {CVPR'24 -- ECCV'24 -- ICPR'24},
     selected = {true}
   }

   CVPR'24 -- ECCV'24 -- ICPR'24

4. PREPRINT Non-Cooperative Backdoor Attacks in Federated Learning: A New Threat Landscape

   Tuan M Nguyen, Dung T Nguyen, Khoa D Doan, and Kok-Seng Wong

   2024

   Abs Bib PDF Code

   Despite the promise of Federated Learning (FL) for privacy-preserving model training on distributed data, it remains susceptible to backdoor attacks. These attacks manipulate models by embedding triggers (specific input patterns) in the training data, forcing misclassification as predefined classes during deployment. Traditional single-trigger attacks and recent work on cooperative multiple-trigger attacks, where clients collaborate, highlight limitations in attack realism due to coordination requirements. We investigate a more alarming scenario: non-cooperative multiple-trigger attacks. Here, independent adversaries introduce distinct triggers targeting unique classes. These parallel attacks exploit FL's decentralized nature, making detection difficult. Our experiments demonstrate the alarming vulnerability of FL to such attacks, where individual backdoors can be successfully learned without impacting the main task. This research emphasizes the critical need for robust defenses against diverse backdoor attacks in the evolving FL landscape. While our focus is on empirical analysis, we believe it can guide backdoor research toward more realistic settings, highlighting the crucial role of FL in building robust defenses against diverse backdoor threats. The code is available at https://anonymous.4open.science/r/nba-980F/.

   @article{nguyen2024noncooperativeBAFL,
     title = {Non-Cooperative Backdoor Attacks in Federated Learning: A New Threat Landscape},
     author = {Nguyen, Tuan M and Nguyen, Dung T and Doan, Khoa D and Wong, Kok-Seng},
     year = {2024},
     bibtex_show = {true},
     abbr = {PREPRINT},
     pdf = {https://arxiv.org/abs/2407.07917},
     code = {https://anonymous.4open.science/r/nba-980F/},
     selected = {true}
   }

5. PREPRINT Less is More: Sparse Watermarking in LLMs with Enhanced Text Quality

   Cao-Duy Hoang, Hung T. Q. Le, Rui Chu, Ping Li, Weijie Zhao, Yingjie Lao, and Khoa D Doan

   2024

   Abs Bib Code

   With the widespread adoption of Large Language Models (LLMs), concerns about potential misuse have emerged. To this end, watermarking has been adapted to LLM, enabling a simple and effective way to detect and monitor generated text. However, while the existing methods can differentiate between watermarked and unwatermarked text with high accuracy, they often face a trade-off between the quality of the generated text and the effectiveness of the watermarking process. In this work, we present a novel type of LLM watermark, \textit{Sparse Watermark}, which aims to mitigate this trade-off by applying watermarks to a small subset of generated tokens distributed across the text. The key strategy involves anchoring watermarked tokens to words that have specific Part-of-Speech (POS) tags. Our experimental results demonstrate that the proposed watermarking scheme achieves high detectability while generating text that outperforms previous LLM watermarking methods in quality across various tasks.

   @article{cao2024sparsellmwatermark,
     title = {Less is More: Sparse Watermarking in LLMs with Enhanced Text Quality},
     author = {Hoang, Cao-Duy and Le, Hung T. Q. and Chu, Rui and Li, Ping and Zhao, Weijie and Lao, Yingjie and Doan, Khoa D},
     year = {2024},
     bibtex_show = {true},
     abbr = {PREPRINT},
     code = {https://github.com/mail-research/sparse-llm-watermarking},
     selected = {true}
   }

6. PREPRINT MetaLLM: A High-performant and Cost-efficient Dynamic Framework for Wrapping LLMs

   Quang H Nguyen, Cao-Duy Hoang, Juliette Decugis, Saurav Manchanda, Nitesh V Chawla, and Khoa D Doan

   2024

   Abs Bib PDF Code

   The rapid progress in machine learning (ML) has brought forth many large language models (LLMs) that excel in various tasks and areas. These LLMs come with different abilities and costs in terms of computation or pricing. Since the demand for each query can vary, e.g., because of the queried domain or its complexity, defaulting to one LLM in an application is not usually the best choice, whether it is the biggest, priciest, or even the one with the best average test performance. Consequently, picking the right LLM that is both accurate and cost-effective for an application remains a challenge. In this paper, we introduce MetaLLM, a framework that dynamically and intelligently routes each query to the optimal LLM (among several available LLMs) for classification tasks, achieving significantly improved accuracy and cost-effectiveness. By framing the selection problem as a multi-armed bandit, MetaLLM balances prediction accuracy and cost efficiency under uncertainty. Our experiments, conducted on popular LLM platforms such as OpenAI's GPT models, Amazon's Titan, Anthropic's Claude, and Meta's LLaMa, showcase MetaLLM's efficacy in real-world scenarios, laying the groundwork for future extensions beyond classification tasks.

   @article{nguyen2024metallm,
     title = {MetaLLM: A High-performant and Cost-efficient Dynamic Framework for Wrapping LLMs},
     author = {Nguyen, Quang H and Hoang, Cao-Duy and Decugis, Juliette and Manchanda, Saurav and Chawla, Nitesh V and Doan, Khoa D},
     year = {2024},
     bibtex_show = {true},
     abbr = {PREPRINT},
     pdf = {https://arxiv.org/abs/2407.10834},
     code = {https://github.com/mail-research/MetaLLM-wrapper/},
     selected = {true}
   }

7. PREPRINT Forget but Recall: Incremental Latent Rectification in Continual Learning

   Nghia D Nguyen, Hieu T Nguyen, Ang Li, Hoang V Pham, Viet Anh Nguyen, and Khoa D Doan

   2024

   Abs Bib PDF Code

   Intrinsic capability to continuously learn a changing data stream is a desideratum of deep neural networks (DNNs). However, current DNNs suffer from catastrophic forgetting, which hinders remembering past knowledge. To mitigate this issue, existing Continual Learning (CL) approaches either retain exemplars for replay, regularize learning, or allocate dedicated capacity for new tasks. This paper investigates an unexplored CL direction for incremental learning called Incremental Latent Rectification or ILR. In a nutshell, ILR learns to propagate with correction (or rectify) the representation from the current trained DNN backward to the representation space of the old task, where performing predictive decisions is easier. This rectification process only employs a chain of small representation mapping networks, called rectifier units. Empirical experiments on several continual learning benchmarks, including CIFAR10, CIFAR100, and Tiny ImageNet, demonstrate the effectiveness and potential of this novel CL direction compared to existing representative CL methods.

   @article{nguyen2024irl,
     title = {Forget but Recall: Incremental Latent Rectification in Continual Learning},
     author = {Nguyen, Nghia D and Nguyen, Hieu T and Li, Ang and Pham, Hoang V and Nguyen, Viet Anh and Doan, Khoa D},
     year = {2024},
     bibtex_show = {true},
     abbr = {PREPRINT},
     pdf = {https://arxiv.org/abs/2312.03419},
     code = {https://github.com/mail-research/IRL-forget-but-recall},
     selected = {true}
   }

8. UAI Cold-start Recommendation by Personalized Embedding Region Elicitation

   Hieu Trung Nguyen, Duy Nguyen, Khoa D Doan, and Viet Anh Nguyen

   In The Conference on Uncertainty in Artificial Intelligence 2023

   Abs Bib PDF Submission History

   Rating elicitation is a success element for recommender systems to perform well at cold-starting, in which the systems need to recommend items to a newly arrived user with no prior knowledge about the user's preference. Existing elicitation methods employ a fixed set of items to learn the user's preference and then infer the users' preferences on the remaining items. Using a fixed seed set can limit the performance of the recommendation system since the seed set is unlikely optimal for all new users with potentially diverse preferences. This paper addresses this challenge using a 2-phase, personalized elicitation scheme. First, the elicitation scheme asks users to rate a small set of popular items in a ``burn-in'' phase. Second, it sequentially asks the user to rate adaptive items to refine the preference and the user's representation. Throughout the process, the system represents the user's embedding value not by a point estimate but by a region estimate. The value of information obtained by asking the user's rating on an item is quantified by the distance from the region center embedding space that contains with high confidence the true embedding value of the user. Finally, the recommendations are successively generated by considering the preference region of the user. We show that each subproblem in the elicitation scheme can be efficiently implemented. Further, we empirically demonstrate the effectiveness of the proposed method against existing rating-elicitation methods on several prominent datasets.

   @inproceedings{nguyen2024cold,
     title = {Cold-start Recommendation by Personalized Embedding Region Elicitation},
     author = {Nguyen, Hieu Trung and Nguyen, Duy and Doan, Khoa D and Nguyen, Viet Anh},
     booktitle = {The Conference on Uncertainty in Artificial Intelligence},
     year = {2023},
     bibtex_show = {true},
     abbr = {UAI},
     pdf = {https://arxiv.org/pdf/2406.00973},
     submissions = {RecSys'23 -- CIKM'23 -- AAAI'24 -- UAI'24},
     selected = {true}
   }

   RecSys'23 -- CIKM'23 -- AAAI'24 -- UAI'24

9. PREPRINT Synthesizing Physical Backdoor Datasets: An Automated Framework Leveraging Deep Generative Models

   Sze Jue Yang, Chinh D La, Quang H Nguyen, Eugene Bagdasaryan, Kok-Seng Wong, Anh T Tran, Chee Seng Chan, and Khoa D Doan

   2024

   Abs Bib PDF Code

   Backdoor attacks, representing an emerging threat to the integrity of deep neural networks, have garnered significant attention due to their ability to compromise deep learning systems clandestinely. While numerous backdoor attacks occur within the digital realm, their practical implementation in real-world prediction systems remains limited and vulnerable to disturbances in the physical world. Consequently, this limitation has given rise to the development of physical backdoor attacks, where trigger objects manifest as physical entities within the real world. However, creating the requisite dataset to train or evaluate a physical backdoor model is a daunting task, limiting the backdoor researchers and practitioners from studying such physical attack scenarios. This paper unleashes a recipe that empowers backdoor researchers to effortlessly create a malicious, physical backdoor dataset based on advances in generative modeling. Particularly, this recipe involves 3 automatic modules: suggesting the suitable physical triggers, generating the poisoned candidate samples (either by synthesizing new samples or editing existing clean samples), and finally refining for the most plausible ones. As such, it effectively mitigates the perceived complexity associated with creating a physical backdoor dataset, transforming it from a daunting task into an attainable objective. Extensive experiment results show that datasets created by our “recipe” enable adversaries to achieve an impressive attack success rate on real physical world data and exhibit similar properties compared to previous physical backdoor attack studies. This paper offers researchers a valuable toolkit for studies of physical backdoors, all within the confines of

   @article{yang2023synthesizing,
     title = {Synthesizing Physical Backdoor Datasets: An Automated Framework Leveraging Deep Generative Models},
     author = {Yang, Sze Jue and La, Chinh D and Nguyen, Quang H and Bagdasaryan, Eugene and Wong, Kok-Seng and Tran, Anh T and Chan, Chee Seng and Doan, Khoa D},
     year = {2024},
     bibtex_show = {true},
     selected = {true},
     abbr = {PREPRINT},
     code = {https://github.com/mail-research/synthetic-physical-backdoor-datasets},
     pdf = {https://arxiv.org/abs/2312.03419}
   }

10. ACL-Findings Fooling the Textual Fooler via Randomizing Latent Representations

    Cao-Duy Hoang, Quang H Nguyen, Saurav Manchanda, Minlong Peng, Kok-Seng Wong, and Khoa D Doan

    In Findings of the Association for Computational Linguistics 2024

    Abs Bib PDF Code Submission History

    Despite outstanding performance in a variety of NLP tasks, recent studies have revealed that NLP models are vulnerable to adversarial attacks that slightly perturb the input to cause the models to misbehave. Among these attacks, adversarial word-level perturbations are well-studied and effective attack strategies. Since these attacks work in black-box settings, they do not require access to the model architecture or model parameters and thus can be detrimental to existing NLP applications. To perform an attack, the adversary queries the victim model many times to determine the most important words in an input text and to replace these words with their corresponding synonyms. In this work, we propose a lightweight and attack-agnostic defense whose main goal is to perplex the process of generating an adversarial example in these query-based black-box attacks; that is to fool the textual fooler. This defense, named AdvFooler, works by randomizing the latent representation of the input at inference time. Different from existing defenses, AdvFooler does not necessitate additional computational overhead during training nor relies on assumptions about the potential adversarial perturbation set while having a negligible impact on the model's accuracy. Our theoretical and empirical analyses highlight the significance of robustness resulting from confusing the adversary via randomizing the latent space, as well as the impact of randomization on clean accuracy. Finally, we empirically demonstrate near state-of-the-art robustness of AdvFooler against representative adversarial word-level attacks on two benchmark datasets.

    @inproceedings{hoang2023advfooler,
      title = {Fooling the Textual Fooler via Randomizing Latent Representations},
      author = {Hoang, Cao-Duy and Nguyen, Quang H and Manchanda, Saurav and Peng, Minlong and Wong, Kok-Seng and Doan, Khoa D},
      year = {2024},
      abbr = {ACL-Findings},
      bibtex_show = {true},
      booktitle = {Findings of the Association for Computational Linguistics},
      pdf = {https://arxiv.org/abs/2310.01452},
      code = {https://github.com/mail-research/AdvFooler-text-defender},
      submissions = {EMNLP'24 -- ICLR'24 -- ACL'24},
      selected = {true}
    }

    EMNLP'24 -- ICLR'24 -- ACL'24

11. ICLR Understanding the Robustness of Randomized Feature Defense Against Query-Based Adversarial Attacks

    Quang H Nguyen, Yingjie Lao, Tung Pham, Kok-Seng Wong, and Khoa D Doan

    In The Twelfth International Conference on Learning Representations 2024

    Abs Bib PDF Code Submission History

    Recent works have shown that deep neural networks are vulnerable to adversarial examples that find samples close to the original image but can make the model misclassify. Even with access only to the model's output, an attacker can employ black-box attacks to generate such adversarial examples. In this work, we propose a simple and lightweight defense against black-box attacks by adding random noise to hidden features at intermediate layers of the model at inference time. Our theoretical analysis confirms that this method effectively enhances the model's resilience against both score-based and decision-based black-box attacks. Importantly, our defense does not necessitate adversarial training and has minimal impact on accuracy, rendering it applicable to any pre-trained model. Our analysis also reveals the significance of selectively adding noise to different parts of the model based on the gradient of the adversarial objective function, which can be varied during the attack. We demonstrate the robustness of our defense against multiple black-box attacks through extensive empirical experiments involving diverse models with various architectures.

    @inproceedings{nguyen2023randomized,
      title = {Understanding the Robustness of Randomized Feature Defense Against Query-Based Adversarial Attacks},
      author = {Nguyen, Quang H and Lao, Yingjie and Pham, Tung and Wong, Kok-Seng and Doan, Khoa D},
      booktitle = {The Twelfth International Conference on Learning Representations},
      year = {2024},
      bibtex_show = {true},
      abbr = {ICLR},
      pdf = {https://openreview.net/forum?id=vZ6r9GMT1n},
      selected = {true},
      code = {https://github.com/mail-research/randomized_defenses},
      submissions = {NeurIPS'23 -- ICLR'24}
    }

    NeurIPS'23 -- ICLR'24

12. NeurIPS Iba: Towards irreversible backdoor attacks in federated learning

    Thuy Dung Nguyen, Tuan M Nguyen, Anh T Tran, Khoa D Doan, and Kok-Seng Wong

    Advances in Neural Information Processing Systems 2024

    Abs Bib PDF Code Submission History

    Federated learning (FL) is a distributed learning approach that enables machine learning models to be trained on decentralized data without compromising end devices' personal, potentially sensitive data. However, the distributed nature and uninvestigated data intuitively introduce new security vulnerabilities, including backdoor attacks. In this scenario, an adversary implants backdoor functionality into the global model during training, which can be activated to cause the desired misbehaviors for any input with a specific adversarial pattern. Despite having remarkable success in triggering and distorting model behavior, prior backdoor attacks in FL often hold impractical assumptions, limited imperceptibility, and durability. Specifically, the adversary needs to control a sufficiently large fraction of clients or know the data distribution of other honest clients. In many cases, the trigger inserted is often visually apparent, and the backdoor effect is quickly diluted if the adversary is removed from the training process. To address these limitations, we propose a novel backdoor attack framework in FL, the Irreversible Backdoor Attack (IBA), that jointly learns the optimal and visually stealthy trigger and then gradually implants the backdoor into a global model. This approach allows the adversary to execute a backdoor attack that can evade both human and machine inspections. Additionally, we enhance the efficiency and durability of the proposed attack by selectively poisoning the model's parameters that are least likely updated by the main task's learning process and constraining the poisoned model update to the vicinity of the global model. Finally, we evaluate the proposed attack framework on several benchmark datasets, including MNIST, CIFAR-10, and Tiny ImageNet, and achieved high success rates while simultaneously bypassing existing backdoor defenses and achieving a more durable backdoor effect compared to other backdoor attacks. Overall, IBA offers a more effective, stealthy, and durable approach to backdoor attacks in FL.

    @article{nguyen2024iba,
      title = {Iba: Towards irreversible backdoor attacks in federated learning},
      author = {Nguyen, Thuy Dung and Nguyen, Tuan M and Tran, Anh T and Doan, Khoa D and Wong, Kok-Seng},
      journal = {Advances in Neural Information Processing Systems},
      volume = {36},
      year = {2024},
      submissions = {NeurIPS'23},
      bibtex_show = {true},
      abbr = {NeurIPS},
      pdf = {https://proceedings.neurips.cc/paper_files/paper/2023/hash/d0c6bc641a56bebee9d985b937307367-Abstract-Conference.html},
      code = {https://github.com/sail-research/iba},
      selected = {true}
    }

    NeurIPS'23

13. EAAI Backdoor attacks and defenses in federated learning: Survey, challenges and future research directions

    Thuy Dung Nguyen, Tuan M Nguyen, Phi Le Nguyen, Hieu H Pham, Khoa D Doan, and Kok-Seng Wong

    Engineering Applications of Artificial Intelligence 2024

    Abs Bib PDF Submission History

    Federated learning (FL) is an approach within the realm of machine learning (ML) that allows the use of distributed data without compromising personal privacy. In FL, it becomes evident that the training data among participants frequently exhibit heterogeneous distribution characteristics. This inherent heterogeneity poses a substantial challenge for the orchestration server as it strives to assess the reliability of each local model update. Due to this challenge, FL becomes susceptible to various potential risks, with the ominous backdoor attack standing out as one of the most menacing threats. Backdoor attacks involve the insertion of malicious functionality into a targeted model through poisoned updates from malicious clients. These attacks can cause the global model to misbehave on specific inputs while appearing normal in other instances. Although the backdoor attacks received significant attention for their potential impact on practical deep learning applications, their exploration within the realm of FL remains limited. This survey seeks to address this gap by offering an all-encompassing examination of prevailing backdoor attack tactics and defenses in the context of FL. We include an exhaustive analysis of diverse approaches to provide a comprehensive understanding of this intricate landscape. Furthermore, we also discuss the challenges and potential future directions for attacks and defenses in the context of FL.

    @article{nguyen2024backdoor,
      title = {Backdoor attacks and defenses in federated learning: Survey, challenges and future research directions},
      author = {Nguyen, Thuy Dung and Nguyen, Tuan M and Le Nguyen, Phi and Pham, Hieu H and Doan, Khoa D and Wong, Kok-Seng},
      journal = {Engineering Applications of Artificial Intelligence},
      volume = {127},
      pages = {107166},
      year = {2024},
      publisher = {Elsevier},
      submissions = {EAAI'24},
      bibtex_show = {true},
      abbr = {EAAI},
      pdf = {https://www.sciencedirect.com/science/article/pii/S0952197623013507},
      selected = {true}
    }

    EAAI'24

14. SIGIR Asymmetric Hashing for Fast Ranking via Neural Network Measures

    Khoa D Doan, Shulong Tan, Weijie Zhao, and Ping Li

    In 46th International ACM SIGIR Conference on Research and Development in Information Retrieval 2023

    Abs Bib PDF Submission History

    Fast item ranking is an important task in recommender systems. In previous works, graph-based Approximate Nearest Neighbor (ANN) approaches have demonstrated good performance on item ranking tasks with generic searching/matching measures (including complex measures such as neural network measures). However, since these ANN approaches must go through the neural measures several times during ranking, the computation is not practical if the neural measure is a large network. On the other hand, fast item ranking using existing hashing-based approaches, such as Locality Sensitive Hashing (LSH), only works with a limited set of measures, such as cosine and Euclidean distance, but not with general search measures such as neural networks. Given an arbitrary searching measure, previous learning-to-hash approaches are also not suitable to solve the fast item ranking problem since they can take a significant amount of time and computation to train the hash functions to approximate the searching measure due to a large number of possible training pairs in this problem. Hashing approaches, however, are attractive because they provide a principal and efficient way to retrieve candidate items. In this paper, we propose a simple and effective learning-to-hash approach for the fast item ranking problem that can be used to efficiently approximate any type of measure, including neural network measures. Specifically, we solve this problem with an asymmetric hashing framework based on discrete inner product fitting. We learn a pair of related hash functions that map heterogeneous objects (e.g., users and items) into a common discrete space where the inner product of their binary codes reveals their true similarity defined via the original searching measure. The fast ranking problem is reduced to an ANN search via this asymmetric hashing scheme. Then, we propose a sampling strategy to efficiently select relevant and contrastive samples to train the hashing model. We empirically validate the proposed method against the existing state-of-the-art fast item ranking methods in several combinations of non-linear searching functions and prominent datasets.

    @inproceedings{doan2023flora,
      bibtex_show = {true},
      abbr = {SIGIR},
      title = {Asymmetric Hashing for Fast Ranking via Neural Network Measures},
      author = {Doan, Khoa D and Tan, Shulong and Zhao, Weijie and Li, Ping},
      booktitle = {46th International ACM SIGIR Conference on Research and Development in Information Retrieval},
      year = {2023},
      submissions = {SIGIR'21 -- RecSys'21 -- WSDM'22 -- WWW'22 -- SIGIR'22 -- VLDB'23 -- CIKM'22 -- WWW'23 -- SIGIR'23},
      pdf = {https://dl.acm.org/doi/abs/10.1145/3539618.3591640},
      selected = {true}
    }

    SIGIR'21 -- RecSys'21 -- WSDM'22 -- WWW'22 -- SIGIR'22 -- VLDB'23 -- CIKM'22 -- WWW'23 -- SIGIR'23

15. AAAI Defending backdoor attacks on vision transformer via patch processing

    Khoa D Doan, Yingjie Lao, and Ping Li

    In AAAI Conference on Artificial Intelligence 2023

    Abs Bib PDF Submission History

    Vision Transformers (ViTs) have a radically different architecture with significantly less inductive bias than Convolutional Neural Networks. Along with the improvement in performance, security and robustness of ViTs are also of great importance to study. In contrast to many recent works that exploit the robustness of ViTs against adversarial examples, this paper investigates a representative causative attack, ie, backdoor. We first examine the vulnerability of ViTs against various backdoor attacks and find that ViTs are also quite vulnerable to existing attacks. However, we observe that the clean-data accuracy and backdoor attack success rate of ViTs respond distinctively to patch transformations before the positional encoding. Then, based on this finding, we propose an effective method for ViTs to defend both patch-based and blending-based trigger backdoor attacks via patch processing. The performances are evaluated on several benchmark datasets, including CIFAR10, GTSRB, and TinyImageNet, which show the proposedds defense is very successful in mitigating backdoor attacks for ViTs. To the best of our knowledge, this paper presents the first defensive strategy that utilizes a unique characteristic of ViTs against backdoor attacks.

    @inproceedings{doan2023bdvitt,
      abbr = {AAAI},
      title = {Defending backdoor attacks on vision transformer via patch processing},
      author = {Doan, Khoa D and Lao, Yingjie and Li, Ping},
      booktitle = {AAAI Conference on Artificial Intelligence},
      year = {2023},
      submissions = {CVPR'22 -- ICCV'22 -- AAAI'23},
      bibtex_show = {true},
      selected = {true},
      teaser = {doan2023bdvits.png},
      pdf = {https://ojs.aaai.org/index.php/AAAI/article/view/25125}
    }

    CVPR'22 -- ICCV'22 -- AAAI'23

16. NeurIPS Marksman Backdoor: Backdoor Attacks with Arbitrary Target Class

    Khoa D Doan, Yingjie Lao, and Ping Li

    In Thirty-Sixth Conference on Neural Information Processing Systems 2022

    Abs Bib PDF Code Slides Submission History

    In recent years, machine learning models have been shown to be vulnerable to backdoor attacks. Under such attacks, an adversary embeds a stealthy backdoor into the trained model such that the compromised models will behave normally on clean inputs but will misclassify according to the adversary's control on maliciously constructed input with a trigger. While these existing attacks are very effective, the adversary's capability is limited: given an input, these attacks can only cause the model to misclassify toward a single pre-defined or target class. In contrast, this paper exploits a novel backdoor attack with a much more powerful payload, denoted as Marksman, where the adversary can arbitrarily choose which target class the model will misclassify given any input during inference. To achieve this goal, we propose to represent the trigger function as a class-conditional generative model and to inject the backdoor in a constrained optimization framework, where the trigger function learns to generate an optimal trigger pattern to attack any target class at will while simultaneously embedding this generative backdoor into the trained model. Given the learned trigger-generation function, during inference, the adversary can specify an arbitrary backdoor attack target class, and an appropriate trigger causing the model to classify toward this target class is created accordingly. We show empirically that the proposed framework achieves high attack performance (e.g., 100% attack success rates in several experiments) while preserving the clean-data performance in several benchmark datasets, including MNIST, CIFAR10, GTSRB, and TinyImageNet. The proposed Marksman backdoor attack can also easily bypass existing backdoor defenses that were originally designed against backdoor attacks with a single target class. Our work takes another significant step toward understanding the extensive risks of backdoor attacks in practice.

    @inproceedings{doan2022marksman,
      bibtex_show = {true},
      abbr = {NeurIPS},
      title = {Marksman Backdoor: Backdoor Attacks with Arbitrary Target Class},
      author = {Doan, Khoa D and Lao, Yingjie and Li, Ping},
      booktitle = {Thirty-Sixth Conference on Neural Information Processing Systems},
      year = {2022},
      submissions = {NeurIPS'22},
      teaser = {doan2022marksman.png},
      pdf = {https://openreview.net/forum?id=i-k6J4VkCDq},
      code = {https://github.com/khoadoan106/backdoor_attacks},
      slides = {https://nips.cc/media/neurips-2022/Slides/52924.pdf},
      selected = {true}
    }

    NeurIPS'22

17. CVPR One Loss for Quantization: Deep Hashing with Discrete Wasserstein Distributional Matching

    Khoa D Doan, Peng Yang, and Ping Li

    In Conference on Computer Vision and Pattern Recognition 2022

    Abs Bib PDF Code Slides Submission History

    Image hashing is a principled approximate nearest neighbor approach to find similar items to a query in a large collection of images. Hashing aims to learn a binary-output function that maps an image to a binary vector. For optimal retrieval performance, producing balanced hash codes with low-quantization error to bridge the gap between the learning stage's continuous relaxation and the inference stage's discrete quantization is important. However, in the existing deep supervised hashing methods, coding balance and low-quantization error are difficult to achieve and involve several losses. We argue that this is because the existing quantization approaches in these methods are heuristically constructed and not effective to achieve these objectives. This paper considers an alternative approach to learning the quantization constraints. The task of learning balanced codes with low quantization error is re-formulated as matching the learned distribution of the continuous codes to a pre-defined discrete, uniform distribution. This is equivalent to minimizing the distance between two distributions. We then propose a computationally efficient distributional distance by leveraging the discrete property of the hash functions. This distributional distance is a valid distance and enjoys lower time and sample complexities. The proposed single-loss quantization objective can be integrated into any existing supervised hashing method to improve code balance and quantization error. Experiments confirm that the proposed approach substantially improves the performance of several representative hashing methods.

    @inproceedings{doan2022hswd,
      bibtex_show = {true},
      abbr = {CVPR},
      title = {One Loss for Quantization: Deep Hashing with Discrete Wasserstein Distributional Matching},
      author = {Doan, Khoa D and Yang, Peng and Li, Ping},
      booktitle = {Conference on Computer Vision and Pattern Recognition},
      year = {2022},
      submissions = {CVPR'22},
      slides = {doan2022hswd-slides.pdf},
      code = {https://github.com/khoadoan106/single_loss_quantization},
      pdf = {https://openreview.net/pdf?id=uaqweIZ-9_k},
      teaser = {doan2022hswd.png},
      selected = {true}
    }

    CVPR'22

18. NeurIPS Backdoor Attack with Imperceptible Input and Latent Modification

    Khoa D Doan, Yingjie Lao, and Ping Li

    In Thirty-Fifth Conference on Neural Information Processing Systems 2021

    Abs Bib PDF VIDEO Code Slides Submission History

    Recent studies have shown that deep neural networks (DNN) are vulnerable to various adversarial attacks. In particular, an adversary can inject a stealthy backdoor into a model such that the compromised model will behave normally without the presence of the trigger. Techniques for generating backdoor images that are visually imperceptible from clean images have also been developed recently, which further enhance the stealthiness of the backdoor attacks from the input space. Along with the development of attacks, defense against backdoor attacks is also evolving. Many existing countermeasures found that backdoor tends to leave tangible footprints in the latent or feature space, which can be utilized to mitigate backdoor attacks.In this paper, we extend the concept of imperceptible backdoor from the input space to the latent representation, which significantly improves the effectiveness against the existing defense mechanisms, especially those relying on the distinguishability between clean inputs and backdoor inputs in latent space. In the proposed framework, the trigger function will learn to manipulate the input by injecting imperceptible input noise while matching the latent representations of the clean and manipulated inputs via a Wasserstein-based regularization of the corresponding empirical distributions. We formulate such an objective as a non-convex and constrained optimization problem and solve the problem with an efficient stochastic alternating optimization procedure. We name the proposed backdoor attack as Wasserstein Backdoor (WB), which achieves a high attack success rate while being stealthy from both the input and latent spaces, as tested in several benchmark datasets, including MNIST, CIFAR10, GTSRB, and TinyImagenet.

    @inproceedings{doan2021wb,
      bibtex_show = {true},
      abbr = {NeurIPS},
      title = {Backdoor Attack with Imperceptible Input and Latent Modification},
      author = {Doan, Khoa D and Lao, Yingjie and Li, Ping},
      booktitle = {Thirty-Fifth Conference on Neural Information Processing Systems},
      year = {2021},
      submissions = {NeurIPS'21},
      pdf = {https://proceedings.neurips.cc/paper/2021/file/9d99197e2ebf03fc388d09f1e94af89b-Paper.pdf},
      code = {https://github.com/khoadoan106/backdoor_attacks},
      video = {https://recorder-v3.slideslive.com/?share=51522&s=8af881c0-56e8-451e-865f-adb1e90e5471},
      slides = {doan2021wb-slides.pdf},
      teaser = {doan2021wb.png},
      selected = {true}
    }

    NeurIPS'21

19. ICCV LIRA: Learnable, Imperceptible and Robust Backdoor Attacks

    Khoa D Doan, Yingjie Lao, Weijie Zhao, and Ping Li

    In International Conference on Computer Vision 2021

    Abs Bib PDF Code Poster Slides Submission History

    Recently, machine learning models have demonstrated to be vulnerable to backdoor attacks, primarily due to the lack of transparency in black-box models such as deep neural networks. A third-party model can be poisoned such that it works adequately in normal conditions but behaves maliciously on samples with specific trigger patterns. However, the trigger injection function is manually defined in most existing backdoor attack methods, e.g., placing a small patch of pixels on an image or slightly deforming the image before poisoning the model. This results in a two-stage approach with a sub-optimal attack success rate and a lack of complete stealthiness under human inspection. In this paper, we propose a novel and stealthy backdoor attack framework, LIRA, which jointly learns the optimal, stealthy trigger injection function and poisons the model. We formulate such an objective as a non-convex, constrained optimization problem. Under this optimization framework, the trigger generator function will learn to manipulate the input with imperceptible noise to preserve the model performance on the clean data and maximize the attack success rate on the poisoned data. Then, we solve this challenging optimization problem with an efficient, two-stage stochastic optimization procedure. Finally, the proposed attack framework achieves 100% success rates in several benchmark datasets, including MNIST, CIFAR10, GTSRB, and T-ImageNet, while simultaneously bypassing existing backdoor defense methods and human inspection.

    @inproceedings{doan2021lira,
      bibtex_show = {true},
      abbr = {ICCV},
      title = {LIRA: Learnable, Imperceptible and Robust Backdoor Attacks},
      author = {Doan, Khoa D and Lao, Yingjie and Zhao, Weijie and Li, Ping},
      booktitle = {International Conference on Computer Vision},
      submissions = {ICCV'21},
      code = {https://github.com/khoadoan106/backdoor_attacks},
      slides = {https://github.com/sunbelbd/invisible_backdoor_attacks/raw/master/resources/ICCV2021-LIRA-Slides.pdf},
      poster = {https://github.com/sunbelbd/invisible_backdoor_attacks/raw/master/resources/ICCV2021-LIRA-Poster.pdf},
      pdf = {https://openaccess.thecvf.com/content/ICCV2021/papers/Doan_LIRA_Learnable_Imperceptible_and_Robust_Backdoor_Attacks_ICCV_2021_paper.pdf},
      year = {2021},
      teaser = {doan2021lira.png},
      selected = {true}
    }

    ICCV'21

20. SIGIR Interpretable Graph Similarity Computation via Differentiable Optimal Alignment of Node Embeddings

    Khoa D Doan, Saurav Manchanda, Suchismit Mahapatra, and Chandan K Reddy

    In 44th International ACM SIGIR Conference on Research and Development in Information Retrieval 2021

    Abs Bib PDF VIDEO Code Slides Submission History

    Computing graph similarity is an important task in many graph-related applications such as retrieval in graph databases or graph clustering. While numerous measures have been proposed to capture the similarity between a pair of graphs, Graph Edit Distance (GED) and Maximum Common Subgraphs (MCS) are the two widely used measures in practice. GED and MCS are domain-agnostic measures of structural similarity between the graphs and define the similarity as a function of pairwise alignment of different entities (such as nodes, edges, and subgraphs) in the two graphs. The explicit explainability offered by the pairwise alignment provides transparency and justification of the similarity score, thus, GED and MCS have important practical applications. However, their exact computations are known to be NP-hard. While recently proposed neural-network based approximations have been shown to accurately compute these similarity scores, they have limited ability in providing comprehensive explanations compared to classical combinatorial algorithms, e.g., Beam search. This paper aims at efficiently approximating these domain-agnostic similarity measures through a neural network, and simultaneously learning the alignments (i.e., explanations) similar to those of classical intractable methods. Specifically, we formulate the similarity between a pair of graphs as the minimal "transformation" cost from one graph to another in the learnable node-embedding space. We show that, if node embedding is able to capture its neighborhood context closely, our proposed similarity function closely approximates both the alignment and the similarity score of classical methods. Furthermore, we also propose an efficient differentiable computation of our proposed objective for model training. Empirically, we demonstrate that the proposed method achieves up to 50%-100% reduction in the Mean Squared Error for the graph similarity approximation task and up to 20% improvement in the retrieval evaluation metrics for the graph retrieval task. The source code is available at https://github.com/khoadoan/GraphOTSim.

    @inproceedings{doan2021interpretable,
      bibtex_show = {true},
      abbr = {SIGIR},
      url = {https://doi.org/10.1145/3404835.3462960},
      code = {https://github.com/khoadoan/GraphOTSim},
      pdf = {doan2021interpretable.pdf},
      slides = {https://github.com/khoadoan/GraphOTSim/raw/main/resources/SIGIR21-fp0937-slides.pdf},
      video = {https://www.youtube.com/watch?v=IWxxsuFPsgs&t=1s},
      teaser = {doan2021interpretable.png},
      selected = {true},
      submissions = {AAAI'21 -- WWW'21 -- SIGIR'21},
      author = {Doan, Khoa D and Manchanda, Saurav and Mahapatra, Suchismit and Reddy, Chandan K},
      title = {Interpretable Graph Similarity Computation via Differentiable Optimal Alignment of Node Embeddings},
      year = {2021},
      isbn = {9781450380379},
      publisher = {Association for Computing Machinery},
      address = {New York, NY, USA},
      doi = {10.1145/3404835.3462960},
      booktitle = {44th International ACM SIGIR Conference on Research and Development in Information Retrieval},
      pages = {665–674},
      numpages = {10},
      keywords = {similarity search, model interpretability, graph similarity, GCN},
      location = {Virtual Event, Canada},
      series = {SIGIR '21}
    }

    AAAI'21 -- WWW'21 -- SIGIR'21

21. WWW Efficient Implicit Unsupervised Text Hashing Using Adversarial Autoencoder

    Khoa D Doan, and Chandan K Reddy

    In Proceedings of The Web Conference 2020

    Abs Bib HTML PDF

    Searching for documents with semantically similar content is a fundamental problem in the information retrieval domain with various challenges, primarily, in terms of efficiency and effectiveness. Despite the promise of modeling structured dependencies in documents, several existing text hashing methods lack an efficient mechanism to incorporate such vital information. Additionally, the desired characteristics of an ideal hash function, such as robustness to noise, low quantization error and bit balance/uncorrelation, are not effectively learned with existing methods. This is because of the requirement to either tune additional hyper-parameters or optimize these heuristically and explicitly constructed cost functions. In this paper, we propose a Denoising Adversarial Binary Autoencoder (DABA) model which presents a novel representation learning framework that captures structured representation of text documents in the learned hash function. Also, adversarial training provides an alternative direction to implicitly learn a hash function that captures all the desired characteristics of an ideal hash function. Essentially, DABA adopts a novel single-optimization adversarial training procedure that minimizes the Wasserstein distance in its primal domain to regularize the encoder’s output of either a recurrent neural network or a convolutional autoencoder. We empirically demonstrate the effectiveness of our proposed method in capturing the intrinsic semantic manifold of the related documents. The proposed method outperforms the current state-of-the-art shallow and deep unsupervised hashing methods for the document retrieval task on several prominent document collections.

    @inproceedings{doan2020efficient,
      bibtex_show = {true},
      abbr = {WWW},
      url = {https://doi.org/10.1145/3366423.3380150},
      html = {https://dl.acm.org/doi/abs/10.1145/3366423.3380150},
      pdf = {https://people.cs.vt.edu/~reddy/papers/WWW20a.pdf},
      teaser = {doan2020efficient.png},
      selected = {true},
      author = {Doan, Khoa D and Reddy, Chandan K},
      title = {Efficient Implicit Unsupervised Text Hashing Using Adversarial Autoencoder},
      year = {2020},
      isbn = {9781450370233},
      publisher = {Association for Computing Machinery},
      address = {New York, NY, USA},
      doi = {10.1145/3366423.3380150},
      booktitle = {Proceedings of The Web Conference},
      pages = {684–694},
      numpages = {11},
      keywords = {autoencoder, Hashing, adversarial training, deep learning.},
      location = {Taipei, Taiwan},
      series = {WWW '20}
    }

22. arXiv Gradient boosting neural networks: Grownet

    Sarkhan Badirli, Xuanqing Liu, Zhengming Xing, Avradeep Bhowmik, Khoa D Doan, and Sathiya K Selvaraj

    arXiv preprint arXiv:2002.07971 2020

    Abs Bib PDF Code

    A novel gradient boosting framework is proposed where shallow neural networks are employed as ``weak learners''. General loss functions are considered under this unified framework with specific examples presented for classification, regression, and learning to rank. A fully corrective step is incorporated to remedy the pitfall of greedy function approximation of classic gradient boosting decision tree. The proposed model rendered outperforming results against state-of-the-art boosting methods in all three tasks on multiple datasets. An ablation study is performed to shed light on the effect of each model components and model hyperparameters.

    @article{badirli2020gradient,
      bibtex_show = {true},
      abbr = {arXiv},
      url = {https://arxiv.org/abs/2002.07971},
      code = {https://github.com/sbadirli/GrowNet},
      pdf = {https://arxiv.org/pdf/2002.07971.pdf},
      teaser = {badirli2020gradient.png},
      selected = {true},
      title = {Gradient boosting neural networks: Grownet},
      author = {Badirli, Sarkhan and Liu, Xuanqing and Xing, Zhengming and Bhowmik, Avradeep and Doan, Khoa D and Selvaraj, Sathiya K},
      journal = {arXiv preprint arXiv:2002.07971},
      year = {2020}
    }

23. CIKM Adversarial Factorization Autoencoder for Look-Alike Modeling

    Khoa D Doan, Pranjul Yadav, and Chandan K Reddy

    In Proceedings of the 28th ACM International Conference on Information and Knowledge Management 2019

    Abs Bib PDF

    Digital advertising is performed in multiple ways, for e.g., contextual, display-based and search-based advertising. Across these avenues, the primary goal of the advertiser is to maximize the return on investment. To realize this, the advertiser often aims to target the advertisements towards a targeted set of audience as this set has a high likelihood to respond positively towards the advertisements. One such form of tailored and personalized, targeted advertising is known as look-alike modeling, where the advertiser provides a set of seed users and expects the machine learning model to identify a new set of users such that the newly identified set is similar to the seed-set with respect to the online purchasing activity. Existing look-alike modeling techniques (i.e., similarity-based and regression-based) suffer from serious limitations due to the implicit constraints induced during modeling. In addition, the high-dimensional and sparse nature of the advertising data increases the complexity. To overcome these limitations, in this paper, we propose a novel Adversarial Factorization Autoencoder that can efficiently learn a binary mapping from sparse, high-dimensional data to a binary address space through the use of an adversarial training procedure. We demonstrate the effectiveness of our proposed approach on a dataset obtained from a real-world setting and also systematically compare the performance of our proposed approach with existing look-alike modeling baselines.

    @inproceedings{doan2019adversarial,
      bibtex_show = {true},
      abbr = {CIKM},
      pdf = {https://dmkd.cs.vt.edu/papers/CIKM19.pdf},
      teaser = {doan2019adversarial.png},
      selected = {true},
      author = {Doan, Khoa D and Yadav, Pranjul and Reddy, Chandan K},
      title = {Adversarial Factorization Autoencoder for Look-Alike Modeling},
      year = {2019},
      isbn = {9781450369763},
      publisher = {Association for Computing Machinery},
      address = {New York, NY, USA},
      url = {https://doi.org/10.1145/3357384.3357807},
      doi = {10.1145/3357384.3357807},
      booktitle = {Proceedings of the 28th ACM International Conference on Information and Knowledge Management},
      pages = {2803–2812},
      numpages = {10},
      keywords = {deep learning, autoencoder, hashing, factorization, adversarial training, look-alike modeling},
      location = {Beijing, China},
      series = {CIKM '19}
    }