Duy H. M. Nguyen

Nguyen Ho Minh Duy

prof_pic.jpg

Universitätsstraße 32

70569 Stuttgart, Germany

Room: 2.321

I am currently a Ph.D. Candidate under the supervision of Prof. Mathias Niepert at Max Planck Research School for Intelligent Systems (IMPRS-IS) and University of Stuttgart. I have also been a Researcher at the German Research Center for Artificial Intelligence (DFKI) since 2021.

My topics of interest are Hybrid Discrete-Continuous Learning (differentiable relaxations for discrete intermediate representations), Scalable Algorithms for Multi-modal Learning with applications for Healthcare, Simulation Science, and Efficient Deep Learning (model compression, accelerated training/inference, etc.)

Please visit my Google Scholar for a full list of publications and GitHub for source codes.

news

Feb 20, 2025 :bell: Excited to share our latest work! 🎉: (i) On Zero-Initialized Attention: Optimal Prompt and Gating Factor Estimation – We introduce a Mixture of Experts (MoE) perspective to explain the mechanism behind LLaMA-Adapter’s prompt learning. (ii) MGPath – A novel multi-granular prompt learning method for few-shot WSI pathology prediction, leveraging the power of foundation vision-language models.
Oct 08, 2024 🇨🇭 Start my visiting research at ETH AI Center, ETH Zurich. The topics are about Multi-Modal LLMs for Healthcare empowered by Retrieval-Augmented Generation.
Oct 07, 2024 :bell: Excited to introduce our latest work on medical multi-modal LLMs: LoGra-Med, a novel pre-training algorithm that incorporates multi-graph alignment to effectively address the data-hungry nature of autoregressive learning.
Oct 06, 2024 :rocket: The paper PiToMe has been accepted at NeurIPS 2024. Our code will be available soon!
Jun 10, 2024 :bell: Our new preprint PiToMe is online. We propose a new method to do token merging in the Transformer with spectrum-preserving.

selected publications

  1. NeurIPS
    NeurIPS_2024.jpg
    Accelerating Transformers with Spectrum-Preserving Token Merging
    Hoai-Chau Tran*, Duy MH Nguyen* , Duy M Nguyen, and 7 more authors
    Advances in Neural Information Processing Systems (NeurIPS), 2024
    PDF Code
  2. ICML
    ICML_2024.jpg
    Structure-aware E(3)-invariant molecular conformer aggregation networks
    Duy MH Nguyen, Nina Lukashina , Tai Nguyen, and 7 more authors
    International Conference on Machine Learning (ICML), 2024
    PDF Code
  3. ICLR (Oral)
    ICLR_2024.jpg
    Energy minimizing-based token merging for accelerating Transformers
    Hoai-Chau Tran*, Duy MH Nguyen* , Manh-Duy Nguyen, and 2 more authors
    5th Workshop on practical ML for limited/low resource settings, International Conference on Learning Representations (ICLR), 2024
    PDF
  4. NeurIPS
    NeurIPS_2023.jpg
    LVM-Med: Learning large-scale self-supervised vision models for medical imaging via second-order graph matching
    Duy MH Nguyen , Hoang Nguyen, Nghiem Diep, and 8 more authors
    Advances in Neural Information Processing Systems (NeurIPS), 2023
    PDF Code
  5. AAAI
    AAAI_2023.jpg
    Joint self-supervised image-volume representation learning with intra-inter contrastive clustering
    Duy MH Nguyen , Hoang Nguyen, Truong TN Mai, and 7 more authors
    Proceedings of the AAAI Conference on Artificial Intelligence, 2023
    PDF Code
  6. CVPR
    CVPR_2022.jpg
    LMGP: Lifted multicut meets geometry projections for multi-camera multi-object tracking
    Duy MH Nguyen, Roberto Henschel, Bodo Rosenhahn, and 2 more authors
    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022
    PDF
  7. MedIA
    MIA_2022.jpg
    TATL: Task agnostic transfer learning for skin attributes detection
    Duy MH Nguyen , Thu T Nguyen, Huong Vu, and 4 more authors
    Medical Image Analysis, 2022
    PDF