Wenxuan Wang   Ph.D. Student ARISE Lab Department of Computer Science and Engineering The Chinese University of Hong Kong Hong Kong SAR, China. E-mail: wxwang[AT]cse.cuhk.edu.hk [Curriculum Vitae][Google Scholar]

Biography

I received my Ph.D degree from the Department of Computer Science and Engineering at The Chinese University of Hong Kong (CUHK), advised by Prof. Michael R. Lyu . I was a visiting Ph.D. student at UCLA, working with Prof. Nanyun Violet Peng . I also work closely with Dr. Zhaopeng Tu from Tencent AI Lab and Prof. Pinjia He from CUHK(SZ). I received my B.E. degree from the Department of Computer Science and Technology, Huazhong University of Science and Technology (HUST).

Research Interests

The reliability and safety of (multi-modal) large language models, agents, and their applications in critical domains, such as medicine, psychology, and code intelligence.

Selected Publications

Preprint

• Wenxuan Wang, Kuiyi Gao, Zihan Jia, Youliang Yuan, Jen-tse Huang, Shuai Wang, Wenxiang Jiao and Zhaopeng Tu.
  Chain-of-Jailbreak Attack for Image Generation Models via Editing Step by Step.
  Preprint
• Wenxuan Wang, Juluan Shi, Chaozheng Wang, Cheryl Lee, Youliang Yuan, Jen-tse Huang and Michael R. Lyu.
  Learning to Ask: When LLMs Meet Unclear Instruction.
  Preprint
• Jie Liu*, Wenxuan Wang*, Zizhan Ma, Guolin Huang, Yihang SU, Kao-Jung Chang, Wenting Chen, Haoliang Li, Linlin Shen, Michael Lyu.
  Medchain: Bridging the Gap Between LLM Agents and Clinical Practice through Interactive Sequential Benchmarking.
  Preprint
• Jie Liu*, Wenxuan Wang*, Yihang Su, Jingyuan Huan, Wenting Chen, Yudi Zhang, Cheng-Yi Li, Kao-Jung Chang, Xiaohan Xin, Linlin Shen, and Michael R. Lyu.
  A Spectrum Evaluation Benchmark for Medical Multi-Modal Large Language Models.
  Preprint
• Wenxuan Wang, Juluan Shi, Zhaopeng Tu, Youliang Yuan, Jen-tse Huang, Wenxiang Jiao and Michael R. Lyu.
  The Earth is Flat? Unveiling Factual Errors in Large Language Models.
  Preprint

2024

• Yuxuan Wan*, Wenxuan Wang*, Wenxiang Jiao, Yiliu Yang, Youliang Yuan, Jen-tse Huang, Pinjia He and Michael R. Lyu.
  LogicAsker: Evaluating and Improving the Logical Reasoning Ability of Large Language Models.
  EMNLP 2024: The 2024 Conference on Empirical Methods in Natural Language Processing.
• Wenxuan Wang, Haonan Bai, Jen-tse Huang, Jingyuan Huang, Haoyi Qiu, Nanyun Violet Peng, and Michael R. Lyu.
  New Job, New Gender? Measuring the Social Bias in Image Generation Models.
  ACM MM 2024 (Oral): ACM Multimedia 2024.
• Wenxuan Wang, Wenxiang Jiao, Jingyuan Huang, Ruyi Dai, Jen-tse Huang, Zhaopeng Tu and Michael R. Lyu.
  Not All Countries Celebrate Thanksgiving: On the Cultural Dominance in Large Language Models.
  ACL 2024: 62th Annual Meeting of the Association for Computational Linguistics.
• Wenxuan Wang, Zhaopeng Tu, Chang Chen, Youliang Yuan, Jen-tse Huang, Wenxiang Jiao and Michael R. Lyu.
  All Languages Matter: On the Multilingual Safety of Large Language Models.
  ACL 2024 Findings: Findings of 62th Annual Meeting of the Association for Computational Linguistics.
• Wenxuan Wang, Wenxiang Jiao, Shuo Wang, Zhaopeng Tu and Michael R. Lyu.
  Understanding and Mitigating the Uncertainty in Zero-Shot Translation.
  TASLP 2024: The IEEE/ACM Transactions on Audio, Speech, and Language Processing.
• Jen-tse Huang, Man Ho Lam, Eric John Li, Shujie Ren, Wenxuan Wang (Corresponding), Wenxiang Jiao, and Michael R. Lyu.
  Apathetic or Empathetic? Evaluating LLMs' Emotional Alignments with Humans.
  NeurIPS 2024: The Thirty-Eighth Annual Conference on Neural Information Processing Systems.
• Jen-tse Huang, Wenxiang Jiao, Man Ho Lam, Eric John Li, Wenxuan Wang (Corresponding) and Michael R. Lyu.
  On the Reliability of Psychological Scales on Large Language Models.
  EMNLP 2024: The 2024 Conference on Empirical Methods in Natural Language Processing.
• Jen-tse Huang, Wenxuan Wang, Eric John Li, Man Ho Lam, Shujie Ren, Youliang Yuan, Wenxiang Jiao, Zhaopeng Tu, Michael R Lyu.
  Who is ChatGPT? Benchmarking LLMs' Psychological Portrayal Using PsychoBench.
  ICLR 2024 (Oral): Twelfth International Conference on Learning Representations.
• Youliang Yuan, Wenxiang Jiao, Wenxuan Wang, Jen-tse Huang, Pinjia He, Shuming Shi, Zhaopeng Tu.
  Gpt-4 Is Too Smart To Be Safe: Stealthy Chat With LLMs Via Cipher.
  ICLR 2024: Twelfth International Conference on Learning Representations.
• Kunyu Wang, Xuanran He, Wenxuan Wang, Xiaosen Wang.
  Boosting Adversarial Transferability by Block Shuffle and Rotation.
  CVPR 2024: IEEE Conference on Computer Vision and Pattern Recognition.
• Chaozheng Wang, Shuzheng Gao, Cuiyun Gao, Wenxuan Wang, Chun Yong Chong, Shan Gao, Michael R. Lyu.
  A Systematic Evaluation of Large Code Models in API Suggestion: When, Which, and How.
  ASE 2024: The 39th IEEE/ACM International Conference on Automated Software Engineering.
• Youliang Yuan, Wenxuan Wang, Qingshuo Guo, Yiming Xiong, Chihao Shen, Pinjia He.
  Does ChatGPT Know that It Does Not Know? Evaluating the Black-Box Calibration of ChatGPT.
  COLING 2024 (Oral): International Conference on Computational Linguistics.

2023

• Wenxuan Wang, Jingyuan Huang, Jen-tse Huang, Chang Chen, Pinjia He, Jiazhen Gu and Michael R. Lyu.
  A Picture is Worth a Thousand Toxic Words: A Metamorphic Testing Framework for Content Moderation Software.
  ASE 2023: The 38th IEEE/ACM International Conference on Automated Software Engineering.
• Yuxuan Wan*, Wenxuan Wang*, Pinjia He, Jiazhen Gu, Haonan Bai and Michael R. Lyu.
  BiasAsker: Measuring the Bias in Conversational AI System.
  FSE 2023: The ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering.
• Wenxuan Wang, Jingyuan Huang, Chang Chen, Jiazhen Gu, Jianping Zhang, Weibin Wu, Pinjia He and Michael R. Lyu.
  Validating Multimedia Content Moderation Software via Semantic Fusion.
  ISSTA 2023: The 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis.
• Wenxuan Wang, Jen-tse Huang, Weibin Wu, Jianping Zhang, Yizhan Huang, Shuqing Li, Pinjia He and Michael R. Lyu.
  MTTM: Metamorphic Testing for Textual Content Moderation Software.
  ICSE 2023: 2023 International Conference on Software Engineering.
• Yun Peng, Chaozheng Wang, Wenxuan Wang, Cuiyun Gao and Michael R. Lyu.
  Generative Type Inference for Python.
  ASE 2023 (ACM Distinguished Paper Award): The 38th IEEE/ACM International Conference on Automated Software Engineering.
• Shuzheng Gao, Xin-cheng Wen, Cuiyun Gao, Wenxuan Wang and Michael R. Lyu.
  Constructing Effective In‐Context Demonstration for Code Intelligence Tasks: An Empirical Study.
  ASE 2023: The 38th IEEE/ACM International Conference on Automated Software Engineering.
• Wenxiang Jiao, Jen-tse Huang, Wenxuan Wang, Xing Wang, Shuming Shi and Zhaopeng Tu.
  Parrot: Translating During Chat Using Large Language Models.
  EMNLP 2023 Findings: The 2023 Conference on Empirical Methods in Natural Language Processing.
• Jianping Zhang, Jen-tse Huang, Wenxuan Wang, Weibin Wu, Xiaosen Wang, Yuxin Su and Michael R. Lyu.
  Improving the Transferability of Adversarial Samples by Path‐Augmented Method.
  CVPR 2023: The 2023 IEEE/CVF Computer Vision and Pattern Recognition Conference.

2022

• Wenxuan Wang, Wenxiang Jiao, Yongchang Hao, Xing Wang, Shuming Shi, Zhaopeng Tu and Michael R. Lyu.
  Understanding and Improving Sequence‐to‐Sequence Pretraining for Neural Machine Translation.
  ACL 2022: 60th Annual Meeting of the Association for Computational Linguistics.
• Jianping Zhang, Weibin Wu, Jen-tse Huang, Yizhan Huang, Wenxuan Wang, Yuxin Su and Michael R. Lyu.
  Improving Adversarial Transferability via Neuron Attribution‐Based Attacks.
  CVPR 2022: The 2022 IEEE/CVF Computer Vision and Pattern Recognition Conference.

2020

• Wenxuan Wang and Zhaopeng Tu.
  Rethinking the Value of Transformer Components.
  COLING 2020: Proceedings of The 28th International Conference on Computational Linguistics.
• Dabiao Ma*, Zhiba Su*, Wenxuan Wang* and Yuhao Lu.
  FPETS: Fully Parallel End-to-End Text-to-Speech System..
  AAAI 2020 (Oral): Proceedings of The Thirty-Fourth AAAI Conference on Artificial Intelligence.

Experiences

• Visiting Student: UCLA NLP Group.
  
  07.2023-02.2024
  It's my pleasure to work with my host professor, Nanyun Violet Peng, on the Evaluation of Multimodal Large Models.
• Research Intern: Tencent AI Lab.
  
  07.2019-07.2023
  It's my pleasure to work with my mentor, Zhaopeng Tu, on Neural Machine Translation.
• Research Engineer: Turing Robot.
  
  07.2017-08.2018
  Working on Multi-modal Emotion Recognition and Text-to-Speech Synthesis..