About Me

I am a third-year Ph.D. student at the University of Maryland, College Park, in the Department of Electrical & Computer Engineering, advised by Prof. Ang Li.
Before that, I obtained my bachelor’s degree at the School of Cyber Science and Engineering, Sichuan University.

I’m always open to collaborations! Feel free to reach out to me at ghsun@umd.edu.

Research Interests

Efficient Foundation Models (LLM / VLM / VLA / WAM)
LLM Privacy and Safety Alignment
Pretraining and Reasoning in LLMs

What’s New

[2026.08] Our work CoIn: Counting the Invisible Reasoning Tokens in Commercial Opaque LLM APIs was accepted at Findings of EMNLP 2026.
[2026.05] Our works Demystifying When Pruning Works via Representation Hierarchies and Invisible Tokens, Visible Bills were accepted at ICML 2026 (the latter in the Position Paper Track).
[2026.04] Our work VeriReason: Reinforcement Learning with Testbench Feedback for Reasoning-Enhanced Verilog Generation was accepted at ACM GLSVLSI 2026.
[2026.04] Our work Arctic-Text2SQL-R1: Simple Rewards, Strong Reasoning in Text-to-SQL was accepted at Findings of ACL 2026.
[2026.03] Our work TOPCELL: Topology Optimization of Standard Cell via LLMs was accepted at DAC 2026.
[2026.02] Our work Uncovering the Redundancy in Transformers via a Unified Study of Layer Dropping was accepted at TMLR.
[2025.11] Our work Invisible Tokens, Visible Bills was selected as one of 9 Orals at the ResponsibleFM Workshop @ NeurIPS 2025.
[2025.10] Our work Enhancing the Security of Large Character Set CAPTCHAs Using Transferable Adversarial Examples was accepted at IEEE TDSC.
[2025.09] Our work SymRTLO: Enhancing RTL Code Optimization with LLMs and Neuron-Inspired Symbolic Reasoning was accepted at NeurIPS 2025.
[2025.05] Started my research internship at Snowflake AI Research, working with Zhewei Yao & Yuxiong He on reinforcement learning for Text-to-SQL.
[2025.05] I am very honored to receive the Qualcomm Innovation Fellowship together with Shwai. Many thanks to Qualcomm for supporting our research on improving the efficiency of the transformer architecture.
[2025.05] Our model Arctic-Text2SQL-R1-32B achieved Top 1 on the BIRD-SQL Leaderboard.
[2024.11] Our work Model-GLUE: Democratized LLM Scaling for A Large Model Zoo in the Wild was accepted at NeurIPS 2024 (Datasets & Benchmarks Track).
[2024.11] Our works SHED and Flora were accepted at NeurIPS 2024.

Publications

* denotes equal contribution † denotes equal supervision

ROCKET: Residual-Oriented Multi-Layer Alignment for Spatially-Aware Vision-Language-Action Models

G. Sun, T. Du, K. Feng, C. Luo, X. Ding, Z. Shen, Z. Wang, Y. He, A. Li
arXiv preprint 2026 [Link]

Drop-Then-Recovery: How Redundant Are Vision-Language-Action Models?

G. Sun, K. Feng, S. He, X. Gong, Y. He, Z. Wang, Z. Shen, W. Ye, R. R. Kompella, G. Liu, A. Li
arXiv preprint 2026 [Link]

Vision-Language-Action in Robotics: A Survey of Datasets, Benchmarks, and Data Engines

Z. Wang, B. Wang, H. Zhang, T. Du, T. Chen, G. Sun, Y. He, Z. Shen, W. Ye, A. Li
Transactions on Machine Learning Research (TMLR) 2026 [Link] [OpenReview] [Code]

CoIn: Counting the Invisible Reasoning Tokens in Commercial Opaque LLM APIs

G. Sun, Z. Wang, B. Tian, M. Liu, Z. Shen, S. He, Y. He, W. Ye, Y. Wang, A. Li
Findings of the Association for Computational Linguistics: EMNLP 2026 [Link]

Position: Invisible Tokens, Visible Bills: The Urgent Need to Audit Hidden Operations in Opaque LLM Services

G. Sun*, Z. Wang*, X. Zhao, B. Tian, Z. Shen, Y. He, J. Xing, A. Li
ICML 2026 (Position Paper Track) | ResponsibleFM Workshop @ NeurIPS 2025 (Oral) [Link] [OpenReview]

Predictive Auditing of Hidden Tokens in LLM APIs via Reasoning Length Estimation

Z. Wang*, G. Sun*, Y. He, Z. Shen, B. Tian, A. Li
arXiv preprint 2025 [Link]

Demystifying When Pruning Works via Representation Hierarchies

S. He, G. Sun, H. Zhang, Y. Fu, A. Li
ICML 2026 [Link]

Uncovering the Redundancy in Transformers via a Unified Study of Layer Dropping

S. He*, G. Sun*, Z. Shen, A. Li
Transactions on Machine Learning Research (TMLR) 2026 [Link] [OpenReview] [Code]
(Early version: What Matters in Transformers? Not All Attention is Needed)

Enhancing the Security of Large Character Set CAPTCHAs Using Transferable Adversarial Examples

G. Sun*, Y. Fu*, H. Yang, J. Huang, R. Zhang, H. Wang
IEEE Transactions on Dependable and Secure Computing (TDSC) 2025 [Link]

VeriReason: Reinforcement Learning with Testbench Feedback for Reasoning-Enhanced Verilog Generation

Y. Wang*, G. Sun*, W. Ye, G. Qu, A. Li
ACM Great Lakes Symposium on VLSI (GLSVLSI) 2026 [Link]

Arctic-Text2SQL-R1: Simple Rewards, Strong Reasoning in Text-to-SQL

Z. Yao*, G. Sun*, L. Borchmann, Z. Shen, M. Deng, B. Zhai, H. Zhang, A. Li, Y. He
Findings of the Association for Computational Linguistics: ACL 2026 [Link] [Code]

TOPCELL: Topology Optimization of Standard Cell via LLMs

Z. Song, Y. Liu, C. Chen, G. Sun, J. Yin, C. Ho, A. Li, H. Ren, C. Yu
63rd ACM/IEEE Design Automation Conference (DAC) 2026 [Link]

SymRTLO: Enhancing RTL Code Optimization with LLMs and Neuron-Inspired Symbolic Reasoning

Y. Wang, W. Ye, P. Guo, Y. He, Z. Wang, B. Tian, S. He, G. Sun, Z. Shen, S. Chen, A. Srivastava, Q. Zhang, G. Qu, A. Li
NeurIPS 2025 [Link]

EdgeLoRA: An Efficient Multi-Tenant LLM Serving System on Edge Devices

Z. Shen, Y. He, Z. Wang, Y. Zhang, G. Sun, W. Ye, A. Li
23rd ACM International Conference on Mobile Systems, Applications, and Services (MobiSys) 2025 [Link]

Router-Tuning: A Simple and Effective Approach for Enabling Dynamic-Depth in Transformers

S. He, T. Ge, G. Sun, B. Tian, X. Wang, D. Yu
EMNLP 2025 (Main) [Link]

Model-GLUE: Democratized LLM Scaling for A Large Model Zoo in the Wild

X. Zhao*, G. Sun*, R. Cai*, Y. Zhou*, P. Li*, P. Wang, B. Tan, Y. He, L. Chen, Y. Liang, B. Chen, B. Yuan, H. Wang†, A. Li†, Z. Wang†, T. Chen†
NeurIPS 2024 (Datasets & Benchmarks) [Link]

Flora: Federated Fine-tuning Large Language Models with Heterogeneous Low-Rank Adaptations

Z. Wang, Z. Shen, Y. He, G. Sun, H. Wang, L. Lyu, A. Li
NeurIPS 2024 [Link]

Y. He, Z. Wang, Z. Shen, G. Sun, Y. Dai, Y. Wu, H. Wang, A. Li
NeurIPS 2024 [Link]

Awards

Qualcomm Innovation Fellowship, Qualcomm, 2025
Dean’s Fellowship, University of Maryland, 2024
National Scholarship, Ministry of Education of China, 2022

Guoheng Sun