About Me

I am a second-year Ph.D. student at the University of Maryland, College Park, in the Department of Electrical & Computer Engineering, advised by Prof. Ang Li.
Before that, I obtained my bachelor’s degree at the School of Cyber Science and Engineering, Sichuan University.

I’m always open to collaborations! Feel free to reach out to me at ghsun@umd.edu.

Research Interests

Efficient LLMs
LLM Privacy and Safety Alignment
Pretraining and Reasoning in LLMs

What’s New

[2025.08] Started my research internship at Snowflake AI Research, working with Zhewei Yao & Yuxiong He on reinforcement learning for Text-to-SQL.
[2025.05] I am very honored to receive the Qualcomm Innovation Fellowship together with Shwai. Many thanks to Qualcomm for supporting our research on improving the efficiency of the transformer architecture.
[2025.05] Our model Arctic-Text2SQL-R1-32B achieved Top 1 on the BIRD-SQL Leaderboard.
[2024.11] Our work Model-GLUE was accepted at NeurIPS 2024 (Datasets & Benchmarks Track).
[2024.11] Our work SHED and Flora were accepted at NeurIPS 2024.

Publications

CoIn: Counting the Invisible Reasoning Tokens in Commercial Opaque LLM APIs

G. Sun, Z. Wang, B. Tian, M. Liu, Z. Shen, S. He, Y. He, W. Ye, Y. Wang, A. Li
arXiv preprint 2025 [Link]

VeriReason: Reinforcement Learning with Testbench Feedback for Reasoning-Enhanced Verilog Generation

Y. Wang*, G. Sun*, W. Ye, G. Qu, A. Li
arXiv preprint 2025 [Link]

Invisible Tokens, Visible Bills: The Urgent Need to Audit Hidden Operations in Opaque LLM Services

G. Sun*, Z. Wang*, X. Zhao, B. Tian, Z. Shen, Y. He, J. Xing, A. Li
arXiv preprint 2025 [Link]

Arctic-Text2SQL-R1: Simple Rewards, Strong Reasoning in Text-to-SQL

Z. Yao, G. Sun, L. Borchmann, Z. Shen, M. Deng, B. Zhai, H. Zhang, A. Li, Y. He
arXiv preprint 2025 [Link] [Code]

What Matters in Transformers? Not All Attention is Needed

S. He*, G. Sun*, Z. Shen, A. Li
arXiv preprint 2024 [Link] [Code]

Model-GLUE: Democratized LLM Scaling for A Large Model Zoo in the Wild

X. Zhao*, G. Sun*, R. Cai*, Y. Zhou*, P. Li*, P. Wang, B. Tan, Y. He, L. Chen, Y. Liang, B. Chen, B. Yuan, H. Wang†, A. Li†, Z. Wang†, T. Chen†
NeurIPS 2024 (Datasets & Benchmarks) [Link]

Router-Tuning: A Simple and Effective Approach for Enabling Dynamic-Depth in Transformers

S. He, T. Ge, G. Sun, et al.
EMNLP 2025 (Main) [Link]

Flora: Federated Fine-tuning Large Language Models with Heterogeneous Low-Rank Adaptations

Z. Wang, Z. Shen, Y. He, G. Sun, et al.
NeurIPS 2024 [Link]

Y. He, Z. Wang, Z. Shen, G. Sun, et al.
NeurIPS 2024 [Link]

Awards

Qualcomm Innovation Fellowship, Qualcomm, 2025
Dean’s Fellowship, University of Maryland, 2024
National Scholarship, Ministry of Education of China, 2022
First-class University Annual Scholarship, Sichuan University, 2022 & 2023
Outstanding Undergraduate Graduate, Sichuan University, 2023
Outstanding Student, Sichuan University, 2021 & 2022
National 1st Prize, China Internet+ Innovation and Entrepreneurship Competition, 2022
National 3rd Prize, China Software Cup, 2022

Guoheng Sun