Hi, I am currently a fourth-year undergraduate student at Sun Yat-sen University, majoring in Artificial Intelligence. My research interests include but are not limited to attention computation theory, large language models, AI security, and learning theory.
📝 Publications / Preprints (Author names in alphabetical order)
- Timothy Chu, Zhao Song, and Chiwun Yang. How to protect copyright data in optimization of large language models? AAAI 2024 Poster, arXiv preprint arXiv:2308.12247
- Timothy Chu, Zhao Song, and Chiwun Yang. Fine-tune language models to approximate unbiased in-context learning. arXiv preprint arXiv:2310.03331, 2023
- Zhao Song and Chiwun Yang. An automatic learning rate schedule algorithm for achieving faster convergence and steeper descent. arXiv preprint arXiv:2310.11291, 2023
- Yichuan Deng, Zhao Song, Shenghao Xie, and Chiwun Yang. Unmasking transformers: A theoretical approach to data recovery via attention weights. arXiv preprint arXiv:2310.12462, 2023
- Chengyang Li, Zhao Song, Weixin Wang, and Chiwun Yang. A theoretical insight into attack and defense of gradient leakage in transformer. arXiv preprint arXiv:2311.13624, 2023
- Raghav Addanki, Chenyang Li, Zhao Song, and Chiwun Yang. One pass streaming algorithm for super long token attention approximation in sublinear space. arXiv preprint arXiv:2311.14652, 2023
- Yichuan Deng, Zhao Song, and Chiwun Yang. Enhancing stochastic gradient descent: A unified framework and novel acceleration methods for faster convergence. arXiv preprint arXiv:2402.01515, 2024
- Yichuan Deng, Zhao Song, and Chiwun Yang. Attention is naturally sparse with gaussian distributed input. arXiv preprint arXiv:2404.02690, 2024
- Yingyu Liang, Zhenmei Shi, Zhao Song, and Chiwun Yang. Towards infinite-long prefix in transformer. arXiv preprint arXiv:2406.14036, 2024
📖 Educations
- 2021.09 - 2025.06, School of Artificial Intelligence, Sun Yat-sen University.
💻 Work Experience
- 2022.08 - 2024.02, Research Assistant, Shenzhen Institute of Artificial Intelligence and Robotics for Society, advised by Dr. Nan Li
- 2024.10 - current, Research Intern, NGai’s Lab, The University of Hong Kong (HKU), advised by Jing Xiong.
📊 Service
- ICLR 2025