|
Junlin Yang Hi, I'm Junlin Yang, a third-year student in Department of Computer Science and Technology at Tsinghua University, . Currently, I am fortunate to be a research intern at the XLANG Lab at The University of Hong Kong, under the guidance of Prof. Tao Yu. In the past, I've had the privilege of interning at the Tsinghua Pervasive HCI Group, where I was advised with Prof. Chun Yu and Prof. Yuanchun Shi. I am actively seeking PhD opportunities for 2026 Fall. Feel free to reach out if you're interested in my research, looking for collaboration, or just want to chat! |
|
ResearchI am particularly interested in Machine Learning and Human-Computer Interaction, with a focus on NLP (especially Language Grounding), Multimodal Learning, Neuro-Symbolic Concepts&Thinkings and Reinforcement Learning. My recent research focuses on building embodied agents, especially computer agents, that can excel in solving human tasks and collaborating effectively with people. I remain curious and open to exploring various research questions in ML and HCI. |
|
AgentNet: Scaling Multimodal
Computer Agent Trajectories Data via Human
Demonstration
2024.7 - present Co-lead Advised by Prof. Tao Yu, The University of Hong Kong We introduce AgentNet, a diverse dataset addressing the limitations of current GUI Agent datasets by focusing on long-horizon tasks, real-world usage scenarios, and application diversity. Using an efficient data collection system, crowdsourcing, and verification, we gathered tens of thousands of long trajectories and analyzed them to gain insights into human-generated GUI data. This dataset enabled us to train Vision-Language Models (VLMs), enhancing their capabilities as computer-use agents and exploring their scaling laws. |
|
EchoMind: Enhancing Group
Discussions through Human-AI Collaborative Issue
Mapping
2023.10 - 2024.9 Advised by Prof. Chun Yu and Prof. Yuanchun Shi, Tsinghua University EchoMind addresses the challenge of unproductive group discussions by aiding facilitators in tracking and structuring conversations. Using a collaborative system powered by Large Language Models (LLMs), it visualizes discussion knowledge in real-time through issue mapping. User studies show that EchoMind improves clarity of objectives and enhances discussion productivity. |
Projects |
|
OSWorld: Benchmarking Multimodal
Agents for Open-Ended Tasks in Real Computer
Environments
2024.8 - present Maintainer OSWorld is a first-of-its-kind scalable, real computer environment for multimodal agents, supporting task setup, execution-based evaluation, and interactive learning across operating systems. It can serve as a unified environment for evaluating open-ended computer tasks that involve arbitrary apps (e.g., task examples in the above Fig). We also create a benchmark of 369 real-world computer tasks in OSWorld with reliable, reproducible setup and evaluation scripts. |
|
MartialArtsLM: Pretraining and
Fine-tuning a Model Capable of Answering Questions about
Martial Arts Novel
2023.8-2023.9 Individual Project Pretrained the LM(Language Model) using preprocessed data from novels by Louis Cha, then SFT the LM with an estimate of 400,000 pieces of synthesized Q\&A data. |
Selected Awards and Honors |
|
Languages |
|
Service and Leadership |
|
|
I founded the first alumni mutual-help platform in my high school. Our platform aims to bridge the information gap for high school students from diverse economic, familial, and cognitive backgrounds by providing them with comprehensive insights into university academic life. By sharing experiences, offering guidance, and fostering a supportive community, the platform has become a vital resource for students who might otherwise lack access to such information. To date, it has received over 50k reads and attracted more than 2.5k followers. |
Miscellanea |
Senior Mentors |
|
At Tsinghua, I was fortunate to meet some incredibly kind, talented, and supportive seniors, including Yuxuan Li and Zirui Cheng. During my internship at HKU, I was lucky to work closely with Tianbao Xie and Yiheng Xu. I'm also grateful to have collaborated with Xinyuan Wang and Bowen Wang on projects like AgentNet. |
Hobbies |
|
|