Teaching

CS7309 Reinforcement Learning Theory and Applications

Graduate Course, Shanghai Jiao Tong University, 2026

With the rapid advancement of artificial intelligence, machine learning is evolving beyond traditional predictive tasks to encompass more complex decision-making challenges, such as game AI, autonomous driving, and intelligent assistants. Unlike predictive tasks, decision-making tasks require agents not only to understand the current state but also to devise long-term strategies based on feedback from the environment. This demands stronger generalization and adaptive capabilities. Reinforcement Learning, as the core technology driving these capabilities, has attracted significant attention in recent years. In the era of AI powered by large language models, RL is recognized as a critical approach for enhancing model generalization and interaction performance. It holds great promise for the development of more intelligent, flexible AI systems and the advancement toward artificial general intelligence. This course offers a systematic introduction to the fundamental principles and key methodologies of Reinforcement Learning. Covered topics include Markov Decision Processes, Dynamic Programming, Temporal Difference Learning, Policy Gradient Methods, Deep Reinforcement Learning, Imitation Learning, and Multi-Agent Reinforcement Learning. The course emphasizes both theoretical understanding and practical implementation, encouraging students to build and optimize agents through programming to solve real-world problems. By completing this course, students will establish a strong foundation for further exploration at the forefront of artificial intelligence research.

CS7346 Algorithms for Big Data

Graduate Course, Shanghai Jiao Tong University, 2026

This course introduces data mining and machine learning algorithms for analyzing big data, with a focus on various types of algorithms that can handle large-scale data. The course consists of seven chapters: Introduction to MapReduce and Spark, Introduction to Locally Sensitive Hashing Algorithms, Large Scale Machine Learning Algorithms, Data and Matrix Dimensionality Reduction, Graph Algorithms and Community Discovery, Introduction to Recommender Systems, and Temporal and Spatio-Temporal Data Mining. Big data algorithms and analytics refer to the methods or tools used to collect, process, and analyze data from different large data sets. These datasets may come from a variety of sources such as the Web, mobile applications, email, social media, and connected smart devices. They typically represent data generated at high speed in a variety of forms, ranging from structured (database tables, Excel sheets) to semi-structured (XML files, web pages) to unstructured (images, audio files). Therefore, this course covers most of the data types likely to be encountered in Big Data processing tasks in seven chapters, helping students understand how to use algorithms wisely to solve large-scale data problems.