Applications of Reinforcement Learning in the Physical Sciences
offered: Winter Semester 2020/21
instructor: Marin Bukov, PhD
time: Sat, 4:30 pm  6:30 pm, ONLINE
exam: scientific presentation, participation in seminar talks
credits: 3 ECTS
List of Talks and Speakers
List of possible papers to present (students should feel free to present any paper of their own choice, but check with the instructor first):
 Breaking Adiabatic Quantum Control with Deep Learning
 Global optimization of quantum dynamics with AlphaZero deep exploration
 FidelityBased Probabilistic QLearning for Control of Quantum Systems
 Reinforcement Learning with Neural Networks for Quantum Feedback
 Coherent transport of quantum states by deep reinforcement learning
 Reinforcement Learning Decoders for FaultTolerant Quantum Computation
 Reinforcement Learning in Different Phases of Quantum Control

Universal quantum control through deep reinforcement learning

Branes with brains: exploring string vacua with deep reinforcement learning
 Flow Navigation by Smart Microswimmers via Reinforcement Learning
 Mastering the game of Go without human knowledge
 Learning handeye coordination for robotic grasping with deep learning and largescale data collection

Deep Reinforcement Learning for Robotic Manipulation with Asynchronous OffPolicy Updates
 Proximal Policy Optimization Algorithms
 Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Review
Description
This special seminar provides an introduction to applications of Reinforcement Learning (RL) in the physical sciences. Students will perform a scientific literature survey and prepare a scientific talk to present to the audience of the seminar. In particular, we will discuss seminal applications of RL in physics. The topics can be chosen according to the students’ interests: e.g., successful navigation of turbulent flows, preparation of the states of quantum bits (qubits), identification of noiserobust quantum channels required for quantum computing, exploring the vast space of string vacua.
Should there be interest by a student, we will also cover important RL publications in the STEM (science, technology, engineering, mathematics) fields.
The seminar is suitable for advanced bachelor students, master students, and PhD students, from all STEM fields.
Check out the accompanying lecture course worth 6 ECTS.
Prerequisites

math prerequisites: linear algebra, analysis in many variables, basic concepts of probability theory. Proficiency in deep learning is not required but can be very useful.

physics prerequisites: general physics, basic statistical physics and basic quantum mechanics knowledge will be helpful for understanding the examples we will discuss.

language: seminar presentations will be in English.
Supplementary Literature
Students can download research papers for free from the arXiv. Additionally, you may want to check these resources:

Sutton and Barto, Reinforcement Learning: an Introduction, 2nd Ed., MIT Press, 2018

YouTube lecture videos of Sergey Levine’s Deep RL class at UC Berkeley

P. Mehta, et al., A HighBias, LowVariance Introduction to Machine Learning for Physicists
Final Grade
Students are required to prepare a scientific presentation of a research paper or topic. Topics / papers will be provided by the instructor, but students are encouraged to make suggestions based on their own interests. The student will be guided by the instructor to prepare the presentation. The presentation should be given during the semester; it will be evaluated and, together with participation in the seminar, they form the final grade for the seminar.