Applications of Reinforcement Learning in the Physical Sciences

offered: Winter Semester 2020/21
instructor: Marin Bukov, PhD
time: Sat, 4:30 pm - 6:30 pm, ONLINE
exam: scientific presentation, participation in seminar talks
credits: 3 ECTS

List of Talks and Speakers

	Title	Speaker	Date	Time	Slides
0.	Breakthrough Applications of Reinforcement Learning in Science	M. Bukov	17/10	4:30pm
1.	Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collection	N. Pashov	7/11	4:30pm	pdf
2.	Dota 2 with Large Scale Deep Reinforcement Learning	A. Mitsov	7/11	5:30pm	pdf
3.	A Deep Reinforcement Learning Approach for Automated Cryptocurrency Trading	S. Gargova	14/11	4:30pm	pdf
4.	Reinforcement Learning with Neural Networks for Quantum Feedback	Z. Abdrahim	21/11	4:30pm	pdf
5.	Solving the Rubik’s cube with approximate policy iteration	S. Petrov	28/11	4:30pm	pdf
6.	Setting up experimental Bell test with reinforcement learning	A. Kirkova	5/12	4:30pm	pdf
7.	Sky surveys scheduling using reinforcement learning	S. Stefanov	12/12	4:30pm	pdf
8.	Mastering the game of Go without human knowledge	D. Davidkov	12/12	5:30pm	pdf
9.	Fidelity-Based Probabilistic Q-Learning for Control of Quantum Systems	H. Tonchev	19/12	4:30pm	pdf
10.	Creation and manipulation of quantized vortices in Bose-Einstein condensates using reinforcement learning	B. Ilik	9/1	4:30pm	pdf

List of possible papers to present (students should feel free to present any paper of their own choice, but check with the instructor first):

• Breaking Adiabatic Quantum Control with Deep Learning
• Global optimization of quantum dynamics with AlphaZero deep exploration
• Fidelity-Based Probabilistic Q-Learning for Control of Quantum Systems
• Reinforcement Learning with Neural Networks for Quantum Feedback
• Coherent transport of quantum states by deep reinforcement learning
• Reinforcement Learning Decoders for Fault-Tolerant Quantum Computation
• Reinforcement Learning in Different Phases of Quantum Control

• Universal quantum control through deep reinforcement learning

• Branes with brains: exploring string vacua with deep reinforcement learning

• Flow Navigation by Smart Microswimmers via Reinforcement Learning

• Learning to soar in turbulent environments

• Mastering the game of Go without human knowledge

• Playing Atari with Deep Reinforcement Learning

• Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collection

• Deep Reinforcement Learning for Robotic Manipulation with Asynchronous Off-Policy Updates

• Proximal Policy Optimization Algorithms

• Trust Region Policy Optimization

• Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Review

Description

This special seminar provides an introduction to applications of Reinforcement Learning (RL) in the physical sciences. Students will perform a scientific literature survey and prepare a scientific talk to present to the audience of the seminar. In particular, we will discuss seminal applications of RL in physics. The topics can be chosen according to the students’ interests: e.g., successful navigation of turbulent flows, preparation of the states of quantum bits (qubits), identification of noise-robust quantum channels required for quantum computing, exploring the vast space of string vacua. Should there be interest by a student, we will also cover important RL publications in the STEM (science, technology, engineering, mathematics) fields. The seminar is suitable for advanced bachelor students, master students, and PhD students, from all STEM fields.

Check out the accompanying lecture course worth 6 ECTS.

Prerequisites

• math prerequisites: linear algebra, analysis in many variables, basic concepts of probability theory. Proficiency in deep learning is not required but can be very useful.

• physics prerequisites: general physics, basic statistical physics and basic quantum mechanics knowledge will be helpful for understanding the examples we will discuss.

• language: seminar presentations will be in English.
  
  

Supplementary Literature

Students can download research papers for free from the arXiv. Additionally, you may want to check these resources:

• Sutton and Barto, Reinforcement Learning: an Introduction, 2nd Ed., MIT Press, 2018

• YouTube lecture videos of Sergey Levine’s Deep RL class at UC Berkeley

• P. Mehta, et al., A High-Bias, Low-Variance Introduction to Machine Learning for Physicists

• M. Nielsen, Neural Networks and Deep Learning, online book

Final Grade

Students are required to prepare a scientific presentation of a research paper or topic. Topics / papers will be provided by the instructor, but students are encouraged to make suggestions based on their own interests. The student will be guided by the instructor to prepare the presentation. The presentation should be given during the semester; it will be evaluated and, together with participation in the seminar, they form the final grade for the seminar.