没有合适的资源？快使用搜索试试~ 我知道了~

文库首页人工智能深度学习An Introduction to Deep Reinforcement Learning

An Introduction to Deep Reinforcement Learning

强化学习

深度学习

需积分: 10 30 下载量 50 浏览量 2018-12-26 13:39:29 上传评论收藏 2.46MB PDF 举报

温馨提示

试读

140页

介绍深度强化学习的教材，非常实用。摘要：Deep reinforcement learning is the combination of reinforce- ment learning (RL) and deep learning. This field of research has been able to solve a wide range of complex decision- making tasks that were previously out of reach for a machine. Thus, deep RL opens up many new applications in domains such as healthcare, robotics, smart grids, finance, and many more. This manuscript provides an introduction to deep reinforcement learning models, algorithms and techniques. Particular focus is on the aspects related to generalization and how deep RL can be used for practical applications. We assume the reader is familiar with basic machine learning concepts.

资源推荐

资源详情

资源评论

强化学习导论中文版增强学习导论中文版 Reinforcement learning an introduction 中文版.

5星 · 资源好评率100%

增强学习导论强化学习导论 Reinforcement learning an introduction 中文版

Deep Reinforcement Learning In Action-CH1, pdf+code,by Alexander Zai(亚马逊工程师)，后续会继续更新。

Deep learning PPT

5星 · 资源好评率100%

CVPR ppt Deep learning 必备资料·真的是特别好，忍不住共享了~

An Introduction to Deep

Reinforcement Learning

Vincent François-Lavet, Peter Henderson, Riashat Islam, Marc G. Bellemare and Joelle

Pineau (2018), “An Introduction to Deep Reinforcement Learning”, Foundations and

Trends in Machine Learning: Vol. 11, No. 3-4. DOI: 10.1561/2200000071.

Vincent François-Lavet

McGill University

vincent.francois-lavet@mcgill.ca

Peter Henderson

McGill University

peter.henderson@mail.mcgill.ca

Riashat Islam

McGill University

riashat.islam@mail.mcgill.ca

Marc G. Bellemare

Google Brain

bellemare@go ogle.com

Joelle Pineau

Faceb ook, McGill University

jpineau@cs.mcgill.ca

Boston — Delft

arXiv:1811.12560v2 [cs.LG] 3 Dec 2018

Contents

1 Introduction 2

1.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . 2

1.2 Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

2 Machine learning and deep learning 6

2.1 Supervised learning and the concepts of bias and overﬁtting 7

2.2 Unsupervised learning . . . . . . . . . . . . . . . . . . . . 9

2.3 The deep learning approach . . . . . . . . . . . . . . . . . 10

3 Introduction to reinforcement learning 15

3.1 Formal framework . . . . . . . . . . . . . . . . . . . . . . 16

3.2 Diﬀerent components to learn a policy . . . . . . . . . . . 20

3.3 Diﬀerent settings to learn a policy from data . . . . . . . . 21

4 Value-based methods for deep RL 24

4.1 Q-learning . . . . . . . . . . . . . . . . . . . . . . . . . . 24

4.2 Fitted Q-learning . . . . . . . . . . . . . . . . . . . . . . 25

4.3 Deep Q-networks . . . . . . . . . . . . . . . . . . . . . . 27

4.4 Double DQN . . . . . . . . . . . . . . . . . . . . . . . . . 28

4.5 Dueling network architecture . . . . . . . . . . . . . . . . 29

4.6 Distributional DQN . . . . . . . . . . . . . . . . . . . . . 31

4.7 Multi-step learning . . . . . . . . . . . . . . . . . . . . . . 32

4.8

Combination of all DQN improvements and variants of DQN

5 Policy gradient methods for deep RL 36

5.1 Stochastic Policy Gradient . . . . . . . . . . . . . . . . . 37

5.2 Deterministic Policy Gradient . . . . . . . . . . . . . . . . 39

5.3 Actor-Critic Methods . . . . . . . . . . . . . . . . . . . . 40

5.4 Natural Policy Gradients . . . . . . . . . . . . . . . . . . 42

5.5 Trust Region Optimization . . . . . . . . . . . . . . . . . 43

5.6 Combining policy gradient and Q-learning . . . . . . . . . 44

6 Model-based methods for deep RL 46

6.1 Pure model-based methods . . . . . . . . . . . . . . . . . 46

6.2 Integrating model-free and model-based methods . . . . . 49

7 The concept of generalization 53

7.1 Feature selection . . . . . . . . . . . . . . . . . . . . . . . 58

7.2

Choice of the learning algorithm and function approximator

selection . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

7.3 Modifying the objective function . . . . . . . . . . . . . . 61

7.4 Hierarchical learning . . . . . . . . . . . . . . . . . . . . . 62

7.5 How to obtain the best bias-overﬁtting tradeoﬀ . . . . . . 63

8 Particular challenges in the online setting 66

8.1 Exploration/Exploitation dilemma . . . . . . . . . . . . . . 66

8.2 Managing experience replay . . . . . . . . . . . . . . . . . 71

9 Benchmarking Deep RL 73

9.1 Benchmark Environments . . . . . . . . . . . . . . . . . . 73

9.2 Best practices to benchmark deep RL . . . . . . . . . . . 78

9.3 Open-source software for Deep RL . . . . . . . . . . . . . 80

10 Deep reinforcement learning beyond MDPs 81

10.1 Partial observability and the distribution of (related) MDPs 81

10.2 Transfer learning . . . . . . . . . . . . . . . . . . . . . . . 86

10.3 Learning without explicit reward function . . . . . . . . . . 89

10.4 Multi-agent systems . . . . . . . . . . . . . . . . . . . . . 91

11 Perspectives on deep reinforcement learning 94

11.1 Successes of deep reinforcement learning . . . . . . . . . . 94

11.2

Challenges of applying reinforcement learning to real-world

problems . . . . . . . . . . . . . . . . . . . . . . . . . . . 95

11.3 Relations between deep RL and neuroscience . . . . . . . . 96

12 Conclusion 99

12.1 Future development of deep RL . . . . . . . . . . . . . . . 99

12.2 Applications and societal impact of deep RL . . . . . . . . 100

Appendices 103

References 106

An Introduction to Deep

Reinforcement Learning

Vincent François-Lavet

, Peter Henderson

, Riashat Islam

, Marc

G. Bellemare

and Joelle Pineau

McGill University; vincent.francois-lavet@mcgill.ca

McGill University; peter.henderson@mail.mcgill.ca

McGill University; riashat.islam@mail.mcgill.ca

Google Brain; bellemare@google.com

Facebook, McGill University; jpineau@cs.mcgill.ca

ABSTRACT

Deep reinforcement learning is the combination of reinforce-

ment learning (RL) and deep learning. This ﬁeld of research

has been able to solve a wide range of complex decision-

making tasks that were previously out of reach for a machine.

Thus, deep RL opens up many new applications in domains

such as healthcare, robotics, smart grids, ﬁnance, and many

more. This manuscript provides an introduction to deep

reinforcement learning models, algorithms and techniques.

Particular focus is on the aspects related to generalization

and how deep RL can be used for practical applications. We

assume the reader is familiar with basic machine learning

concepts.

剩余139页未读，继续阅读

评论收藏

内容反馈

资源评论

资源反馈

评论星级较低，若资源使用遇到问题可联系上传者，3个工作日内问题未解决可申请退款~

江南小白龙

粉丝: 57
资源: 14

上传资源快速赚钱

我的内容管理展开

我的资源快来上传第一个资源

我的收益

登录查看自己的收益

我的积分登录查看自己的积分

我的C币登录后查看C币余额

我的收藏

我的下载

下载帮助

前往需求广场，查看用户热搜

An Introduction to Deep Reinforcement Learning

强化学习导论中文版 增强学习导论中文版 Reinforcement learning an introduction 中文版.

deep reinforcement learning

introduction to deep learning

An Introduction to Deep Reinforcement Learning.pdf

2017强化学习英文最新综述 Deep Reinforcement Learning: An Overview

DEEP REINFORCEMENT LEARNING

Deep Reinforcement Learning In Action-CH1 pdf+code(by Alexander Zai)

Deep learning PPT

An Introduction to Reinforcement Learning

Introduction to Reinforcement Learning

Reinforcement Learning: An Introduction

Reinforcement Learning An Introduction

Deep Reinforcement Learning

强化学习：介绍 Reinforcement Learning: An Introduction

deep_reinforcement_learning

Reinforcement Learning：An Introduction.pdf

Reinforcement Learning An Introduction.pdf

强化学习入门（Introduction to Deep Reinforcement Learning by Shenglin Zhao）

Reinforcement Learning：An Introduction PDF文档+源代码

Reinforcement Learning: An Introduction November 和Deep Learning

Reinforcement Learning: An Introduction最新版习题解答（第一版本）

Reinforcement Learning - An Introduction

最新资源

强化学习导论中文版增强学习导论中文版 Reinforcement learning an introduction 中文版.