SC²S Colloquium - March 22, 2010

From Sccswiki
Jump to navigation Jump to search
Date: March 22
Room: 02.07.023
Time: 15:00 pm, s.t.

Irene Klompmaker: Sparse Grids in Reinforcement Learning and Optimal Control

Reinforcement Learning (RL) is an aspect of machine learning, where the underlying problem is that of an agent that must learn behaviour in order to achieve a specific aim. The agent learns through trial-and-error-interaction with a dynamic environment which is described by a state space. So RL- methods are methods for solving optimal control (OC) problems for which only a partial amount of initial data are available to the system that learns. To solve such optimal control problems, Dynamic Programming (DP) meth- ods like for example Value Iteration, are used to determine the optimal value function and therefore the optimal control policy.

In case of a continuous state space, appropriate discretization methods are needed. Especially in regard of making progress for higher dimensional prob- lems, it is one aim of our project to introduce Sparse Grids in this context. So in this talk, a short overview of the mathematical background of RL (or OC, resp.) will be given, and than an approach using adaptive sparse grids to approximate the value function of some OC problem via Dynamic Pro- gramming will be presented. By means of some low dimensional examples from deterministic and stochastic optimal control, we will see that there are some difficulties concerning the convergence of the sparse grid approximation scheme.