Overview of Reinforcement Learning approaches and methods. It is important to start by mentioning that RL is not a mature field; there are many scattered methods and variants, and researchers refer to the same methods by different names. Hence, the perspective and highlighted methods provided in this overview might not be consensual, even among experts in the field. Here, we follow more closely [2] rather than the perhaps most standard reference [3], although some methods from [3] are also highlighted. This choice is motivated by the fact that [2] considers mostly simulation-based settings whereas [3] focusses on experimental-based setting. Therefore [2] is closer in spirit to the present project XXXXXX. However, there are many other important methods in the literature not covered in [2] and for those we rely mostly on the survey paper [21]. Specific methods that use digital twins are also discussed. In [2], reinforcement learning is divided into several subfields. First, there are approaches that rely on: Version Status Date Page 2.0 Non-Confidential 2024.05.1172022.03.1 23/100 1. Approximations in value space, where the cost-to-go or value function that encapsulates the reward/cost associated with a given action is the main focus. Once the value function is determined or approximated, the control policy (specifying actions as function of states) follows easily from searching for the decision that optimizes such value function. The approximation technique can be based on (deep) neural networks [14] or other forms of parametric approximation (an architecture that is off-line fitted based on data), or on-line simulation-based, which is especially interesting in the context of XXXXXX.
