Focus-stacking.Β Basic Q-learning is fundamentally based on the assumption that the state adheres to the Markov property. All information the agent needs to make an optimal decision is encapsulated in that state. (Illustrated in Figure 57). Our problem needs to be able to be framed as a Markov Decision Problem [BB17] (MDP), where the future only depends on the current state, not the history. Formally: π[ ππ‘+1 β£ ππ‘ ] = π[ ππ‘+1 β£ π1 β¦ ππ‘ ]. Or put differently, the future and past are conditionally independent given the present, as the current state encapsulates all information [BB22]. For Ronchigrams this is clearly not the case. This can be seen in Figures 40 and 41. The fact that a multitude of aberrations may lead to the same wavefront is grounded in physical reality and thus something we must deal with. This situation may be resolved by either using a sequence of measurements and logged microscope settings through time, or by taking multiple measurements while willfully changing the so-called defocus setting on the microscope. These combined Ronchigrams uniquely identify the A1 magnitude parameter. This last option comes at the cost of taking seemingly superfluous measurements but is technically simpler to implement. The first pixel-based RL algorithm for XXXXXX uses it. Figure 57: Base DQN architecture with a Resnet 18 encoder and a linear head.The latter is dependent on the number of actions that is available. Version Status Date Page 2.0 Non-Confidential 2024.05.1172022.03.1 73/100 Figure 58 General DQN learning architecture adapted from [1].
