Compressed Sensing in Ptychography and Transmission Electron Microscopy
Final Report Abstract
In the context of near-field ptychography with structured illumination, a phase plate was manufactured, by texturing a SiN-membrane with a focused ion beam. We found out how to design such phase plates systematically, so they produce electron beams of predefined width and convergence angle reliably. A dedicated Matlab program was written for this purpose. The quality of these phase plates was confirmed experimentally. Due to severe hardware problems, the avenue of near-field ptychography with structured illumination was not pursued further. This project carried out the first investigation of distributed compressed sensing under Poisson noise from an information theory point of view, and it is the first that started from the principle that a proper comparison between techniques requires keeping constant the electron dose in order to account for beam damage. As a result many of the usual promising claims regarding compressed sensing were refuted, especially its viability as a dose reduction mechanism. Furthermore, a formalism was developed allowing researchers to design their compressed sensing experiment in an optimal way, given the noise properties of the detector. In response to these unexpected and surprising results, the plans of implementing distributed compressed sensing for far-field ptychography were abandoned. It was worked out analytically how the overdetermination of the mathematical problem underlying ptychography is reduced by a host of experimentally more desirable settings, thus guiding researchers in their experimental design. Furthermore, a derivative-based ptychographic algorithm was published that takes into account finite sample thickness, optimizes for nuisance parameters (beam shape and positions), and includes various regularizations, thus achieving reliable results from noisy data with severely reduced and underdetermined information. This was confirmed through simulation and experiment. It was shown that deep reinforcement learning could be used to train, on-the-go, a recurrent neural network that determines, from information gathered during the ongoing scan, what the optimal next positions for the scan are. So far adaptive scanning often requires a coarser scan to produce a first guess of the regions of interest. During this stage beam damage already occurs and dose budget is hence wasted. Most machine learning techniques require ground-truth data to train on, and this is notoriously difficult to come by in TEM. This work bypasses the former problem by predicting new regions of interest from the ongoing scan, and the latter by training on the recorded data itself. Preliminary results have been successful on both simulated and experimental data.
Publications
- Production of arbitrary phase apertures for electron ptychography beams. Proceedings of QSORT conference (2018), Jülich, Germany
W. Van den Broek, P.M. Pelz, P.-H. Lu, M. Kruth, V. Grillo, R.E. Dunin-Borkowski, C.T. Koch, R.J. Dwayne Miller
- Towards Ptychography with Structured Illumination, and a Derivative-Based Reconstruction Algorithm. Microscopy and Microanalysis 25 (S2) (2019) 58–59
W. Van den Broek, M. Schloz, T.C. Pekin, P. M. Pelz, P.-H. Lu, M. Kruth, V. Grillo, R. E. Dunin-Borkowski, R. J. D. Miller, C.T. Koch
(See online at https://doi.org/10.1017/S1431927619001028) - Various Compressed Sensing Setups Evaluated Against Shannon Sampling Under Constraint of Constant Illumination. IEEE Transactions on Computational Imaging (2019) 5 502-514
W. Van den Broek, B.W. Reed and A. Béché, A. Velazco, J. Verbeeck, C.T. Koch
(See online at https://doi.org/10.1109/TCI.2019.2894950) - Viability of Compressed Sensing as a Dose Reduction Strategy in STEM. Microscopy and Microanalysis 25 (S2) (2019), 1686–1687
W. Van den Broek, B.W. Reed, A. Béché, J. Verbeeck, C.T. Koch
(See online at https://doi.org/10.1017/S1431927619009164) - Comparison of Ptychography vs. Center-of-mass Analysis of Registered 4D-STEM Series. Microscopy and Microanalysis (2020) 26 (S2), 1898-1900
B. Haas, M. Schloz, A. Mittelberger, T. Lovejoy, J. Müller, O. Krivanek, L. Jones, W. Van den Broek, C.T. Koch
(See online at https://doi.org/10.1017/S1431927620019765) - Derivation of an expression for the oversampling ratio for ptychography. Proceedings of European Microscopy Congress (2020) paper no. 862
W. Van den Broek, C.T. Koch
- Overcoming information reduced data and experimentally uncertain parameters in ptychography with regularized optimization. Optics Express (2020) 28 19 28306
M. Schloz, T.C. Pekin, Z. Chen, W. Van den Broek, D A. Muller, C.T. Koch
(See online at https://doi.org/10.1364/OE.396925) - Adaptive Scanning in Ptychography through Deep Multi-Objective Reinforcement Learning. Proceedings of OSA Optical Sensors and Sensing Congress (2021) paper no. CTu2H.3
M. Schloz, J. Müller, T.C. Pekin, W. Van den Broek, C.T. Koch
- Adaptive Scanning in Ptychography through Deep Reinforcement Learning. Microscopy and Microanalysis (2021) 27 (S1) 818–821
M. Schloz, J. Müller, T.C. Pekin, W. Van den Broek, C.T. Koch
(See online at https://doi.org/10.1017/S1431927621003238)
