Advances in Probe Subsampling for 4D-STEM

Session

Poster Session One

Authors

Mr Alex Robinson (2), Dr Amirafashar Moshtaghpour (1), Mr Jack Wells (2, 4), Mr Daniel Nicholls (2), Professor Angus Kirkland (3, 1), Professor Nigel Browning (2, 4, 5)

Affiliations

1. Rosalind Franklin Institute
2. University of Liverpool
3. University of Oxford
4. Sivananthan Laboratories
5. Pacific Northwest National Laboratory

Keywords

4D-STEM, STEM, ptychography, compressive sensing, subsampling, sparse sampling, dictionary learning, differential phase contrast, low-dose, fast acquisition 

Abstract text

Four-dimensional scanning transmission electron microscopy (4D-STEM) is the acquisition of a convergent beam diffraction pattern at each scanned probe location [1, 2]. 4D-STEM is a powerful imaging method as multi-modal analysis can be performed without the need of multiple detectors. Common methods include virtual imaging [3], differential phase contrast imaging [4], and ptychography [5, 6] for phase retrieval.  The later method being popular due to its effectiveness for extracting weak phase contrast from light elements such as those found in battery materials and biological specimen [7-9].

Furthermore, the advent of much faster pixelated detectors with typical read out speeds at over 10,000 frames per second has further advanced method development in 4D-STEM [10, 11]. However, there still exists at least an order of magnitude which makes radial detectors more effective in terms of time resolution. It is because of this that the study of beam sensitive materials or dynamic processes are limited, and therefore either more advancements in detectors are required, or perhaps a new method. 

In this work, a method of probe subsampling is given where only a subset of resulting diffraction patterns is acquired. Probe subsampling in STEM has been demonstrated in several works to promote lower dose, faster scanning, and faster computation [12- 17]. The subsampled data is passed through an inpainting algorithm that recovers the fully sampled data. Common classes of inpainting algorithms include dictionary learning (with sparse pursuit) [18], interpolation [19], and deep learning [20]. In this case, an implementation of the Beta Process Factor Analysis [21, 22] method is used since an alternative sparse representation (known as a dictionary or basis) of the dataset can be generated from subsampled measurements. The full dataset is recovered as a combination of weights multiplied by the elements (atoms) of the learnt basis. 

We show that it is possible to reach the same acquisition speeds of radial detectors, with the advantage of far more data analysis techniques. We give examples of these techniques applied to both the subsampled and fully sampled data sets to show functionally identical results. We also present potential future works in the field of in-situ electron microscopy combined with 4D-STEM, as well as the potential to improve older hardware through subsampling. 

References

[1] Nellist, P.D., McCallum, B.C. and Rodenburg, J.M., 1995. Resolution beyond the 'information limit' in transmission electron microscopy. nature, 374(6523), pp.630-632.

[2] Ophus, C., 2019. Four-dimensional scanning transmission electron microscopy (4D-STEM): From scanning nanodiffraction to ptychography and beyond. Microscopy and Microanalysis, 25(3), pp.563-582.

[3] Ribet, S.M., Ophus, C., Reis, R.D. and Dravid, V.P., 2022. Defect contrast with 4D-STEM: Applications of virtual detectors and beam modification. arXiv preprint arXiv:2211.06511.

[4] Shibata, N., Findlay, S.D., Kohno, Y., Sawada, H., Kondo, Y. and Ikuhara, Y., 2012. Differential phase-contrast microscopy at atomic resolution. Nature Physics, 8(8), pp.611-615.

[5] Rodenburg, J.M. and Bates, R.H.T., 1992. The theory of super-resolution electron microscopy via Wigner-distribution deconvolution. Philosophical Transactions of the Royal Society of London. Series A: Physical and Engineering Sciences, 339(1655), pp.521-553.

[6] Rodenburg, J.M., McCallum, B.C. and Nellist, P.D., 1993. Experimental tests on double-resolution coherent imaging via STEM. Ultramicroscopy, 48(3), pp.304-314.

[7] Li, G., Zhang, H. and Han, Y., 2022. 4D-STEM Ptychography for Electron-Beam-Sensitive Materials. ACS Central Science, 8(12), pp.1579-1588.

[8] Yang, H., MacLaren, I., Jones, L., Martinez, G.T., Simson, M., Huth, M., Ryll, H., Soltau, H., Sagawa, R., Kondo, Y. and Ophus, C., 2017. Electron ptychographic phase imaging of light elements in crystalline materials using Wigner distribution deconvolution. Ultramicroscopy, 180, pp.173-179.

[9] Zhou, L., Song, J., Kim, J.S., Pei, X., Huang, C., Boyce, M., Mendonça, L., Clare, D., Siebert, A., Allen, C.S. and Liberti, E., 2020. Low-dose phase retrieval of biological specimens using cryo-electron ptychography. Nature Communications, 11(1), p.2773.

[10] Philipp, H.T., Tate, M.W., Shanks, K.S., Mele, L., Peemen, M., Dona, P., Hartong, R., van Veen, G., Shao, Y.T., Chen, Z. and Thom-Levy, J., 2022. Very-high dynamic range, 10,000 frames/second pixel array detector for electron microscopy. Microscopy and Microanalysis, 28(2), pp.425-440.

[11] Plotkin-Swing, B., Haas, B., Mittelberger, A., Dellby, N., Hotz, M., Hrncirik, P., Meyer, C., Zambon, P., Hoermann, C., Meffert, M. and Bachevskaya, D., 2022. 100,000 Diffraction patterns per second with live processing for 4D-STEM. Microscopy and Microanalysis, 28(S1), pp.422-424.

[12]      A. Robinson, D. Nicholls, J. Wells, A. Moshtaghpour, A. Kirkland, and N. D. Browning. “SIM-STEM Lab: Incorporating Compressed Sensing Theory for Fast STEM Simulation”, vol. 242, pp. 113625, 2022

[13]      A. Stevens, H.  Yang, L. Carin, I. Arslan, and N. D. Browning. “The potential for Bayesian compressive sensing to significantly reduce electron dose in high-resolution STEM images”, Microscopy, vol. 63, no. 1, pp.41-51, 2014.

[14]     L. Donati, M. Nilchian, S. Trépout, C. Messaoudi, S. Marco, and M. Unser. “Compressed sensing for STEM tomography”, Ultramicroscopy, vol. 179, pp.47-56, 2017.

[15]     L. Kovarik, A. Stevens, A. Liyu, and N. D. Browning. “Implementing an accurate and rapid sparse sampling approach for low-dose atomic resolution STEM imaging”, Applied Physics Letters, vol. 109, no. 16, p.164102, 2016. 

[16]      D. Nicholls, A. Robinson, J. Wells, A. Moshtaghpour, M. Bahri, A. Kirkland, and N. Browning, “Compressive Scanning Transmission Electron Microscopy,” In ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1586-1590, IEEE, 2022

[17]      D. Nicholls, J. Wells, A.W. Robinson, A. Moshtaghpour, M. Kobylynska, R.A. Fleck, A.I. Kirkland, and N.D. Browning, “A Targeted Sampling Strategy for Compressive Cryo Focused Ion Beam Scanning Electron Microscopy,” arXiv preprint, arXiv:2211.03494, 2022.

[18] Mairal, J., Bach, F., Ponce, J. and Sapiro, G., 2009, June. Online dictionary learning for sparse coding. In Proceedings of the 26th annual international conference on machine learning (pp. 689-696).

[19] Jassim, F.A., 2013. Image inpainting by Kriging interpolation technique. arXiv preprint arXiv:1306.0139.

[20] Xie, J., Xu, L. and Chen, E., 2012. Image denoising and inpainting with deep neural networks. Advances in neural information processing systems, 25.

[21]      S. Sertoglu and J. Paisley, “Scalable Bayesian nonparametric dictionary learning,” in 2015 23rd European Signal Processing Conference (EUSIPCO), pp. 2771–2775, 2015.

[22]      D. Nicholls, J. Wells, A. Stevens, Y. Zheng, J. Castagna, and N. D. Browning, “Sub-sampled imaging for STEM: maximising image speed, resolution and precision through reconstruction parameter refinement,” Ultramicroscopy, vol. 233, p.113451, 2022.