A A Pilot Review of State-of-the-Art Deep Learning Applications to Nuclear Engineering & Technology

Authors

  • Tatiana Tambouratzis

DOI:

https://doi.org/10.14738/aivp.94.10798

Keywords:

Deep Learning, Nuclear Energy, Review, Fault Detection and Diagnostics, Monitoring, Proliferation and Resistance

Abstract

Given the advent of big data (BD), the focus of this “pilot” review of the most recent literature is upon deep learning (DL) implementations/solutions to important, as well as inherently complex, nuclear engineering and technology (NET) problems. The potential advancement from implementing large/real-scale DL-based algorithms is critically detailed, as well as highlighted, by inspecting the relevant body of latest research which appears in two major – overlapping, yet also complementary to each other in terms of scope and audience - nuclear energy (NE)-dedicated scientific research means of publication, namely Elsevier’s “Annals of Nuclear Energy” and “Progress in Nuclear Energy” journals. It is demonstrated that, given their (I) accuracy, robustness-to-noise and resilience to incomplete/erroneous information, as well as (II) competence under non-stationary and transient nuclear reactor (NR) operating conditions, deep and convolutional neural networks (DNNs and CNNs, respectively), as well as their variants, are becoming increasingly important, in their own right, for the swift and reliable solution of NET problems. It will be of great interest for – while also advantageous to - the NE, BD, and computational intelligence (CI)/DL communities, to join forces, follow up, and contribute to future research in – as well as application to - this particularly interesting and fruitful collaboration between disciplines.

References

Samuel A.L. (1959). Some studies in machine learning using the game of checkers, IBM Journal of Research and Development, vol. 3, pp. 210-229

Russell S.J., Norvig P. (2009). Artificial intelligence: a modern approach (3rd Edition), Upper Saddle River, New Jersey: Prentice-Hall

Siddique N., Adeli H. (2013). Computational intelligence: synergies of fuzzy logic, neural networks and evolutionary computing, John-Wiley & Sons ISBN: 978-1-118-33784-4

https://cis.ieee.org/about/what-is-ci

Zadeh L.A. (1965), Fuzzy sets, information and control, vol. 8, pp. 338–353. doi:10.1016/S0019-9958(65)90241-X

McCulloch W., Pitts W. (1943). A logical calculus of ideas immanent in nervous activity, Bulletin of Mathematical Biophysics, vol. 5, pp. 115–133. doi:10.1007/BF02478259

Fogel D.B. (2005). Evolutionary computation: toward a new philosophy of machine intelligence, Wiley-IEEE Press (3rd edition)

Lieberman H. (2016). Symbolic vs subsymbolic AI, https://courses.media.mit.edu/2016spring/mass63/wp-content/uploads/sites/40/2016/ 02/Symbolic-vs.-Subsymbolic.pptx_.pdf

Takeuti, G., Titani S. (1984). Intuitionistic fuzzy logic and intuitionistic fuzzy set theory, Journal of Symbolic Logic, vol. 49, pp. 851–866

Garavelli A.C., Gorgoglione M., Scozzi B. (1999). Fuzzy logic to improve the robustness of decision support systems under uncertainty, Computers & Industrial Engineering, vol. 37, pp. 477-480

Rosenblatt F. (1958). The perceptron: a probabilistic model for information storage and organization in the brain, Psychological Review, vol. 65 , pp. 386-406

Rumelhart D.E., Hinton G.E., Williams R.J. (1986). Learning representations by back-propagating errors, Nature, vol. 323, pp. 533–536

Schmidhuber J. (2014). Deep learning in neural networks: an overview, Neural Networks, vol. 61, pp. 85-117

Fogel L.J., Owens A.J., Walsh M.J. (1966). Artificial intelligence through simulated evolution. New York: John Wiley

Rechenberg I. (1973). Evolutionstrategie: Optimierung Technischer Systeme nach Prinzipien des Biologischen Evolution. Fromman-Hozlboog Verlag, Stuttgart.

Holland J.H. (1975). Adaptation in natural and artificial systems, University of Michigan Press, Ann Arbor, 1975.

Bäck Th., Fogel D.B., Michalewicz Z. eds. (1997), Handbook of Evolutionary Computation, ISBN 0750303921

Yang X-S, He X.S. (2015). Swarm intelligence and evolutionary computation: overview and analysis, “Studies in computational intelligence, Recent Advances in Swarm Intelligence and Evolutionary Computation”, (SCI book series, vol. 585, pp. 1-23, Springer. ISBN 9783319138251. (doi:10.1007/978-3-319-13826-8_1)

Dorigo M. (1992). Optimization, learning and natural algorithms, PhD thesis, Politecnico di Milano, Italy

Kennedy J., Eberhart R. (1995). Particle swarm optimization, Proceedings of IEEE International Conference on Neural Networks. IV. pp. 1942-1948. doi:10.1109/ ICNN. 1995. 488968

Reynolds C. (1987). Flocks, herds and schools: a distributed behavioral model, in SIGGRAPH '87: Proceedings of the 14th Annual Conference on Computer Graphics and Interactive Techniques. Association for Computing Machinery, pp. 25–34.

Beni, G., Wang, J. (1993). "Swarm Intelligence in Cellular Robotic Systems". Proceed. NATO Advanced Workshop on Robots and Biological Systems, Tuscany, Italy, June 26–30 (1989). Berlin, Heidelberg: Springer. pp. 703–712. doi:10.1007/978-3-642-58069-7_38. ISBN 978-3-642-63461-1.

Engelbrecht A.P. (2016). Particle swarm optimization with crossover: a review and empirical analysis, Artificial Intelligence Review, vol. 45, pp. 131–165

https://www.space.com/four-fundamental-forces.html

Newton I. (1687). Philosophiae Naturalis Principia Mathematica (“Mathematical Principles of Natural Philosophy”), London, U.K.

Maxwell J.C. 1861. On physical lines of force, Philosophical Magazine, vol. 90, pp. 11-23 (doi:10.1080/14786431003659180)

https://www.britannica.com/science /electromagnetism/Special-theory-of-relativity

Wikipedia; search keyword: isotope

Bodanis D. (2009). E=mc2: A biography of the world's most famous equation, Bloomsbury Publishing. ISBN 978-0-8027-1821-1

https://www.world-nuclear.org/information-library/current-and-future-generation//nuclear-fusion-power. aspx

https://www.iea.org/reports/nuclear-power-in-a-clean-energy-system

https://www.world-nuclear.org/information-library/current-and-future-generation/nuclear-power-in-the-world-today.aspx

https://www.nrc.gov/materials/fuel-cycle-fac/ur-deconversion/fu|cdeaq-depleted-ur-decon.html https://www.eia.gov/energyexplained/nuclear/nuclear-power-plants.php

https://en.wikipedia.org/wiki/Neutron_flux

https://www.eia.gov/energyexplained/nuclear/nuclear-power-plants.phpNuclear Power in the World Today (Updated February 2021)

Tambouratzis T., Giannatzis G., Kyriazis A., Siotropos P. (2020). Applying the computational intelligence paradigm to nuclear power plant operation: a review (1990-2015), International Journal of Energy Optimization and Engineering, Vol. 9, pp. 27-109

https://aiimpacts.org/scale-of-the-human-brain/

Cybenko G. (1989). Approximation by superpositions of a sigmoidal function, Mathematics of Control, Signals, and Systems, vol. 2, pp. 303-314. doi:10.1007/BF02551274

Hornik K. (1991). Neural Networks, vol. 4, pp. 251–257. doi:10.1016/0893-6080(91 )9 0009-T

Hadamard J. (1908). Mémoire sur le problème d'analyse relatif à Véquilibre des plaques élastiques encastrées, Mémoires présentés par divers savants éstrangers à l'Académie des Sciences de l'Institut de France. 33.

Haykin S. (2008). Adaptive filter theory. Pearson Education India, p. 108-142, & 217-242

Kohavi, Ron (1995). "A study of cross-validation and bootstrap for accuracy estimation and model selection". Proceedings of the Fourteenth International Joint Conference on Artificial Intelligence. San Mateo, CA: Morgan Kaufmann. 2 (12): 1137–1143.

Montufar G.F., Razvan Pascanu R., Cho K.G., Bengio Y. (2014). On the number of linear regions of deep neural networks, in Advances in Neural Information Processing Systems, pp. 2924–2932, 2014.

Ivakhnenko A.G. (1971). Polynomial theory of complex systems. IEEE Transactions on Systems, Man and Cybernetics, (4): 364-378.

He T., Fan Y., Qian Y., Tan T., Yu K. (2014). Reshaping deep neural network for fast decoding by node-pruning,2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2014, pp. 245-249, doi: 10.1109/ICASSP.2014.6853595.)

G.E. Dahl, T.N. Sainath and G.E. Hinton (2013). Improving deep neural networks for LVCSR using rectified linear units and dropout, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 8609-8613, doi: 10.1109/ICASSP.2013.6639346.

Shorten C., Khoshgoftaar T.M. (2019). A survey on image data augmentation for deep learning. Journal of Big Data 6, 60 https://doi.org/10.1186/s40537-019-0197-0)

Hubel D. H., Wiesel T. N. 1962 Receptive fields, binocular interaction and functional architecture in the cat's visual cortex, Journal of Physiology, Vol. 160, pp. 106–154, doi: 10.1113/jphysiol.1962.sp006837

https://en.wikipedia.org/wiki/Frobenius_inner_product

https://www.enertiv.com/resources/faq/what-is-fault-detection-diagnostics OK UP TO HERE

Wang H., Peng M-J, Ayodeli A., Xia H., Wang X-K, Li Z.K. (2020). Advanced fault diagnosis method for nuclear power plant based on convolutional gated recurrent neural network and enhanced particle swarm optimization, Annals of Nuclear Energy, vol. 151, Article 107934

Z. Huang, F. Yang, F. Xu, X. Song and K. Tsui, "Convolutional Gated Recurrent Unit–Recurrent Neural Network for State-of-Charge Estimation of Lithium-Ion Batteries," IEEE Access, vol. 7, pp. 93139-93149, 2019, doi: 10.1109/ACCESS.2019.2928037]

Zhang B.W. (2020). Novel fault diagnosis scheme utilizing deep learning networks, Progress in Nuclear Energy, vol.118, Article 103066

Perez M., Allison C.M., Wagner R.J., Martinez V., Fu Z., Hohorst J.K., Abarca A.(2015). The Development of RELAP5/SCDAPSIM/MOD4.0 for Advanced Fluid Systems Design and Analysis. The 23rd International Conference on Nuclear Engineering (ICONE 23), May 17th - 21st, 2015, Makuhari Messe, Chiba, Japan

Hotelling, H. (1933). Analysis of a complex of statistical variables into principal components. Journal of Educational Psychology, vol. 24, pp. 417–441

https://www.analyticsvidhya.com/blog/2017/12/fundamentals-of-deep-learning-introduction-to-lstm/

https://www.iaea.org/publications/7790/on-line-monitoring-for-improving-performance-of-nuclear-power-plants-part-1-instrument-channel-monitoring

Liang D., Gong P., Zhang R. (2019). Rapid nuclide identification algorithm based on convolutional neural network, Annals of Nuclear Energy, vol. 133, pp. 483-496, 2019.

https://www.nrc.gov/about-nrc/radiation/health-effects/detection-radiation.html#riid

https://en.wikipedia.org/wiki/Hilbert_curve

Saeed H.A., Wang H., Nawaz A. (2020). Online fault monitoring based on DNN and sliding window technique, Progress in Nuclear Energy, vol. 121, Article No. 103236.

https://www.un.org/disarmament/wmd/nuclear/npt/text/

https://www.iaea.org

Kim S.H., Lim S.C., Kim D.Y. (2017). Intelligent intrusion detection system featuring a virtual fence, active intruder detection, classification, tracking, and action recognition, Annals of Nuclear Energy, vol. 112, pp. 845-855, Article 107934

J.G.Kim, S.C.Jang, H.S.Park A study of object recognition using DL for optimizing categorization of radioactive waste, PNE vol.130, Dec. 2020, Article 103528.

Downloads

Published

2021-09-09

How to Cite

Tambouratzis, T. (2021). A A Pilot Review of State-of-the-Art Deep Learning Applications to Nuclear Engineering & Technology . European Journal of Applied Sciences, 9(4), 207–227. https://doi.org/10.14738/aivp.94.10798