The realization of an interoperable and scalable virtual platform, currently known as the “metaverse,” is inevitable, but many technological challenges need to be overcome first. With the metaverse still in a nascent phase, research currently indicates that building a new 3D social environment capable of interoperable avatars and digital transactions will represent most of the initial investment in time and capital. The return on investment, however, is worth the financial risk for firms like Meta, Google, and Apple. While the current virtual space of the metaverse is worth $6.30 billion, that is expected to grow to $84.09 billion by the end of 2028. But the creation of an entire alternate virtual universe of 3D avatars, objects, and otherworldly cityscapes calls for a new development pipeline and workflow. Existing 3D modeling and digital twin processes, already well-established in industry and gaming, will be ported to support the need to architect and furnish this new digital world. The current development pipeline, however, is cumbersome, expensive and limited in output capacity. This paper proposes a new and innovative immersive development pipeline leveraging the recent advances in artificial intelligence (AI) for 3D model creation and optimization. The previous reliance on 3D modeling software to create assets and then import into a game engine can be replaced with nearly instantaneous content creation with AI. While AI art generators like DALL-E 2 and DeepAI have been used for 2D asset creation, when combined with game engine technology, such as Unreal Engine 5 and virtualized geometry systems like Nanite, a new process for creating nearly unlimited content for immersive reality is possible. New processes and workflows, such as those proposed here, will revolutionize content creation and pave the way for Web 3.0, the metaverse and a truly 3D social environment.
References
[1]
Mondal, B. (2020) Artificial Intelligence: State of the Art. In: Balas, V.E., Kumar, R. and Srivastava, R., Eds., Recent Trends and Advances in Artificial Intelligence and Internet of Things, Springer, Berlin, 389-425.
https://doi.org/10.1007/978-3-030-32644-9_32
[2]
Zweig, B. (2015) Forgotten Genealogies: Brief Reflections on the History of Digital Art History. International Journal for Digital Art History, No. 1.
[3]
Mikalonytė, E.S. and Kneer, M. (2022) Can Artificial Intelligence Make Art? Folk Intuitions as to whether AI-Driven Robots Can Be Viewed as Artists and Produce Art. ACM Transactions on Human-Robot Interaction, 11, Article No. 43.
https://doi.org/10.1145/3530875
[4]
Li, S. and Zhou, Y. (2022) Pipeline 3D Modeling Based on High-Definition Rendering Intelligent Calculation. Mathematical Problems in Engineering, 2022, Article ID: 4580363. https://doi.org/10.1155/2022/4580363
[5]
Burova, A., Mäkelä, J., Heinonen, H., Palma, P.B., Hakulinen, J., Opas, V. and Turunen, M. (2022) Asynchronous Industrial Collaboration: How Virtual Reality and Virtual Tools Aid the Process of Maintenance Method Development and Documentation Creation. Computers in Industry, 140, Article ID: 103663.
https://doi.org/10.1016/j.compind.2022.103663
[6]
Tchomdji, L.O.K., Park, S.J. and Kim, R. (2022) Developing Virtual Tour Content for the Inside and Outside of a Building Using Drones and Matterport. International Journal of Contents, 18, 74-84.
[7]
Harrington, M.C., Jones, C. and Peters, C. (2022) Course on Virtual Nature as a Digital Twin: Botanically Correct 3D AR and VR Optimized Low-Polygon and Photogrammetry High-Polygon Plant Models. ACM SIGGRAPH 2022 Courses, Vancouver, 7-11 August 2022, 1-69. https://doi.org/10.1145/3532720.3535663
[8]
Oussalah, M. (2021) AI Explainability. A Bridge between Machine Vision and Natural Language Processing. In: International Conference on Pattern Recognition, Springer, Cham, 257-273. https://doi.org/10.1007/978-3-030-68796-0_19
[9]
Beer, R.D. (1995) A Dynamical Systems Perspective on Agent-Environment Interaction. Artificial Intelligence, 72, 173-215.
https://doi.org/10.1016/0004-3702(94)00005-L
[10]
Salehi, M., Hooli, K., Hulkkonen, J. and Tolli, A. (2022) Enhancing Next-Generation Extended Reality Applications with Coded Caching.
[11]
Schmidt, R., Isenberg, T., Jepp, P., Singh, K. and Wyvill, B. (2007) Sketching, Scaffolding, and Inking: A Visual History for interactive 3D Modeling. Proceedings of the 5th International Symposium on Non-Photorealistic Animation and Rendering, San Diego, 4-5 August 2007, 23-32. https://doi.org/10.1145/1274871.1274875
[12]
Madhavadas, V., Srivastava, D., Chadha, U., Raj, S.A., Sultan, M.T.H., Shahar, F.S. and Shah, A.U.M. (2022) A Review on Metal Additive Manufacturing for Intricately Shaped Aerospace Components. CIRP Journal of Manufacturing Science and Technology, 39, 18-36. https://doi.org/10.1016/j.cirpj.2022.07.005
[13]
Cheng, R., Wu, N., Chen, S. and Han, B. (2022) Reality Check of Metaverse: A First Look at Commercial Social Virtual Reality Platforms. 2022 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW), Christchurch, 12-16 March 2022, 141-148. https://doi.org/10.1109/VRW55335.2022.00040
[14]
Medina, J.J., Maley, J.M., Sannapareddy, S., Medina, N.N., Gilman, C.M. and McCormack, J.E. (2020) A Rapid and Cost-Effective Pipeline for Digitization of Museum Specimens with 3D Photogrammetry. PLOS ONE, 15, e0236417.
https://doi.org/10.1371/journal.pone.0236417
[15]
Korbel, J.J., Siddiq, U.H. and Zarnekow, R. (2022) Towards Virtual 3D Asset Price Prediction Based on Machine Learning. Journal of Theoretical and Applied Electronic Commerce Research, 17, 924-948. https://doi.org/10.3390/jtaer17030048
[16]
Zhang, Y. (2021, December 10) Cross-Platform Methods in Computer Graphics That Boost Experimental Film Making. Thesis, Institute of Technology, Rochester.
https://scholarworks.rit.edu/cgi/viewcontent.cgi?article=12190&context=theses
[17]
Favreau, J. (2019) The Mandalorian. [Film] Disney.
[18]
Turnock, J.A. (2022) Conclusion. Unreal Engine. In: The Empire of Effects, University of Texas Press, Austin, 216-222. https://doi.org/10.7560/325308
[19]
Anderson, K. (2020, August 24) The Mandalorian’s Virtual Sets Are Wild. Nerdist.
https://nerdist.com/article/mandalorian-virtual-production-star-wars
[20]
Dent, S. (2021, May 13) See How “The Mandalorian” Used Unreal Engine for Its Real-Time Digital Sets. Engadget.
https://www.engadget.com/2020-02-21-mandalorian-ilm-stagecraft-real-time-digital-sets.html
[21]
Ronfard, R. (2021, May) Film Directing for Computer Games and Animation. Computer Graphics Forum, 40, 713-730. https://doi.org/10.1111/cgf.142663
[22]
Cohen, J.L. (2022) Film/Video-Based Therapy and Trauma: Research and Practice. Taylor & Francis, Abingdon-on-Thames. https://doi.org/10.4324/9781315622507
[23]
Kuusela, V. (2022) 3D Modeling Pipeline for Games. Turku University of Applied Sciences, Turku.
[24]
Hevko, I.V., Potapchuk, O.I., Lutsyk, I.B., Yavorska, V.V., Hiltay, L.S. and Stoliar, O.B. (2022) The Method of Teaching Graphic 3D Reconstruction of Architectural Objects for Future IT Specialists. AET 2020, 1, 119-131.
https://doi.org/10.5220/0010921800003364
[25]
Sharif, M.M., Haas, C. and Walbridge, S. (2022) Using Termination Points and 3D Visualization for Dimensional Control in Prefabrication. Automation in Construction, 133, Article ID: 103998. https://doi.org/10.1016/j.autcon.2021.103998
[26]
Abboud, A. and Nasrullah, M. (2022) 3D Modeling for the Largest Stone in the World Using Laser Scanning Technique. Doctoral Dissertation, Полоцкий государственный университет имени Евфросинии Полоцкой.
[27]
Karami, A., Menna, F. and Remondino, F. (2022) Combining Photogrammetry and Photometric Stereo to Achieve Precise and Complete 3D Reconstruction. Sensors, 22, 8172. https://doi.org/10.3390/s22218172
[28]
Wachowski, L. and Wachowski, L. (1999) The Matrix [Film] Warner Bros. Pictures.
[29]
Mamani, A.R.C., Polanco, S.R.R. and Ordonez, C.A.C. (2022) Systematic Review on Photogrammetry, Streaming, Virtual and Augmented Reality for Virtual Tourism. HCI International 2022—Late Breaking Papers: Interacting with Extended Reality and Artificial Intelligence: 24th International Conference on Human-Computer Interaction, HCII 2022, Vol. 13518, 46.
[30]
OpenAI (2022, April 14) Dall·E 2. https://openai.com/dall-e-2
[31]
Blance, A. (2022, July 27) Using AI to Generate 3D Models, Fast! Medium.
https://towardsdatascience.com/using-ai-to-generate-3d-models-2634398c0799
[32]
Ha, H. and Song, S. (2022) Semantic Abstraction: Open-World 3d Scene Understanding from 2d Vision-Language Models. 6th Annual Conference on Robot Learning, Cornell University, Ithaca.
[33]
McKenzie, T. (2022, August 17) Kaedim: A Cool AI for Turning 2D Images into 3D Models.
https://80.lv/articles/kaedim-a-cool-ai-for-turning-2d-images-into-3d-models
[34]
Bebeshko, B., Khorolska, K., Kotenko, N., Desiatko, A., Sauanova, K., Sagyndykova, S. and Tyshchenko, D. (2021) 3D Modelling by Means of Artificial Intelligence. Journal of Theoretical and Applied Information Technology, 99, 1296-1308.
[35]
Salian, I. (2022, August 12) Nerf Research Turns 2D Photos into 3D Scenes. NVIDIA Blog.
https://blogs.nvidia.com/blog/2022/03/25/instant-nerf-research-3d-ai
[36]
Touloupaki, E. and Theodosiou, T. (2017) Performance Simulation Integrated in Parametric 3D Modeling as a Method for Early Stage Design Optimization—A Review. Energies, 10, 637. https://doi.org/10.3390/en10050637
[37]
Raitt, B. and Minter, G. (2000) Digital Sculpture Techniques. Interactivity Magazine, 4, 1-13. https://doi.org/10.14733/cadconfP.2019.392-396
[38]
Rossoni, M., Barsanti, S.G., Colombo, G. and Guidi, G. (2020) Retopology and Simplification of Reality-Based Models for Finite Element Analysis.
[39]
Tasdizen, T., Whitaker, R., Burchard, P. and Osher, S. (2003) Geometric Surface Processing via Normal Maps. ACM Transactions on Graphics (TOG), 22, 1012-1033.
https://doi.org/10.1145/944020.944024
[40]
Jean-Pierre, J. (2021) The Invasion: Applying the Aesthetics of Horror Films in a Virtual Reality Gaming Environment. Doctoral Dissertation, Florida Atlantic University, Boca Raton.
[41]
Choquette, J., Gandhi, W., Giroux, O., Stam, N. and Krashinsky, R. (2021) NVIDIA A100 Tensor Core GPU: Performance and Innovation. IEEE Micro, 41, 29-35.
https://doi.org/10.1109/MM.2021.3061394
[42]
Nvidia Developer (2019, December 3) NVIDIA NGX Technology—AI for Visual Applications. https://developer.nvidia.com/rtx/ngx
[43]
Wu, J., Dick, C. and Westermann, R. (2015) A System for High-Resolution Topology Optimization. IEEE Transactions on Visualization and COMPUTER Graphics, 22, 1195-1208. https://doi.org/10.1109/TVCG.2015.2502588
[44]
Haar, U. and Aaltonen, S. (2015) GPU-Driven Rendering Pipelines—Real-Time Rendering.
http://advances.realtimerendering.com/s2015/aaltonenhaar_siggraph2015_combined_final_footer_220dpi.pdf
[45]
Barrier, R. (2022, May 17) What Unreal Engine 5 MEANS for the Games Industry. IGN.
https://www.ign.com/articles/what-unreal-engine-5-means-for-the-games-industry
[46]
El-Wajeh, Y.A., Hatton, P.V. and Lee, N.J. (2022) Unreal Engine 5 and Immersive Surgical Training: Translating Advances in Gaming Technology into Extended-Reality Surgical Simulation Training Programmes. British Journal of Surgery, 109, 470-471.
https://doi.org/10.1093/bjs/znac015
[47]
Beardsall, R. (2020, September 4) Unreal Engine 5 and Nanite Virtualized Geometry—What Does It Mean for Content Creators? Medium.
https://medium.com/xrlo-extended-reality-lowdown/unreal-engine-5-and-nanite-virtualized-geometry-what-does-it-mean-for-content-creators-b4106accd306
[48]
Grba, D. (2022) Deep Else: A Critical Framework for AI Art. Digital, 2, 1-32.
https://doi.org/10.3390/digital2010001
