Abstract

Large Language Models (LLMs) have advanced reasoning, adaptation, and generative abilities despite their limited training on next-token prediction, which is the only way they are trained in world simulation. There are recent developments in transformer architectures and representation learning that indicate that such systems can learn structured internal representations, but it is not clear how much LLMs learn a coherent latent world model. Much existing work has focused on benchmark performance and emergent behaviors, and less on the internal representational dynamics involved in inference. This study proposes the Latent World Model Emergence (LWME) framework to explore the emergence of latent world representations in transformer-based LLMs. The framework is based on linear probing, activation patching, geometric manifold analysis, and causal ablation to examine representational organization for various knowledge domains. The study proposes a World Model Fidelity Score (WMFS) that is multidimensional and measures structural coherence, causal consistency, and inferential reliability to quantify the quality of representation. Results from experiments show that latent world-model representations are consistently observed for different transformer architectures and become more powerful when the parameter scales are larger and the training data is more varied. The causal ablation experiments demonstrate that ablation of mid-layer attention circuits is essential for inferential coherence and that the targeted ablations have a significant impact on reasoning performance. Additionally, there is also a problem that the accuracy of the benchmarks is not aligned with the actual reasoning abilities of the WMFS, which means that traditional assessment of actual reasoning ability may be misleading and merge memorization with structured inference. The study offers a systematic interpretability framework and empirical evidence of the ability of advanced LLMs to form organized internal representations that have implications for the fields of AI interpretability, alignment, and safety.

References

• Hinton, G. E., Deng, L., Yu, D., et al. (2012). Deep neural networks for acoustic modeling in speech recognition. IEEE Signal Processing Magazine, 29(6), 82–97. https://doi.org/10.1109/MSP.2012.2205597
• LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature, 521(7553), 436–444. https://doi.org/10.1038/nature14539
• Vaswani, A., Shazeer, N., Parmar, N., et al. (2017). Attention is all you need. Advances in Neural Information Processing Systems, 30. https://doi.org/10.48550/arXiv.1706.03762
• Devlin, J., Chang, M. W., Lee, K., & Toutanova, K. (2019). BERT: Pre-training of deep bidirectional transformers for language understanding. NAACL-HLT. https://doi.org/10.48550/arXiv.1810.04805
• Radford, A., Narasimhan, K., Salimans, T., & Sutskever, I. (2018). Improving language understanding by generative pre-training. OpenAI Technical Report. https://doi.org/10.48550/arXiv.1801.06146
• Brown, T. B., Mann, B., Ryder, N., et al. (2020). Language models are few-shot learners. Advances in Neural Information Processing Systems, 33, 1877–1901. https://doi.org/10.48550/arXiv.2005.14165
• Kaplan, J., McCandlish, S., Henighan, T., et al. (2020). Scaling laws for neural language models. arXiv. https://doi.org/10.48550/arXiv.2001.08361
• Bubeck, S., Chandrasekaran, V., Eldan, R., et al. (2023). Sparks of artificial general intelligence: Early experiments with GPT-4. arXiv. https://doi.org/10.48550/arXiv.2303.12712
• Bengio, Y., Courville, A., & Vincent, P. (2013). Representation learning: A review and new perspectives. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(8), 1798–1828. https://doi.org/10.1109/TPAMI.2013.50
• Geva, M., Schuster, R., Berant, J., & Levy, O. (2022). Transformer feed-forward layers build predictions by promoting concepts in the vocabulary space. EMNLP. https://doi.org/10.48550/arXiv.2203.14680
• McClelland, J. L., Rumelhart, D. E., & PDP Research Group. (1987). Parallel distributed processing. Psychological and Biological Models, 2. https://doi.org/10.7551/mitpress/5236.001.0001
• Tenenbaum, J. B., Kemp, C., Griffiths, T. L., & Goodman, N. D. (2011). How to grow a mind: Statistics, structure, and abstraction. Science, 331(6022), 1279–1285. https://doi.org/10.1126/science.1192788
• Bahdanau, D., Cho, K., & Bengio, Y. (2014). Neural machine translation by jointly learning to align and translate. arXiv. https://doi.org/10.48550/arXiv.1409.0473
• OpenAI. (2023). GPT-4 technical report. arXiv. https://doi.org/10.48550/arXiv.2303.08774
• Friston, K. (2010). The free-energy principle: A unified brain theory? Nature Reviews Neuroscience, 11(2), 127–138. https://doi.org/10.1038/nrn2787
• Lake, B. M., Ullman, T. D., Tenenbaum, J. B., & Gershman, S. J. (2017). Building machines that learn and think like people. Behavioral and Brain Sciences, 40, e253. https://doi.org/10.1017/S0140525X16001837
• Griffiths, T. L., Lieder, F., & Goodman, N. D. (2015). Rational use of cognitive resources: Levels of analysis between the computational and the algorithmic. Topics in Cognitive Science, 7(2), 217–229. https://doi.org/10.1111/tops.12142
• Zador, A. M. (2019). A critique of pure learning and what artificial neural networks can learn from animal brains. Nature Communications, 10, 3770. https://doi.org/10.1038/s41467-019-11786-6
• Mischler, G., Li, Y. A., Bickel, S., et al. (2024). Contextual feature extraction hierarchies converge in large language models and the brain. Nature Machine Intelligence, 6, 1467–1477. https://doi.org/10.1038/s42256-024-00925-4
• Kumar, P. (2024). Large language models (LLMs): Survey, technical frameworks, and future challenges. Artificial Intelligence Review, 57, 260. https://doi.org/10.1007/s10462-024-10888-y
• Wang, C., Zhao, J., & Gong, J. (2024). A survey on large language models from concept to implementation. arXiv. https://doi.org/10.48550/arXiv.2403.18969
• Zhao, W. X., Zhou, K., Li, J., et al. (2023). A survey of large language models. arXiv. https://doi.org/10.48550/arXiv.2303.18223
• Mahowald, K., Ivanova, A., Blank, I. A., et al. (2024). Dissociating language and thought in large language models. Trends in Cognitive Sciences, 28(6), 517–540. https://doi.org/10.1016/j.tics.2024.01.011
• Connell, L., & Lynott, D. (2024). What can language models tell us about human cognition? Current Directions in Psychological Science, 33(3), 169–176. https://doi.org/10.1177/09637214241242746
• Silver, D., Hubert, T., Schrittwieser, J., et al. (2018). A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play. Science, 362(6419), 1140–1144. https://doi.org/10.1126/science.aar6404
• Bommasani, R., Hudson, D. A., Adeli, E., et al. (2021). On the opportunities and risks of foundation models. arXiv. https://doi.org/10.48550/arXiv.2108.07258
• Dosovitskiy, A., Beyer, L., Kolesnikov, A., et al. (2021). An image is worth 16x16 words: Transformers for image recognition at scale. ICLR. https://doi.org/10.48550/arXiv.2010.11929
• Fan, L., Li, L., Ma, Z., et al. (2023). A bibliometric review of large language models research from 2017 to 2023. arXiv. https://doi.org/10.48550/arXiv.2304.02020
• Schmidhuber, J. (2015). Deep learning in neural networks: An overview. Neural Networks, 61, 85–117. https://doi.org/10.1016/j.neunet.2014.09.003.
• Tu, X., He, Z., Huang, Y., et al. (2024). An overview of large AI models and their applications. Visual Intelligence, 2, 34. https://doi.org/10.1007/s44267-024-00065-8