Abstract

Querying data stored by database systems has been one of the most important tasks of these systems. Hence, there were always efforts to generate such queries using text written in a natural language. In this article we use a part of the SQL Standard called SQL/OLAP and ChatGTP as a representative of Large Language Models (LLMs) to evaluate the process described above. Our evaluation shows that different subgroups of queries deliver significantly different results. There are two subgroups of queries, for which ChatGTP generates many erroneous queries. The first subgroup comprises queries related to non-standardized temporal features and predicates. In other words, the main property of the queries in this group is that they are not specified in the standard, but are supported by several RDBMSs. The second subgroup contains queries in relation to standardized temporal features and predicates not implemented in the chosen RDBMS. Our evaluation shows that for the subgroup of queries related to non-standardized temporal features, ChatGTP generates only one query correctly. If the generation process demands the use of standardized temporal features, which are not implemented in the DBMS, the error rate is 43% and 75%, respectively.

References

• ISO/IEC 9075-2:2011, Information technology –Database languages–SQL–Part 2:Foundation
• Kulkarni, K.,Michels, J., Temporal Features in SQL:2011, SIGMOD Records,Vol. 41(3), 2012
• Petković, D., Was lange währt, wird endlich gut: Temporale Daten im SQL-Standard (in German), Datenbank-Spektrum 13(2), 2013, https://doi.org/10.1007/s13222-013-0120-3
• Melton, J., SQL: 1999 – Advanced SQL:1999, Understanding Object-relational and Other Advanced Features, Morgan-Kaufman, 2002.
• Sannik, G., Daniels, F., Enabling the Temporal Data Warehouse, 2011, https://cs.ulb.ac.be/public/_media/teaching/infoh415/teradata_enabling_temporal.pdf
• E. R. Nascimento and G. García, 2024. LLM‑Based Text‑to‑SQL for Real‑World Databases, SN Computer Science. https://doi.org/10.1007/s42979-025-03662-6.
• Petković, D., Applying Text-to-SQL to SQL/OLAP Using LLMs, (MLNLP 2025), Hangzhou, China, November 2025, DOI: 10.1109/MLNLP66797.2025.11389090
• A. Liu and X. Hu.. A comprehensive evaluation of ChatGTP’s zero-shot Text2SQL Capability,. https://doi.org/10.48550/arXiv.2303.13547.
• T. Yu and R. Zhang and K. Yang. 2018. Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL, In EMNLP, 2018.
• J. Li and B. Hui., Can LLM Already Serve as A Database Interface? A Big Bench for Large-Scale Database Grounded Text-to-SQLs, 2023, https://doi.org/10.48550/ARXIV.2305.03111
• Petković, D., SQL Server 2019 A Beginner’s Guide, McGraw Hill, 978-1-260-45887-3, 2019.
• Saracco, C., Nicola, M., Gandhi, L, A matter of time: Temporal data management in DB2, www.ibm.com/developerworks/data/library/techarticle/dm-1204db2temporaldata/dm-1204db2temporaldatapdf.
• Clark, D. Historical records with PostgreSQL, temporal tables and SQL:2011, https://clarkdave.net/2015/02/historical-records-with-postgresql-and-temporal-tables-and-sql-2011/.
• MariaDB: Temporal Tables, https://mariadb.com/docs/server/reference/sql-structure/temporal-tables
• Allen J.F., Maintaining knowledge about temporal intervals. Com. ACM Vol. 26(11), 1983.