Presentation: Using AI to Optimize SQL Query Plans and Performance

Digital transformation is enabling organizations to discover valuable new ways to leverage structured and unstructured data. Queries are very powerful since they specify what information is desired from a data source. Applications issue queries to read and write tables that transform and summarize Big Data into actionable information and insights. Queries using the high level SQL language and Hive let the query engine build an execution plan, avoiding the complexity of MapReduce and promising greater efficiency and faster results with less effort.

However, cluster resources are not unlimited, and at some point the operator (and developer) will observe a query that is running more slowly than usual, or has stopped altogether. An inefficient query may pose a burden on the database’s resources, cause slow query performance, or even impact other cluster users if the query contains errors. Poor performance can result from many sources, including CPU and memory bottlenecks, an excessive number of rows, and failed stages, forcing the query operator to intervene. But without clear visibility into the query plan, resolving these issues can prove to be difficult and time-consuming.

In this informative session, Pepperdata field engineer Kirk Lewis will discuss how Pepperdata’s query performance management solution, Query Spotlight, uses AI to continuously monitor and analyze queries, providing operators with a top-down view of cluster resources combined with detailed insight into query plans, including duration time, memory/CPU usage, stages, and critical path for every query. Learn why the real-time correlation of application and infrastructure performance is essential to optimizing query plans and ensuring efficient cluster operations.