Oracle today announced MySQL HeatWave Lakehouse, enabling customers to process and query hundreds of terabytes of data in object store in a variety of file formats, such as CSV and Parquet, as well as Aurora and Redshift backups. MySQL HeatWave Lakehouse is the newest addition to the MySQL HeatWave portfolio, the only cloud service that combines transaction processing, analytics, machine learning, and machine learning-based automation within a single MySQL database.
Powered by the massively parallel scale-out MySQL HeatWave architecture, MySQL HeatWave Lakehouse delivers significantly better performance than competitive cloud database services for running queries and loading data, as demonstrated by industry standard benchmarks. In addition, in a single query, customers can query transactional data in the MySQL database and combine it with data in the object store using standard MySQL syntax. Oracle also announced new MySQL Autopilot capabilities that improve performance and make MySQL HeatWave Lakehouse easy to use. MySQL HeatWave Lakehouse is now available in Beta for customers to try and is slated for general availability in 1HCY23.
“At Oracle, being a customer centric organisation, we have always aimed to evolve our services and make it easier for our customers to move towards digitization. The announcement around MySQL HeatWave Lakehouse is a primary example of the same. While the service was well received by our customers globally as well as in India but we wanted to increase our customers’ ROI by offering a single service for transaction processing as well as analytics across data warehouse and data lake, and machine learning – thus saving the amount and effort that the customers were spending for ETL across cloud services.” said P Saravanan, Vice-President, of Cloud Engineering, Oracle India
He added, “For our Indian customer especially, we understand that saving time and receiving efficient results is one of the top priorities therefore we believe that MySQL Autopilot’s capabilities such as Auto provisioning, and Auto query plan improvement will prove useful and the users will receive increased performance of MySQL HeatWave Lakehouse while saving significant time.”
Customers migrating from AWS, Google, and on-premises have been using MySQL HeatWave for a broad set of use cases including marketing analytics, particularly real-time analysis of advertising campaign performance and customer data analytics to build effective campaigns. Customers migrating from AWS include leaders in the automotive, telecommunications, retail, high-tech, and healthcare industries.
“MySQL HeatWave is the result of years of research and advanced development, which we are turning into breakthrough innovations to address a bigger set of challenges for all MySQL customers. In fact, MySQL HeatWave Lakehouse is our third major MySQL HeatWave announcement this year,” said Edward Screven, chief corporate architect, Oracle. “There is a huge growth in data stored outside of databases, and with MySQL HeatWave Lakehouse, customers can leverage all the benefits of HeatWave on data residing in object store. MySQL HeatWave now provides one integrated service on multiple clouds for transaction processing, analytics across data warehouses and data lakes, and machine learning without ETL. This combination helps deliver massive improvements in performance, automation, and cost—further distancing MySQL HeatWave from other cloud database services.”
“We are excited to continue our collaboration with Oracle, evolving it into supporting their new MySQL HeatWave Lakehouse offering, which is optimized to run on AMD EPYC-powered Oracle cloud instances and leverage the latest innovations in our processors,” said Mark Papermaster, chief technology officer and executive vice president at AMD. “The collective work of the AMD and Oracle engineering teams has helped create an impressive MySQL solution that can support great scalability and performance for transaction processing, analytics, machine learning, and machine learning-based automation within a single MySQL database.”
Oracle is also publishing new lakehouse benchmarks and introducing several innovative capabilities for MySQL HeatWave Lakehouse and MySQL Autopilot.
Benchmarks
- Faster than Snowflake & Amazon Redshift in both query performance and data loadingAs demonstrated by a fully transparent, publicly available 400 TB TPC-H* benchmark, the query performance of MySQL HeatWave Lakehouse is:
*17X faster than Snowflake
*6X faster than Amazon Redshift
Loading data from object store into MySQL HeatWave Lakehouse is also significantly faster. For a 400 TB TPC-H* workload, load performance of MySQL HeatWave Lakehouse is:
*8X faster than Amazon Redshift
*2.7X faster than Snowflake
All of these fully transparent benchmark scripts are available on GitHub for customers to replicate.
“MySQL HeatWave Lakehouse sets the competition on fire by blazing the trail to the previously uncharted territory of 400 TB cloud database benchmarks at breakneck speeds,” said Ron Westfall, senior analyst and research director, Futurum Research. “MySQL HeatWave Lakehouse is a quantum leap for HeatWave in terms of processing capacity and computing power: from 32TB and 64 nodes to 400 TB and 512 nodes with performance and price performance that handily beat Amazon Redshift and Snowflake. Meanwhile, the cloud database competitors have yet to respond to the in-database convergence and the multi-cloud presence of MySQL HeatWave. How will they cope with the 400 TB MySQL HeatWave Lakehouse?”
Innovative new capabilities for MySQL HeatWave Lakehouse
- Larger data size, standard MySQL syntax: Customers can query up to 400 TB of data with MySQL HeatWave Lakehouse, and the HeatWave cluster scales to 512 nodes. Customers use standard MySQL syntax for querying the data.
- Identical performance and compression: MySQL HeatWave offers the same query performance for data stored inside MySQL database or on object store—as demonstrated by both 10TB and 30TB TPC-H benchmarks. Furthermore, the amount of compression achieved and the amount of data which can be processed per node is the same in both instances.
- Support for multiple file formats: With MySQL HeatWave Lakehouse, customers can load and process data stored in a variety of file formats, such as CSV and Parquet, as well as Aurora and Redshift backups from AWS. This enables customers to leverage the benefits of MySQL HeatWave even when their data is not stored inside a MySQL database. The query performance is the same regardless of the file format in which the data is stored.
- Ability to query data in MySQL and combine it with data in object store: With MySQL HeatWave Lakehouse, customers can query their OLTP data stored inside MySQL database and combine it with data stored in the object store. Any change made to the OLTP data is updated in real-time and reflected in the query result.
New MySQL Autopilot capabilities for MySQL HeatWave Lakehouse
MySQL Autopilot provides machine learning-based automation for MySQL HeatWave. Existing MySQL Autopilot capabilities such as auto-provisioning and auto query plan improvement have been enhanced for MySQL HeatWave Lakehouse, which further reduces database administration overhead and improve performance. In addition, a number of new MySQL Autopilot capabilities are now available for MySQL HeatWave Lakehouse.
- Auto schema inference: Autopilot automatically infers the mapping of the file data to datatypes in the database. As a result, customers don’t need to manually specify the mapping for each new file to be queried by MySQL HeatWave Lakehouse—thereby saving time and effort.
- Adaptive data sampling: Autopilot intelligently samples portions of files in object storage, collecting accurate statistics with minimal data access. MySQL HeatWave uses these statistics to generate and improve query plans, determine the optimal schema mapping, and for other purposes.
- Auto load: Autopilot analyzes the data to predict the load time into MySQL HeatWave, determines the mapping of the datatypes, and automatically generates the loading scripts. Users don’t have to manually specify the mapping of files to database schemas and tables.
- Adaptive data flow: MySQL HeatWave Lakehouse dynamically adapts to the performance of the underlying object store. As a result, MySQL HeatWave can get the maximum available performance from the underlying cloud infrastructure which improves overall performance, price performance, and availability.
