Название: Scalable Data Management for Future Hardware
Автор: Kai-Uwe Sattler, Alfons Kemper, Thomas Neumann, Jens Teubner
Издательство: Springer
Год: 2025
Страниц: 255
Язык: английский
Формат: pdf (true), epub
Размер: 22.8 MB

This book presents the results of the DFG priority program on Scalable Data Management for Future Hardware. It details requirements and solutions of how modern and future hardware architectures can be leveraged to address the challenges in modern data management.

The nine chapters of the book present a wide range of data management architectures in conjunction with current hardware developments, often related to applications in data analytics or Machine Learning. They cover topics such as hardware-accelerated query or event processing on FPGA, GPU, and multicore CPUs, scalable data management in data center networks or on modern memory and storage technologies, and operating system support.

Computer science and the IT industry are currently undergoing significant and fundamental changes at many levels. On the one hand, the business model has evolved from a manufacturing industry (software and hardware) to a service industry based on cloud providers offering hardware and software as a service. This alone has enormous implications for the technology as the economies of scale and the highly shared infrastructure of the cloud create opportunities for optimization and specialization not available before. On the other hand, the relevant applications, use cases, and workloads have become mainly data driven, involving extraordinary amounts of data and very tight performance as well as efficiency requirements. These changes are happening at a time where limitations at the physical and economic level prevent us from producing faster general-purpose processors (CPUs), a drawback that is being addressed through hardware specialization where use-case-specific hardware is developed to be able to meet the requirements of common applications.

The Scalable Data Management for Future Hardware of the DFG priority program has tackled these challenges from the perspective of data processing, a cornerstone of data science and the machine learning revolution we are experiencing. Data management systems, and especially relational databases, are still one of the largest software industries. Neither the cloud nor Machine Learning has changed that. However, their role and how they are used have changed and are quickly evolving. For instance, on the software side, now all major database machines support vector search and large-scale vectors, something that would have been difficult to imagine just 5 years ago given the characteristics of vector data and the operations performed over it. On the hardware side, there is a growing amount of examples from both industry and academia of tailoring database engines to new hardware (networking, new memory models, new processors, accelerators, etc.). The researchers in the program have pioneered and led many of these efforts, their work having now become a major reference for anybody exploring the area. The program has also been a great, and much needed, source of talent covering both the data management and the hardware specialization side, creating the basis for future research and education programs.

The chapters in this book provide a glimpse of the highly innovative work done as part of the program, covering topics that encompass from new memory models and new memory technologies to heterogeneous computing, tackling research questions related to high-scale parallelism, consistency, bottlenecks caused by interconnects, reconfigurable computing, modern networking, and non-volatile memory in the context of various applications (query processing, graph databases, event processing, etc.). The work reported provides a comprehensive coverage of the topics and provides excellent examples of the opportunities new hardware has to offer. It is important to note that the topics explored do so in great detail, not only in terms of describing a design but also exploring the practical implications for data management systems and database engines in terms of how their architecture needs to evolve. It is also interesting that the book covers a very insightful spectrum between ideas that can be applied to existing engines without having to do major changes to radically new designs that open up intriguing opportunities for future systems.

Data management is and remains a key component of the data science revolution, maybe attracting less attention than machine learning and large language models but still acting as a key enabler and a source of ideas to improve the performance and efficiency of what are now the largest data processing engines ever built. But data management systems need to evolve with the times, and that requires to adapt and adopt new hardware.

This book provides researchers in academia and industry with a comprehensive combination of data management, operating systems, distributed systems and computer architecture issues necessary to address the requirements from practice as well as to propel innovative ideas and challenging research questions.

Contents: