Название: Hypothesis Generation and Interpretation: Design Principles and Patterns for Big Data Applications
Автор: Hiroshi Ishikawa
Издательство: Springer
Серия: Studies in Big Data
Год: 2024
Страниц: 380
Язык: английский
Формат: pdf (true)
Размер: 11.1 MB

This book focuses in detail on Data Science and data analysis and emphasizes the importance of data engineering and data management in the design of Big Data applications. The author uses patterns discovered in a collection of Big Data applications to provide design principles for hypothesis generation, integrating Big Data processing and management, Machine Learning and data mining techniques.

The book proposes and explains innovative principles for interpreting hypotheses by integrating micro-explanations (those based on the explanation of analytical models and individual decisions within them) with macro-explanations (those based on applied processes and model generation). Practical case studies are used to demonstrate how hypothesis-generation and -interpretation technologies work. These are based on “social infrastructure” applications like in-bound tourism, disaster management, lunar and planetary exploration, and treatment of infectious diseases.

The novel methods and technologies proposed in Hypothesis Generation and Interpretation are supported by the incorporation of historical perspectives on science and an emphasis on the origin and development of the ideas behind their design principles and patterns.

In the Big Data era, characterized by volume, variety, and velocity, which generates a large amount of diverse data at high speed, the role of a hypothesis is more important to generate the final value, and such a hypothesis is more complicated and complex than ever. At the same time, the era of Big Data creates new vague concerns for end users as to whether Big Data relevant to them will be used appropriately.

Generating hypotheses in advance determines not only the success of scientific discoveries but also the success of investments in business and government. A model is constructed by abstracting the generated hypothesis. By executing the model, individual values and judgments are made as results.

Recently, a framework that allows development by low code has been proposed instead of traditional high-code development. In a low-code framework, it is possible to build an application by combining the components provided in advance through an interactive user interface. If low code is available, it may be possible to use it as it is, in order to explain the use of big data. However, this method is not always applicable because the low-code framework has limitations to flexible customization and not all applications can be developed only with low code. In any case, a method that is more abstract than a program and does not depend on individual programming languages (e.g., Python and Java) is required. In other words, it is necessary to be able to describe the procedures as the meanings of the programs.

Academic investigators and practitioners working on the further development and application of hypothesis generation and interpretation in Big Data computing, with backgrounds in Data Science and engineering, or the study of problem solving and scientific methods or who employ those ideas in fields like Machine Learning will find this book of considerable interest.

Contents: