Название: Designing the AI-Driven Data Foundations: Architecture, Principles, and Practice
Автор: Sanjeev Mohan
Издательство: Wiley
Год: 2026
Страниц: 272
Язык: английский
Формат: pdf, epub (true)
Размер: 10.1 MB

Build AI-powered data platforms that unify analytics and intelligence.

The playbook for data and AI architecture has been rewritten. This is the guide to what comes next.

For decades, data architectures were optimized for a predictable set of workloads. AI changes every layer of the stack where autonomous agents, not just humans, consume data. The architectures that worked for traditional analytics cannot meet these demands.

Designing the AI-Driven Data Foundations is a comprehensive guide to architecting platforms for this new reality.

Drawing on his experience as principal analyst at SanjMo and former VP of Research at Gartner, where he advised thousands of enterprises on data strategy, renowned analyst Sanjeev Mohan delivers vendor-neutral guidance for navigating a landscape where yesterday's best practices no longer apply. The book systematically unpacks each layer of contemporary data stacks, from operational and analytical data stores through ingestion, integration, analytics, generative AI, governance, security, privacy, and operations. The data architecture landscape is experiencing unprecedented disruption. AI has catalyzed changes across the entire data management stack: hardware optimized for vector operations, engineering practices reimagined for unstructured content, and consumption patterns transformed by autonomous agents. Designing the AI-Driven Data Foundations examines how the rise of unstructured data and AI agents as first-class consumers are redefining architectural principles that have guided the industry for decades.

If you are a data architect being told to “make the data AI-ready” without a clear definition of what that means, this book is for you. If you are a CDO watching your organization spin up AI pilots that keep stalling because the underlying data is fragmented, ungoverned, or simply inaccessible to agents, this book is for you. If you have spent the last two years hearing that AI will transform everything, while you struggle to get a single RAG implementation into production, this book is for you. The gap between AI ambition and data reality has never been wider, and closing it requires more than enthusiasm. It requires architecture, discipline, and honest evaluation of trade-offs. This book exists because I confronted those same gaps. Writing about a subject forces a different kind of rigor than speaking or advising about it.

Apache Arrow has emerged as the standard for in-memory analytical data interchange. Arrow's columnar layout matches how analytical engines process data, so multiple compute engines can share data without serialization overhead. When a Spark job prepares features for model training and passes them to a Python-based training framework, such as PyTorch or TensorFlow, Arrow eliminates the conversion cost that would occur with row-oriented protocols. By storing data in columns, Arrow enables vectorized processing with SIMD instructions, direct access to relevant columns without deserializing entire rows, and efficient compression. The adoption of Arrow extends beyond data transfer protocols to query engine implementations. Apache DataFusion, a query engine written in Rust, uses Arrow as its native in-memory format and offers both SQL and DataFrame interfaces through Python. Velox, a C++ database acceleration library, provides reusable high-performance components for filters, joins, and aggregations using Arrow for memory representation.

You'll also discover:
• A unified and comprehensive data and AI strategy
• Techniques to avoid vendor lock-in
• Frameworks for evaluating data stores
• Contextual data integration pattern that replaces ETL for AI workloads
• Coverage of AI-ready data governance, quality and security
• DataOps and observability practices that operationalize trust at scale

Who This Book Is For:
This book lays out the process of building an AI-driven data foundation in a linear, practical way. It balances generalized principles with actionable advice. The orientation is largely bottom-up: it examines individual technologies and then pieces them together into cohesive solutions and operational models. Many chapters also lay out essential evaluation criteria, challenges unique to the AI data landscape, and emerging trends. Every successful project combines people, process, and technology; this book primarily addresses the technical concerns of the AI era. It is specifically intended for practitioners and decision-makers within organizations deeply involved with projects that increasingly hinge on AI.

The roles that will find this book most relevant include:
• Executives steering AI strategy (CDO, CTO, CIO, CDAO)
• Architects designing data and AI blueprints
• Data and ML engineering teams
• Analytics and data science professionals
• Governance and security leaders ensuring responsible AI deployment
• Operations teams spanning DevOps, DataOps, and related disciplines

There is another reason this book matters now. As AI tools become embedded in daily workflows, work is not shrinking. It is intensifying. AI accelerates individual tasks, raising expectations for speed and output; in turn, this drives greater reliance on AI, expanding the scope of what each person attempts. It is a self-reinforcing cycle. Meanwhile, traditional role boundaries are dissolving. Product managers are writing code. Data scientists are building pipelines. Researchers are doing engineering work. Everyone touches data infrastructure now, whether they planned to or not. This book is written for that world. Whether you are an architect rethinking your blueprints or a product leader who suddenly finds yourself making data platform decisions, the foundations covered here will ground your work in principles that hold up under AI-driven intensity.

Perfect for data architects, and technical leaders who must make consequential platform decisions, Designing the AI-Driven Data Foundations translates the complexity of data and AI infrastructure into actionable architectural decisions that serve both human analysts and autonomous agents.