Название: Language Models in Plain English: Humans Understanding How Machines Understand Language
Автор: Austin Eovito, Marina Danilevsky
Издательство: O’Reilly Media, Inc.
Год: 2021-10-22
Страниц: 76
Язык: английский
Формат: epub
Размер: 10.2 MB

Recent advances in Machine Learning (ML) have lowered the barriers to creating and using ML models. But understanding what these models are doing has only become more difficult. We discuss technological advances with little understanding of how they work and struggle to develop a comfortable intuition for new functionality.

The recent explosion in advances in machine learning has brought a myriad of interesting, powerful, and increasingly opaque models. Simultaneously, the recent movement toward democratization of AI has lowered the barriers for being a data scientist and using machine learning models. It is simple to deploy a model in the real world without being concerned about explaining the output or without exploring the ethical implications of decisions that will be made on the basis of that output. The ease of creating and using machine learning models is going up; the ease of understanding what machine learning models are doing is going down.

If you have read about neural networks, you are probably aware that they are so called because their general structure is inspired by the observed behavior of biological neurons. However, this is not the same as saying that a neural network constructed to perform machine learning tasks works in the same way as a human brain. An artificial neural network is intended to mimic certain things the human brain does. But in the same way that eating bugs and using sonar cannot let a human know what it is like to be a bat, learning to predict the next word does not allow an LM to know what it is like to be a human. This goes both ways: the fact that our behavior inspires neural network architecture, and there is convergence in the observed outcome (a word is, indeed, predicted), does not mean that we know what it is like to be an LM.

We therefore introduce neural network language models, and all subsequent techniques in this report, by first making critical observations of how humans approach language, memory, and communication. It is important to develop good intuition about the basic building blocks of a neural network language model so that we may take on the increasingly complex models that are coming to the forefront of language modeling.