Jürgen Schmidhuber's "Matters Computational" provides a comprehensive overview of computer science, spanning its theoretical foundations and practical applications. It delves into topics like algorithmic information theory, computability, complexity theory, and the history of computation, including discussions of Turing machines and the Church-Turing thesis. The book also explores the nature of intelligence and the possibilities of artificial intelligence, covering areas such as machine learning, neural networks, and evolutionary computation. It emphasizes the importance of self-referential systems and universal problem solvers, reflecting Schmidhuber's own research interests in artificial general intelligence. Ultimately, the book aims to provide a unifying perspective on computation, bridging the gap between theoretical computer science and the practical pursuit of artificial intelligence.
Jürgen Schmidhuber's "Matters Computational (2010)" presents a comprehensive, and at times highly technical, overview of theoretical computer science, with a particular focus on its intersection with artificial intelligence and machine learning, specifically Schmidhuber's own contributions to these fields. The book, structured as a collection of previously published papers and new material, explores fundamental computational principles through the lens of algorithmic information theory, Kolmogorov complexity, and the concept of universal problem solvers.
The narrative begins with an exploration of the theoretical foundations of computation, delving into the nature of computability, Turing machines, and the limits of what can be computed. It emphasizes the importance of universality and self-reference in computation, drawing connections to Gödel's incompleteness theorems and the halting problem. Schmidhuber introduces the speed prior, a concept central to his research, which favors programs that not only solve a given problem but do so efficiently. This leads into a discussion of optimal universal search algorithms, such as Levin Search, and their implications for artificial general intelligence.
A substantial portion of the book is dedicated to algorithmic information theory and its application to inductive inference and learning. Schmidhuber argues that the shortest program capable of generating a given data sequence is the best explanation for that sequence, aligning with Occam's Razor and the principle of Minimum Description Length. This theoretical framework is used to analyze the problem of learning from data, including supervised learning, unsupervised learning, and reinforcement learning. He discusses various algorithms for finding short programs, including variations of Levin Search and methods based on Monte Carlo sampling.
The text also delves into the practical implications of these theoretical concepts, examining the design and implementation of artificial neural networks, recurrent neural networks, and other learning systems. Schmidhuber meticulously details his own contributions to the field, including the development of Long Short-Term Memory (LSTM) networks, a type of recurrent neural network architecture designed to address the vanishing gradient problem, which had previously hampered the training of deep networks. He also discusses other architectures like self-referential neural networks and the concept of Gödel machines, self-improving AI systems inspired by Gödel's incompleteness theorems.
Throughout the book, Schmidhuber emphasizes the importance of formal mathematical frameworks for understanding intelligence and the potential for creating truly intelligent machines. He rigorously analyzes the computational complexity of various learning algorithms and discusses the limitations imposed by finite resources. The text is replete with mathematical formulas, algorithms, and proofs, providing a deep dive into the theoretical underpinnings of the field. He concludes with a forward-looking perspective on the future of artificial intelligence, emphasizing the potential for creating increasingly sophisticated and capable learning systems based on the principles outlined in the book. The overarching theme is a pursuit of general purpose artificial intelligence through a formal, mathematically grounded approach centered on optimal program search and algorithmic information theory.
Summary of Comments ( 11 )
https://news.ycombinator.com/item?id=43288861
HN users discuss the density and breadth of "Matters Computational," praising its unique approach to connecting diverse computational topics. Several commenters highlight the book's treatment of randomness, floating-point arithmetic, and the FFT as particularly insightful. The author's background in physics is noted, contributing to the book's distinct perspective. Some find the book challenging, requiring multiple readings to fully grasp the concepts. The free availability of the PDF is appreciated, and its enduring relevance a decade after publication is also remarked upon. A few commenters express interest in a physical copy, while others suggest potential updates or expansions on certain topics.
The Hacker News post titled "Matters Computational (2010) [pdf]" linking to a PDF of Jörg Fliege's book "Matters Computational" has a moderate number of comments, discussing various aspects of the book and computational mathematics in general.
Several commenters praise the book's comprehensive nature and clarity. One user highlights its value as a reference for "all sorts of basic algorithms and data structures," appreciating the detailed explanations and pseudocode provided. They specifically mention its usefulness for understanding fundamental concepts like numerical stability.
Another commenter focuses on the book's treatment of linear algebra, noting its depth and accessibility, even for those without a strong mathematical background. They contrast it with other resources they found less helpful.
A few comments delve into specific topics covered in the book. One user discusses the exploration of floating-point arithmetic and its associated challenges, acknowledging the importance of understanding these concepts for anyone working with numerical computations. Another highlights the chapter on optimization, mentioning its practical value and the inclusion of various optimization algorithms.
Some commenters offer broader perspectives on computational mathematics and its role in computer science. One reflects on the importance of a strong mathematical foundation for software engineers, advocating for more emphasis on these concepts in education.
The discussion also touches on the book's availability. The author's decision to make it freely available is commended, with some users expressing gratitude for open access to such valuable educational resources. A link to the author's webpage is shared, offering further context.
While a number of commenters express interest in the book based on the description and other comments, there isn't extensive engagement in deep technical discussions. The overall sentiment is positive, with the comments primarily focusing on the book's breadth, clarity, and value as a resource for understanding fundamental computational concepts.