Stories with Tag mathematics

• Portrait of the Hilbert Curve (2010)

  permalink

  Posted: 2025-01-18 01:25:40

  Verbose tl;dr

  This post explores the Hilbert curve, a continuous fractal space-filling curve. The author visualizes its construction through iterative rotations and connections of smaller, U-shaped segments, demonstrating how this process generates increasingly complex patterns that effectively fill a square grid. The post further examines how points in 2D space can be mapped to a 1D position along the curve and vice-versa, highlighting the curve's applications in image processing and data organization by providing Python code examples for these conversions. The intricate visuals and detailed explanations offer a compelling portrait of the Hilbert curve's properties and practical utility.

  This blog post, "Portrait of the Hilbert Curve (2010)," delves into the fascinating mathematical construct known as the Hilbert curve, providing an in-depth exploration of its properties and an elegant Python implementation for generating its visual representation. The author begins by introducing the Hilbert curve as a continuous fractal space-filling curve, emphasizing its remarkable ability to map a one-dimensional sequence onto a two-dimensional plane while preserving locality. This means that points close to each other in the linear sequence are generally mapped to points close together in the two-dimensional space. This property makes the Hilbert curve highly relevant for diverse applications, such as image processing and spatial indexing.

  The post then meticulously dissects the recursive nature of the Hilbert curve, explaining how it's constructed through repeated rotations and concatenations of a basic U-shaped motif. It illustrates this process with helpful diagrams, showcasing the curve's evolution through successive iterations. This recursive definition forms the foundation of the Python code presented later.

  The core of the post lies in the provided Python implementation, which elegantly translates the recursive definition of the Hilbert curve into a concise and efficient algorithm. The code generates a sequence of points representing the curve's path for a given order (level of recursion), effectively mapping integer indices to corresponding coordinates in the two-dimensional plane. The author takes care to explain the logic behind the coordinate calculations, highlighting the bitwise operations used to manipulate the input index and determine the orientation and position of each segment within the curve.

  Furthermore, the post extends the basic implementation by introducing a method to draw the Hilbert curve visually. It utilizes the calculated coordinate sequence to produce a graphical representation, allowing for a clear visualization of the curve's intricate structure and space-filling properties. The author discusses the visual characteristics of the resulting curve, noting its self-similar nature and the increasing complexity with higher orders of recursion.

  In essence, "Portrait of the Hilbert Curve (2010)" provides a comprehensive and accessible introduction to this fascinating mathematical concept. It combines a clear theoretical explanation with a practical Python implementation, enabling readers to not only understand the underlying principles but also to generate and visualize the Hilbert curve themselves, fostering a deeper appreciation for its elegance and utility. The post serves as an excellent resource for anyone interested in exploring fractal geometry, space-filling curves, and their applications in various fields.

  • Hilbert Curve
  • Space-filling Curves
  • Fractals
  • mathematics
  • Visualization
  • Algorithm
  • Python
  • image processing
  • 2010
  • programming
  • computer graphics
  • Curve Generation

  Summary of Comments ( 5 )
  https://news.ycombinator.com/item?id=42744932

  Verbose tl;dr

  Hacker News users generally praised the visualization and explanation of Hilbert curves in the linked blog post. Several appreciated the interactive nature and clear breakdown of the curve's construction. Some comments delved into practical applications, mentioning its use in mapping and image processing due to its space-filling properties and locality preservation. A few users pointed out its relevance to Morton codes (Z-order curves) and their applications in databases. One commenter linked to a Python implementation for generating Hilbert curves. The overall sentiment was positive, with users finding the post educational and well-presented.

  The Hacker News post titled "Portrait of the Hilbert Curve (2010)" has a modest number of comments, focusing primarily on the mathematical and visual aspects of Hilbert curves, as well as some practical applications.

  Several commenters appreciate the beauty and elegance of Hilbert curves, describing them as "mesmerizing" and "aesthetically pleasing." One points out the connection between the increasing order of the curve and the emerging visual detail, resembling a "fractal unfolding." Another emphasizes the self-similarity aspect, where parts of the curve resemble the whole.

  The discussion also touches on the practical applications of Hilbert curves, particularly in mapping and image processing. One comment mentions their use in spatial indexing, where they can improve the efficiency of database queries by preserving locality. Another comment delves into how these curves can be used for dithering and creating visually appealing color gradients. A further comment references the use of Hilbert curves in creating continuous functions that fill space.

  A few comments delve into the mathematical properties. One commenter discusses the concept of "space-filling curves" and how the Hilbert curve is a prime example. Another explains how these curves can map a one-dimensional interval onto a two-dimensional square. The continuous nature of the curve and its relationship to fractal dimensions are also briefly mentioned.

  One commenter highlights the author's clear explanations and interactive visualizations, making the concept accessible even to those without a deep mathematical background. The code provided in the article is also praised for its clarity and simplicity.

  While there's no single overwhelmingly compelling comment, the collective discussion provides a good overview of the Hilbert curve's aesthetic, mathematical, and practical significance. The commenters generally express admiration for the curve's properties and the author's presentation.

• An alternative construction of Shannon entropy

  permalink

  Posted: 2024-11-13 16:45:13

  Verbose tl;dr

  This blog post presents a different way to derive Shannon entropy, focusing on its property as a unique measure of information content. Instead of starting with desired properties like additivity and then finding a formula that satisfies them, the author begins with a core idea: measuring the average number of binary questions needed to pinpoint a specific outcome from a probability distribution. By formalizing this concept using a binary tree representation of the questioning process and leveraging Kraft's inequality, they demonstrate that -∑pᵢlog₂(pᵢ) emerges naturally as the optimal average question length, thus establishing it as the entropy. This construction emphasizes the intuitive link between entropy and the efficient encoding of information.

  This blog post presents a different perspective on deriving Shannon entropy, distinct from the traditional axiomatic approach. Instead of starting with desired properties and deducing the entropy formula, it begins with a fundamental problem: quantifying the average number of bits needed to optimally represent outcomes from a probabilistic source. The author argues this approach provides a more intuitive and grounded understanding of why the entropy formula takes the shape it does.

  The post meticulously constructs this derivation. It starts by considering a source emitting symbols from a finite alphabet, each with an associated probability. The core idea is to group these symbols into sets based on their probabilities, specifically targeting sets where the cumulative probability is a power of two. This allows for efficient representation using binary codes, as each set can be uniquely identified by a binary prefix.

  The process begins with the most probable symbol and continues iteratively, grouping less probable symbols into progressively larger sets until all symbols are assigned. The author demonstrates how this grouping mirrors the process of building a Huffman code, a well-known algorithm for creating optimal prefix-free codes.

  The post then carefully analyzes the expected number of bits required to encode a symbol using this method. This expectation involves summing the product of the number of bits assigned to a set (which relates to the negative logarithm of the cumulative probability of that set) and the cumulative probability of the symbols within that set.

  Through a series of mathematical manipulations and approximations, leveraging the properties of logarithms and the behavior of probabilities as the number of samples increases, the author shows that this expected number of bits converges to the familiar Shannon entropy formula: the negative sum of each symbol's probability multiplied by the logarithm base 2 of that probability.

  Crucially, the derivation highlights the relationship between optimal coding and entropy. It demonstrates that Shannon entropy represents the theoretical lower bound on the average number of bits needed to encode messages from a given source, achievable through optimal coding schemes like Huffman coding. This construction emphasizes that entropy is not just a measure of uncertainty or information content, but intrinsically linked to efficient data compression and representation. The post concludes by suggesting this alternative construction offers a more concrete and less abstract understanding of Shannon entropy's significance in information theory.

  • information theory
  • Shannon entropy
  • entropy
  • mathematics
  • probability
  • coding theory
  • data compression
  • alternative construction
  • axiomatic definition
  • foundations of information theory
  • Statistics
  • Information Measurement
  • axiomatic systems
  • axiomatic derivation

  Summary of Comments ( 11 )
  https://news.ycombinator.com/item?id=42127609

  Verbose tl;dr

  Hacker News users discuss the alternative construction of Shannon entropy presented in the linked article. Some express appreciation for the clear explanation and visualizations, finding the geometric approach insightful and offering a fresh perspective on a familiar concept. Others debate the pedagogical value of the approach, questioning whether it truly simplifies understanding for those unfamiliar with entropy, or merely offers a different lens for those already versed in the subject. A few commenters note the connection to cross-entropy and Kullback-Leibler divergence, suggesting the geometric interpretation could be extended to these related concepts. There's also a brief discussion on the practical implications and potential applications of this alternative construction, although no concrete examples are provided. Overall, the comments reflect a mix of appreciation for the novel approach and a pragmatic assessment of its usefulness in teaching and application.

  The Hacker News post titled "An alternative construction of Shannon entropy," linking to an article exploring a different way to derive Shannon entropy, has generated a moderate discussion with several interesting comments.

  One commenter highlights the pedagogical value of the approach presented in the article. They appreciate how it starts with desirable properties for a measure of information and derives the entropy formula from those, contrasting this with the more common axiomatic approach where the formula is presented and then shown to satisfy the properties. They believe this method makes the concept of entropy more intuitive.

  Another commenter focuses on the historical context, mentioning that Shannon's original derivation was indeed based on desired properties. They point out that the article's approach is similar to the one Shannon employed, further reinforcing the pedagogical benefit of seeing the formula emerge from its intended properties rather than the other way around. They link to a relevant page within a book on information theory which seemingly discusses Shannon's original derivation.

  A third commenter questions the novelty of the approach, suggesting that it seems similar to standard treatments of the topic. They wonder if the author might be overselling the "alternative construction" aspect. This sparks a brief exchange with another user who defends the article, arguing that while the fundamental ideas are indeed standard, the specific presentation and the emphasis on the grouping property could offer a fresh perspective, especially for educational purposes.

  Another commenter delves into more technical details, discussing the concept of entropy as a measure of average code length and relating it to Kraft's inequality. They connect this idea to the article's approach, demonstrating how the desired properties lead to a formula that aligns with the coding interpretation of entropy.

  Finally, a few comments touch upon related concepts like cross-entropy and Kullback-Leibler divergence, briefly extending the discussion beyond the scope of the original article. One commenter mentions an example of how entropy is useful, by stating how optimizing for log-loss in a neural network can be interpreted as an attempt to make the predicted distribution very similar to the true distribution.

  Overall, the comments section provides a valuable supplement to the article, offering different perspectives on its significance, clarifying some technical points, and connecting it to broader concepts in information theory. While not groundbreaking, the discussion reinforces the importance of pedagogical approaches that derive fundamental formulas from their intended properties.