Stories with Tag EDA

• Things I would have told myself before building an autorouter

  permalink

  Posted: 2025-03-28 00:38:53

  Verbose tl;dr

  Building an autorouter is significantly more complex than it initially appears. It's crucial to narrow the scope drastically, focusing on a specific problem subset like single-layer PCBs or a particular routing style. Thorough upfront research and experimentation with existing tools and algorithms is essential, as is a deep understanding of graph theory and computational geometry. Be prepared for substantial debugging and optimization, especially around performance bottlenecks, and recognize the importance of iterative development with constant testing and feedback. Don't underestimate the value of visualization for both debugging and user interaction, and choose your data structures and algorithms wisely with future scalability in mind. Finally, recognize that perfect routing is often computationally intractable, so aim for "good enough" solutions and prioritize practical usability.

  The author of "13 Things I Would Have Told Myself Before Building an Autorouter" reflects on the arduous journey of developing an autorouting tool, offering a comprehensive list of lessons learned through experience. He emphasizes the immense complexity inherent in such a project, cautioning against underestimating the scope. The initial naive assumption that existing algorithms would suffice proved drastically incorrect, leading to a deeper appreciation for the nuances and intricacies of routing.

  He stresses the critical importance of meticulously defining the problem and establishing clear objectives before diving into development. Without a precise understanding of the constraints and desired outcomes, the process can quickly become unwieldy. Furthermore, he underscores the value of iterative development and incremental progress, advocating for a phased approach with continuous testing and refinement rather than striving for an all-encompassing solution from the outset.

  The author details the challenges of data representation, highlighting the significance of choosing an appropriate data structure that can efficiently handle the vast amounts of information involved in routing. He also emphasizes the need for robust error handling and debugging strategies, given the inevitability of encountering unexpected issues and edge cases. Performance optimization becomes a paramount concern, requiring careful consideration of algorithms and data structures to ensure responsiveness and scalability.

  A key takeaway is the realization that achieving perfection is an unrealistic goal in autorouting. Instead, the focus should shift towards finding a balance between performance, accuracy, and practicality. The author learned the hard way that attempting to cater to every possible scenario can lead to an overly complex and ultimately less effective solution.

  He further discusses the importance of understanding the specific requirements of the target application. Generic autorouting algorithms often fall short when applied to specialized domains, necessitating customization and adaptation. He also advises against neglecting the user interface and user experience, recognizing that even the most powerful autorouting engine is useless if it's not accessible and intuitive to use. Finally, the author emphasizes the continuous nature of development, acknowledging that an autorouter is never truly "finished" and requires ongoing maintenance, updates, and improvements to stay relevant and effective.

  • autorouting
  • PCB Design
  • electronic design automation
  • EDA
  • Software Development
  • lessons learned
  • algorithms
  • Graph Theory
  • Routing
  • optimization
  • Computer Science
  • Engineering

  Summary of Comments ( 76 )
  https://news.ycombinator.com/item?id=43499992

  Verbose tl;dr

  Hacker News users generally praised the author's transparency and the article's practical advice for aspiring software developers. Several commenters highlighted the importance of focusing on a specific niche and iterating quickly based on user feedback, echoing the author's own experience. Some discussed the challenges of marketing and the importance of understanding the target audience. Others appreciated the author's honesty about the struggles of building a business, including the financial and emotional toll. A few commenters also offered technical insights related to autorouting and pathfinding algorithms. Overall, the comments reflect a positive reception to the article's pragmatic and relatable approach to software development and entrepreneurship.

  The Hacker News post "Things I would have told myself before building an autorouter" sparked a brief but interesting discussion with a few insightful comments. No one seemed to directly challenge the author's points from the original blog post, but rather expanded upon them or offered related anecdotes and perspectives.

  One commenter highlighted the importance of understanding the specific problem domain before diving into a complex project like autorouting. They mentioned that often, 80% of the routes are "trivial" and easily solved, while the remaining 20% present the real challenge. This commenter emphasized focusing on that difficult 20% and understanding its nuances, rather than getting bogged down in optimizing for the easy cases. They provided an example from their experience with PCB routing, where certain high-speed signals required specialized handling and couldn't be treated generically.

  Another commenter echoed this sentiment by discussing the "long tail" of edge cases in software development. They agreed that optimizing for the common scenarios is often straightforward, but dealing with the unusual or unexpected situations is where the real difficulty lies. This reinforces the idea of needing deep domain expertise to anticipate and handle these less frequent but crucial scenarios in autorouting.

  A third comment shifted the focus slightly to the importance of iterative development and testing. They described a similar project where they initially aimed for a perfect solution but later realized the value of incremental improvements. This commenter advocated for starting with a simple, working system and gradually enhancing it based on real-world feedback and testing. They suggested that this approach allows for faster learning and adaptation, ultimately leading to a more robust and practical solution.

  Finally, one commenter briefly mentioned the concept of constraint solvers and their potential relevance to autorouting problems. While they didn't elaborate extensively, this comment hinted at alternative approaches and tools that might be valuable in this domain.

  Overall, the comments on Hacker News provided valuable additions to the original blog post by emphasizing the importance of understanding the problem domain, focusing on edge cases, and adopting an iterative development approach. They offered practical insights and relatable experiences, further enriching the discussion around the challenges of building an autorouter.

• Your AI Can't See Gorillas

  permalink

  Posted: 2025-02-05 16:33:55

  Verbose tl;dr

  Large language models (LLMs) excel at mimicking human language but lack true understanding of the world. The post "Your AI Can't See Gorillas" illustrates this through the "gorilla problem": LLMs fail to identify a gorilla subtly inserted into an image captioning task, demonstrating their reliance on statistical correlations in training data rather than genuine comprehension. This highlights the danger of over-relying on LLMs for tasks requiring real-world understanding, emphasizing the need for more robust evaluation methods beyond benchmarks focused solely on text generation fluency. The example underscores that while impressive, current LLMs are far from achieving genuine intelligence.

  Chiraag Gohel's blog post, "Your AI Can't See Gorillas," delves into the critical yet often overlooked aspect of exploratory data analysis (EDA) when working with large language models (LLMs). The central argument revolves around the inherent limitations of LLMs in fully comprehending the nuances and complexities within datasets, particularly those containing unstructured or semi-structured data like text. Gohel utilizes the metaphor of a gorilla in a dataset, representing an unexpected or anomalous pattern that, while potentially obvious to a human observer conducting thorough EDA, might remain entirely invisible to an LLM.

  He meticulously illustrates this point through several practical examples. He demonstrates how relying solely on aggregate metrics, like average sentiment or topic distribution, can mask underlying issues. A seemingly positive average sentiment, for instance, could conceal a significant subset of highly negative sentiments within the dataset. He further emphasizes the importance of visualizing the data through histograms and scatter plots, techniques that allow for the identification of outliers, unusual distributions, and other irregularities that could indicate data quality problems or reveal hidden insights. These visualizations, Gohel argues, are analogous to a human "seeing" the gorilla, something an LLM, operating primarily on statistical patterns, might miss.

  The post elaborates on the crucial role of human intuition and domain expertise in interpreting the findings from EDA. While LLMs excel at processing vast quantities of data and identifying statistical correlations, they lack the contextual understanding and critical thinking abilities necessary to make sense of these correlations in a meaningful way. Gohel stresses that EDA should not be viewed as a mere preprocessing step but as an iterative and interactive process involving continuous exploration, questioning, and refinement of understanding. This involves going beyond simply calculating summary statistics and diving deeper into the data to uncover hidden patterns and potential biases.

  Furthermore, the post highlights the dangers of deploying LLMs without adequate EDA, warning that this can lead to biased, inaccurate, or even harmful outcomes. By bypassing thorough EDA, developers risk perpetuating existing biases present in the data, leading to models that reinforce these biases and produce unfair or discriminatory results.

  In conclusion, Gohel's "Your AI Can't See Gorillas" serves as a potent reminder of the indispensable role of human-driven EDA in the age of LLMs. It underscores the limitations of relying solely on automated analysis and advocates for a more nuanced and iterative approach that combines the computational power of LLMs with the critical thinking and domain expertise of human analysts. This combined approach, he argues, is essential for developing robust, reliable, and ethically sound AI systems.

  • AI
  • artificial intelligence
  • Large Language Models
  • LLMs
  • computer vision
  • Image Recognition
  • Bias in AI
  • AI Limitations
  • Data Bias
  • Exploratory Data Analysis
  • EDA
  • machine learning
  • Gorilla
  • Adversarial Examples
  • AI Safety

  Summary of Comments ( 119 )
  https://news.ycombinator.com/item?id=42950976

  Verbose tl;dr

  Hacker News users discussed the limitations of LLMs in visual reasoning, specifically referencing the "gorilla" example where models fail to identify a prominent gorilla in an image while focusing on other details. Several commenters pointed out that the issue isn't necessarily "seeing," but rather attention and interpretation. LLMs process information sequentially and lack the holistic view humans have, thus missing the gorilla because their attention is drawn elsewhere. The discussion also touched upon the difference between human and machine perception, and how current LLMs are fundamentally different from biological visual systems. Some expressed skepticism about the author's proposed solutions, suggesting they might be overcomplicated compared to simply prompting the model to look for a gorilla. Others discussed the broader implications of these limitations for safety-critical applications of AI. The lack of common sense reasoning and inability to perform simple sanity checks were highlighted as significant hurdles.

  The Hacker News post "Your AI Can't See Gorillas" (linking to an article about LLMs and Exploratory Data Analysis) has several comments discussing the limitations of LLMs, particularly in tasks requiring visual or spatial reasoning.

  Several commenters point out that the "gorilla" problem isn't specific to AI, but a broader issue of attention and perception. Humans, too, can miss obvious details when their focus is elsewhere, referencing the famous "invisible gorilla" experiment. This suggests the issue is less about the type of intelligence (artificial or biological) and more about the nature of attention itself.

  One commenter suggests the article title is misleading, arguing that the problem lies not in the LLM's inability to "see," but its lack of training on tasks requiring visual analysis and object recognition. They argue that specialized models, like those trained on image data, can "see" gorillas.

  Another commenter highlights the importance of incorporating diverse data sources and modalities into LLMs, moving beyond text to encompass images, videos, and other sensory inputs. This would allow the models to develop a more comprehensive understanding of the world and perform tasks requiring visual or spatial reasoning, like identifying a gorilla in an image.

  The discussion also touches upon the challenges of evaluating LLM performance. One commenter emphasizes that standard metrics may not capture the nuances of complex real-world tasks, and suggests focusing on specific capabilities rather than general intelligence.

  Some commenters delve into the technical aspects of LLMs, discussing the role of attention mechanisms and the potential for future development. They suggest that incorporating external tools and APIs could augment LLM capabilities, enabling them to access and process visual information.

  A few comments express skepticism about the article's premise, arguing that LLMs are simply tools and should not be expected to possess human-like perception or intelligence. They emphasize the importance of understanding the limitations of these models and using them appropriately.

  Finally, there's a brief discussion about the practical implications of these limitations, particularly in fields like data analysis and scientific discovery. Commenters suggest that LLMs can still be valuable tools, but human oversight and critical thinking remain essential.