Stories with Tag RAG

• Show HN: Morphik – Open-source RAG that understands PDF images, runs locally

  permalink

  Posted: 2025-04-22 16:18:41

  Verbose tl;dr

  Morphik is an open-source Retrieval Augmented Generation (RAG) engine designed for local execution. It differentiates itself by incorporating optical character recognition (OCR), enabling it to understand and process information contained within PDF images, not just text-based PDFs. This allows users to build knowledge bases from scanned documents and image-heavy files, querying them semantically via a natural language interface. Morphik offers a streamlined setup process and prioritizes data privacy by keeping all information local.

  The GitHub repository introduces Morphik, an open-source Retrieval Augmented Generation (RAG) system designed for comprehensive document understanding, particularly excelling in processing Portable Document Format (PDF) files, including those containing image-based content. Unlike cloud-based RAG solutions, Morphik emphasizes local execution, offering enhanced privacy and control over data. Its functionality is built around efficient vector embeddings that capture the semantic meaning of the text and image components within PDF documents. These embeddings facilitate rapid and accurate retrieval of relevant information when queried. The system's ability to interpret images within PDFs differentiates it from many existing RAG implementations that primarily focus on textual data. By leveraging optical character recognition (OCR), Morphik extracts textual information from scanned documents and images, enabling them to be included in the knowledge base and subsequently retrieved via semantic search. This local, image-aware approach empowers users to build knowledge bases from their own PDF collections without relying on external services, maintaining data security and confidentiality. The open-source nature of Morphik encourages community contributions and allows for customization and adaptability to diverse use cases, from personal knowledge management to enterprise-level document processing. The project aims to provide a robust and versatile tool for leveraging the information locked within complex PDF documents, making it readily accessible and searchable through a local, privacy-preserving architecture.

  • RAG
  • Retrieval Augmented Generation
  • Open Source
  • PDF
  • Image Understanding
  • OCR
  • Local Deployment
  • document analysis
  • information retrieval
  • knowledge management
  • Morphik
  • HN
  • Hacker News
  • AI
  • artificial intelligence
  • machine learning

  Summary of Comments ( 1 )
  https://news.ycombinator.com/item?id=43763814

  Verbose tl;dr

  HN users generally expressed interest in Morphik, praising its local operation and potential for privacy. Some questioned the licensing (AGPLv3) and its suitability for commercial applications. Several commenters discussed the challenges of accurate OCR, particularly with complex or unusual PDFs, and hoped for future improvements in this area. Others compared it to existing tools, with some suggesting integration with tools like LlamaIndex. There was significant interest in its ability to handle images within PDFs, a feature lacking in many other RAG solutions. A few users pointed out potential use cases, such as academic research and legal document analysis. Overall, the reception was positive, with many eager to experiment with Morphik and contribute to its development.

  The Hacker News post "Show HN: Morphik – Open-source RAG that understands PDF images, runs locally" (https://news.ycombinator.com/item?id=43763814) has generated a modest number of comments, primarily focusing on the practicalities and potential applications of the Morphik project.

  One commenter expressed enthusiasm for the project, highlighting the challenge of extracting information from image-based PDFs and appreciating Morphik's local processing capability. They specifically mentioned the difficulty of dealing with scanned documents and the desire for a self-hosted solution, praising Morphik for addressing these needs.

  Another commenter questioned the method used for OCR, wondering if it relied on Tesseract or a different approach. This commenter also inquired about the handling of mathematical formulas within the PDFs, indicating an interest in the project's ability to extract and understand complex information.

  A further comment delved into the performance aspects of the project, particularly regarding memory usage. The commenter inquired about the RAM requirements, expressing concern about potential memory limitations, especially with large PDF files. They also touched upon scalability and the ability to process a high volume of documents.

  One user provided a concise but valuable comment, pointing out a potential licensing issue. They suggested that the project's use of Apache 2.0 licensed Tesseract might conflict with the AGPLv3 license chosen for Morphik. This raises a significant legal consideration for the project maintainers.

  Finally, another commenter made a brief, neutral observation about the project's reliance on Docker for deployment. While not expressing an opinion, this comment highlights a technical aspect of Morphik's implementation.

  Overall, the comments on Hacker News demonstrate genuine interest in the Morphik project, focusing on its practical utility, technical aspects, and potential licensing issues. They highlight the demand for tools that can effectively process image-based PDFs locally, while also raising important questions about performance, scalability, and licensing compliance.

• Mayo Clinic's secret weapon against AI hallucinations: Reverse RAG in action

  permalink

  Posted: 2025-03-11 20:21:43

  Verbose tl;dr

  Mayo Clinic is combating AI "hallucinations" (fabricating information) with a technique called "reverse retrieval-augmented generation" (Reverse RAG). Instead of feeding context to the AI before it generates text, Mayo's system generates text first and then uses retrieval to verify the generated information against a trusted knowledge base. If the AI's output can't be substantiated, it's flagged as potentially inaccurate, helping ensure the AI provides only evidence-based information, crucial in a medical context. This approach prioritizes accuracy over creativity, addressing a major challenge in applying generative AI to healthcare.

  The VentureBeat article, "Mayo Clinic's secret weapon against AI hallucinations: Reverse RAG in action," details a novel approach employed by the Mayo Clinic to combat the pervasive issue of "hallucinations" in large language models (LLMs), specifically within the context of medical applications. These hallucinations, technically known as fabrications, manifest as the LLM confidently generating factually incorrect or entirely invented information, posing a significant risk in a field where accuracy is paramount. Rather than relying solely on traditional Retrieval Augmented Generation (RAG), which retrieves relevant information from a knowledge base to inform the LLM's response, the Mayo Clinic has pioneered a technique referred to as "reverse RAG."

  In traditional RAG, the LLM receives a user query, searches a connected knowledge base for pertinent information, and then uses this retrieved information to construct its response. Reverse RAG inverts this process. After the LLM generates its initial response, the system employs a secondary retrieval step. This secondary retrieval uses the LLM-generated answer as the query to search the knowledge base. The goal is to locate corroborating evidence within the established, trusted medical knowledge base that supports the LLM’s assertions. If the system finds supporting documentation, it bolsters confidence in the LLM's response. Conversely, if the system cannot find supporting evidence, it flags the LLM’s output as potentially unreliable, alerting users to the possibility of a hallucination.

  This approach offers several advantages. It provides a mechanism for verifying the factual accuracy of the LLM's output, thereby mitigating the risk of propagating misinformation. It also allows for the identification of the source material supporting the LLM's claims, enhancing transparency and facilitating further investigation if needed. Furthermore, this reverse retrieval process doesn't merely confirm or deny; it also allows for refinement. If the retrieved information partially supports the LLM's answer but also contains additional relevant details, the system can use these details to augment and improve the initial response, leading to more comprehensive and accurate information delivery. The article underscores that this methodology is particularly crucial in healthcare, where misinformation can have serious consequences. By implementing reverse RAG, the Mayo Clinic is working towards harnessing the power of LLMs while simultaneously safeguarding against their inherent fallibility, paving the way for more responsible and dependable AI integration in the medical field.

  • Mayo Clinic
  • AI
  • artificial intelligence
  • hallucinations
  • RAG
  • Retrieval Augmented Generation
  • Reverse RAG
  • Large Language Models
  • LLMs
  • Medical AI
  • Healthcare AI
  • AI Safety
  • AI Reliability
  • Fact Verification
  • information retrieval
  • VentureBeat

  Summary of Comments ( 42 )
  https://news.ycombinator.com/item?id=43336609

  Verbose tl;dr

  Hacker News commenters discuss the Mayo Clinic's "reverse RAG" approach, expressing skepticism about its novelty and practicality. Several suggest it's simply a more complex version of standard prompt engineering, arguing that prepending context with specific instructions or questions is a common practice. Some question the scalability and maintainability of a large, curated knowledge base for every specific use case, highlighting the ongoing challenge of keeping such a database up-to-date and relevant. Others point out potential biases introduced by limiting the AI's knowledge domain, and the risk of reinforcing existing biases present in the curated data. A few commenters note the lack of clear evaluation metrics and express doubt about the claimed 40% hallucination reduction, calling for more rigorous testing and comparisons to simpler methods. The overall sentiment leans towards cautious interest, with many awaiting further evidence of the approach's real-world effectiveness.

  The Hacker News post titled "Mayo Clinic's secret weapon against AI hallucinations: Reverse RAG in action" has generated several comments discussing the concept of Reverse Retrieval Augmented Generation (Reverse RAG) and its application in mitigating AI hallucinations.

  Several commenters express skepticism about the novelty and efficacy of Reverse RAG. One commenter points out that the idea of checking the source material isn't new, and that existing systems like Perplexity.ai already implement similar fact-verification methods. Another echoes this sentiment, suggesting that the article is hyping a simple concept and questioning the need for a new term like "Reverse RAG." This skepticism highlights the view that the core idea isn't groundbreaking but rather a rebranding of existing fact-checking practices.

  There's discussion about the practical limitations and potential downsides of Reverse RAG. One commenter highlights the cost associated with querying a vector database for every generated sentence, arguing that it might be computationally expensive and slow down the generation process. Another commenter raises concerns about the potential for confirmation bias, suggesting that focusing on retrieving supporting evidence might inadvertently reinforce existing biases present in the training data.

  Some commenters delve deeper into the technical aspects of Reverse RAG. One commenter discusses the challenges of handling negation and nuanced queries, pointing out that simply retrieving supporting documents might not be sufficient for complex questions. Another commenter suggests using a dedicated "retrieval model" optimized for retrieval tasks, as opposed to relying on the same model for both generation and retrieval.

  A few comments offer alternative approaches to address hallucinations. One commenter suggests generating multiple answers and then selecting the one with the most consistent supporting evidence. Another commenter proposes incorporating a "confidence score" for each generated sentence, reflecting the strength of supporting evidence.

  Finally, some commenters express interest in learning more about the specific implementation details and evaluation metrics used by the Mayo Clinic, indicating a desire for more concrete evidence of Reverse RAG's effectiveness. One user simply states their impression that the Mayo Clinic is making impressive strides in using AI in healthcare.

  In summary, the comments on Hacker News reveal a mixed reception to the concept of Reverse RAG. While some acknowledge its potential, many express skepticism about its novelty and raise concerns about its practicality and potential drawbacks. The discussion highlights the ongoing challenges in addressing AI hallucinations and the need for more robust and efficient solutions.

• Show HN: In-Browser Graph RAG with Kuzu-WASM and WebLLM

  permalink

  Posted: 2025-03-10 15:12:57

  Verbose tl;dr

  This blog post demonstrates a Retrieval Augmented Generation (RAG) pipeline running entirely within a web browser. It uses Kuzu-WASM, a WebAssembly build of the Kuzu graph database, to store and query a knowledge graph, and WebLLM, a library for running large language models (LLMs) client-side. The demo allows users to query the graph using natural language, with Kuzu translating the query into its native query language and retrieving relevant information. This retrieved context is then fed to a local LLM (currently, a quantized version of Flan-T5), which generates a natural language response. This in-browser approach offers potential benefits in terms of privacy, reduced latency, and offline functionality, enabling new possibilities for interactive and personalized AI applications.

  This blog post introduces a novel approach to implementing Retrieval Augmented Generation (RAG) entirely within a web browser, leveraging the power of Kuzu-WASM, a WebAssembly port of the Kuzu graph database, and WebLLM, a library for running large language models (LLMs) client-side. The post demonstrates how these technologies can be combined to create a powerful and privacy-preserving question-answering system that operates without server-side components.

  The core concept revolves around using Kuzu-WASM to store and query a knowledge graph directly in the browser. This eliminates the need for a remote database server and keeps sensitive data localized to the user's machine. The post uses a movie dataset as an example, showcasing how relationships between actors, movies, and genres can be represented within the graph. Queries written in Cypher, KuzuDB's query language, retrieve relevant information from this local graph based on user questions.

  WebLLM then enters the scene, taking the results retrieved by Kuzu-WASM and feeding them to a locally running LLM. This LLM uses the retrieved information as context to generate a comprehensive and accurate answer to the user's query. The post highlights the use of a smaller, quantized LLM model optimized for browser execution, emphasizing the potential for performance and efficiency in this client-side architecture.

  The post details the technical steps involved in setting up this in-browser RAG pipeline. It covers loading Kuzu-WASM and the chosen LLM model, populating the graph database with the movie dataset, and constructing the logic that connects user queries, graph traversal with Cypher, and LLM-powered answer generation. Code snippets are provided to illustrate the implementation.

  The authors emphasize the benefits of this approach, particularly its privacy implications. By keeping data and processing local, user information is never transmitted to a server, offering a significantly more private user experience. Furthermore, the post hints at the potential for offline functionality, suggesting that this architecture could enable powerful knowledge-based applications even without an internet connection. Finally, the post encourages readers to explore and experiment with this technology, positioning it as an exciting development in the evolution of web-based applications and AI.

  • Graph Databases
  • KuzuDB
  • Wasm
  • WebAssembly
  • WebLLM
  • RAG
  • Retrieval Augmented Generation
  • In-browser
  • client-side
  • Database
  • Knowledge Graph
  • natural language processing
  • NLP
  • javascript
  • Frontend
  • Web Development

  Summary of Comments ( 4 )
  https://news.ycombinator.com/item?id=43321523

  Verbose tl;dr

  HN commenters generally expressed excitement about the potential of in-browser graph RAG, praising the demo's responsiveness and the possibilities it opens up for privacy-preserving, local AI applications. Several users questioned the performance and scalability with larger datasets, highlighting the current limitations of WASM and browser storage. Some suggested potential applications, like analyzing personal knowledge graphs or interacting with codebases. Concerns were raised about the security implications of running LLMs client-side, and the challenge of keeping WASM binaries up-to-date. The closed-source nature of KuzuDB also prompted discussion, with some advocating for open-source alternatives. Several commenters expressed interest in trying the demo and exploring its capabilities further.

  The Hacker News post discussing in-browser graph RAG with Kuzu-WASM and WebLLM has generated several comments, offering a range of perspectives on the project.

  One commenter expresses excitement about the potential of WebAssembly for database applications, specifically highlighting the possibility of running complex queries client-side without server dependencies. They see this as a significant step toward enabling powerful and responsive web applications. They also inquire about the feasibility of using this technology with larger datasets, acknowledging the current limitations of browser storage.

  Another commenter raises a practical concern about the performance implications of handling large graph datasets within the browser. They question whether the current implementation can efficiently manage substantial graphs and suggest that server-side processing might be more suitable for complex graph operations on large datasets. This comment highlights a common trade-off between client-side convenience and server-side performance when dealing with data-intensive applications.

  A further comment delves into the specifics of the technology, mentioning the use of Apache Arrow for data serialization. They posit that this choice could be contributing to performance bottlenecks, particularly when transferring data between JavaScript and WebAssembly. They suggest exploring alternative serialization methods or optimizing the data transfer process to improve overall efficiency.

  Another individual inquires about the licensing of the project, expressing interest in its potential applications. This highlights the importance of clear licensing information for open-source projects to encourage adoption and collaboration.

  The discussion also touches upon the security implications of running database queries within the browser environment. One comment raises the concern of potential vulnerabilities arising from client-side execution and suggests that careful consideration should be given to security best practices.

  Finally, a commenter expresses enthusiasm for the project's potential to democratize access to graph databases, making them more accessible to developers and users without requiring specialized server infrastructure. They see this as a positive step towards empowering individuals and smaller organizations to leverage the power of graph technology.

  In summary, the comments on the Hacker News post reflect a general interest in the project while also raising important questions and concerns regarding performance, scalability, security, and licensing. The discussion highlights the potential benefits and challenges of bringing graph database technology to the browser environment.

• Enhancing Frame Detection with Retrieval Augmented Generation

  permalink

  Posted: 2025-02-28 17:25:06

  Verbose tl;dr

  This paper introduces FRAME, a novel approach to enhance frame detection – the task of identifying predefined semantic roles (frames) and their corresponding arguments (roles) in text. FRAME leverages Retrieval Augmented Generation (RAG) by retrieving relevant frame-argument examples from a large knowledge base during both frame identification and argument extraction. This retrieved information is then used to guide a large language model (LLM) in making more accurate predictions. Experiments demonstrate that FRAME significantly outperforms existing state-of-the-art methods on benchmark datasets, showing the effectiveness of incorporating retrieved context for improved frame detection.

  The arXiv preprint "Enhancing Frame Detection with Retrieval Augmented Generation" introduces a novel approach to improve the performance of frame detection, a crucial task in Natural Language Processing (NLP) that involves identifying and classifying semantic frames, which represent stereotyped situations and their participants. Frame detection encompasses identifying the presence of a frame within a given text and subsequently labeling the semantic roles (frame elements) of the words or phrases that fill the frame's slots. The traditional methods for frame detection, primarily relying on supervised machine learning models trained on annotated data, often struggle with data scarcity, especially for less common frames. Furthermore, these models can exhibit brittleness when faced with out-of-distribution examples or nuanced language variations.

  This paper proposes leveraging the power of Retrieval Augmented Generation (RAG) to address these limitations. RAG combines the strengths of information retrieval and sequence-to-sequence generation. Instead of relying solely on trained parameters, the proposed method retrieves relevant contextual examples from a large corpus based on the input text. These retrieved examples, which may contain instances of the target frame or semantically related frames, provide valuable contextual information that can guide the frame detection process. The core idea is to augment the input to the frame detection model with these retrieved examples, effectively enriching the input representation with external knowledge and enabling the model to make more informed decisions.

  The authors implement this RAG-based frame detection approach using a two-stage process. The first stage involves retrieving relevant sentences from a large text corpus using a dense retrieval method. These retrieved sentences are then used to create a prompt for the second stage, which employs a sequence-to-sequence generation model. The prompt consists of the input sentence concatenated with the retrieved sentences, effectively providing the generation model with additional contextual information. The generation model is then tasked with generating the frame and corresponding frame element labels for the input sentence.

  The authors evaluate their proposed method on two benchmark datasets commonly used in frame detection research, demonstrating significant improvements in performance compared to existing state-of-the-art methods. These results suggest that the integration of retrieved contextual information through RAG significantly enhances the ability of the model to identify and classify frames, especially in scenarios with limited training data or complex linguistic phenomena. Furthermore, the paper explores different retrieval strategies and prompt engineering techniques to optimize the effectiveness of the RAG framework for frame detection, providing valuable insights into the practical implementation and optimization of this approach. The authors conclude that the proposed RAG-based framework offers a promising avenue for improving frame detection and potentially other related NLP tasks by effectively leveraging external knowledge and contextual information.

  • Frame Detection
  • Retrieval Augmented Generation
  • RAG
  • computer vision
  • artificial intelligence
  • AI
  • image processing
  • Object Detection
  • deep learning
  • machine learning
  • information retrieval

  Summary of Comments ( 3 )
  https://news.ycombinator.com/item?id=43208096

  Verbose tl;dr

  Several Hacker News commenters express skepticism about the claimed improvements in frame detection offered by the paper's retrieval-augmented generation (RAG) approach. Some question the practical significance of the reported performance gains, suggesting they might be marginal or attributable to factors other than the core RAG mechanism. Others point out the computational cost of RAG, arguing that simpler methods might achieve similar results with less overhead. A recurring theme is the need for more rigorous evaluation and comparison against established baselines to validate the effectiveness of the proposed approach. A few commenters also discuss potential applications and limitations of the technique, particularly in resource-constrained environments. Overall, the sentiment seems cautiously interested, but with a strong desire for further evidence and analysis.

  The Hacker News post "Enhancing Frame Detection with Retrieval Augmented Generation" (linking to arXiv preprint 2502.12210) has generated a modest number of comments, primarily focusing on the practicality and potential limitations of the proposed method.

  One commenter questions the real-world applicability of the technique, specifically in situations with a large number of classes (e.g., hundreds or thousands). They express skepticism that maintaining a separate retrieval database for each class would be scalable or efficient. This concern highlights the potential trade-off between improved accuracy and computational cost, a common theme in machine learning applications.

  Another comment builds on this concern by pointing out that the approach seems tailored to very specific, pre-defined scenarios, making it less generalizable than desired. They suggest that the need for pre-defined "frames" limits its adaptability to novel situations or unforeseen contexts. This resonates with a broader discussion in AI about the balance between specialized solutions and more adaptable, general-purpose models.

  A further comment delves into the technical details, questioning the choice of cosine similarity as the primary metric for retrieval. They propose exploring alternative metrics that might be more suitable for certain data types or problem domains. This comment underscores the importance of carefully considering the underlying assumptions and limitations of specific mathematical tools within a larger machine learning framework.

  Finally, one commenter raises a fundamental question about the overall value proposition of the proposed approach. They wonder if the performance gains achieved justify the added complexity of incorporating a retrieval component. This comment highlights the need for rigorous evaluation and comparison with simpler, more established methods to demonstrate the actual benefits of the new technique.

  Overall, the comments on the Hacker News post express a cautious but curious perspective on the proposed method. While acknowledging the potential for improved frame detection, they raise important concerns about scalability, generalizability, and overall efficiency that warrant further investigation. The comments refrain from directly evaluating the core research within the paper, focusing instead on the practical implications and potential limitations of applying the presented techniques.