Stories with Tag key-value store

• Show HN: A bi-directional, persisted KV store that is faster than Redis

  permalink

  Posted: 2025-03-17 12:35:16

  Verbose tl;dr

  HPKV is a new key-value store boasting faster performance than Redis, achieved through a novel lock-free B+ tree implementation. It's bi-directional, allowing efficient retrieval by both key and value, and offers persistence to disk. Designed for embedded and server-side use cases, HPKV supports multiple languages (C, C++, Python, Java, Go, and JavaScript) and provides various features like range scans, prefix scans, and TTL. It's available under the Apache 2.0 license, promoting open-source contribution and adoption.

  The Hacker News post introduces hpkv (High-Performance Key-Value store), a novel key-value data store boasting superior performance compared to established solutions like Redis. Hpkv is designed as a bi-directional, persistent store, meaning data flows seamlessly in both directions (read and write) and is durably saved to prevent data loss upon system restart. The developers emphasize speed as a primary differentiator, claiming faster performance than Redis, a popular in-memory data store known for its speed. This performance gain is attributed to hpkv's architectural design, although the specifics are not detailed in the brief announcement.

  Persistence is a crucial feature, ensuring data reliability and eliminating the need for complex data recovery mechanisms in case of failures. Hpkv manages this persistence internally, abstracting away the complexities of data durability from the user.

  The project is open-source, inviting community contributions and scrutiny. The source code is available on GitHub, providing transparency and allowing developers to examine the implementation details. This openness fosters collaboration and potentially accelerates the project's development and refinement.

  The announcement itself is concise, primarily focusing on the core features: bi-directionality, persistence, and speed. It serves as an initial introduction to hpkv, aiming to pique interest and encourage further exploration via the provided GitHub repository. While the post highlights the key advantages, it lacks extensive technical details regarding the underlying architecture, storage engine, and specific performance benchmarks to substantiate the performance claims. This suggests the project is likely in its early stages, with more comprehensive documentation and performance evaluations expected as it matures. The emphasis on being faster than Redis positions hpkv as a potential contender in the high-performance key-value store space, targeting applications requiring rapid data access and guaranteed data durability.

  • key-value store
  • Database
  • persistence
  • High Performance
  • redis alternative
  • data storage
  • kv store
  • fast data access
  • bi-directional
  • hpkv
  • Show HN

  Summary of Comments ( 9 )
  https://news.ycombinator.com/item?id=43387834

  Verbose tl;dr

  Hacker News users discussed the performance claims of hpkv, questioning the benchmark methodology and the choice of Redis as a comparison point. Several commenters pointed out that using redis-benchmark with a pipeline size of 1 is unfair to Redis, significantly hindering its performance. Others suggested alternative benchmarking tools and emphasized the importance of real-world workload simulations. The lack of detail about hpkv's persistence mechanism and data safety guarantees also drew scrutiny. Some expressed interest in the project but desired more information about its architecture and use cases. A few users pointed out potential bugs in the benchmarking script itself, further questioning the validity of the presented results.

  The Hacker News post "Show HN: A bi-directional, persisted KV store that is faster than Redis" linking to hpkv.io generated a moderate number of comments, primarily focusing on technical aspects and comparisons to existing solutions.

  Several commenters expressed skepticism regarding the performance claims, particularly the assertion of being "faster than Redis." They pointed out the need for more rigorous benchmarking and detailed methodology to substantiate such a claim. Specific concerns included the lack of clarity on the types of benchmarks run, the hardware used, and the specific Redis configuration being compared against. Some users requested benchmark results using established tools like redis-benchmark to provide a more standardized comparison.

  Discussion also arose around the choice of language (Rust) and its impact on performance. While some lauded Rust's speed and memory safety, others questioned whether these advantages alone could justify the performance claims, suggesting that algorithmic optimizations and data structures likely played a more significant role.

  The project's novelty and potential use cases were also points of discussion. Some commenters saw value in the bi-directional nature of the key-value store, exploring potential applications in areas like graph databases and indexing. Others questioned the practical benefits of bi-directionality, suggesting that existing solutions with appropriate indexing could achieve similar functionality.

  The persistence aspect of hpkv also drew some attention, with queries about the specific mechanisms employed for data persistence and the potential performance implications of these choices. Commenters also inquired about data durability guarantees and crash recovery capabilities.

  A few commenters shared their own experiences with similar projects and offered alternative approaches to achieving high-performance key-value storage. They mentioned existing databases and libraries known for their speed and efficiency, suggesting that the author explore these for potential inspiration or comparison.

  Overall, the comments reflect a cautious but curious reception to the project. While acknowledging the potential of hpkv, many commenters highlighted the need for more robust evidence to support the performance claims and a more in-depth explanation of the technical details. The discussion ultimately centered around the importance of thorough benchmarking, clear documentation, and careful consideration of existing solutions when introducing a new database technology.

• Show HN: A Database Written in Golang

  permalink

  Posted: 2025-02-26 14:28:26

  Verbose tl;dr

  AtomixDB is a new open-source, embedded, distributed SQL database written in Go. It aims for high availability and fault tolerance using a Raft consensus algorithm. The project features a SQL-like query language, support for transactions, and a focus on horizontal scalability. It's intended to be embedded directly into applications written in Go, offering a lightweight and performant database solution without external dependencies.

  Sahil Bansal has introduced AtomixDB, a novel database management system implemented entirely in the Go programming language. This project aims to provide a performant and robust database solution, leveraging the strengths of Go, such as its concurrency features and efficient garbage collection. AtomixDB is designed with a focus on simplicity and ease of use, aiming to be a practical choice for various applications.

  While the full feature set is still under development, AtomixDB currently supports core database functionalities, including data persistence and retrieval. It employs a key-value store architecture, enabling efficient storage and access of data through unique keys. The database utilizes an on-disk storage mechanism, ensuring data durability across sessions.

  The project emphasizes a clean and modular codebase, promoting maintainability and extensibility. The choice of Go contributes to this by offering a statically typed language with a relatively simple syntax and a rich standard library. AtomixDB's implementation details suggest a commitment to performance optimization, although specific benchmarks are not yet available.

  Furthermore, the project is open-source, encouraging community involvement and contributions. This fosters transparency and allows for collaborative development, potentially leading to a more robust and feature-rich database system over time. The project's current stage suggests it is actively being developed, with future plans likely encompassing expanded functionalities and performance improvements. While the current implementation provides a foundational key-value store, the project leaves room for potential exploration of other database paradigms in the future.

  • Golang
  • Database
  • distributed systems
  • key-value store
  • Raft
  • Show HN
  • Open Source
  • atomixdb
  • fault tolerance
  • Go

  Summary of Comments ( 24 )
  https://news.ycombinator.com/item?id=43183891

  Verbose tl;dr

  HN commenters generally expressed interest in AtomixDB, praising its clean Golang implementation and the choice to avoid Raft. Several questioned the performance implications of using gRPC for inter-node communication, particularly for write-heavy workloads. Some users suggested benchmarks comparing AtomixDB to established databases like etcd or FoundationDB would be beneficial. The project's novelty and apparent simplicity were seen as positive aspects, but the lack of real-world testing and operational experience was noted as a potential concern. There was some discussion around the chosen consensus protocol and its trade-offs compared to Raft.

  The Hacker News post titled "Show HN: A Database Written in Golang" (linking to the AtomixDB GitHub repository) has generated several comments, offering a mix of praise, critique, and inquiries.

  Several commenters express initial positive impressions, appreciating the project's ambition and the apparent clean codebase. One commenter highlights the clear documentation as a strong point, making the project approachable for those wanting to understand its inner workings. Another emphasizes the value of having a learning-oriented database project in Go, contrasting it with the complexity of established databases, and thus making it a good resource for educational purposes.

  However, some commenters raise concerns and offer constructive criticism. A recurring theme is the lack of performance comparisons. Commenters question how AtomixDB stacks up against existing database solutions, emphasizing that benchmarks are essential for assessing the project's viability. They suggest comparing it with established Go-based databases like BadgerDB and BoltDB, or even more broadly with databases like SQLite. The absence of this data leaves potential users unsure of AtomixDB's practical applications.

  Another point of discussion revolves around the choice of using Raft for distributed consensus. While acknowledging Raft's robustness, some commenters inquire about the performance implications and suggest exploring alternative consensus algorithms that might be more efficient for certain workloads. Related to this, questions are raised about the single-leader limitation in the current Raft implementation.

  Further points of interest include questions regarding the project's maturity level and future plans. Commenters inquire about the roadmap, planned features, and the author's long-term vision for the database. There's also a discussion around potential use cases, with commenters speculating about scenarios where AtomixDB could be a good fit, such as embedded systems or specific niche applications.

  Finally, some commenters offer practical advice and suggestions for improvement. One commenter points out the importance of testing and suggests incorporating property-based testing to ensure correctness. Another advises considering compatibility with WireGuard for secure communication.

  In summary, the comments reflect a genuine interest in the AtomixDB project, appreciating the effort while also highlighting key areas for improvement, particularly regarding performance evaluation and providing a clearer picture of the project’s future direction.

• Kronotop: Redis-compatible, transactional document store backed by FoundationDB

  permalink

  Posted: 2025-01-20 18:12:24

  Verbose tl;dr

  Kronotop is a new open-source database designed as a Redis-compatible, transactional document store built on top of FoundationDB. It aims to offer the familiar interface and ease-of-use of Redis, combined with the strong consistency, scalability, and fault tolerance provided by FoundationDB. Kronotop supports a subset of Redis commands, including string, list, set, hash, and sorted set data structures, along with multi-key transactions ensuring atomicity and isolation. This makes it suitable for applications needing both the flexible data modeling of a document store and the robust guarantees of a distributed transactional database. The project emphasizes performance and is actively under development.

  Kronotop introduces itself as a novel document store that strives to bridge the gap between the simplicity and performance of Redis and the robust transactional guarantees and scalability offered by FoundationDB. It aims to provide a familiar Redis-compatible interface while leveraging the underlying power of FoundationDB for data persistence and consistency.

  The project's core objective is to offer a streamlined developer experience for building applications requiring both the flexible data modeling capabilities of a document store and the strong ACID properties of a transactional database. By emulating the Redis API, Kronotop allows developers already versed in Redis to leverage their existing knowledge and tools without a steep learning curve. This compatibility encompasses a wide range of Redis commands, enabling developers to perform common operations like setting and retrieving key-value pairs, working with various data structures such as lists, sets, and hashes, and leveraging features like Pub/Sub messaging.

  Under the hood, Kronotop leverages FoundationDB's distributed architecture and transactional engine. This allows Kronotop to provide strong consistency and durability guarantees, ensuring data integrity even in the face of failures. FoundationDB's scalability features also translate to Kronotop, allowing it to handle large datasets and high throughput demands. This combination of Redis compatibility and FoundationDB's robustness positions Kronotop as a potential solution for applications requiring high performance, scalability, and data consistency.

  The project is open-source and written in Rust, a language known for its performance and safety features. This choice of language contributes to Kronotop's efficiency and reliability. The developers emphasize that the project is still under active development, with ongoing efforts to expand Redis compatibility and enhance performance. They also highlight the project's potential for various use cases, including caching, real-time analytics, and microservices architectures. While acknowledging the project's ongoing development status, the stated goal is to eventually provide a production-ready solution for applications needing a powerful and dependable document store.

  • Kronotop
  • FoundationDB
  • Redis
  • Database
  • document store
  • transactional
  • distributed database
  • key-value store
  • NoSQL
  • data storage
  • scalability
  • fault tolerance
  • Open Source

  Summary of Comments ( 1 )
  https://news.ycombinator.com/item?id=42771403

  Verbose tl;dr

  HN commenters generally expressed interest in Kronotop, praising its use of FoundationDB for its robustness and the project's potential. Some questioned the need for another database when Redis already exists, suggesting the value proposition wasn't entirely clear. Others compared it favorably to Redis' JSON support, highlighting Kronotop's transactional nature and ACID compliance as significant advantages. Performance concerns were raised, with a desire for benchmarks to compare it to existing solutions. The project's early stage was acknowledged, leading to discussions about potential feature additions like secondary indexes and broader API compatibility. The choice of Rust was also lauded for its performance and safety characteristics.

  The Hacker News post titled "Kronotop: Redis-compatible, transactional document store backed by FoundationDB" generated a moderate amount of discussion, with several commenters expressing interest and raising relevant questions.

  Several commenters focused on the choice of FoundationDB as the backing store. One questioned why FoundationDB was chosen over something simpler like SQLite, prompting a response from the project author explaining that FoundationDB provides distributed consistency and scalability, crucial for the intended use cases of Kronotop. The author also clarified that while starting with a simpler backing store might seem easier, it would eventually become a limitation. This exchange highlighted the project's emphasis on robust scalability and fault tolerance.

  Another commenter expressed curiosity about the compatibility layer with Redis and whether it was challenging to implement. The author responded, detailing that the Redis protocol's simplicity made the implementation relatively straightforward, though managing client connections efficiently was a key aspect of their work. They elaborated on their use of Tokio and the complexities of handling multiple simultaneous connections within that framework.

  Further discussion centered on the specific features of Kronotop and their potential applications. The transactional nature of the database garnered attention, with users exploring use cases where data integrity is paramount. Questions about data modeling and querying capabilities were raised, with the author outlining their approach to document storage and retrieval. They clarified that Kronotop utilizes JSON for document representation and supports a subset of Redis commands.

  Performance and benchmarking were also topics of interest, with one commenter suggesting a comparison with existing Redis implementations. While acknowledging the value of such benchmarks, the author stated that their current focus was on stability and feature completeness. They indicated that formal benchmarking would be a future priority.

  The project's open-source nature and the invitation for community contributions were welcomed by several commenters. The overall tone of the discussion was positive, with a general sense of intrigue surrounding Kronotop's potential and the novel approach of combining Redis compatibility with the robustness of FoundationDB.