This blog post details the author's successful endeavor to create audiobooks from EPUB files using an open-source large language model (LLM) called Kokoro-82M. The author meticulously outlines the entire process, motivated by a desire to listen to e-books while engaged in other activities. Dissatisfied with existing commercial solutions due to cost or platform limitations, they opted for a self-made approach leveraging the power of locally-run AI.
The process begins with converting the EPUB format, which is essentially a zipped archive containing various files like HTML and CSS for text formatting and images, into a simpler, text-based format. This stripping-down of the EPUB is achieved through a Python script utilizing the ebooklib library. The script extracts the relevant text content, discarding superfluous elements like images, tables, and formatting, while also ensuring proper chapter segmentation. This streamlined text serves as the input for the LLM.
The chosen LLM, Kokoro-82M, is a relatively small language model, specifically designed for text-to-speech synthesis. Its compact size makes it suitable for execution on consumer-grade hardware, a crucial factor for the author's local deployment. The author specifically highlights the selection of Kokoro over larger, more resource-intensive models for this reason. The model is loaded and utilized through a dedicated Python script, processing the extracted text chapter by chapter. This segmented approach allows for manageable processing and prevents overwhelming the system's resources.
The actual text-to-speech generation is accomplished using the piper functionality provided within the transformers library, a popular Python framework for working with LLMs. The author provides detailed code snippets demonstrating the necessary configurations and parameters, including voice selection and output format. The resulting audio output for each chapter is saved as a separate WAV file.
Finally, these individual chapter audio files are combined into a single, cohesive audiobook. This final step involves employing the ffmpeg command-line tool, a powerful and versatile utility for multimedia processing. The author's process uses ffmpeg to concatenate the WAV files in the correct order, generating the final audiobook output, typically in the widely compatible MP3 format. The blog post concludes with a reflection on the successful implementation and the potential for future refinements, such as automated metadata tagging. The author emphasizes the accessibility and cost-effectiveness of this method, empowering users to create personalized audiobooks from their e-book collections using readily available open-source tools and relatively modest hardware.
Summary of Comments ( 174 )
https://news.ycombinator.com/item?id=42708773
Commenters on Hacker News largely discuss alternative methods and tools for converting ebooks to audiobooks. Several suggest using pre-trained models available through services like Google Cloud or Amazon Polly, noting their superior quality compared to the Kokoro model mentioned in the article. Others recommend exploring open-source solutions like Coqui TTS. Some commenters also delve into the technical aspects, discussing different voice synthesis techniques and the importance of pre-processing ebook text for optimal results. A few raise concerns about the potential misuse of AI-generated audiobooks for copyright infringement or creating deepfakes. The overall sentiment leans towards acknowledging the author's ingenuity while suggesting more robust and readily available solutions for achieving higher quality audiobook generation.
The Hacker News post "Generate audiobooks from E-books with Kokoro-82M" has a modest number of comments, sparking a discussion around the presented method of creating audiobooks from ePubs using the Kokoro-82M speech model.
Several commenters focus on the quality of the generated audio. One user points out the robotic and unnatural cadence of the example audio provided, noting specifically the odd intonation and unnatural pauses. They express skepticism about the current feasibility of generating truly natural-sounding speech, especially for longer works like audiobooks. Another commenter echoes this sentiment, suggesting that the current state of the technology is better suited for shorter clips rather than full-length books. They also mention that even small errors become very noticeable and grating over a longer listening experience.
The discussion also touches on the licensing and copyright implications of using such a tool. One commenter raises the question of whether generating an audiobook from a copyrighted ePub infringes on the rights of the copyright holder, even for personal use. This sparks a small side discussion about the legality of creating derivative works for personal use versus distribution.
Some users discuss alternative methods for audiobook creation. One commenter mentions using Play.ht, a commercial service offering similar functionality, while acknowledging its cost. Another suggests exploring open-source alternatives or combining different tools for better control over the process.
One commenter expresses excitement about the potential of the technology, envisioning a future where easily customizable voices and reading speeds could enhance the accessibility of audiobooks. However, they acknowledge the current limitations and the need for further improvement in terms of naturalness and expressiveness.
Finally, a few comments delve into more technical aspects, discussing the specific characteristics of the Kokoro-82M model and its performance compared to other text-to-speech models. They touch on the complexities of generating natural-sounding prosody and the challenges of training models on large datasets of high-quality speech. One commenter even suggests specific technical adjustments that could potentially improve the quality of the generated audio.