Rebuilding Ubuntu packages from source with sccache, a compiler cache, can drastically reduce compile times, sometimes up to 90%. The author demonstrates this by building the Firefox package, achieving a 7x speedup compared to a clean build and a 2.5x speedup over using the system's build cache. This significant performance improvement is attributed to sccache's ability to effectively cache and reuse compilation results, both locally and remotely via cloud storage. This approach can be particularly beneficial for continuous integration and development workflows where frequent rebuilds are necessary.
The post "Make Ubuntu packages 90% faster by rebuilding them" explores a significant performance improvement achieved by rebuilding Ubuntu Debian packages using a modified configuration. The author, frustrated by the slow build times of certain packages, specifically gcsfuse, observed that the default build process utilized a single core, severely underutilizing the available processing power of modern multi-core systems.
The post then meticulously details the process of accelerating the build process. The core of the optimization lies in leveraging the DEB_BUILD_OPTIONS environment variable. By setting this variable to parallel=N, where N represents the desired number of concurrent build jobs (ideally matching the number of CPU cores), the compilation process can be parallelized. This modification instructs the build system, specifically dpkg-buildpackage, to utilize multiple cores during the compilation and linking stages, dramatically reducing the overall build time.
The author showcases the remarkable impact of this simple change. A rebuild of the gcsfuse package, which initially took an hour and a half, completed in a mere ten minutes after implementing the parallelization. This represents an approximately 90% reduction in build time. Furthermore, the post demonstrates how to make this optimization permanent by setting the DEB_BUILD_OPTIONS variable within the user's shell configuration file (e.g., .bashrc or .zshrc). This ensures that all future builds automatically benefit from parallel processing.
The post concludes by emphasizing the ease and effectiveness of this optimization, suggesting that it could significantly improve the development workflow for anyone building Debian packages on multi-core systems. The author also notes the potential for further performance gains by exploring other compiler flags and build optimizations, although these are not explored within the post itself. The provided example focuses on a specific package, gcsfuse, but the technique is applicable to any Debian package built with tools like dpkg-buildpackage that respect the DEB_BUILD_OPTIONS variable.
Summary of Comments ( 10 )
https://news.ycombinator.com/item?id=43406710
Hacker News users discuss various aspects of the proposed method for speeding up Ubuntu package builds. Some express skepticism, questioning the 90% claim and pointing out potential downsides like increased rebuild times after initial installation and the burden on build servers. Others suggest the solution isn't practical for diverse hardware environments and might break dependency chains. Some highlight the existing efforts within the Ubuntu community to optimize build times and suggest collaboration. A few users appreciate the idea, acknowledging the potential benefits while also recognizing the complexities and trade-offs involved in implementing such a system. The discussion also touches on the importance of reproducible builds and the challenges of maintaining package integrity.
The Hacker News post "Make Ubuntu packages 90% faster by rebuilding them" generated a significant discussion with several compelling comments exploring various facets of the proposed speed improvements.
Several commenters focused on the reproducibility aspect. One user questioned the reproducibility of builds using
ccachegiven its potential to mask underlying issues that might manifest differently on different systems. This concern stemmed from the idea that whileccachemight speed up builds, it could also hide bugs that would otherwise be caught during a clean build. Another commenter echoed this sentiment, emphasizing the importance of clean builds for verifying package integrity and catching errors. They also highlighted the inherent tension between build speed and ensuring correct and reproducible builds across diverse environments.Another thread of conversation revolved around the technical details of the proposed speed improvements. One commenter inquired about the specific changes implemented to achieve the 90% speed increase, prompting the original poster (OP) to provide more context. The discussion delved into the mechanics of
ccacheand how it leverages caching mechanisms to accelerate compilation times. This technical exchange shed light on the underlying principles enabling the performance gains.The practicality and applicability of the proposed changes were also discussed. One commenter questioned whether the changes would be upstreamed, given the potential benefits for a wider audience. This prompted a conversation about the challenges and considerations involved in integrating such changes into the broader Ubuntu ecosystem. Further discussion focused on the trade-offs between build speed and resource consumption, specifically memory usage. Some users raised concerns about the potential impact on systems with limited resources, while others argued that the benefits outweighed the drawbacks.
Finally, some comments focused on alternative approaches and existing best practices. One commenter mentioned that using
ccacheis already a common practice within the community and suggested that the observed speed improvements might not be entirely novel. Another commenter pointed out the importance of distributing build processes to further enhance performance, especially for larger projects. These comments provided valuable context and expanded the discussion beyond the specific approach presented in the original post.