Attend QCon Plus online conference (May 10-20) and find practical inspiration from software leaders. Register

Facilitating the Spread of Knowledge and Innovation in Professional Software Development

Avdi Grimm describes the future of development, which is already here. Get a tour of a devcontainer, and contrast it with a deployment container.

The panelists reflect on various microservices topics.

Monte Zweben proposes a whole new approach to MLOps that allows to scale models without increasing latency by merging a database, a feature store, and machine learning.

In this article we will be sharing our experience learned from 12 months of adopting certain management and organisational insights from the book Team Topologies. It explores how we identified areas of responsibility and assigned those into mostly customer facing domains which could be given to our teams. It shows how an inverse Conway manoeuvre can be used to improve the architecture.

The panelists discuss the security for the software supply chain and software security risk measurement.

Uncover emerging trends and practices from software leaders. Attend online on May 10-20, 2022.

Learn how cloud architectures achieve cost savings, improve reliability & deliver value. Register Now.

Understand the emerging software trends you should pay attention to. Attend in-person on Oct 24-28, 2022.

InfoQ Homepage News Mold is a New Linux Linker Aiming to Outperform Lld

Mold, a modern drop-in replacement for current Unix linkers, has reached version 1.0. Written by the original creator of the LLVM lld linker, mold aims to be several times faster than its predecessor.

According to its author, Rui Ueyama, mold would allow developers to increase their productivity by significantly speeding up the debug-edit-rebuild cycles:

Concretely speaking, I want to use the linker to link a Chromium executable (1.8 GiB in size) just in 1 second. LLVM's lld, the fastest open-source linker which I originally created a few years ago, takes about 12 seconds to link Chromium on my machine. So the goal is 12x performance bump over lld. Compared to GNU gold, it's more than 50x.

Ueyama published a benchmark showing that mold is already able to link Chrome, 2 GB in size, in slightly over 2 seconds, meaning with a 5x speed-up over lld and over 25 times faster than GNU gold. While this result is still more than double the stated goal, it definitely makes for very impressive credentials.

While the Chrome benchmark is particularly promising, it must be also noted that results vary widely across programs. For example, mold is only twice as fast as lld compiling Clang 13, while still 30+ times faster than GNU gold at the same task.

Mold achieves its performance by aggressively leveraging parallelism on multi-core CPUs. As Ueyama notes, existing linkers aren't doing a great job at scaling with available cores.

As you can see, mold uses all available cores throughout its execution and finishes quickly. On the other hand, lld failed to use available cores most of the time. In this demo, the maximum parallelism is artificially capped to 16 so that the bars fit in the GIF.

Ueyama benchmarks also show that mold takes roughly double time to link a program than cp to copy it to a different location, which shows the real goal here is to achieve cp-like performance.

To this aim, mold explores a number of alternative design choices which include the idea of preloading input object files from disk and process them as soon as possible, that is before all the relevant input object files are ready. So, linking has two steps, with a first step consisting of speculatively parsing and preprocessing input files, which does not require all of them to be ready. By executing this first step as early as possible, the second step may complete sooner as input files become progressively ready.

Additional speedup is granted by executing some computationally intensive tasks in the first step, such as resolving symbols using string interning and merging string sections.

There are also approaches that Ueyama analyzed and finally rejected, such as adopting incremental linking, creating a new file format alternative to ELF, or using inotify to watch for object files.

There are lots of other low-level details that affect a linker performance that become relevant when the goal is to shave off every millisecond, such as using forks to make the main linker process exit faster when an output file has been written to disk. Besides mold's design document, you can find an interesting read in this Hacker News thread where Ueyama answers a number of questions.

Mold is still a very young project and, in spite of its 1.0 versioning, not yet ready to replace lld in production according to the author. Still, if you want to try it out, that is really easy using clang -fuse-ld/-ld-path command line arguments. gcc requires you to use instead the -B flag to specify the directory where your custom ld is to be found.

Becoming an editor for InfoQ was one of the best decisions of my career. It has challenged me and helped me grow in so many ways. We'd love to have more people join our team.

Presented by: Johnson Noel - Senior Product Marketing Manager at Hazelcast

A round-up of last week’s content on InfoQ sent out every Tuesday. Join a community of over 250,000 senior developers. View an example

You need to Register an InfoQ account or Login or login to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

A round-up of last week’s content on InfoQ sent out every Tuesday. Join a community of over 250,000 senior developers. View an example

Real-world technical talks. No product pitches. Practical ideas to inspire you and your team. QCon Plus - May 10-20, Online. QCon Plus brings together the world's most innovative senior software engineers across multiple domains to share their real-world implementation of emerging trends and practices. Find practical inspiration (not product pitches) from software leaders deep in the trenches creating software, scaling architectures and fine-tuning their technical leadership to help you make the right decisions.

InfoQ.com and all content copyright © 2006-2022 C4Media Inc. InfoQ.com hosted at Contegix, the best ISP we've ever worked with. Privacy Notice, Terms And Conditions, Cookie Policy