Performance Measured: How good is your WebAssembly?

WebAssembly adoption is exploding. Almost every week, at least one startup, SaaS vendor, or existing software platform provider starts offering Wasm tools or has already introduced Wasm options in their portfolio, it seems. But how do all the different offerings compare in terms of performance?

The good news is that due to the simplicity of Wasm’s runtime, actual performance at least uptime can be directly compared between different WebAssembly offerings. This direct comparison is certainly much easier to do when benchmarking distributed applications running on or with Kubernetes, containers, and microservices.

This means whether Wasm is running on a browser, edge device, or server, the computing optimization Wasm offers in each case is end-to-end, and its runtime environment is in a tunnel of sorts – obviously good for security – and unaffected. environments in which it operates directly at the hardware level on the CPU.

Historically, Wasm was also around for a while, before the World Wide Web Consortium (W3C) called it a web standard in 2019, thus becoming the fourth web standard along with HTML, CSS, and JavaScript. But while web browser apps are Wasm’s central and historical use case, again, the point is that it’s designed to run anywhere on a properly configured CPU.

In the case of a PaaS or SaaS service in which Wasm is used to improve compute performance – whether it is running in a browser or not – the compute improvement offered by Wasm at runtime can be measured directly among the different options.

At least one implementation that can be used to measure different runtimes, compilers, and JITs is increasingly being adopted for different versions of Wasm: libsodium. Despite his anecdotal stories, this writer has contacted at least 10 companies that have used him or know him.

libsodium consists of a library for encryption, decryption, signatures, password hashes, and other security-related applications. Its maintainers describe it in the documentation as a portable, compilable, compilable, and packageable fork of NaCl, with an extended API for improved usability.

Since its introduction, the libsodium benchmark has been widely used as a benchmark for choosing the best runtimes, said security consultant Frank Dennis. libsodium It includes 70 tests, covering a large number of optimizations that code builders can implement, Dennis points out. None of these tests perform any kind of I/O (disk or network), so they actually measure the true efficiency of compilers and runtimes, in a platform-independent way. “The same uptimes will be arranged on a local laptop and on a server in the cloud,” Dennis said.

In fact, libsodium is useful for testing some Wasm applications, Fermyon Technologies CEO and co-founder Matt Butcher told New Stack. “Any good reference tool has three desirable characteristics: it should be repeatable, fair (or unbiased toward a particular runtime), and reflective of production use,” Butcher said. “Libsodium is an excellent candidate for scaling. Not only is cryptography itself an appropriate use case, but the algorithms used in cryptography will detect real computational properties at runtime.”

Libsodium is also useful for testing some Wasm environments because it includes benchmarking tasks with a wide range of different requirement profiles, some of which look for raw CPU or memory performance, while others look for more granular performance profiles,” Torsten Volk, Analyst for Management “The current results show the group’s ability to detect significant differences in performance between different runtimes, for both compiled languages ​​and interpreted languages,” said Falk, Institutions Associates (EMA), to The New Stack. Running, without WASM in the middle, gives us a good idea of ​​the potential for future improvement of these run times.”

real specs

in a blog post. Dennis described how different Wasm applications were evaluated in the tests he completed. Among them were:

  • iwasm, which is part of the WAMR (“WebAssembly micro runtime”) package – pre-compiled files downloaded from its repository.
  • Wasm2c, included in the Zig source code for booting the compiler.
  • Wasmer 3.0 was installed using the command shown on their website. The three backgrounds were individually tested.
  • Wasmtime 4.0, compiled from source.
  • Node 18.7.0 is installed via the Ubuntu package.
  • Bun 0.3.0, installed via viewing commands on their website.
  • Waziro from 796fca4689be6 review portal, compiled from source.

Which came first in the runtime tests? Iwasm, which is part of the WebAssembly Micro Runtime (WAMR), according to Dennis’ findings. iwasm VM core is used to run WASM applications. It supports interpreter mode, pre-compilation mode (AOT), just-in-time (JIT) compilation modes, LLVM JIT, and Fast JIT, according to the project documentation.

This is not to say that Iwasim wins recognition for its simplicity of use. Compared to other options, [iwasm] Dennis Books. “It looks like a kitchen sink, including various ingredients.” These include IDE integration, remote management of the application framework library and SDK “that make it look like a complex solution to simple problems. The documentation is also a bit messy and overwhelming,” Dennis writes.

Runtime isn’t everything

There are other benchmarks to measure the performance differences between the different Wasm alternatives. Testing alternatives reported by Denise include:

However, the runtime benchmark is not a core metric for all WebAssembly applications. There are other test alternatives for testing different Wasm runtimes that focus on very specific tasks, such as calculating the Fibonacci sequence, sorting data matrices, or summing up integers, Volk noted. Other, more comprehensive benchmarks consist of analyzing complete use cases, such as video processing, PDF editing, or even deep learning-based object recognition, Volk said.

“Wasm comes with the ability to offer near instantaneous startup and scalability, and thus can be used for cost effective provisioning and scaling of network bandwidth and functional capabilities,” Volk said. “Assessing this fast startup capability based on the requirements of a specific use case can show the direct impact of Wasm’s runtime on the end user experience.”

Some Wasm applications are used in networks to improve latency. Of course, the uptime performance is important, but it is the latency performance that matters in this case, said Sehyo Chang, Chief Technology Officer of InfinyOn. This is because, Zhang said, latency plays a significant role in determining the overall user experience in an application. “Slow response time can greatly affect user interaction and lead to dissatisfaction, which can lead to lost sales opportunities,” Zhang said.

During the recent KubeCon + CloudNativeCon conference, Zhang gave a talk about using Wasm to replace Kafka for lower latency data streaming. Zhang said that Java-based streaming technologies, such as Kafka, may experience high latency due to garbage collection and the JVM. However, the use of WebAssembly (WASM) technology allows broadcast processing without these penalties, resulting in significantly reduced response time while providing more flexibility and security, Zhang said.

a group Created with Sketch.

Leave a Comment