The team at Paris-based Snips has created a voice assistant that can be embedded in a single device or used in a home network to control lights, thermostat, music, and more. You can build a home hub on a Raspberry Pi and ask it for a weather report, to play your favorite song, or to brew up a double espresso. Manufacturers like Keecker are adding Snips’ technology to products like multimedia home robots. And Snips works closely with leaders across the value chain, like NVIDIA, EBV, and Analog Devices, in order to voice-enable an increasingly wider range of device types, from speakers to home automation systems to cars.
Snips’ solution is different from other voice assistants, in that it runs its entire code base on a single device and can work without an Internet connection. Snips’ software stack includes its wake word (“Hey Snips”), application logic, speech recognition engine, and language understanding module.
By comparison, products like Amazon Echo and Google Home just run code for the wake word, and they are dependent on the cloud to process queries and generate responses. That approach opens the door for companies to potentially collect users’ speech data, raising privacy concerns.
How can Snips embed all the code for a voice assistant onto a single device? They wrote it using the Rust systems programming language.
Rust is a highly efficient programming language that was developed in an open source project and is sponsored by Mozilla. The first stable release of Rust was in May 2015. Now, the Rust community is seeing companies adopt Rust to build commercial software, often at the cutting edge of their fields.
Rust is compelling because it combines attributes from different kinds of languages, so it can offer high performance and low memory overhead as well as memory safety and cross-compilation to different platforms. That made it a great fit for Snips’ use case: embedding code into a range of device types with limited memory and processing power.
Why Rust?
Snips Principal Engineer Mathieu Poumeyrol had used Rust at a previous job, writing multi-platform code. Instead of having to write and then rewrite for each platform, he used Rust’s cross-compilation capability. That let him write once and translate his code so it could run well on different machines, without days or weeks of hand-coding rework.
Poumeyrol pushed hard for Snips to consider adopting Rust. It had the traits Snips needed – efficiency, portability, and safety – and it had the performance characteristics to be able to run wicked fast, even on small devices.
“Snips was already using very modern languages for both mobile development and the back end, like Swift, Kotlin, and Scala,” Poumeyrol said. “That played a big part in convincing our engineers to try Rust.”
After more investigation, the Snips technical team was convinced that Rust was the right way to go. “We went all-in on Rust in 2016,” said Snips CTO Joseph Dureau. “And we are very happy with that decision.”
Performance and Portability
The primary challenge for Snips’ engineering team was this: How can we embed a voice assistant so it runs efficiently and safely on all of our clients’ connected devices, regardless of the operating system and architecture they use?
Rust was the answer to that challenge. The language offered a combination of virtues: the performance of a low-level language like C/C++, the capability to port code to new platforms, and memory safety features designed to enhance security, even when code is running on connected devices that are relatively exposed. (See how crockpots were hacked in 2016.)
Performance: Rust code is fast and efficient. It can run on resource-constrained devices with no degradation in performance. The language manages zero-cost abstraction in the same spirit as C++, Poumeyrol said, while maintaining the same safety level as a language with garbage collection. Rust delivers high-level features without a runtime performance penalty, which was exactly what Snips needed.
Portability: Rust’s first-class compiler, rustc, allows Snips’ engineers to write code once and port it to new devices. This is critical, because the company adds new device platforms to its solution every few weeks. Under the hood, rustc is implemented on top of LLVM, a solid, proven compiler framework. LLVM enables programmers to cross-compile code to most any modern hardware architecture, from mobile devices to desktops and servers.
“We must be able to code once and run our code on many platforms in an optimal and secure way,” Dureau said. “Everything we write for the embedded voice assistant, we write in Rust.”
Safety: Rust has a unique ownership model that makes its code, once compiled, safer than C/C++ and easier to maintain over time. The language uses concepts of ownership, moves, and borrows to keep track of memory resources and make sure they are being used appropriately.
Here’s how Rust’s memory safety features work. After programmers write new code, they run it through a compiler. The rustc compiler checks the code for errors. If it finds code that does not handle memory resources correctly, the compile step will not complete. That makes it more difficult to put memory-unsafe code into a production environment. The compiler helps in another way: It gives some feedback about the error alerts, and when possible, suggests fixes. This feedback saves a lot of time and lets new programmers learn by doing, with a lowered risk of introducing security vulnerabilities.
Poumeyrol is a fan of the Rust compiler. “At compilation time, it can make sure the resource management is done correctly, so we have no surprises at runtime,” he said.
One Fast Development Cycle
Working in Rust, the Snips technical team was able to complete its voice platform in record time: It took Snips less than a year to complete the coding work in Rust and put its voice assistant into production.
Memory safety played a large role in accelerating Snips’ development process. The developers could find and fix bugs using feedback from the Rust compiler. Those early corrections made the development cycle much shorter, because it’s simpler to fix bugs early, rather than waiting until runtime. It also speeded up the QA (quality assurance) phase of the process, so Snips was able to move new features into production more quickly.
Snips’ solution currently supports a dozen different device platforms, including the Raspberry Pi 3, DragonBoard, Sagemcom, Jetson TX2, IMX.8M, and others. Rust has made it simpler for the team to extend support to new boards, because they can reuse the same code base rather than writing custom implementations for each architecture.
Learning Rust
Today, all Snips’ embedded code is written in Rust. Over time, Poumeyrol has trained the embedded software engineers to code in Rust, as well as a significant number of the company’s Machine Learning scientists. As they all got more familiar with the language, the team’s go-to reference was the second edition of The Rust Programming Language Book, published online by the open source Rust Project.
The whole training process was fairly quick and organic, Poumeyrol said. The engineers he trained in turn shared their expertise with others, until the entire embedded software engineering team was actively learning the language.
“Rust is a language of its time,” Poumeyrol said. “Once one has a taste for these modern languages, it can be very frustrating to come back to C or C++ when you suddenly need portability and efficiency.” Poumeyrol has seen broad adoption of Rust in the larger industry as well, as software engineers and machine learning scientists see it as a useful tool that can solve persistent coding problems.
