The Temptation of Unsafe

Recently, another round of discussion concerning the use of Rust’s unsafe features in the Actix web framework happened, or rather erupted, on Reddit, even more heated and acrimonious than the first time around. (I am not linking to any of the threads, as I believe that they don’t need any more exposure. Use your favorite search engine.) This proves, if more proof is needed, that people hold passionate beliefs about the matter.

This post is not about Actix as such. It’s about unsafe, and its allure as a tool for unlocking good performance. It also attempts to propose some guidelines for reasoning about its use (a tall order, I know). But first, a refresher on the concept, which is also a check on my understanding. If it’s familiar to you, skip the following several paragraphs.

Let’s first mention unsafe’s treacherous sometime companion, undefined behavior (UB), which amounts to a breakdown of the contract with the language, giving the compiler a license to do a lot of strange things with your code. There is a set of behaviors which precipitate that breakdown, like instantiating multiple aliased mutable references or reading uninitialized values. Rust is not fully or formally specified, so there is no definitive UB list, but many instances have long been agreed upon.

UB is never OK. Having it anywhere means that the program is essentially unpredictable. It may work today, with the specific combination of your CPU, OS, compiler version, library versions, and set of inputs. Change any of those and it may stop working. Tomorrow. Next week. Six months hence. Perhaps never—​but you don’t know that. "Stop working" includes crashing, corrupting data, or letting an attacker exfiltrate your whole user database. You don’t know that either.

People occasionally confuse or conflate UB with unsafe. The two are not the same, and neither implies the other. To reiterate, UB is never OK. Unsafe, however, sometimes is. Unsafe marks the limit of safe Rust’s expressive power or its protection domain. A canonical example of exceeding the former is a data structure with backreferences, and of the latter, a call to an external C library.

In the source code, unsafe can appear in several ways.

An unsafe block means "I (the author of the block) vouch for the semantics of the code inside the block". "Semantics" are the union of avoiding UB and upholding the program’s and Rust’s invariants. This is trickier than it seems: the scope of unsafety is not limited to the block in which unsafe operations appear, but stretches at least to the enclosing module boundary.

An unsafe function means "You (the caller of the function) must guarantee the semantics of the code calling the function".

An unsafe trait means "this trait requires special consideration when implementing in order to preserve language and/or program semantics".

An unsafe trait implementation means "I (the implementor of the trait) guarantee that the trait’s semantics are followed in the code".

One goal of safe Rust is to make UB impermissible. An unsafe block which contains UB and/or makes it possible to introduce UB in safe code is about the worst imaginable violation of Rust’s safety guarantees, because the potential for unsoundness is hidden behind a safe façade. An unsafe function or trait implementation with UB are not any better, but at least the "unsafe" keyword prompts the user to be vigilant.

A sub-category of "expressive power unsafe" is code where safety is sidestepped to achieve better performance. This is legitimate if supported by evidence of meaningful performance improvement. Since benchmarking is difficult and not everyone’s criteria for adequate performance are in alignment, this is the most contentious use of unsafe. Rust is attractive to programmers who reflexively seek to eliminate perceived inefficiencies, and the temptation of unsafe always beckons.

As a lifelong C programmer, I’ll admit to succumbing now and then.

I have a demonstration, too. A crate of mine, env_proxy, extracts the proxy server parameters from the environment variables, and is used in rustup, exposing it to all kinds of unconventional setups and unearthing an occasional edge case or bug. A recent issue arose as a result of having an HTTP proxy on port 80: since that’s the default HTTP port, the URL parser would remove it when explicitly specified, setting it to None in the Url struct, but env_proxy would treat its absence as a sign of not having been specified at all, and set the port to its default, 8080. Effectively, setting http://proxy.example.com:80 would give you back http://proxy.example.com:8080.

The proper way to fix this is to tell the URL parser to use a different default port, not currently possible in url. As a workaround, I chose to replace the http or https scheme with the bogus xttp/xttps, which doesn’t have a default port, so anything specified is retained by the parser.

Now, look at it as a C programmer. You probably have the URL in a string (char *), call it s. Changing the scheme is just changing the first character of the string.

What about Rust? If the URL is in a String (also named s), you can’t change it easily in the same way. A String can’t be indexed, for good reasons. Rebuilding the string requires an unnecessary allocation. So inefficient! Then there’s the unsafe way…​

It must be unsafe, since messing with the raw bytes of a String can break the "always valid UTF-8" invariant. But the effects have been carefully checked. It’s efficient. It’s bulletproof.

It is also completely unnecessary. The most likely use of this code will have an HTTP(S) connection in the future. Right there, the cost of an extra allocation will be lost in the noise. Venturing into unsafety, however well technically justified, is simply not worth it in this case. After a mild remonstrance by the rustup maintainer, and thinking about the tradeoffs more thoroughly, I replaced the unsafe block with a brief but allocating call.

Case closed and disaster averted? In this instance, yes (although no disaster was in the cards). But I would like to turn attention to the wider picture. As mentioned upfront, any discussion of unsafe stirs up passions. To me, that is a sign of vagueness of understanding the full implications of its use, and immaturity of the tooling which could help gain that understanding.

I don’t think that the answer should be to choose the strictly pro- or anti-unsafe camp, and accuse the other side of various character failings (which, depressingly, happens). While understanding and tooling slowly improve, having a solid set of guidelines reflecting the current situation should help assess each use of unsafe without having to resort to gut feeling too much. Let’s try to sketch some rules, from most to least strict.

Thou shalt not commit UB. For the third time. Just don’t.

Unsafe is not easy. It gives you the tools to potentially introduce UB. The rules are not fully specified, and are sometimes quite subtle. Acknowledge and respect the complexity of the area.

Remember that the scope of unsafety is larger than the unsafe block. You may be depending on state which is mutable by safe code within the whole module.

Don’t be less vigilant when you must use unsafe. You can’t avoid unsafe for FFI, and no one will berate you for using it there. Which doesn’t absolve you from the responsibility of adapting the semantics of the external library to the rest of the Rust code.

If using unsafe as an optimization, treat it as such. In addition to everything else, try not to use it before measuring. (Guilty!)

Document your assumptions thoroughly. It will have to be a comment, and we all know that the coupling between code and comments tends to rot. That’s another case where using unsafe requires more attention.

Be conservative when using unsafe. It’s never easy and requires more attention from everyone. Don’t burden yourself and others unthinkingly.

Try not be dogmatic when someone else does. Give them the benefit of doubt. Don’t condone or tolerate UB—​but you can still be diplomatic about it.