Linux Fu: Remote Execution Made Easy

If you have SSH and a few other tools set up, it is pretty easy to log into another machine and run a few programs. This could be handy when you are using a machine that might not have a lot of memory or processing power and you have access to a bigger machine somewhere on the network. For example, suppose you want to reencode some video on a box you use as a media server but it would go much faster on your giant server with a dozen cores and 32 GB of RAM.

Remote Execution

However, there are a few problems with that scenario. First, you might not have the software on the remote machine. Even if you do, it might not be the version you expect or have all the same configuration as your local copy. Then there’s the file problem. the input file should come from your local file system and you’d like the output to wind up there, too. These aren’t insurmountable, of course. You could install the program on the remote box and copy your files back and forth manually. Or you can use Outrun.

There are a few limitations, though. You do need Outrun on both machines and both machines have to have the same CPU architecture. Sadly, that means you can’t use this to easily run jobs on your x86-64 PC from a Raspberry Pi. You’ll need root access to the remote machine, too. The system also depends on having the FUSE file system libraries set up.

A Simple Idea

The idea is simple. You could do a video encoding like this:

outrun user@host ffmpeg -i input.mp4 -vcodec libx265 -crf 28 output.mp4

This will work even if ffmpeg isn’t on the remote machine and the input and output files will be on your local box where you expect them. Here’s a screencast from the project’s GitHub page:

A Complex Implementation

How does this work? A FUSE file system mounts your local filesystem remotely using a lightweight RPC file system. Then a chroot makes the remote machine look just like your local machine but — presumably — faster. There are a few other things done, such as setting up the environment and current directory.

The chroot, by the way, is why you need root on the remote machine. As an ordinary user, you can’t pivot the root file system to make this trick work.

To improve performance, Outrun caches system directories and assumes they won’t change over the life of the command. It also aggressively prefetches using some heuristics to guess what files you’ll need in addition to the one that the system asked for.

The Future

We wish there was an option to assume the program will execute on the remote machine and only set up the input and output files. This would make it easier to do things like slice a 3D print on a remote PC from a Raspberry Pi running Octoprint, for example. Of course, this is all open source, so maybe we should go make that fix ourselves.

Then again, you could do something like this pretty easily with sshfs and some other tricks. If you want to run a program on a bunch of remote machines, there are ways to do that, too.