Running System Processes with Command

Published: August 26, 2024
Written by:

Natan Rolnik

Sponsored by

Get the best apps for your Mac

Setapp is a platform that provides a wide range of quality software for Mac users. The membership gives you access to a large library of curated apps and tools for Mac, web and iOS. My favorite ones are: Tableplus, Cleanshot X, Proxyman, and Bartender.

Check out Setapp

CLI tools often need to run other commands behind the scenes. For example, it might want to check if other tools such as SwiftLint or SwiftFormat are installed and run them. Maybe it needs to start a local server using npm and node. Or possibly it just needs to check the currently selected Xcode path using the xcode-select -p.

These operations have a few things in common: they might take a few moments to return some content (and keep outputting every few seconds); the command’s executable might not be present in the environment; or the command might fail early with a non-zero error code.

Foundation’s Process

This is relatively simple to perform programmatically using Foundation’s Process. However, in some advanced use cases, getting it right might be tricky: it’s not always trivial to handle concurrency, avoid pipe deadlocks, correctly manage environment variables and working directories. To ensure reliable execution, all these pitfalls require careful synchronization and error handling.

The Swift community has developed a few packages that wrap this API. Shellout by John Sundell is probably the most known, and there are a few others worth checking, such as SwiftShell, Shwift. There’s even an implementation which doesn’t use Process at all: SwiftSlash.

Command

In this post we will cover the Command package, maintained by the folks at Tuist. It has some nice advantages:

• Modern Swift’s structured concurrency support, using AsyncStream, allowing any output to be displayed as soon as it’s available
• Ensures thread safety when reading standard output and error
• Integrated with swift-log, allowing a Logger instance to be passed and help debugging commands
• Scoped errors make it easier to both the developer and the end user understand what has gone wrong
• Provides the process with the running environment variables
• It has been extensively used by Tuist itself over the years

The Sample Project

To demo a simplified, but real world use case, a sample project is available. The final goal of the sample project will be a tool that pulls a Swift Docker image of the current Swift version in the environment. This will enable us to learn how to:

• Run a short command, that returns almost immediately to check the current Swift version
• Run a longer command, that pulls the image, and outputs content as a stream
• Handle error cases such as not having Docker installed

If you want to follow along, clone the sample project and open the Package.swift file present in the starter directory.

Download the sample project

In Terminal, at the starter directory, you can run the executable with the command below:

swift run

After SPM fetches Command and the other dependencies, the executable should output To be implemented. If you open the project, you’ll see that’s the only line in the static main() function.

Reading the Current Swift Version

To start adding the desired functionality, implement a function that will call swift --version. To do so, you will need an object that can run commands. The Command package provides a CommandRunning protocol, which provides an interface for receiving arguments, environment variables, and a working directory path, and returns an asynchronous stream of events. These events can originate either from the standard output (regular outputs) or standard errors. The package provides also a concrete implementation of this protocol, which you’ll use later on.

The starter project already imports the Command package. So, in the beginning of the DockerSwiftPull struct, declare a property that conforms to the CommandRunning protocol:

let runner: CommandRunning

Now, below the run() function, add the following method:

//1
func swiftVersion() async throws -> String {
    //2
    try await runner.run(arguments: ["swift", "--version"])
        .concatenatedString() //3
        .extractSwiftVersion() //4
}

Here’s what this code block means:

1. The function signature tells us it is asynchronous, might throw an error, and returns a string with the Swift version
2. Call the run method on the runner, passing the arguments we want. The first argument must always be an executable
3. Because run returns an AsyncStream of events, use the helper concatenatedString method to wait aggregate all the events, and form a single stream with the output
4. Finally, call the extension function in the String+SwiftVersion file. It uses RegexBuilder to extract the short version from the whole swift --version output, which includes much more information

When running commands that are short and do not require iterating over every element in the stream, the concatenatedString() method is very useful and might be just enough for your usage.

Using Docker to Pull the Image

The second part consists of calling the pull command available in Docker. It downloads an image from the Docker registry, if it hasn’t been previously pulled. This is a great example of an operation that outputs new content every few moments, being a perfect fit for an AsyncStream.

After the method you just added, append the following method:

//1
func pullDockerImage(swiftVersion: String) async throws {
    //2 
    let arguments = ["docker", "pull", "swift:(swiftVersion)-amazonlinux2"]

    //3
    for try await output in runner.run(arguments: arguments) {
        // 4
        guard let outputString = output.string() else { return }

        //5
        if output.isError {
            print("Error: (outputString)")
        } else {
            print(outputString)
        }
    }
}

This function is a bit longer, but there’s not reason to be scared. Let’s go over it, bit by bit:

1. This method receives a Swift version as a string. It’s also asynchronous and throwing, as running commands requires both
2. Build the arguments array, composed of three elements: docker is the executable, pull is the command, and the swift:x.y-amazonlinux2 argument is the image tag
3. Call the runner’s run() method, passing the arguments you just declared. Notice the for try await output syntax, that will wait until each output is provided by the stream
4. Then, use the method on CommandEvent that converts the output into a string
5. Lastly, print the output. If it’s an error, prepend the message indicating it’s an error

Calling Both Methods

Once you have both methods in place, you can call actually call them. You can notice there’s an empty run method, and that it’s already marked with async throws. Implement that function with the following code:

//1
let swiftVersion = try await swiftVersion()
print("Swift version: (swiftVersion)")

//2
do {
    try await pullDockerImage(swiftVersion: swiftVersion)
} catch CommandError.executableNotFound { //3
    print("Docker is not available, install it before running this tool.")
    exit(1)
} catch { //4
    print("Error: (error)")
    exit(1)
}

Step by step, this is what it does:

1. Call the swiftVersion() method, and print it
2. Using a do-catch statement to catch possible errors, call the second function you added, pullDockerImage, passing the swift version you obtained in the first step
3. In case Docker is not available in the environment, Command will throw the executableNotFound error. Catch it, and print a special message in this case
4. For any other eventual errors, print them with the Error: prefix. In both error cases, call the exit function with 1 as the code, to indicate there was an error.

Finally, the last thing left is to call the run method. The starting point of the executable is the static main() function, so call the run function from there:

try await Self(runner: CommandRunner()).run()

This will initialize the DockerSwiftPull struct with the CommandRunner concrete implementation of CommandRunning. This is useful as the struct is agnostic to the implementation, and can call the methods using the protocol interface, allowing for better testability and code isolation.

Running the Tool

All the pieces are set up! It’s time to run the tool again using swift run.

If you don’t have Docker installed, you’ll get the error messaged we intentionally thought of:

Docker is not available, install it before running this tool.

When Docker is available, this is what the tool will do:

Testing

To learn how you can test a tool that depends on asynchronous commands, you can read the follow up post:

Testing Commands Using Mockable

Explore techniques to test a tool that runs system commands