Writing a Bitrise Step in Rust - Scripting, Building and Caching

In the previous article, we saw how to create a simple script in Bash to make the Rust Toolchain available in our workflows.

Now it's time to move forward, install rust-script and run our script written in Rust.

Step creation (take two!)

Once again, we create a Bitrise step via the Bitrise CLI with bitrise :step create and still select Bash as programming language in the interactive wizard. The file structure is the same as last time, so we're first going to update the step.yml file with inputs and outputs, the step.sh script that serves as entry point for our script, and an example file that will be used in our test run.

toolkit:
  bash:
    entry_file: step.sh


inputs:
  - RUST_SCRIPT_VERSION:
    opts:
      title: "rust-script version"
      summary: Enforce a specific rust-script version.
      description: |
        If set, makes cargo install the specified version of rust-script..
      is_expand: true

  - RUST_SCRIPT_AUTO_UPDATE: false
    opts:
      title: "rust-script auto update"
      summary: Auto update rust-script at every step invocation.
      description: |
        If `true`, runs cargo install every time to make sure the crate is updated.
      is_expand: true
      is_required: false
      value_options: [false, true]

  - RUST_SCRIPT_FILE_PATH: "rust_script_example.rs"
    opts:
      title: "Source script file"
      summary: Specify the Rust script file to run.
      description: |
        Path shall be relative to the root of the project.

outputs:
  - RUST_SCRIPT_CACHE_PATH:
    opts:
      title: "rust-script cache path"
      summary: The rust-script local cache used to build the script.

The inputs are straightforward:

• With RUST_SCRIPT_VERSION, we allow the user to select a specific version of the rust-script crate, if it's empty it keeps the one already present (if any)
• With RUST_SCRIPT_AUTO_UPDATE, we allow them to auto-update rust-script to get the latest version installed
• Last but not least, with RUST_SCRIPT_FILE_PATHwe get the path of the file that needs to be run as script. The path is relative to the project root.

As for the outputs, we locate the rust-script cache folder and export it (as seen in Flawless scripting with Rust) to speed up subsequent runs of the same unchanged script. The built-in Cache Push and Cache Pull will help us achieve this.

Calling rust-script via step.sh

In our step.sh file, after performing the required ceremonies to make sure the rust-script version is valid and the one requested by the user, we can run our script.

The only thing we need to make sure of is that the Rust Toolchain has been installed on the current worker. The default Bitrise Docker images do not have it pre-installed, so you need to do it in a prior step or use a custom image.

#!/bin/bash
set -e

[...] 

if [ -z "${RUST_SCRIPT_FILE_PATH}" ]; then
  printf "No file path provided, make sure RUST_SCRIPT_FILE_PATH is set.\n"
  exit 1
fi

# If no Rust Toolchain is present, fail the step.
if ! command -v rustup &> /dev/null; then
  printf "Rust Toolchain is not installed, exiting...\n"
  exit 1
fi

[...]

rust-script --cargo-output $BITRISE_STEP_SOURCE_DIR/$RUST_SCRIPT_FILE_PATH

case "$OSTYPE" in
  darwin*)  
    envman add --key RUST_SCRIPT_CACHE_PATH --value '~/Library/Caches/rust-script'
    ;; 
  
  linux*)   
    envman add --key RUST_SCRIPT_CACHE_PATH --value '~/.cache/rust-script'
    ;;

  *)        
    printf "Cache path not supported for OS: $OSTYPE, skipping...\n\n" 
    exit 0
    ;;
esac

exit $?

As you can see, rust-script runs the content of $BITRISE_STEP_SOURCE_DIR/$RUST_SCRIPT_FILE_PATH, exports the cache path to RUST_SCRIPT_CACHE_PATH and terminates.

Time to summon the crab 🦀

After making sure the step works as expected with bitrise run test, we're ready to commit our step and import it in our workflow. Even this time, the step won't be part of the standard step library, so we have to reference it in our YML file via the git:: notation.

Still, we don't have a Rust file to run. For the sake of simplicity, in the test repository I have added rust_script_example.rs which is just a copy of the random numbers generator shown in last week's post. You can find it here if you want to take a look.

Moving forward we need to create a new workflow, and this time we will include the Rust Toolchain installer, the rust-script runner and, as mentioned before, Bitrise.io Cache: Pull and Bitrise.io Cache: Push.

Our bitrise.yml file will look like this:

workflows:
  another_workflow:
    steps:
    - cache-pull@2: {}
    - git::https://github.com/nick0602/bitrise-step-rustup@main:
        title: Rustup Install
    - git::https://github.com/nick0602/bitrise-step-rust-script-runner@main:
        title: Rust script runner
        inputs:
        - RUST_SCRIPT_FILE_PATH: rust_script_example.rs
    - cache-push@2:
        inputs:
        - cache_paths: {}

• We first pull the cache: on our first run, it will silently fail as there's no cache to retrieve (and it's expected)
• We then run the Rust Toolchain installer and run our script by setting RUST_SCRIPT_FILE_PATH for the second step
• We keep cache_paths empty for now, we'll get back to it later.

Let's run! 🚀

Our script did actually run! However, the length of the step might a bit worrisome. This is caused by the fact that rust-script is not installed by default with rustup, and cargo takes 1m 12s to build a release version of its binary.

The script, on the other hand, takes nothing to compile as it only has one direct dependency and a few lines of code (Finished release [optimized] target(s) in 3.57s).

How about we make it faster?

The Bitrise cache steps help in this regard. By adding the Cache:Pull and Cache:Push steps respectively at the beginning and the end of the workflow, we can re-use build artifacts and make cargo skip some compilation steps.

Going back to the YML workflow file, we can specify the local cargo and rustup paths as entries for the cache-push step:

workflows:
  another_workflow:
    steps:
    
    # [...]
    
    - cache-push@2:
        inputs:
        - cache_paths: |-
            ~/.cargo
            ~/.rustup

If we run our workflow again and take a look at the App settings page on Bitrise, in fact, the cache gets populated:

Running the workflow another time now, we should see a significant overall reduction in the execution time:

As you can see, the step does not need to reinstall rust-script since it's already there, cached (the step does not perform auto-updates here), and the work required by rust-script only takes a few seconds.

If we look at the cache-pull step, it actually did download and unpack the archive (in 8.36s):

And even if we were to sum the overhead introduced by using the Cache steps, it does not even come close to 1.3 minutes, so it's a huge win in this regard:

The rust-script cache

Taking it to the next level, another optimization could be related to rust-script. Let's go back to our custom step and remember that it exports RUST_SCRIPT_CACHE_PATH: that's the cache folder used by the crate when it compiles our script.

Let's add the env to the cache_paths of the cache-push step:

    - cache-push@2:
        inputs:
        - cache_paths: |-
            ~/.cargo
            $RUST_SCRIPT_CACHE_PATH

And now we run the workflow again:

This last change is not really impactful in our case, as the script is rather simple and has one dependency (rand), but by adding $RUST_SCRIPT_CACHE_PATH to  cache_paths we're making rust-script skip the compilation step and run the binary right away!

With a simple Bash script and a few other instructions, we managed to get Bitrise running a Rust script. At first, the entire workflow seemed a bit slow, but thanks to the local cache, setting up rust-script and running Rust files gets actually quite fast.

That being said since our script is very simple and doesn't do much more than just a few println! commands, caching does not really shine, but I'm now more curious than ever to try this approach with future scripts of a way bigger scale to see what kind of improvements it brings. 🚀