Random Thoughts by Fabien Penso

Asset pipeline for Rust

Rust’s performance is insane but requires a significant amount of manual work, unlike Rails which is highly opinionated but gives you everything.

My server-side HTML templates for Constellations used Bootstrap with a CDN but I wanted to move it to a classic asset pipeline. This post explains how I built it.

The way it works:

  1. All needed assets are built by your asset builder then copied in assets
  2. Assets are then copied to public/assets
  3. Assets are delivered by Actix
  4. An helper for templates allows to link asset files

I’d be very interested if you found this helpful or have improvement suggestions.


1. Build assets

build.js builds my own SASS and Typescript assets then save them into assets. Javascript/Typescript files are stored in javascript and CSS in css.

It uses esbuild but you could probably adapt it to use webpack.

#!/usr/bin/env node

const esbuild = require('esbuild');
const glob = require("tiny-glob");
const sassPlugin = require('esbuild-plugin-sass');

(async () => {
  let entryPoints = await glob("./javascript/*.{ts,js}");

  esbuild.build({
    entryPoints: entryPoints,
    bundle: true,
    outdir: 'assets/',
  }).catch((e) => console.error(e.message))

  entryPoints = await glob("./css/*.css");

  esbuild.build({
    entryPoints: entryPoints,
    bundle: true,
    outdir: 'assets/',
    loader: {
      '.woff': 'file',
      '.woff2': 'file',
    },
    plugins: [sassPlugin()],
  }).catch((e) => console.error(e.message))
})();

You’ll want to have a package.json and run npm install:

{
  "devDependencies": {
    "esbuild": "^0.18.17",
    "esbuild-plugin-manifest": "^0.6.0",
    "tailwindcss": "^3.3.3",
    "sass": "^1.64.1",
    "esbuild-plugin-sass": "^1.0.1",
    "tiny-glob": "^0.2.9"
  }
}

2. Copy assets and add a hash

This will copy all assets in assets to public/assets including a SHA1 hash of the file content in its filename to prevent caching issue on new deploys, then add a manifest.json file.

// build.rs
use sha1::{Digest, Sha1};
use std::collections::HashMap;
use std::fs;
use std::io::Write;
use std::path::Path;
use walkdir::WalkDir;

fn main() -> Result<(), anyhow::Error> {
    // Place the directory containing your asset files here
    let assets_dir = Path::new("assets");

    // The output directory
    let dest_path = Path::new("./").join("public/assets");
    fs::remove_dir_all(&dest_path).ok();
    fs::create_dir_all(&dest_path).expect("Can't create directory");

    let mut asset_map = HashMap::new();

    WalkDir::new(assets_dir).into_iter().for_each(|entry| {
        let Ok(entry) = entry else { return; };

        if !entry.file_type().is_file() {
            return;
        };

        let file_name = entry.file_name().to_string_lossy().to_string();
        let root_file = entry
            .path()
            .file_stem()
            .unwrap()
            .to_string_lossy()
            .to_string();
        let extension = entry
            .path()
            .extension()
            .unwrap()
            .to_string_lossy()
            .to_string();

        if ["woff", "woff2"].contains(&extension.as_str()) {
            // Those file already have hashes in their filenames
            let source = entry.path().to_string_lossy().to_string();
            let dest = dest_path.join(&file_name).to_string_lossy().to_string();
            fs::copy(source, dest).expect("Can't copy file");
            return;
        }

        let file_content = fs::read_to_string(entry.path()).expect("Can't read file");

        let hash = calculate_sha1(file_content.as_str());
        let new_filename = format!("{}.{}.{}", root_file, hash, extension);

        let source = entry.path().to_string_lossy().to_string();
        let dest = dest_path.join(&new_filename).to_string_lossy().to_string();

        fs::copy(source, dest).expect("Can't copy file");

        // Keep track of old and new filenames
        asset_map.insert(file_name, new_filename);
    });

    // Write the map to a manifest.json
    let mut file = fs::File::create(Path::new(&dest_path).join("manifest.json"))
        .expect("Can't write asset_map.rs");

    let data = serde_json::to_string(&asset_map)?;

    file.write_all(data.as_bytes())
        .expect("Can't write content");

    Ok(())
}

fn calculate_sha1(input: &str) -> String {
    let mut hasher = Sha1::new();
    hasher.update(input);
    let result = hasher.finalize();
    format!("{:x}", result)
}

You also need to change your Cargo.toml file:

# Cargo.toml
[build-dependencies]
sha1 = "0.10"
walkdir = "2.3"
anyhow = "1.0"
serde_json = { version = "^1" }

3. Deliver asset with actix

The static files feature for Actix allows you to easily deliver those assets yourself.

4. Helper method

The following code helps me linking assets in HTML templates.

// assets_decorator.rs
use std::path::Path;
use std::{collections::HashMap, fs::File, io::Read};

const DIR: &str = "public/assets";
const PUBLIC_DIR: &str = "/assets";

pub fn asset_path(filename: &str) -> Result<String, anyhow::Error> {
    let mut file_content = String::new();
    let mut file = File::open(format!("{}/manifest.json", DIR))?;
    file.read_to_string(&mut file_content)?;

    let data: HashMap<String, String> = serde_json::from_str(&file_content)?;

    match data.get(filename) {
        Some(filename) => Ok(format!("{}/{}", PUBLIC_DIR, filename)),
        None => Err(anyhow::anyhow!("Asset not found")),
    }
}

pub fn asset_tag(filename: &str) -> String {
    let filename = match asset_path(filename) {
        Ok(path) => path,
        Err(_) => return Default::default(),
    };
    let extension = Path::new(&filename)
        .extension()
        .expect("Can't get filename extension")
        .to_string_lossy()
        .to_string();

    match extension.as_str() {
        "js" => format!("<script src=\"{}\"></script>", filename),
        "css" => format!("<link rel=\"stylesheet\" href=\"{}\">", filename),
        _ => Default::default(),
    }
}

I currently use askama, the template looks like:

<!-- layout.html -->
<head>
  {{ crate::decorators::assets_decorator::asset_tag("custom.css")|safe }}
</head>

and the generated output will look like:

<head>
  <link rel="stylesheet" href="/assets/custom.617aacf2e80dea7b5970121248dfe3aadcd844ef.css">
</head>

Automate it all

I automate all this with a Makefile entry. I first manually copy static assets from node packages like preline or flowbite. I also use the tailwind cli tool builder.

# Makefile
build_assets:
	rm -rf assets public/assets
	mkdir assets
	cp node_modules/preline/dist/preline.js assets/
	cp node_modules/flowbite/dist/flowbite.min.js assets/
	cp node_modules/bootstrap-icons/bootstrap-icons.svg assets/
	./build.js
	npx tailwindcss --minify -i ./css/tailwind.css -o ./assets/tailwind.css
	touch build.rs # force rebuilding

CI

I use Drone for the CI and the asset building is done with the following:

  - name: build assets
    image: node:18-bullseye
    commands:
      - npm install
      - make build_assets