Extremely Static Embeddings (ESE)

ESE, our first take on a compiler oriented approach to static embedding. It’s a flattening of a tokenizer and map of embeddings into a perfect hash function. Static embeddings, while less contextually accurate than attention based models, are much faster, and our version is by far the fastest that we know about.

Let’s take a look at a simple, complete, example:

// ese exposes two entry points: encode_single for one string,
// and encode for a batch. both return fixed-size f32 vectors
// whose length is ese::DIMENSIONS (512 by default).

fn main() {
    let v = ese::encode_single("hello world");
    println!("dims: {}", v.len());
    // 512

    // batch encode: one embedding per input, same order
    let embeddings = ese::encode(["potato", "root vegetable"]);
    println!("{} embeddings, each {} dims", embeddings.len(), embeddings[0].len());
    // 2 embeddings, each 512 dims
}

Things to know

• Play around with chunk sizes that you feed into ESE, you may find a sweet spot with a smaller size.
• Be sure to run any rust programs using ESE with --release it will make the embedding calls many times faster than running it in debug.
• ESE has a few crate features for changing output dimensions and enabling quantization