Fold, our take on an incremental programming framework, is the engine that powers Bog. It’s a rust crate with iterator like primitives for materializing a stream of ever changing data into views. Statically typed and very, very fast.
Fold’s incrementally updates its views using diffs. As new data is inserted into a stream, fold marks it with a positive or negative integer representing the ‘diff’ amount this new data represents (+ for insert, - for removal). This allows every materialized view to only do work proportional to the actual change in data. We are incrementally ‘folding’ in the changes to specific data.
Let’s take a look at a simple, complete, example:
use fold::{FlatMap, Stream, terminals::Bag};
// this toy fold usage uses the simple 'Bag' terminal
// to track occurances of individual words in a string.
fn main() {
// Every fold starts with a Stream definition, which takes two args:
// a location of the db file, and a Graph definition.
// The graph here is using a simple Fold Flatmap, which defines how the
// incoming data should be manipulated (func), and the terminal that should
// be used for reading the data (next)
let mut st = Stream::new(
"intro.db",
FlatMap {
func: |doc: &String| doc.split_whitespace().map(String::from).collect::<Vec<_>>(),
next: Bag::<String>::new("word_counts"),
},
);
let (a, b) = ("to be or not to be".to_string(), "let it be".to_string());
// under the hood, these inserts are represented as (data, +1), for inserts
st.wtx(|tx| {
tx.insert(&a);
tx.insert(&b);
});
st.rtx(|words| println!("{:?}", words.iter().collect::<Vec<_>>()));
// [("be", 3), ("it", 1), ("or", 1), ("to", 2), ("let", 1), ("not", 1)]
// under the hood, this remove call is represented as (data, -1)
st.wtx(|tx| tx.remove(&a));
st.rtx(|words| println!("{:?}", words.iter().collect::<Vec<_>>()));
// [("be", 1), ("it", 1), ("let", 1)]
st.wtx(|tx| tx.remove(&b)); // cleanup
}