reflexo-typst Documentation

Claim: the Typst compiling functions is provided by official typst

The rust library help you build compiler services for Vector Format.

• precompiler CLI: shiroa precompiles (prepares) artifacts for static websites.
• incremental compilation server: typst-preview compiles and streams typst document data to a web browser to provide fast preview of typst documents.

Note: the following content is for typst.ts >=v0.6.0. To use rust library in <v0.6.0, check the section.

The unofficial compiler library provides the world implementation for the official typst. To help parallelize the compilation, it provides a Universe that spawns multiple World instances. They are references to the resources to be used by the typst compiler. You can run a compiler task with the world reference in another thread:

std::thread::spawn(move || {

    let doc = typst::compile(&world)?;

});

std::thread::spawn(move || {

    let doc = typst::compile(&world)?;

});

We usually use the reflexo-typst crate to build native tools. Add the following dependency to your Cargo.toml:

[dependencies]

reflexo-typst = { version = "0.x.y", features = ["system"] }

[dependencies]

reflexo-typst = { version = "0.x.y", features = ["system"] }

The example of using the crate natively is the typst-ts-node-compiler.

It can also compiled to be run in browsers by changing the features to browser:

reflexo-typst = { version = "0.x.y", features = ["browser"] }

reflexo-typst = { version = "0.x.y", features = ["browser"] }

The example is the typst-ts-web-compiler.

It can also be compiled as a Wasm typst plugin by changing the features to an empty list:

reflexo-typst = { version = "0.x.y", features = [] }

reflexo-typst = { version = "0.x.y", features = [] }

The example is the embedded typst, the Rust part and the typst part.

To align the CLI flags with typst-cli, we directly provide a convenient struct CompileOnceArgs to directly resolve a universe from system arguments.

First, parse the system arguments:

let args = CompileOnceArgs::parse();

let args = CompileOnceArgs::parse();

Then, simply resolve the universe from CompileOnceArgs:

let verse = args.resolve_system()?;

let verse = args.resolve_system()?;

You can also extend the CLI flags with the power of clap. For example, crityp, a benchmark tool for typst scripts, has an additional argument --bench-output to specify the output directory of the benchmark results:

/// Common arguments of crityp benchmark.

#[derive(Debug, Clone, Parser, Default)]

pub struct BenchArgs {

    /// Arguments for compiling the document once, compatible with `typst-cli compile`.

    #[clap(flatten)]

    pub compile: CompileOnceArgs,

    /// Path to output file for benchmarks

    #[clap(long, default_value = "target/crityp")]

    pub bench_output: String,

}

/// Common arguments of crityp benchmark.

#[derive(Debug, Clone, Parser, Default)]

pub struct BenchArgs {

    /// Arguments for compiling the document once, compatible with `typst-cli compile`.

    #[clap(flatten)]

    pub compile: CompileOnceArgs,

    /// Path to output file for benchmarks

    #[clap(long, default_value = "target/crityp")]

    pub bench_output: String,

}

More topics about building:

• Configuring fonts: Font Searchers and Resolvers
• Configuring packages: Package Registries

The universe provides a synchronous view of compiler resources. It can be easily modified for incremental compilations, at the cost of not being a Sync. To start (compiler) tasks, you need to spawn a World instance from the universe.

let world = verse.snapshot();

let doc = typst::compile(&world)?;

let world = verse.snapshot();

let doc = typst::compile(&world)?;

The World instance is Send and Sync, so you can use it in another thread.

std::thread::spawn(move || {

    let doc = typst::compile(&world)?;

});

std::thread::spawn(move || {

    let doc = typst::compile(&world)?;

});

Sometimes, you would like to start a task that compiles a different document.

let world = verse.snapshot_with(Some(reflexo_typst::TaskInputs {

    ..Default::default()

}));

let world = verse.snapshot_with(Some(reflexo_typst::TaskInputs {

    ..Default::default()

}));

You can change the entry (the entry file) or inputs (the sys.inputs in typst documents).

The entry field has EntryState type, and you can get and mutate current world's entry by:

let entry = verse

  .entry_state()

  .select_in_workspace(Path::new("/main.typ"));

let world = verse.snapshot_with(Some(TaskInputs {

  entry: Some(entry),

  ..Default::default()

}));

let entry = verse

  .entry_state()

  .select_in_workspace(Path::new("/main.typ"));

let world = verse.snapshot_with(Some(TaskInputs {

  entry: Some(entry),

  ..Default::default()

}));

There is also a fallible version of select_in_workspace to apply paths from user input:

let another_entry = current_dir()?.join("main.typ");

let entry = verse

    .entry_state()

    .try_select_path_in_workspace(&another_entry)?;

let world = verse.snapshot_with(Some(TaskInputs {

    entry,

    ..Default::default()

}));

let another_entry = current_dir()?.join("main.typ");

let entry = verse

    .entry_state()

    .try_select_path_in_workspace(&another_entry)?;

let world = verse.snapshot_with(Some(TaskInputs {

    entry,

    ..Default::default()

}));

another_entry is required to be an absolute path, so that we can check and select the entry file without ambiguity.

There are also constructors for EntryState, reflecting the possible state of the entry (an optional root, and an optional entry path):

• EntryState::new_rooted_by_parent(entry). It accepts an absolute path to the entry file, and sets typst root to the parent directory. The typst document cannot access the files outside of the parent directory (the root).
• EntryState::new_rooted(root, main). It accepts a path to the root directory and a path relative to the root.
• EntryState::new_rootless(main). It accepts an absolute path to the entry file, and the typst document cannot access any other files (other than package files).
• EntryState::new_workspace(root). It accepts root directory, but the entry file is still not determined. You can use select_in_workspace and its variants to select the entry file later.
• EntryState::new_detached(), where neither the root nor the entry file are determined.

You can create a new sys.inputs either from the dict! macro:

let world = verse.snapshot_with(Some(TaskInputs {

    inputs: Some(Arc::new(LazyHash::new(dict! {

        "my-target" => "markdown"

    }))),

    ..Default::default()

}));

let world = verse.snapshot_with(Some(TaskInputs {

    inputs: Some(Arc::new(LazyHash::new(dict! {

        "my-target" => "markdown"

    }))),

    ..Default::default()

}));

or from string pairs:

let pairs = [("my-target", "docx")].map(|(k, v)| (k.into(), v.into_value()));

let inputs = Arc::new(LazyHash::new(pairs.into_iter().collect()));

let pairs = [("my-target", "docx")].map(|(k, v)| (k.into(), v.into_value()));

let inputs = Arc::new(LazyHash::new(pairs.into_iter().collect()));

You can use the official typst::compile to compile a typst document:

let result = typst::compile::<reflexo_typst::TypstPagedDocument>(&world)?;

let result = typst::compile::<reflexo_typst::TypstPagedDocument>(&world)?;

By default, it is targeting paged, i.e., the sys.target() == "paged" is true. To compile the document targeting html, you could further modify the world for HTML compilation:

let world = world.html_task();

let result = typst::compile::<reflexo_typst::TypstHtmlDocument>(world.as_ref())?;

let world = world.html_task();

let result = typst::compile::<reflexo_typst::TypstHtmlDocument>(world.as_ref())?;

The CompileActor is a wrapper around a universe providing convenient way to run the watch compilation loop. The server watches for filesystem changes and compiles documents again on demand.

let (intr_tx, intr_rx) = tokio::sync::mpsc::unbounded_channel();

let actor = reflexo_typst::CompileActor::new(verse, intr_tx, intr_rx).with_watch(args.watch);

tokio::spawn(actor.run());

let (intr_tx, intr_rx) = tokio::sync::mpsc::unbounded_channel();

let actor = reflexo_typst::CompileActor::new(verse, intr_tx, intr_rx).with_watch(args.watch);

tokio::spawn(actor.run());

Note: it is not as stable as the above other APIs and may change in future.

The intr_tx can be used for sending interrupts to the server.

pub enum Interrupt {

    /// Compile anyway.

    Compile,

    /// Memory file changes.

    Memory(MemoryEvent),

    /// File system event.

    Fs(FilesystemEvent),

    /// Request compiler to stop.

    Settle(oneshot::Sender<()>),

    // ... and others

}

pub enum Interrupt {

    /// Compile anyway.

    Compile,

    /// Memory file changes.

    Memory(MemoryEvent),

    /// File system event.

    Fs(FilesystemEvent),

    /// Request compiler to stop.

    Settle(oneshot::Sender<()>),

    // ... and others

}

A concurrent compiler compiles multiple documents at the same time. This is not exposed yet, but you can check the tinymist test.

let result = start_project(verse, None, move |c, mut i, next| {

    if let Interrupt::Compiled(artifact) = &mut i {

        let files = artifact.documents();

        let res = test_once(&world, &files);

    }

    // Notifies all the dependences touched by `test_once`.

    next(c, i)

});

let result = start_project(verse, None, move |c, mut i, next| {

    if let Interrupt::Compiled(artifact) = &mut i {

        let files = artifact.documents();

        let res = test_once(&world, &files);

    }

    // Notifies all the dependences touched by `test_once`.

    next(c, i)

});

You can shadow paths with content in memory to avoid filesystem accesses or patch the files temporarily, using the ShadowApi. For example, shadowing the main (entry) file using map_shadow_by_id:

let source = Source::new(verse.main_id().unwrap(), "Hello World.".into());

verse

    .map_shadow_by_id(source.id(), Bytes::from_string(source.text().to_owned()))?;

let source = Source::new(verse.main_id().unwrap(), "Hello World.".into());

verse

    .map_shadow_by_id(source.id(), Bytes::from_string(source.text().to_owned()))?;

The shadow has two layers:

• {map,unmap}_shadow: to map/unmap resources on absolute paths about filesystem.
• {map,unmap}_shadow_by_id: to map/unmap resources on virtual file ids.

You can revise the universe, which generates new revisions:

verse.increment_revision(|verse| {

  verse.vfs().invalidate_path(_);

  verse.vfs().invalidate_file_id(_);

  verse.vfs().reset_shadow();

  verse.vfs().map_shadow(_, _);

  verse.vfs().unmap_shadow(_);

  verse.vfs().map_shadow_by_id(_, _);

  verse.vfs().unmap_shadow_by_id(_);

  verse.vfs().notify_fs_changes(_);

  verse.set_fonts(_);

  verse.set_package(_);

  verse.set_inputs(_);

  verse.set_entry_file(_);

  verse.mutate_entry(_);

});

verse.increment_revision(|verse| {

  verse.vfs().invalidate_path(_);

  verse.vfs().invalidate_file_id(_);

  verse.vfs().reset_shadow();

  verse.vfs().map_shadow(_, _);

  verse.vfs().unmap_shadow(_);

  verse.vfs().map_shadow_by_id(_, _);

  verse.vfs().unmap_shadow_by_id(_);

  verse.vfs().notify_fs_changes(_);

  verse.set_fonts(_);

  verse.set_package(_);

  verse.set_inputs(_);

  verse.set_entry_file(_);

  verse.mutate_entry(_);

});

New revisions may be created if the resources are changed by the callback function.

We provide a print_diagnostics function to print the diagnostics:

let res = typst::compile::<TypstPagedDocument>(&world);

let errors = res.output.err();

let diag = res.warnings.iter().chain(errors.iter().flatten());

let _ = print_diagnostics(&world, diag, reflexo_typst::DiagnosticFormat::Human);

let res = typst::compile::<TypstPagedDocument>(&world);

let errors = res.output.err();

let diag = res.warnings.iter().chain(errors.iter().flatten());

let _ = print_diagnostics(&world, diag, reflexo_typst::DiagnosticFormat::Human);

It accepts a reflexo_typst::DiagnosticFormat to specify the format of the output, either Human to pretty print the output or Short to print suitable to be reidentified by editors or other tools.

You must carefully evcit cache to avoid memory leak using CompilerUniverse::evict, 10 is suggested after each compilation.

verse.evict(10);

verse.evict(10);

Also, you have to reset the universe to react filesystem changes if necessary:

verse.increment_revision(|verse| {

  verse.vfs().invalidate_path(changed_path);

  verse.vfs().invalidate_file_id(changed_file_id);

});

verse.increment_revision(|verse| {

  verse.vfs().invalidate_path(changed_path);

  verse.vfs().invalidate_file_id(changed_file_id);

});

Alternatively, you can use CompilerUniverse::reset to reset all the caches and resources:

verse.reset();

verse.reset();

CompilerUniverse::reset is a heavy operation, so it is not recommended to call it frequently. Insteadly, fined-grained control such as using invalidate_{path,file_id} or CompileActor for watch compilation is suggested.

After compilation, you can do rendering with the artifacts, the reflexo_typst::TypstPagedDocument or reflexo_typst::TypstHtmlDocument.

It is pretty easy to get such artifacts for rendering if you can access the universe directly, using the typst::compile. To customized the way of rendering in watch compilation, passing a compilation handler to reflexo_typst::CompileActor::new_with(opts).

Gather all of above code together, a minimal complete example of using the compiler library is like this:

let verse = CompileOnceArgs::parse().resolve_system()?;

let doc = typst::compile(&verse.snapshot())?;

verse.evict(10);

let verse = CompileOnceArgs::parse().resolve_system()?;

let doc = typst::compile(&verse.snapshot())?;

verse.evict(10);

Or incrementally:

let (intr_tx, intr_rx) = tokio::sync::mpsc::unbounded_channel();

let actor = reflexo_typst::CompileActor::new(verse, intr_tx, intr_rx).with_watch(true);

tokio::spawn(actor.run());

let (intr_tx, intr_rx) = tokio::sync::mpsc::unbounded_channel();

let actor = reflexo_typst::CompileActor::new(verse, intr_tx, intr_rx).with_watch(true);

tokio::spawn(actor.run());

Note: the following content is for typst.ts <v0.6.0

Note: The struct TypstSystemUniverse can create multiple snapshots at the same time, struct TypstSystemWorld, implementing trait typst::World.

Example: fn create_driver in compile.rs

let verse = TypstSystemUniverse::new(CompileOpts {

  root_dir: workspace_dir.clone(),

  font_paths: args.font.paths.clone(),

  with_embedded_fonts: EMBEDDED_FONT.to_owned(),

  ..CompileOpts::default()

})

.unwrap_or_exit();

// usage

let mut tracer = Tracer::default();

typst::compile(&verse.snapshot(), tracer);

let verse = TypstSystemUniverse::new(CompileOpts {

  root_dir: workspace_dir.clone(),

  font_paths: args.font.paths.clone(),

  with_embedded_fonts: EMBEDDED_FONT.to_owned(),

  ..CompileOpts::default()

})

.unwrap_or_exit();

// usage

let mut tracer = Tracer::default();

typst::compile(&verse.snapshot(), tracer);

Note: The struct PureCompiler implements trait Compiler.

Example:

std::marker::PhantomData.compile(

  &verse.snapshot(), &mut Default::default());

std::marker::PhantomData.compile(

  &verse.snapshot(), &mut Default::default());

Note: The struct CompileExporter implements trait CompileMiddleware.
Note: The struct CompileExporter derives trait Compiler.

Retrieve an exporter instance that is executed on each successful compilation (more useful for incremental compilation).

let driver = CompileExporter::default()

  .with_exporter(exporter)

let driver = CompileExporter::default()

  .with_exporter(exporter)

See exporter.rs for usage of the exporter feature.

Glance at current available exporters:

type WithAst = reflexo_typst::AstExporter;

type WithPdf = reflexo_typst::PdfDocExporter;

type WithSvg = reflexo_typst::PureSvgExporter;

type WithSvgHtml = reflexo_typst::SvgHtmlExporter<DefaultExportFeature>;

type WithSIR = reflexo_typst::SvgModuleExporter;

type WithText = reflexo_typst::TextExporter;

type WithAst = reflexo_typst::AstExporter;

type WithPdf = reflexo_typst::PdfDocExporter;

type WithSvg = reflexo_typst::PureSvgExporter;

type WithSvgHtml = reflexo_typst::SvgHtmlExporter<DefaultExportFeature>;

type WithSIR = reflexo_typst::SvgModuleExporter;

type WithText = reflexo_typst::TextExporter;

Note: The struct DynamicLayoutCompiler implements trait CompileMiddleware.
Note: The struct DynamicLayoutCompiler derives trait Compiler.

Enable dynamic layout based on a trait Compiler.

let driver = DynamicLayoutCompiler::new(driver, output_dir);

let driver = DynamicLayoutCompiler::new(driver, output_dir);

Specifical for incremental compilation (Specifically, it watches files and compiles on demand) based on some universe instance.

Example: use of struct CompileActor in tool/preview.rs in tinymist

let (intr_tx, intr_rx) = mpsc::unbounded_channel();

let actor = CompileActor::new(verse,

  intr_tx, intr_rx).with_watch(Some(handle.clone()));

let (intr_tx, intr_rx) = mpsc::unbounded_channel();

let actor = CompileActor::new(verse,

  intr_tx, intr_rx).with_watch(Some(handle.clone()));

Example: use of intr_tx in tool/preview.rs in tinymist

Access the service of the CompileActor instance.

/// Updates the overlay layer of VFS (Virtual File System)

let _ = self.intr_tx.send(Interrupt::Memory(event));

/// Reads the snapshot of the current compilation

let (tx, rx) = oneshot::channel();

self.intr_tx.send(Interrupt::SnapshotRead(tx))?;

let snapshot = rx.await

/// Updates the overlay layer of VFS (Virtual File System)

let _ = self.intr_tx.send(Interrupt::Memory(event));

/// Reads the snapshot of the current compilation

let (tx, rx) = oneshot::channel();

self.intr_tx.send(Interrupt::SnapshotRead(tx))?;

let snapshot = rx.await