Testing Feature

The testing feature is available since tinymisttinymist v0.13.10.

You can run tests and check coverage in the IDE or CLI.

Given a file, tinymist will try to discover tests related to the file.

• All dependent files in the same workspace will be checked.

  • For example, if file a.typa.typ contains import "b.typ"import "b.typ" or include "b.typ"include "b.typ", tinymist will check b.typb.typ for tests as well.

• For each file including the entry file itself, tinymist will check the file for tests.

  • If a file is named example-*.typexample-*.typ, it is considered an example document and will be compiled using typst::compiletypst::compile.

    • Both png export and html export may be called.
    • For now, png export is always called for each example file.
    • If the label <test-html-example><test-html-example> can be found in the example file, html export will be called.

  • Top-level functions will be checked for tests.

    • If a function is named test-*test-*, it is considered a test function and will be called directly.
    • If a function is named bench-*bench-*, it is considered a benchmark function and will be called once to collect coverage.
    • If a function is named panic-on-*panic-on-*, it will only pass the test if a panic occurs during execution.

Example Entry File:

#import "example-hello-world.typ"

#let test-it() = {

  "test"

}

#let panic-on-panic() = {

  panic("this is a panic")

}

#import "example-hello-world.typ"

#let test-it() = {

  "test"

}

#let panic-on-panic() = {

  panic("this is a panic")

}

#import "example-hello-world.typ"

#let test-it() = {

  "test"

}

#let panic-on-panic() = {

  panic("this is a panic")

}

#import "example-hello-world.typ"

#let test-it() = {

  "test"

}

#let panic-on-panic() = {

  panic("this is a panic")

}

Example Output:

Running example(example-hello-world)

Running test(test-it)

Running test(panic-on-panic)

 Passed test(test-it)

 Passed test(panic-on-panic)

 Passed example(example-hello-world)

   Info All test cases passed...

Running example(example-hello-world)

Running test(test-it)

Running test(panic-on-panic)

 Passed test(test-it)

 Passed test(panic-on-panic)

 Passed example(example-hello-world)

   Info All test cases passed...

Running example(example-hello-world)

Running test(test-it)

Running test(panic-on-panic)

 Passed test(test-it)

 Passed test(panic-on-panic)

 Passed example(example-hello-world)

   Info All test cases passed...

Running example(example-hello-world)

Running test(test-it)

Running test(panic-on-panic)

 Passed test(test-it)

 Passed test(panic-on-panic)

 Passed example(example-hello-world)

   Info All test cases passed...

Since it requires some heavy framework to run benchmarks, a standalone tool is provided to run benchmarks.

Check crityp for more information.

• Run and collect file coverage using command tinymist.profileCurrentFileCoveragetinymist.profileCurrentFileCoverage in VS Cod(e,ium).

• Run and collect test coverage using command tinymist.profileCurrentTestCoveragetinymist.profileCurrentTestCoverage in VS Cod(e,ium).

  • Check Test Discovery to learn how tinymist discovers tests.

VS Cod(e,ium) will show the overall coverage in the editor.

You can run tests and check coverage in the CLI.

tinymist test tests/main.typ

...

  Info: All test cases passed...

tinymist test tests/main.typ

...

  Info: All test cases passed...

tinymist test tests/main.typ

...

  Info: All test cases passed...

tinymist test tests/main.typ

...

  Info: All test cases passed...

You can pass same arguments as typst compiletypst compile to tinymist testtinymist test.

To watch for changes and run tests automatically, use the --watch--watch option:

tinymist test --watch tests/main.typ

tinymist test --watch tests/main.typ

tinymist test --watch tests/main.typ

tinymist test --watch tests/main.typ

When running in the watch mode, input command lines to control the test runner:

   Hint Press 'h' for help

$ h

h/r/u/c/q: help/run/update/quit

   Hint Press 'h' for help

$ h

h/r/u/c/q: help/run/update/quit

   Hint Press 'h' for help

$ h

h/r/u/c/q: help/run/update/quit

   Hint Press 'h' for help

$ h

h/r/u/c/q: help/run/update/quit

For example, according to the help message, update the reference files using the command uu (update).

You can collect coverage using the --coverage--coverage option.

tinymist test tests/main.typ --coverage

...

   Info Written coverage to target/coverage.json ...

    Cov Coverage Summary 9/10 (90.00%)

   Info All test cases passed...

tinymist test tests/main.typ --coverage

...

   Info Written coverage to target/coverage.json ...

    Cov Coverage Summary 9/10 (90.00%)

   Info All test cases passed...

tinymist test tests/main.typ --coverage

...

   Info Written coverage to target/coverage.json ...

    Cov Coverage Summary 9/10 (90.00%)

   Info All test cases passed...

tinymist test tests/main.typ --coverage

...

   Info Written coverage to target/coverage.json ...

    Cov Coverage Summary 9/10 (90.00%)

   Info All test cases passed...

Use --print-coverage=full--print-coverage=full to print the coverage of each file.

tinymist test tests/main.typ --coverage --print-coverage=full

...

    Cov  6     / 6     (100.00%)  tests/example-hello-world.typ

    Cov  3     / 4     ( 75.00%)  tests/main.typ

    Cov Coverage Summary 9/10 (90.00%)

   Info All test cases passed...

tinymist test tests/main.typ --coverage --print-coverage=full

...

    Cov  6     / 6     (100.00%)  tests/example-hello-world.typ

    Cov  3     / 4     ( 75.00%)  tests/main.typ

    Cov Coverage Summary 9/10 (90.00%)

   Info All test cases passed...

tinymist test tests/main.typ --coverage --print-coverage=full

...

    Cov  6     / 6     (100.00%)  tests/example-hello-world.typ

    Cov  3     / 4     ( 75.00%)  tests/main.typ

    Cov Coverage Summary 9/10 (90.00%)

   Info All test cases passed...

tinymist test tests/main.typ --coverage --print-coverage=full

...

    Cov  6     / 6     (100.00%)  tests/example-hello-world.typ

    Cov  3     / 4     ( 75.00%)  tests/main.typ

    Cov Coverage Summary 9/10 (90.00%)

   Info All test cases passed...

If any test fails, the CLI will return a non-zero exit code.

tinymist test tests/main.typ

...

 Fatal: Some test cases failed...

tinymist test tests/main.typ

...

 Fatal: Some test cases failed...

tinymist test tests/main.typ

...

 Fatal: Some test cases failed...

tinymist test tests/main.typ

...

 Fatal: Some test cases failed...

To update the reference files, you can run:

tinymist test tests/main.typ --update

tinymist test tests/main.typ --update

tinymist test tests/main.typ --update

tinymist test tests/main.typ --update

To get image files to diff you can use grep to find the image files to update:

tinymist test tests/main.typ 2> >(grep Hint) > >(grep "compare image")

   Hint example(example-hello-world): compare image at refs/png/example-hello-world.png

   Hint example(example-other): compare image at refs/png/example-other.png

tinymist test tests/main.typ 2> >(grep Hint) > >(grep "compare image")

   Hint example(example-hello-world): compare image at refs/png/example-hello-world.png

   Hint example(example-other): compare image at refs/png/example-other.png

tinymist test tests/main.typ 2> >(grep Hint) > >(grep "compare image")

   Hint example(example-hello-world): compare image at refs/png/example-hello-world.png

   Hint example(example-other): compare image at refs/png/example-other.png

tinymist test tests/main.typ 2> >(grep Hint) > >(grep "compare image")

   Hint example(example-hello-world): compare image at refs/png/example-hello-world.png

   Hint example(example-other): compare image at refs/png/example-other.png

You can use your favorite image diffdiff tool to compare the images, e.g. magick comparemagick compare.

To ensure that the rendering is reproducible, you can ignore system fonts.

tinymist test tests/main.typ --ignore-system-fonts

tinymist test tests/main.typ --ignore-system-fonts

tinymist test tests/main.typ --ignore-system-fonts

tinymist test tests/main.typ --ignore-system-fonts

Adds font paths using the --font-paths--font-paths option if you want to use custom fonts:

tinymist test tests/main.typ --font-paths /path/to/fonts

tinymist test tests/main.typ --font-paths /path/to/fonts

tinymist test tests/main.typ --font-paths /path/to/fonts

tinymist test tests/main.typ --font-paths /path/to/fonts

tinymist testtinymist test only compares hash files to check whether content is changed. Therefore, you can ignore rendered files and only keep the hash files to compare them on CI. Putting the following content in .gitignore.gitignore will help you to ignore the files:

# png files

refs/png/**/*.png

# html files

refs/html/**/*.html

# hash files

!refs/**/*.hash

# png files

refs/png/**/*.png

# html files

refs/html/**/*.html

# hash files

!refs/**/*.hash

# png files

refs/png/**/*.png

# html files

refs/html/**/*.html

# hash files

!refs/**/*.hash

# png files

refs/png/**/*.png

# html files

refs/html/**/*.html

# hash files

!refs/**/*.hash

Install tinymisttinymist on CI and run tinymist testtinymist test to check whether the content is changed.

- name: Install tinymist

  env:

    TINYMIST_VERSION: 0.13.x # to test with typst compiler v0.13.x, tinymist v0.14.x for typst v0.14.x, and so on.

  run: curl --proto '=https' --tlsv1.2 -LsSf https://github.com/Myriad-Dreamin/tinymist/releases/download/${TINYMIST_VERSION}/tinymist-installer.sh | sh

- name: Run tests (Typst)

  run: tinymist test tests/main.typ --root . --ppi 144 --ignore-system-fonts

- name: Upload artifacts

  uses: actions/upload-artifact@v4

  with:

    name: refs

    path: refs

- name: Install tinymist

  env:

    TINYMIST_VERSION: 0.13.x # to test with typst compiler v0.13.x, tinymist v0.14.x for typst v0.14.x, and so on.

  run: curl --proto '=https' --tlsv1.2 -LsSf https://github.com/Myriad-Dreamin/tinymist/releases/download/${TINYMIST_VERSION}/tinymist-installer.sh | sh

- name: Run tests (Typst)

  run: tinymist test tests/main.typ --root . --ppi 144 --ignore-system-fonts

- name: Upload artifacts

  uses: actions/upload-artifact@v4

  with:

    name: refs

    path: refs

- name: Install tinymist

  env:

    TINYMIST_VERSION: 0.13.x # to test with typst compiler v0.13.x, tinymist v0.14.x for typst v0.14.x, and so on.

  run: curl --proto '=https' --tlsv1.2 -LsSf https://github.com/Myriad-Dreamin/tinymist/releases/download/${TINYMIST_VERSION}/tinymist-installer.sh | sh

- name: Run tests (Typst)

  run: tinymist test tests/main.typ --root . --ppi 144 --ignore-system-fonts

- name: Upload artifacts

  uses: actions/upload-artifact@v4

  with:

    name: refs

    path: refs

- name: Install tinymist

  env:

    TINYMIST_VERSION: 0.13.x # to test with typst compiler v0.13.x, tinymist v0.14.x for typst v0.14.x, and so on.

  run: curl --proto '=https' --tlsv1.2 -LsSf https://github.com/Myriad-Dreamin/tinymist/releases/download/${TINYMIST_VERSION}/tinymist-installer.sh | sh

- name: Run tests (Typst)

  run: tinymist test tests/main.typ --root . --ppi 144 --ignore-system-fonts

- name: Upload artifacts

  uses: actions/upload-artifact@v4

  with:

    name: refs

    path: refs