Skip to content

Automatic notebook testing

Given that we now have solutions to all the notebooks (https://gitlab.ewi.tudelft.nl/mmc-tudelft/education/MMSRNotebooks/-/issues/9) we should introduce notebook testing.

The main idea is that whenever we introduce a change to a notebook we would like to run it under all likely environments (e.g. Colab, VM, Docker etc) to make sure that the notebook outputs match what we expect/latest commit didn't break anything.

The main difficulty is replicating the Colab environment with all of its binaries/packages. Unfortunately, Colab team have not released their Docker image. Whilst it may happen eventually, currently the only way that comes to mind is to connect Colab to a local runtime (https://research.google.com/colaboratory/local-runtimes.html). Further examination is needed but a potential source of conflict will also be the fact that we would have to make sure that the testing is able to interact with/use local files, not those on Colab.

Running testing code on VM is easier but we also introduce a split between Colab and local infrastructure. Again, Docker containers for everything would be ideal but this is currently not possible.

We should decide on whether this issue is a priority after we rule on unifying colab and VM notebooks as well as our transition to Docker.

Edited by Norman
To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information