Skip to content

Docker parsing

Luc Everse requested to merge docker-parsing into development

Description

Adds Docker output parsing.

Introduces placeholder metric names for tools like JUnit, checkstyle, and JaCoCo. For now, only JUnit is implemented.

Note that the Docker logs metric is required for the Tests metric to run.

Read about it on the wiki

This is an example Dockerfile that runs Gradle on AuTA and extracts test reports:

FROM f00f/auta-build-env

WORKDIR /var/auta/submission

CMD ["sh", "-c", "chmod +x ./gradlew \
    && ./gradlew test; \
    rm -Rf /var/auta/out; \
    mkdir -p /var/auta/out/junit; \
    cp -Rv core/build/test-results /var/auta/out/junit/core; \
    cp -Rv worker/build/test-results /var/auta/out/junit/worker; \
    cp -Rv build/test-results /var/auta/out/junit/common"]

When given a failing test, it outputs the errors inside the test method entities:

image

Additions

  • OutputProcessors - classes that consume Docker output, translating it to metrics
  • The Docker runner now extracts and passes the output to the output processors
  • Automatic loading for output processors. This also means more manifests. Yay!
  • XSD -> Java bean translation. Read more about it on the wiki
  • The Docker runner can use both TCP and UNIX sockets to communicate with the API. Settings are no longer TCP-only and have to be specified as URIs
  • Submission directory traversal up to the first directory with contents. Resolves #206 (closed) as a happy little accident
  • A helper for tests generating directories, which can recursively delete these directories safely
  • Tests for what can be tested

Changes

  • @Unmodifiable is also applicable to parameters
  • Metric criteria scripts now expect ECMAScript 6 to allow IntelliJ to intelligently highlight strings
  • The Docker runner now archives the entire submission instead of mounting the temporary directory, as this did not work for DinD
  • Quality refactoring of the Docker classes
  • Split slim.Dockerfile into separate files for the core and worker, because the core image was growing too large
  • Fixed the Docker containers using locale C instead of C.UTF-8, Unicode characters are now supported in paths
  • Hackishly disabled job path ASCII-fying using DI (#219)
  • Added a magic "local" tag for build-slim-image that tags the image as "latest" and does not push to the Docker Hub
  • The worker no longer drops privileges, running as root instead. While unfortunate, it is necessary in order to allow the worker to communicate with the Docker daemon (#220)

Test and Review

To be filled in by the reviewers

  • All of the methods are commented to expectation

  • The methods are tested to satisfaction

  • There are no unnecessary files present in the MR

  • The continuous integration has no problems with the MR

  • The MR is filled in as requested (including labels, milestones, and reviewers)

  • The documentation is up-to-date

  • All nullable parameters are marked as such

Edited by Luc Everse

Merge request reports