Skip to main content

Architecture

The system consists of several components, which are briefly explained in the following sections.

Evaluation Stack​

An evaluation stack is a configuration used for analysis. It consists of an environment and an evaluator. The environment provides the software required for analysis. This includes compilers for programming or markup languages, frameworks, and test runners. Docker images and the containers that instantiate them are examples of environments. However, Eva is not tightly coupled to Docker and could in theory also use the host system as an environment.

An evaluator runs bash scripts and commands on the environment. It then processes the return values and outputs to produce the analysis result. It standardizes the results through dedicated exchange formats. An evaluator can support all analysis capabilities (compilation, test, style, coverage) or only a subset of them.

A stack is uniquely identified by a selector:

<evaluator_id>[@(<environment_id>)]

Examples:

  • java: Refers to the evaluator java with the default environment configured in the evaluator declaration
  • java@java19: Refers to the evaluator java with the java19 environment

Dispatcher​

The dispatcher processes analysis requests that it receives from clients through a RabbitMQ broker. Multiple dispatcher instances can exist, running on different systems (to enable horizontal scalability). Overall, the dispatcher can be viewed as a worker that acts as the local master on a machine, utilizing the available system resources to process its assigned requests in parallel. The dispatcher ensures that a new instance of the requested environment (e.g. a Docker container) is created for each request. In addition, it creates an instance of the requested evaluator and runs it on the environment. Once the analysis is complete, the result is sent back to the client through the RabbitMQ broker.

Registry​

The registry provides information about the available evaluation stacks and exposes their metadata. It serves as a centralized source for discovering and managing evaluation stacks and was designed with the following objectives:

  • Single source of truth: Eva can be used by multiple clients. Maintaining a separate registry within each client can lead to duplication and inconsistencies. Thus, the registry is managed centrally, ensuring that all clients access the same information.
  • Encapsulation: The registry exposes relevant information, such as the set of available environments, while abstracting away implementation details. This separation allows underlying resources, such as Docker images, to be relocated or modified without affecting clients.

Technical Context and Distribution​