Coverage Exchange Format (CEXF)

CEXF is an exchange format for representing code coverage metrics, both overall and per test case. The format was created to address the lack of standardization among existing code coverage tools. Current tools vary significantly in both their data formats and the way they define and report coverage metrics (see this paper).

Structure

<?xml version="1.0" encoding="UTF-8"?>
<coverage tool="clover" version="4.4.1">
    <testsuite>
        <testcase classname="name.panitz.oose.StarTest" name="test1">
            <file name="name/panitz/oose/Star.java" fqcn="name.panitz.oose.Star">
                <line nr="6" type="line" covered="2" missed="0"/>
                <line nr="6" type="method" covered="1" missed="0"/>
                <line nr="6" type="stmt" covered="1" missed="0"/>
                <line nr="7" type="line" covered="1" missed="0"/>
                <line nr="7" type="stmt" covered="1" missed="0"/>
            </file>
        </testcase>
        <testcase classname="name.panitz.oose.StarTest" name="test2">
            <file name="name/panitz/oose/Star.java" fqcn="name.panitz.oose.Star">
                <line nr="6" type="line" covered="2" missed="0"/>
                <line nr="6" type="method" covered="1" missed="0"/>
                <line nr="6" type="stmt" covered="1" missed="0"/>
                <line nr="7" type="line" covered="1" missed="0"/>
                <line nr="7" type="stmt" covered="1" missed="0"/>
            </file>
        </testcase>
    </testsuite>
</coverage>

file elements within a test element contain coverage metrics for a specific source file captured during the execution of that test case. Test cases are uniquely identified by their classname and name attributes. line elements represent coverage metrics associated with individual lines in the source file. Each element specifies the number of covered and missed items for a given metric identified by the type attribute (method, stmt, ...). While multiple line elements may reference the same nr (line number), each metric is only reported once per line.

The meaning of individual coverage metrics is defined by the instrumentation strategy and the coverage tool that produced them. In particular, metrics that appear similar across tools may represent different concepts and therefore cannot always be compared directly. For example, the definition of C0 coverage depends fundamentally on the type of instrumentation performed by the coverage tool. As such, there is no single universal definition of C0. While JaCoCo defines C0 as instruction coverage, OpenClover defines it as statement coverage. To provide a more general approximation of executed source lines, CEXF introduces the line metric, which represents line coverage. Depending on the used coverage tool, this metric corresponds either to instruction coverage or statement coverage.

OpenClover

Metrics: line, method, stmt, cond, class

JaCoCo

Metrics: line, instr, cond