Core Concepts
ReqCover compares the requirements in your specification with the requirements your passing tests claim to verify, and fails the build when too few are covered.
Requirements
Section titled “Requirements”A requirement is a piece of functionality the system should have.
ReqCover does not invent requirements: it loads them from a specification through a requirements source, such as an OpenAPI document read by reqcover-source-openapi.
The locations are set with reqCover.requirementsUris, a comma-separated list of classpath resources.
Requirement IDs
Section titled “Requirement IDs”Every requirement has a string ID. Tests refer to requirements by this ID.
The OpenAPI source builds IDs from each operation’s operationId:
| OpenAPI content | Requirement ID |
|---|---|
Operation without responses |
#getUsers |
| Each response of an operation | #getUsers/200, #getUsers/404 |
Each key under a response’s x-requirements |
#getUsers/200/sortedByName |
paths: /users: get: operationId: getUsers responses: '200': description: The list of users x-requirements: sortedByName: Users are sorted by name paginated: At most 50 users per page '404': description: No users foundThis document defines #getUsers/200/sortedByName, #getUsers/200/paginated and #getUsers/404.
When a response has x-requirements, the response itself (#getUsers/200) is not a separate requirement.
Operations without an operationId are skipped with a warning. Details are in Requirement sources.
Linking tests to requirements
Section titled “Linking tests to requirements”- JUnit and TestNG: annotate a test method or (since 0.0.12) a test class with
@ForRequirement(packagedev.reqcover.api). The annotation is repeatable. - Cucumber: tag a scenario, rule or feature with
@req:<id>, for example@req:#getUsers/200.
Expected, verified, covered, unexpected
Section titled “Expected, verified, covered, unexpected”At the end of a test run, ReqCover works with these sets:
| Set | Meaning |
|---|---|
| Expected | All requirements loaded from the specification |
| Verified | All requirement IDs named by passing tests |
| Covered | Requirements that are both expected and verified |
| Not covered | Expected but not verified; logged as a warning |
| Unexpected | Verified but not in the specification, for example a typo in an ID; logged as a warning and not counted |
The coverage is covered / expected × 100. With no expected requirements, the coverage is 100%.
When a requirement counts as covered
Section titled “When a requirement counts as covered”A requirement is covered when at least one passing test names it. Failing, skipped and aborted tests never count.
- A test with several
@ForRequirementannotations covers all of them when it passes. @ForRequirementon a test class (JUnit, TestNG; since 0.0.12) is covered when at least one test in the class passes.- For parameterized tests,
@RepeatedTest, the dynamic tests of a@TestFactoryand TestNG data providers, one passing run is enough. - A passing Cucumber scenario covers the requirements of its own tags and the tags inherited from its feature and rule.
The coverage gate
Section titled “The coverage gate”After the run, ReqCover:
- logs the requirements that are not covered and the unexpected ones,
- logs
Requirements coverage: <covered> out of <expected> = <percent>%, - writes the reports,
- compares the coverage with
reqCover.minimumRequiredCoveragePercent(default0), - if the coverage is below the minimum and
reqCover.failIfBelowMinimumistrue(the default), logs an error and ends the test JVM with exit code 1.
A coverage equal to the minimum passes. Because the reports are written first, you can always inspect them after a failed gate.
Problems with the specification are stricter: a requirements file that is missing, malformed or in an unsupported format fails the build right away.
If reqCover.requirementsUris is not set, ReqCover logs a warning. With no requirements loaded, coverage is reported as 100% and the gate always passes.
Configuration precedence
Section titled “Configuration precedence”Each setting is looked up in order; the first place that sets it wins.
| Order | JUnit Jupiter | TestNG | Cucumber |
|---|---|---|---|
| 1 | JVM system property | JVM system property | JVM system property |
| 2 | JUnit configuration parameter (junit-platform.properties, launcher or build tool) |
testng.xml suite <parameter> |
cucumber.properties |
| 3 | Default | Default | junit-platform.properties |
| 4 | Default |
All settings are listed in Configuration.
Extension points (SPI)
Section titled “Extension points (SPI)”ReqCover finds its plug-ins with the Java ServiceLoader, so adding a module to the test classpath is enough.
| Interface | Purpose | Built-in implementations |
|---|---|---|
dev.reqcover.engine.spi.RequirementsSource |
Load requirement IDs from a URI | OpenApiRequirementsSource |
dev.reqcover.engine.spi.RequirementsCoverageReporter |
Write a report from the coverage result | HTML and XML reporters |
To write your own, see Requirement sources and Reporters.
Next steps
Section titled “Next steps”- Getting Started
- Configuration
- JUnit, TestNG, Cucumber guides