ETSI EN 303 645 Test evidence for consumer IoT assessments

EN 303 645 is the baseline requirements standard. Annex B classifies provisions as mandatory, recommended, conditional, and feature-dependent and lets the user record Y, N, or N/A. The detail field records implemented measures, reasons for non-support, or the rationale for N/A. Provision 5.0-1 separately requires a justification for each recommendation considered not applicable or not fulfilled.

TS 103 701 is the compatible assessment methodology. It defines the Device Under Test (DUT), Supplier Organization, Test Laboratory, ICS, IXIT, conceptual and functional tests, external evidence, and verdict rules. An evidence pack should therefore identify the DUT and software version, associated services and processes in scope, each ICS claim, supporting IXIT detail, applied test group or accepted external evidence, indications, and verdict.

The Supplier Organization completes DUT identification and the ICS, then supplies the necessary IXIT information for provisions claimed Y. The Test Laboratory checks the IXIT for completeness, consistency, and soundness with the supplier. It also verifies that no mandatory provision is claimed N and tests whether N/A claims match the provision's stated condition or feature.

The IXIT must be specific enough to select test methods, equipment, conditions, and instructions. A referenced document has to be supplied to the laboratory. Missing or insufficient information can produce INCONCLUSIVE at test-case level because the test cannot be properly executed; it should not be recorded as PASS merely because no failure was observed.

Sometimes, but only for the test group covered by the TS 103 701 external-evidence rules. Existing security certifications or third-party evaluations of parts of the DUT can reduce assessment effort. The Supplier Organization must identify the evidence in the relevant ICS detail field and provide the certificate, scope, test reports, and other information needed for verification.

The Test Laboratory decides whether the evidence is adequate for that test group. It checks whether the scope matches the objective, the prior test activities satisfy every test purpose, and the test depth or evaluation assurance level fits the baseline assessment. Evidence accepted for one component or test group does not establish an overall verdict for the whole DUT.

Verdicts roll up in a fixed order. A test group passes when every test case passes or accepted external evidence satisfies clause 4.7. Any failed test case fails its test group. The overall result passes only if the ICS is valid and every test group for a provision claimed Y passes; one failed applicable group produces FAIL, while an inconclusive applicable group produces INCONCLUSIVE when no fail criterion applies.