SRFI-64 Test Results

Running a test sets various status properties in the current test-runner. This can be examined by a custom test-runner, or (more rarely) in a test-suite.

Result Kind

Running a test may yield one of the following status symbols:

'pass

The test passed, as expected.

'fail

The test failed (and was not expected to).

'xfail

The test failed and was expected to.

'xpass

The test passed, but was expected to fail.

'skip

The test was skipped.

Scheme Procedure: test-result-kind [runner]

Returns one of the above result codes from the most recent tests. Returns #f if no tests have been run yet. If we’ve started on a new test, but don’t have a result yet, then the result kind is 'xfail if the test is expected to fail, 'skip if the test is supposed to be skipped, or #f otherwise.

Scheme Procedure: test-passed? [runner]

True if the value of ‘(test-result-kind [runner])’ is one of 'pass or 'xpass. This is a convenient shorthand that might be useful in a test suite to only run certain tests if the previous test passed.

Test result properties

A test runner also maintains a set of more detailed “result properties” associated with the current or most recent test. (I.e. the properties of the most recent test are available as long as a new test hasn’t started.) Each property has a name (a symbol) and a value (any value). Some properties are standard or set by the implementation; implementations can add more.

Scheme Procedure: test-result-ref runner pname [default]

Returns the property value associated with the pname property name (a symbol). If there is no value associated with pname return default, or #f if default isn’t specified.

Scheme Syntax: test-result-set! runner pname value

Sets the property value associated with the pname property name to value. Usually implementation code should call this function, but it may be useful for a custom test-runner to add extra properties.

Scheme Procedure: test-result-remove runner pname

Remove the property with the name pname.

Scheme Procedure: test-result-clear runner

Remove all result properties. The implementation automatically calls test-result-clear at the start of a test-assert and similar procedures.

Scheme Procedure: test-result-alist runner

Returns an association list of the current result properties. It is unspecified if the result shares state with the test-runner. The result should not be modified; on the other hand, the result may be implicitly modified by future test-result-set! or test-result-remove calls. However, a test-result-clear does not modify the returned alist. Thus you can “archive” result objects from previous runs.

Standard result properties

The set of available result properties is implementation-specific. However, it is suggested that the following might be provided:

'result-kind

The result kind, as defined previously. This is the only mandatory result property. (test-result-kind runner) is equivalent to: (test-result-ref runner 'result-kind)

'source-file

'source-line

If known, the location of test statements (such as test-assert) in test suite source code.

'source-form

The source form, if meaningful and known.

'expected-value

The expected non-error result, if meaningful and known.

'expected-error

The error-typespecified in a test-error, if it meaningful and known.

'actual-value

The actual non-error result value, if meaningful and known.

'actual-error

The error value, if an error was signalled and it is known. The actual error value is implementation-defined.