README.md 4.54 KB
Newer Older
Camil Staps's avatar
Camil Staps committed
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88
# clean-test [![pipeline status](https://gitlab.science.ru.nl/clean-and-itasks/clean-test/badges/master/pipeline.svg)](https://gitlab.science.ru.nl/clean-and-itasks/clean-test/commits/master)

This projects hosts tools that can be used to test Clean programs.
The available tools are:

- `cleantest` which can run test programs
- `makecleantest` which can generate test programs from dcl modules

## cleantest

This tool can run other test programs and combine the results.
The tool was conceived in
[clean-platform#17](https://gitlab.science.ru.nl/clean-and-itasks/clean-platform/issues/19).

It is assumed that the other programs adhere to the interface described in
`Testing.TestEvents`. For [Gast][] tests, this can be done easily with the
`exposeProperties` function. For example:

```clean
Start w = exposeProperties [OutputTestEvents] [Tests 500000]
	[ EP prop_one
	, EP prop_two
	, EP prop_three
	]
	w
```

Here, `prop_one`, `prop_two` and `prop_three` are Gast properties (not
necessarily of the same type). `exposeProperties` generates a command-line
interface. `cleantest` can run this kind of test programs (and others) by
adding their executables with the `-r` flag, e.g.:

```bash
cleantest -r testModuleA -r testModuleB
```

The tool has several command-line options; check `cleantest --help` for
up-to-date details.

## makecleantest

This tool can generate test programs using [Gast][]'s `exposeProperties` (see
above) from dcl modules. The properties to test are described in [CleanDoc][]
blocks: comments above function types. This works as follows:

- On the module documentation block, add a `@property-bootstrap` field. This
  field should contain the 'header' of the generated test program: required
  imports and helper definitions. The programmer must not add a module header
  (`definition module ...`) and does not need to import Gast modules or the
  module under test itself.

- On each function to test, add one or more `@property` fields to the docblock.
  This property for now always starts with `A.`, followed by a number of
  parameters and their types. The first line ends with a colon `:`. The
  implementation of the property follows on the next line(s). For example:

  ```clean
  /**
   * @property member_correct: A.x :: a; xs :: [a]:
   *   member x (fromList xs) <==> isMember x xs
   */
  member :: !a !(Set a) -> Bool | < a
  ```

  Note that one can use Gast's `<==>` here without import, nor does one need to
  import the module itself for `member` (and `fromList`). However, `isMember`
  requires `import StdList` in the `@property-bootstrap` of the module.

  It is possible to use multiple lines for the implementation of the property.
  `makecleantest` will guess the right indentation level.

- When parameters in `@property` fields contain type variables, the programmer
  is required to suggest types to instantiate these variables with when
  testing &mdash; otherwise, how would Gast be able to generate test cases?
  This can be done with `@property-test-with`, either locally on the function
  docblock or globally on the module docblock. For example:

  ```clean
  /**
   * @property-test-with a = Int
   * @property-test-with a = Real
   */
  ```

  For all relevant tests using the type variable `a`, two versions will be
  generated: one with `Int` and one with `Real`. When multiple type variables
  are used, all combinations are tested.

89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114
- Sometimes, Gast's generic test generation method is not very useful. This is
  for instance the case with types for which automatically generated values are
  not 'valid'.

  Take `Set a` as an example. It is possible to write tests on `[a]` instead,
  then in every test use `fromList :: [a] -> Set a` to get the set. However,
  this is not very modular, and also clutters up the properties.

  What you can do instead is writing tests as you would normally do,
  quantifying over `Set a` in the property signature, and then add a
  `@property-test-generator` to the module. This field tells the test generator
  how to generate values of a certain type. An example is:

  ```clean
  /**
   * @property-test-generator [a] -> Set a | < a
   *   gen xs = fromList xs
   */
  ```

  Currently, only generators with arity 1 are supported (but this can be worked
  around using a tuple or an auxiliary data type in the `@property-bootstrap`).
  When the generator has class constraints, you are expected to resolve them
  yourself by providing the required instances, depending on the type variable
  instantiations.

Camil Staps's avatar
Camil Staps committed
115 116 117

[Gast]: https://gitlab.science.ru.nl/clean-and-itasks/gast
[CleanDoc]: https://github.com/clean-cloogle/Cloogle#clean-documentation