Verified Commit 2f42a4f4 authored by Camil Staps's avatar Camil Staps 🚀

Merge branch 'master' into ssl

parents 9bfbda8f 06b6c528
Clean System Files
_Tests
_Tests.*
test:
before_script:
- install_clean.sh bundle-complete && apt-get update -qq && apt-get install -y -qq build-essential
- install_clean.sh bundle-complete
- apt-get update -qq
- apt-get install -y -qq build-essential git coreutils libsnappy-dev
- git clone --recursive https://gitlab.science.ru.nl/clean-and-itasks/clean-test-properties /opt/clean-test-properties
- make -C /opt/clean-test-properties/src -f Makefile.linux64
image: "camilstaps/clean:nightly"
script:
- make -C tests/linux64
- make -C tests/linux64 run
- stdbuf -o0 -e0 /opt/clean-test-properties/src/testproperties -IL Dynamics -d src/libraries/OS-Independent -P Quiet -r -T 'Tests 100000' -C -h -C 100m
# Clean documentation
Cloogle indexes documentation of the syntax elements it stores, through
functions in `Clean.Doc`. Docblocks are comments that start with `/**` and have
a leading asterisk on every line (leading whitespace is ignored). The first
part of the docblock is taken as a general description. Below the description,
documentation fields can be added with `@`. Currently, documentation fields
should have only one line.
An example is below:
```clean
/**
* Apply a function to every element in a list.
*
* @param The function
* @param The list
* @result The new list
*/
map :: (a -> b) [a] -> [b]
```
`@result` can be given multiple times for tuples.
For short documentation items, doclines, starting with `//*` can be used. When
documenting a constructor, or record field, they should be placed *after* the
item they document. Doclines are only supported for constructors and record
fields. For example:
```clean
/**
* A date in the Gregorian calendar
*/
:: Date =
{ day :: Int //* The day of the month, starting with 1
, month :: Int //* The month (January is 1)
, year :: Int //* The year
}
```
## Markup in documentation
Some simple Markdown-inspired markup is allowed in documentation:
- `` `foo` `` renders `foo` in monospaced font.
- Code blocks can be surrounded by `` ``` `` on separate lines. The start of a
code block can indicate the language (for highlighting purposes), as in
`` ```clean ``.
- `{{bar}}` marks `bar` as a defined entity (that can be searched for).
- Double newlines distinguish paragraphs; single newlines are ignored unless
followed by a hyphen.
## Documentation fields
The tables below describe which fields and documentation types can be used for
different syntax elements, and what they should document. An extension, to
document test properties, is discussed below.
What fields are accepted for what syntax elements is defined by the records in
`Clean.Doc`; how they are parsed in the instances of the generic function
`docBlockToDoc`. The below is merely a convenient representation of the same
information.
| | Description | `@param` | `@result` | `@type` | `@var` | `@representation` | `@throws` | `@complexity`
|--------------|-------------|----------|-----------|---------|--------|-------------------|-----------|--------------
| Class | ![][y] | ![][y]<sup>1</sup> | ![][y]<sup>1</sup> | | ![][y] | |
| Class member | ![][y] | ![][y] | ![][y] | | | | ![][y] | ![][y]
| Constructor | ![][y] | | | | | | |
| Function | ![][y] | ![][y] | ![][y] | | | | ![][y] | ![][y]
| Generic | ![][y] | ![][y] | ![][y] | | ![][y] | | |
| Instance | | | | | | | |
| Macro | ![][y] | ![][y] | ![][y] | ![][y]<sup>2</sup> | | | |
| Module | ![][y] | | | | | | |
| Record field | ![][y] | | | | | | |
| Type | ![][y] | | | | ![][y] | ![][y], for type synonyms | |
<sup>1: only for shorthand classes like `class zero a :: a`, where there is no
other place for the documentation of the class member.</sup>
<sup>2: for simple macros (depending on what the type deriver in
`Clean.Types.CoclTransform` can do), Cloogle will derive the type if it is not
given.</sup>
| Field | Description
|-------------------|-------------
| `@complexity` | E.g. "O(n log n)".
| `@param` | Parameters of a function(-like). Name a parameter using `@param name: description`.
| `@representation` | The representation of a synonym type.
| `@result` | The result of a function.
| `@return` | A deprecated synonym of `@result`.
| `@throws` | iTasks exceptions that can be thrown.
| `@type` | The type of a macro (without name and `::`).
| `@var` | Type variables of types, classes and generics.
### Property documentation
With [clean-test-properties][]' `testproperties` tool, [Gast][] test programs
can be generated with properties from docblocks. For this, several additional
fields can be used, which are further documented by [clean-test-properties][].
[clean-test-properties]: https://gitlab.science.ru.nl/clean-and-itasks/clean-test-properties
[Gast]: https://gitlab.science.ru.nl/clean-and-itasks/gast
[y]: http://i.stack.imgur.com/iro5J.png
......@@ -70,11 +70,11 @@ id x = x
```
Several JavaDoc like parameters are supported such as `@param`, `@result`,
`@type`, `@var` and `@representation`. More info about this can be found
[here](https://github.com/clean-cloogle/Cloogle#clean-documentation).
We use `@complexity` for the complexity order. Some other special fields are
used, like `@gin-icon`, but one should be reluctant with inventing new field
names. If there is a general use case, adding it can be discussed.
`@type`, `@var` and `@representation`. More info about this can be found in
[DOCUMENTATION.md](DOCUMENTATION.md). We use `@complexity` for the complexity
order. Some other special fields are used, like `@gin-icon`, but one should be
reluctant with inventing new field names. If there is a general use case,
adding it can be discussed.
## Layout
......@@ -101,7 +101,7 @@ The applicable instances for the _general_ classes should be exported in the mod
This means that for example the `Functor` instance of `Maybe` should be defined in `Data.Maybe` and not in `Data.Functor`.
For _specific_ classes the instances for types should be exported in submodules.
For example, `JSONEncode` for `Map` should be exported in `Data.Map.JSONEncode` and not in `Data.Map` nor in `Text.JSON`.
For example, `JSONEncode` for `Map` should be exported in `Data.Map.GenJSON` and not in `Data.Map` nor in `Text.GenJSON`.
This rule also holds for types that have multiple valid instances such as the `Monoid` for `Int`.
_general_ classes are:
......
......@@ -2,4 +2,5 @@ definition module System._Platform
import System.Platform
//* @type Platform
CURRENT_PLATFORM :== Android32
......@@ -2,4 +2,5 @@ definition module System._Platform
import System.Platform
//* @type Platform
CURRENT_PLATFORM :== Android64
definition module Clean.Doc
/**
* Parsing and storing Clean documentation
*/
import StdGeneric
from StdOverloaded import class toString
from Data.Either import :: Either
from Data.GenDefault import generic gDefault
from Data.Maybe import :: Maybe
from Clean.Types import :: Type
/**
* A wrapper around the {{`String`}} type which makes sure that multi-line
* documentation blocks get trimmed w.r.t. whitespace.
*/
:: MultiLineString = MultiLine !String
class docDescription d :: !d -> Maybe Description
class docComplexity d :: !d -> Maybe String
class docParams d :: !d -> [ParamDoc]
class docVars d :: !d -> [Description]
class docResults d :: !d -> [Description]
class docType d :: !d -> Maybe Type
class docThrows d :: !d -> [Description]
class docMembers d :: !d -> [Maybe ClassMemberDoc]
class docFields d :: !d -> Maybe [Maybe Description]
class docConstructors d :: !d -> Maybe [Maybe ConstructorDoc]
class docRepresentation d :: !d -> Maybe (Maybe Description)
class docPropertyBootstrap d :: !d -> Maybe String
class docPropertyTestWith d :: !d -> [PropertyVarInstantiation]
class docPropertyTestGenerators d :: !d -> [PropertyTestGenerator]
class docProperties d :: !d -> [Property]
/**
* Documentation of a Clean module.
*/
:: ModuleDoc =
{ description :: !Maybe Description
, property_bootstrap :: !Maybe MultiLineString //* For generating unit tests with clean-test
, property_test_with :: ![PropertyVarInstantiation] //* With which types to test the properties
, property_test_generators :: ![PropertyTestGenerator]
//* Functions to generate values of types for which Gast's {{`ggen`}} is not good enough, like {{`Map`}}
}
instance docDescription ModuleDoc
instance docPropertyBootstrap ModuleDoc
instance docPropertyTestWith ModuleDoc
instance docPropertyTestGenerators ModuleDoc
derive gDefault ModuleDoc
/**
* Documentation of a Clean function.
*/
:: FunctionDoc =
{ description :: !Maybe Description
, complexity :: !Maybe String //* E.g. "O(n log n)"
, params :: ![ParamDoc] //* Descriptions of the parameters
, vars :: ![Description] //* Descriptions of the type variables (for generics)
, results :: ![Description] //* Descriptions of the result(s, for tuples)
, type :: !Maybe Type //* The type (for macros)
, throws :: ![Description] //* The exceptions it may throw (iTasks)
, properties :: ![Property] //* Properties of this function
, property_test_with :: ![PropertyVarInstantiation] //* With which types to test the properties
, preconditions :: ![String] //* Preconditions for the properties
}
instance docDescription FunctionDoc
instance docComplexity FunctionDoc
instance docParams FunctionDoc
instance docVars FunctionDoc
instance docResults FunctionDoc
instance docType FunctionDoc
instance docThrows FunctionDoc
instance docPropertyTestWith FunctionDoc
instance docProperties FunctionDoc
/**
* Documentation of a function parameter.
*/
:: ParamDoc =
{ name :: !Maybe String //* An optional name for the parameter
, description :: !Maybe Description //* An optional description
}
instance toString ParamDoc
instance docDescription ParamDoc
/**
* A property of a function.
* Typically, the property can be tested with Gast.
*
* - `ForAll`: the right-hand side (the third argument) holds for all values of
* the arguments (the second argument). The first argument is the name.
*/
:: Property
= ForAll !String ![(!String,!Type)] !String
/**
* When a property type contains type variables, a `PropertyVarInstantiation`
* can be used to instantiate those variables when generating test cases.
*/
:: PropertyVarInstantiation = PropertyVarInstantiation !(!String, !Type)
/**
* A test generator generates values of some type. The first argument of the
* constructor is the function type of the generator, for instance
* `[(k,v)] -> {{Map}} k v`. The second argument is the implementation, which
* should assume the generator is called `gen` (e.g.: `gen elems = ...`).
*/
:: PropertyTestGenerator = PropertyTestGenerator !Type !String
derive gDefault FunctionDoc, Property, PropertyVarInstantiation, PropertyTestGenerator
/**
* Documentation of a Clean class member.
* For an explanation of the fields, see the documentation on {{`FunctionDoc`}}.
*/
:: ClassMemberDoc =
{ description :: !Maybe Description
, complexity :: !Maybe String
, params :: ![ParamDoc]
, results :: ![Description]
, type :: !Maybe Type
, throws :: ![Description]
}
instance docDescription ClassMemberDoc
instance docComplexity ClassMemberDoc
instance docParams ClassMemberDoc
instance docResults ClassMemberDoc
instance docType ClassMemberDoc
instance docThrows ClassMemberDoc
derive gDefault ClassMemberDoc
/**
* Documentation of a Clean ADT constructor.
* For an explanation of the fields, see the documentation on {{`FunctionDoc`}}.
*/
:: ConstructorDoc =
{ description :: !Maybe Description
, params :: ![ParamDoc]
}
instance docDescription ConstructorDoc
instance docParams ConstructorDoc
derive gDefault ConstructorDoc
/**
* Documentation of a Clean class.
*/
:: ClassDoc =
{ description :: !Maybe Description
, vars :: ![Description] //* The type variables
, members :: ![Maybe ClassMemberDoc] //* Documentation on the members
}
instance docDescription ClassDoc
instance docVars ClassDoc
instance docMembers ClassDoc
derive gDefault ClassDoc
/**
* Documentation of a Clean type.
*/
:: TypeDoc =
{ description :: !Maybe Description
, vars :: ![Description] //* Type variables
, representation :: !Maybe (Maybe Description) //* For synonym types
, fields :: !Maybe [Maybe Description] //* For records
, constructors :: !Maybe [Maybe ConstructorDoc] //* For ADTs
, invariants :: ![Property] //* For Gast test generation
}
instance docDescription TypeDoc
instance docVars TypeDoc
instance docFields TypeDoc
instance docConstructors TypeDoc
instance docRepresentation TypeDoc
derive gDefault TypeDoc
/**
* Description of a Clean syntax element
*/
:: Description :== String
/**
* Parse error for parsing Clean documentation; no documentation could be found
*/
:: ParseError
= MissingAsterisk !String //* At least one line did not start with a *
| MissingField !String //* A required field was missing
| UnknownError !String //* Another error
| InternalNoDataError
/**
* Parse warning while parsing Clean documentation; the parser has made a
* best-effort result nevertheless
*/
:: ParseWarning
= UnknownField !String //* Unknown @-field
| IllegalField !String //* This @-field is not allowed in this docblock
| NoDescription //* The main description is missing
| UsedReturn //* Used @return instead of @result
| UnparsableType !String //* Could not parse a @type field as a type
/**
* Convert a ConstructorDoc to a FunctionDoc.
*/
constructorToFunctionDoc :: !ConstructorDoc -> FunctionDoc
/**
* Convert a FunctionDoc to a ClassMemberDoc.
*/
functionToClassMemberDoc :: !FunctionDoc -> ClassMemberDoc
/**
* Add a class member to an existing class definition
*
* @param The documentation to add the member to
* @param The documentation on the class member
* @result The new ClassDoc
*/
addClassMemberDoc :: !ClassDoc !(Maybe ClassMemberDoc) -> ClassDoc
/**
* Parse a single docstring, removing the asterisk and trimming whitespace.
*/
parseSingleLineDoc :: (String -> String)
/**
* Parse a documentation block. The magic happens in {{`docBlockToDoc`}}.
*/
parseDoc :: !String -> Either ParseError (!d, ![ParseWarning]) | docBlockToDoc{|*|} d
/**
* A documentation block.
* @representation An order list of key-value pairs. A key can occur multiple
* times. The description has key `description`.
*/
:: DocBlock :== [(!String, !String)]
/**
* The magic for {{`parseDoc`}}. Usually, a record type like {{`FunctionDoc`}}
* will derive a convenient parser. In some cases, it may be necessary to
* override the default, such as in the instance for {{`Type`}}, where parsing
* of the type happens.
* @var The thing to parse
*/
generic docBlockToDoc d :: !(Either [String] DocBlock) -> Either ParseError (!d, ![ParseWarning])
derive docBlockToDoc UNIT, PAIR, EITHER, CONS, OBJECT, FIELD of {gfd_name}, RECORD
derive docBlockToDoc String, [], Maybe, Type
derive docBlockToDoc ModuleDoc, FunctionDoc, ClassMemberDoc, ClassDoc,
ConstructorDoc, TypeDoc
/**
* Print a documentation block as a string. The magic happens in
* {{`docToDocBlock`}}.
*/
printDoc :: !d -> String | docToDocBlock{|*|} d
/**
* The magic for {{`printDoc`}}.
* @param If true, return a `Left`. If false, return a `Right`.
*/
generic docToDocBlock d :: Bool d -> Either [String] DocBlock
derive docToDocBlock ModuleDoc, FunctionDoc, ClassMemberDoc, ClassDoc,
ConstructorDoc, TypeDoc
/**
* Trace a list of ParseWarnings like StdDebug might do it
*/
traceParseWarnings :: ![ParseWarning] !a -> a
/**
* Trace a ParseError like StdDebug might do it
*/
traceParseError :: !ParseError !a -> a
This diff is collapsed.
definition module Clean.ModuleFinder
/**
* This module provides functionality to search for Clean modules in the file
* system.
*/
from System.FilePath import :: FilePath
from System.Options import :: Option
from System.OSError import :: OSError, :: OSErrorMessage, :: OSErrorCode
/**
* Options to tweak the searching for Clean modules.
*/
:: ModuleFindingOptions =
{ include_paths :: ![FilePath] //* Complete paths to search in (clm's `-I`)
, include_libraries :: ![String] //* Libraries to search in (combined with CLEAN_HOME; clm's `-IL`)
, clean_home :: !FilePath //* Override CLEAN_HOME
, include_applications :: !Bool //* Whether to include modules that do not have a definition module or not
}
/**
* Get the default {{`ModuleFindingOptions`}}. This requires the World because
* {{`clean_home`}} needs to be set correctly, for which the `CLEAN_HOME`
* environment variable is read.
*/
defaultModuleFindingOptions :: !*World -> *(!ModuleFindingOptions, !*World)
/**
* An option description ({{System.Options}}) for {{`ModuleFindingOptions`}},
* supporting clm's `-I` and `-IL`, as well as long forms, possibility to
* override `CLEAN_HOME`, etc.
*/
moduleFindingOptionDescription :: Option ModuleFindingOptions
/**
* Find a specific module in the file system.
*
* @param The module name
* @param The options to search for the module
* @result A list of all matching file paths
*/
findModule :: !String !ModuleFindingOptions !*World -> *(![FilePath], !*World)
/**
* Find all modules in the file system.
*
* @param The options to search for modules
* @result {{`OSError`}}s that occurred during searching
* @result File paths of all modules found
*/
findAllModules :: !ModuleFindingOptions !*World -> *(![OSError], ![FilePath], !*World)
implementation module Clean.ModuleFinder
import StdArray
import StdList
import StdString
import Clean.Parse.ModuleName
import Data.Error
from Data.Func import $, mapSt
import System.Directory
import System.Environment
import System.File
import System.FilePath
import System.Options
import System.OS
import Text
defaultModuleFindingOptions :: !*World -> *(!ModuleFindingOptions, !*World)
defaultModuleFindingOptions w
# (home,w) = getEnvironmentVariable "CLEAN_HOME" w
# home = fromMaybe (IF_WINDOWS "C:\\Clean" "/opt/clean") home
# opts =
{ include_paths = []
, include_libraries = ["StdEnv"]
, clean_home = home
, include_applications = False
}
= (opts, w)
moduleFindingOptionDescription :: Option ModuleFindingOptions