Commit 53d02ea6 authored by benoit's avatar benoit
Browse files

cleaning

parent df4b46ab
This diff is collapsed.
Dune is an community orientated open source project. It was originally
developed at [Jane Street][js] and is now maintained by Jane Street,
[OCamlLabs][ocl] as well as several developers from the OCaml
community.
Contributions to Dune are welcome and should be submitted via GitHub
pull requests against the `master` branch. Dune is distributed under
the MIT license and contributors are required to sign their work in
order to certify that they have the right to submit it under this
license. See the following section for more details.
Signing contributions
---------------------
We require that you sign your contributions. Your signature certifies
that you wrote the patch or otherwise have the right to pass it on as
an open-source patch. The rules are pretty simple: if you can certify
the below (from [developercertificate.org][dco]):
```
Developer Certificate of Origin
Version 1.1
Copyright (C) 2004, 2006 The Linux Foundation and its contributors.
1 Letterman Drive
Suite D4700
San Francisco, CA, 94129
Everyone is permitted to copy and distribute verbatim copies of this
license document, but changing it is not allowed.
Developer's Certificate of Origin 1.1
By making a contribution to this project, I certify that:
(a) The contribution was created in whole or in part by me and I
have the right to submit it under the open source license
indicated in the file; or
(b) The contribution is based upon previous work that, to the best
of my knowledge, is covered under an appropriate open source
license and I have the right under that license to submit that
work with modifications, whether created in whole or in part
by me, under the same open source license (unless I am
permitted to submit under a different license), as indicated
in the file; or
(c) The contribution was provided directly to me by some other
person who certified (a), (b) or (c) and I have not modified
it.
(d) I understand and agree that this project and the contribution
are public and that a record of the contribution (including all
personal information I submit with it, including my sign-off) is
maintained indefinitely and may be redistributed consistent with
this project or the open source license(s) involved.
```
Then you just add a line to every git commit message:
```
Signed-off-by: Joe Smith <joe.smith@email.com>
```
Use your real name (sorry, no pseudonyms or anonymous contributions.)
If you set your `user.name` and `user.email` git configs, you can sign
your commit automatically with `git commit -s`.
[dco]: http://developercertificate.org/
[js]: https://www.janestreet.com/
[ocl]: http://ocamllabs.io/
Coding style
------------
- wrap lines at 80 characters,
- use `[Ss]nake_case` over `[Pp]ascalCase`.
# Hacking on Dune
This section is for people who want to work on Dune itself.
## Setup
### Create an Opam switch with the required dependencies
From the project root, execute
```sh
$ make dev-switch
# This takes some time
```
## Building
### Common usage
To build and install the development version, run the following while in the
project's root directory:
```sh
$ make dev
```
Note that this will build the `dune` binary twice: once using the
bootstrap build procedure and then one normal dune build using the
freshly produced `dune binary`.
For a short synopsis on the usual `make` commands used in development, run
```sh
$ make
# or
$ make help
```
### A note on bootstrapping
In order to build itself, Dune uses a micro dune written as a single
[boot/duneboot.ml](boot/duneboot.ml) file. This micro build system
cannot read `dune` files and instead has the configuration hard-coded
in [boot/libs.ml](boot/libs.ml). This latter file is automatically
updated during development when we modify the `dune` files in the
repository. [boot/duneboot.ml](boot/duneboot.ml) itself is built with
a single invocation of `ocamlopt` or `ocamlc` via the
[bootstrap.ml](bootstrap.ml) ocaml script.
[boot/duneboot.ml](boot/duneboot.ml) builds a `dune.exe` binary at the
root of the source tree and uses this binary to build everything else.
`make dev` takes care of bootstrapping if needed, but if you want to just run
the bootstrapping step itself, build the `dune.exe` target with
```sh
make dune.exe
```
## OCaml compatibility test
Install opam switches for all the entries in the
[dune-workspace.dev](dune-workspace.dev) file and run:
```sh
$ make all-supported-ocaml-versions
```
## Repository organization
- `vendor/` contains dependencies of Dune, that have been vendored
- `plugin/` contains the API given to `dune` files that are OCaml
scripts
- `src/` contains the core of Dune
- `bin/` contains the command line interface
- `doc/` contains the manual and rules to generate the manual pages
## Design
Dune (nee "JBuilder") was initially designed to sort out the public release of
Jane Street packages which became incredibly complicated over time. It is still
successfully used for this purpose.
One necessary feature to achieve this is the ability to precisely
report the external dependencies necessary to build a given set of
targets without running any command, just by looking at the source
tree. This is used to automatically generate the `<package>.opam`
files for all Jane Street packages.
To implement this, the build rules are described using a build data type,
which is defined in [src/dune/build.mli](src/dune/build.mli). In the end it
makes the development of the internal rules of Dune very composable and
quite pleasant.
To deal with process multiplexing, Dune uses a simplified
Lwt/Async-like monad, implemented in [src/fiber/fiber.mli](src/fiber/fiber.mli).
## Tests
Dune uses [cram style](https://blog.janestreet.com/testing-with-expectations/)
tests for its test suite. The test suite is contained in
[test/blackbox-tests](test/blackbox-tests). A single test consists of a test
directory in the test-cases/ sub directory which contains a run.t file defining
the test.
An example `run.t` file:
```
A description of the test. The command running the tests is preceeded by
two spaces and a $. The expected output of the command is also indented by
two spaces and is right below the command (note the absence of a $)
$ echo "the expected output is below"
the expected output is below
```
Running the entire test suite is done with `$ make test`. A particular test can
be executed with `$ dune build @<test-name>`.
Running the test will execute the command after the `$` and its output will be
compared against the expected output right below the command. Any differences
will result in a test failure.
### Adding a Test
Simply add a new directory in test/blackbox-tests/test-cases and then `$ make`
generate the rules for the test , followed by `$ make promote` to accept the new
rules.
### Accepting Corrections
A failing expect test will generate a diff between the expected and actual
output. If the new output generated by the command is satisfactory, it can be
*promoted* with the `$ make promote` command.
## Documentation
### User documentation
User documentation lives in the [./doc](./doc) directory.
In order to build the user documentation, you must install
- [`python-sphinx`][python-sphinx], and
- [`sphinx_rtd_theme`][sphinx_rtd_theme]
Build the documentation with
```sh
make doc
```
For automatically updated builds, you can [install
`sphinx-autobuild`][sphinx-autobuild], and run
```sh
make livedoc
```
[python-sphinx]: http://www.sphinx-doc.org/en/master/usage/installation.html
[sphinx_rtd_theme]: https://sphinx-rtd-theme.readthedocs.io/en/stable/
[sphinx-autobuild]: https://pypi.org/project/sphinx-autobuild/
## Code flow
- [src/dune/dune_file.mli](src/dune/dune_file.mli) contains the internal
representation of `dune` files and the parsing code
- [src/dune/dune_load.mli](src/dune/dune_load.mli) contains the code to scan
a source tree and build the internal database by reading
the `dune` files
- [src/dune/gen_rules.mli](src/dune/gen_rules.mli) contains all the build rules
of Dune
- [src/dune/build_system.mli](src/dune/build_system.mli) contains a trivial
implementation of a Build system. This is what Jenga will provide
when implementing the bridge
The MIT License
Copyright (c) 2016 Jane Street Group, LLC <opensource@janestreet.com>
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
jbuilder has been renamed to dune. Refer to the manual for details of the
migration:
[manual](http://dune.readthedocs.io/en/latest/migration.html).
.DEFAULT_GOAL := help
PREFIX_ARG := $(if $(PREFIX),--prefix $(PREFIX),)
LIBDIR_ARG := $(if $(LIBDIR),--libdir $(LIBDIR),)
DESTDIR_ARG := $(if $(DESTDIR),--destdir $(DESTDIR),)
INSTALL_ARGS := $(PREFIX_ARG) $(LIBDIR_ARG) $(DESTDIR_ARG)
BIN := ./dune.exe
# Dependencies used for developing and testing dune
DEV_DEPS := \
core_bench \
menhir \
merlin \
ocamlformat \
odoc \
ppx_expect \
ppx_inline_test \
ppxlib \
utop
-include Makefile.dev
help:
@cat doc/make-help.txt
release: $(BIN)
$(BIN) build -p dune --profile dune-bootstrap
dune.exe: bootstrap.ml boot/libs.ml boot/duneboot.ml
ocaml bootstrap.ml
dev: $(BIN)
$(BIN) build @install
all: $(BIN)
$(BIN) build
install:
$(BIN) install $(INSTALL_ARGS) dune
uninstall:
$(BIN) uninstall $(INSTALL_ARGS) dune
reinstall: uninstall install
dev-switch:
opam switch create -y . --deps-only --with-test
opam install -y $(DEV_DEPS)
test: $(BIN)
$(BIN) runtest
test-js: $(BIN)
$(BIN) build @runtest-js
test-coq: $(BIN)
$(BIN) build @runtest-coq
test-all: $(BIN)
$(BIN) build @runtest @runtest-js @runtest-coq
check: $(BIN)
$(BIN) build @check
fmt: $(BIN)
$(BIN) build @fmt --auto-promote
promote: $(BIN)
$(BIN) promote
accept-corrections: promote
all-supported-ocaml-versions: $(BIN)
$(BIN) build @install @runtest --workspace dune-workspace.dev --root .
clean: $(BIN)
$(BIN) clean || true
rm -rf _boot dune.exe
distclean: clean
rm -f src/dune/setup.ml
doc:
cd doc && sphinx-build . _build
livedoc:
cd doc && sphinx-autobuild . _build \
-p 8888 -q --host $(shell hostname) -r '\.#.*'
update-jbuilds: $(BIN)
$(BIN) build @doc/runtest --auto-promote
# If the first argument is "run"...
ifeq (dune,$(firstword $(MAKECMDGOALS)))
# use the rest as arguments for "run"
RUN_ARGS := $(wordlist 2,$(words $(MAKECMDGOALS)),$(MAKECMDGOALS))
# ...and turn them into do-nothing targets
$(eval $(RUN_ARGS):;@:)
endif
dune: $(BIN)
$(BIN) $(RUN_ARGS)
.PHONY: default install uninstall reinstall clean test doc dev-switch
.PHONY: promote accept-corrections opam-release dune check fmt
opam-release:
dune-release distrib --skip-build --skip-lint --skip-tests -n dune
# See https://github.com/ocamllabs/dune-release/issues/206
DUNE_RELEASE_DELEGATE=github-dune-release-delegate dune-release publish distrib --verbose -n dune
dune-release opam pkg -n dune
dune-release opam submit -n dune
Dune - A composable build system
================================
Dune is a build system designed for OCaml/Reason projects only. It
focuses on providing the user with a consistent experience and takes
care of most of the low-level details of OCaml compilation. All you
have to do is provide a description of your project and dune will
do the rest.
The scheme it implements is inspired from the one used inside Jane
Street and adapted to the open source world. It has matured over a
long time and is used daily by hundreds of developers, which means
that it is highly tested and productive.
Dune comes with a [manual][manual]. If you want to get started
without reading too much, you can look at the [quick start
guide][quick-start] or watch [this introduction video][video].
The [example][example] directory contains examples of projects using
dune.
[![Travis status][travis-img]][travis]
[![AppVeyor status][appveyor-img]][appveyor]
[![Release][release-img]][release]
[manual]: https://dune.readthedocs.io/en/latest/
[quick-start]: https://dune.readthedocs.io/en/latest/quick-start.html
[example]: https://github.com/ocaml/dune/tree/master/example
[travis]: https://travis-ci.org/ocaml/dune
[travis-img]: https://travis-ci.org/ocaml/dune.svg?branch=master
[appveyor]: https://ci.appveyor.com/project/diml/dune/branch/master
[appveyor-img]: https://ci.appveyor.com/api/projects/status/rsxayce22e8f2jkp?svg=true
[release]: https://github.com/ocaml/dune/releases
[release-img]: https://img.shields.io/github/release/ocaml/dune.svg
[merlin]: https://github.com/ocaml/merlin
[opam]: https://opam.ocaml.org
[issues]: https://github.com/ocaml/dune/issues
[topkg-jbuilder]: https://github.com/diml/topkg-jbuilder
[video]: https://youtu.be/BNZhmMAJarw
Overview
--------
Dune reads project metadata from `dune` files, which are either
static files in a simple S-expression syntax or OCaml scripts. It uses
this information to setup build rules, generate configuration files
for development tools such as [merlin][merlin], handle installation,
etc...
Dune itself is fast, has very low overhead and supports parallel
builds on all platforms. It has no system dependencies: all you need
to build dune and packages using dune is OCaml. You don't need
`make` or `bash` as long as the packages themselves don't use `bash`
explicitly.
In particular, one can install OCaml on Windows with a binary installer
and then use only the Windows Console to build dune and packages
using dune.
Strengths
---------
### Composable
Take n repositories that use dune, arrange them in any way on the
file system and the result is still a single repository that dune
knows how to build at once.
This make simultaneous development on multiple packages trivial.
### Gracefully handles multi-package repositories
Dune knows how to handle repositories containing several
packages. When building via [opam][opam], it is able to correctly use
libraries that were previously installed even if they are already
present in the source tree.
The magic invocation is:
```sh
$ dune build --only-packages <package-name> @install
```
### Building against several configurations at once
Dune is able to build a given source code repository against
several configurations simultaneously. This helps maintaining packages
across several versions of OCaml as you can test them all at once
without hassle.
In particular, this makes it easy to handle [cross-compilation](https://dune.readthedocs.io/en/latest/advanced-topics.html#cross-compilation).
This feature requires [opam][opam].
Requirements
------------
Dune requires OCaml version 4.07.0 to build itself and can build OCaml
projects using ocaml 4.02.3 or greater.
Installation
------------
The recommended way to install dune is via the [opam package manager][opam]:
```sh
$ opam install dune
```
You can also build it manually with:
```sh
$ make release
$ make install
```
If you do not have `make`, you can do the following:
```sh
$ ocaml bootstrap.ml
$ ./dune.exe build -p dune --profile dune-bootstrap
$ ./dune.exe install dune
```
The first command builds the `dune.exe` binary. The second builds the
additional files that are installed by dune, such as the man pages, and
the last simply installs all of that on the system.
Note that unless you ran the optional `./configure` script, you can
simply copy `dune.exe` anywhere and it will just work. `dune` is
fully relocatable and discovers its environment at runtime rather than
hard-coding it at compilation time.
Support
-------
If you have questions about dune, you can send an email to
ocaml-core@googlegroups.com or [open a ticket on GitHub][issues].
Migration from jbuilder
-----------------------
Dune was formerly known as jbuilder. Migration from jbuilder to dune is
described in the [manual](http://dune.readthedocs.io/en/latest/migration.html).
Status
------
Dune is now fairly stable and is used by the majority of packages on
opam. Note that dune retains backward compatibility with Jbuilder, and
in particular existing Jbuilder projects will continue to be buildable
with dune.
# AppVeyor config using https://github.com/Chris00/ocaml-appveyor
install:
- cd "%APPVEYOR_BUILD_FOLDER%"
- install_ocaml.cmd
build_script:
- cd "%APPVEYOR_BUILD_FOLDER%"
- ocaml bootstrap.ml
- dune.exe build @test\blackbox-tests\windows-diff
artifacts:
- path: _build/log
name: build-log
- path: _boot/log
name: build_bootstrap-log
# Uncomment this to enable Remote Desktop on the build worker at the end of the
# build. The worker is available for the remainder of the allocated hour.
# on_finish:
# - ps: $blockRdp = $true; iex ((new-object net.webclient).DownloadString('https://raw.githubusercontent.com/appveyor/ci/master/scripts/enable-rdp.ps1'))
(executable
(name main)
(modules main)
(libraries dune_bench core_bench.inline_benchmarks))
(library
(name dune_bench)
(libraries stdune fiber dune)
(library_flags -linkall)
(preprocess
(pps ppx_bench)))
(* Benchmark File_tree.t's internal cache *)
open Stdune
open Dune
let deep_path = "a1/a2/a3/a4/a5/a6/a7/a8/a9/10"
let setup =
lazy
(let tmp = Path.External.of_string (Filename.get_temp_dir_name ()) in
Path.External.mkdir_p (Path.External.relative tmp deep_path);
Path.set_root tmp;
Path.Build.set_build_dir (Path.Build.Kind.of_string "_build");
let _conf = Dune_load.load ~ancestor_vcs:None () in
let path = Path.Source.of_string deep_path in
at_exit (fun () -> Sys.remove "./dune-project");
path)
let%bench_fun "File_tree.find_dir" =
let path = Lazy.force setup in
fun () ->
ignore
(Sys.opaque_identity (File_tree.find_dir path) : File_tree.Dir.t option)
(* Benchmark the scheduler *)
open Stdune
open Dune
let setup =
lazy
( Path.set_root (Path.External.cwd ());
Path.Build.set_build_dir (Path.Build.Kind.of_string "_build") )
let prog = Option.value_exn (Bin.which ~path:(Env.path Env.initial) "true")