build-by-date

My docker images are getting out of date, and this causes issues for example when building Cloogle: code that works on a fresh installation does not work in the container, because of frontwards incompatible changes. That is why I'd like to modify the build scripts to be able to build a Clean system for a particular date, with the goal to provide a Docker image that can build any Clean distribution by using clean-build. It is not possible to base a Docker image on the nightlies, because the nightlies are removed the next day (and Docker images should be stable and reproducible). This is an enhancement and does not change the default behaviour of these scripts.

This MR makes the change for linux-x64 for the libraries, bundle-complete and bundle-itasks distributions.

This is (should be) fully backwards-compatible and does (should) not break the jenkins build: when $CLEANDATE is not present, the result is exactly the same as before. I did a diff -r on bundle-complete for the old and the new build scripts and the only changes were in generated assembly, because the Jenkins build uses GCC 5 and I'm on GCC 6 (bundle-complete.diff).

Is this something useful to have upstream (@baslijns)? If yes, would it be a problem if $CLEANDATE is only used for linux-x64? The other platforms are not interesting from a Docker perspective.

Changes:

• Every fetch.sh for libraries checks if $CLEANDATE exists. If so, it checks out the repositories at that point in time. For this --depth 1 had to be removed from the git clone calls. When $CLEANDATE is unset, the behaviour is the same as before.
• The fetch.sh, setup.sh and build.sh of bundle-complete now call scripts from the lib-* directories to fetch, setup and build the libraries. For base, the old approach (downloading the release) is used, because this build script does not seem to work for me, presumably because I'm on GCC 6.
• This change requires that every library-fetch.sh cleans up the directories it's going to create, because several fetch scripts pull the same code.
• The build.sh of bundle-complete uses rsync to move everything to one directory, which is needed to merge directories (which mv cannot do).
• The fetch.sh of base only uses $CLEANDATE for StdEnv; the tools (clm, compiler, etc.) and other libraries (not included in the distribution) use the latest version, s.t. you always have an up to date build environment and only old libraries.

Minor problems with this setup:

• Because fetch.sh of bundle-complete downloads the latest base, $CLEANDATE does not work on StdEnv. That is not such a big issue since it's updated rarely. This will be fixed when base is built instead of downloaded like the libraries in bundle-complete.
• This breaks when using dates far in the past, when the repository directory structure has changed (like is the case for iTasks). There is not much we can do about that without changing the setup (like adding build rules in the source repositories).

Note: I have now created the Docker image for nightlies, the readme shows a usage example at the bottom. The image currently uses my fork of clean-build.