README.md

# Preliminaries

## Install rust
See: [https://www.rust-lang.org/tools/install](https://www.rust-lang.org/tools/install)

## Install anaconda
See: [https://docs.anaconda.com/anaconda/install/index.html](https://docs.anaconda.com/anaconda/install/index.html)

## (Recommended) install mamba for faster virtual environment installations
See: [https://github.com/mamba-org/mamba](https://github.com/mamba-org/mamba)

## Install and activate the virtual environment:
```bash
<mamba/conda> env create -f conda_environment.yml && \
conda activate natural_computing
```

# The dataset

## Obtain the raw data
Run the following commands
```bash
mkdir -p data
wget https://nilsgolembiewski.nl/public_files/uploads/2IGcXY8HeE69JlgFk1QLCvBh7NRxAV/full_export.txt.gz -O - | gunzip -c > data/raw_data.txt
```

## Generate dataset from raw data
```bash
cargo run --manifest-path=data_generation/Cargo.toml --release -- -d ./data/raw_data.txt -o ./data/dataset -l 20
```

This will result in the following dataset:

Folder structure: `folder/<canvas_id>/<canvas_id>_<user_id>_<idx>_<label>_<info>.<data_type>`.

### `before.png`
the state of the canvas as it was before the modification

### `delta.png`
the modifications since the canvas was moved

### `mask_points.txt` columns
`x`, `y`. The first y (y=0) is the top of the image. 

### `sequence.txt` columns
Concatenated information of: the latest placed pixel, the previously placed pixel (if any)
placed pixel: `canvas_id`, `user_id`, `x`, `y`, `r`, `g`, `b`, `timestamp`, `is_grief`
previous pixel: `exists`, `user_id`, `r`, `g`, `b`, `timestamp`, if `exists` is zero, all other values are also -1


## Downloads
A pregenerated dataset can be downloaded here: [https://nilsgolembiewski.nl/public_files/uploads/fDhANiJtdVw7EZSoW3sFyunk6mRL9q/dataset.zip](https://nilsgolembiewski.nl/public_files/uploads/fDhANiJtdVw7EZSoW3sFyunk6mRL9q/dataset.zip).

The corresponding `train_metadata.yml` can be downloaded from [https://nilsgolembiewski.nl/public_files/uploads/dXuJ7lqc6WPKh4ebgfVOw523vnSAjN/train_metadata.yml.gz](https://nilsgolembiewski.nl/public_files/uploads/dXuJ7lqc6WPKh4ebgfVOw523vnSAjN/train_metadata.yml.gz)

Or use the following commands (unzipping may take a while):
```bash
mkdir -p data
cd data
wget https://nilsgolembiewski.nl/public_files/uploads/fDhANiJtdVw7EZSoW3sFyunk6mRL9q/dataset.zip -O dataset.zip \
    && unzip -q dataset.zip \
    && rm dataset.zip
wget https://nilsgolembiewski.nl/public_files/uploads/dXuJ7lqc6WPKh4ebgfVOw523vnSAjN/train_metadata.yml.gz -O - | gunzip -c > train_metadata.yml
```


# Training

## Vision model
### Train for the folds
```bash
python train_vision.py -t data/train_metadata.yml -j 4 -e configurations/train_vision/resnet_18.yml -o output/vision_models
```

### Inspect results
Run:
```
mlflow ui
```
And view the results in a browser by clicking on the link which is printed. Each fold is a separate run, but they share a common `unique_id`, which can be found in the parameters.

The best models for each fold can be found in the output folder (`output/vision_models/<unique_id>`) if the command above was used.


# TODO
mistake analysis, mistake correlated?