Commit 3d74d356 authored by Nils Golembiewski's avatar Nils Golembiewski
Browse files

final commit

parent c3beb87b
......@@ -69,21 +69,54 @@ wget https://nilsgolembiewski.nl/public_files/uploads/dXuJ7lqc6WPKh4ebgfVOw523vn
## Vision model
### Train for the folds
```bash
python train_vision.py -t data/train_metadata.yml -j 4 -e configurations/train_vision/resnet_18.yml -o output/vision_models
python train_vision.py -t data/train_metadata.yml -j 4 -e configurations/train_vision/<configuration> -o output/vision_models
```
For the experiments in the report, configuration `resnet_18.yml` and `simple.yml` were used.
### Inspect results
Run:
## Sequence model
### Train for the folds
```bash
python train_sequence.py -t data/train_metadata.yml -j 4 -e configurations/train_vision/resnet_18.yml -o output/sequence_models
```
For the experiments in the report, configuration `simple.yml` and `simple_lstm_400.yml` were used.
## Inspect results
Run:
```bash
mlflow ui
```
And view the results in a browser by clicking on the link which is printed. Each fold is a separate run, but they share a common `unique_id`, which can be found in the parameters.
The best models for each fold can be found in the output folder (`output/vision_models/<unique_id>`) if the command above was used.
The best models for each fold can be found in the output folder (`output/<type>_models/<unique_id>`) if the commands above were used, where `<type>` is either `vision` or `sequence`.
## Run an ensemble
### Put the relevant checkpoints in a directory
This can either be done manually, by copying the checkpoints from the directory.
Or the `scripts/extr_best_models.py` script can be used. WARNING: Best models are extracted based on the filename. If the filenames of the models in the output directory changed, the script won't work.
For example:
```bash
python scripts/extr_best_models.py -i <unique_id> -o models/<type>
```
# TODO
mistake analysis, mistake correlated?
performance
compare resnet, sequence
compare simple model, sequence
\ No newline at end of file
Create a `yaml` ensemble configuration file (the files used for the report can be found in `configurations/ensemble`).
Example:
```yml
vision_model_paths:
- "models/vision"
sequence_model_paths:
- "models/sequence"
ensemble_strategy: "avg_prob"
```
### Run the ensemble and generate results
```bash
python ensemble_analysis.py -t data/train_metadata.yml -e '<path to ensemble configuration>'
```
### Inspect the ensemble results
Run (if not done previously):
```bash
mlflow ui
```
And inscpect the results in browser (default http://localhost:5000), in the `ensemble_analyisis` experiment.
\ No newline at end of file
vision_model_paths: []
sequence_model_paths:
- "models/seq_cnn_final"
ensemble_strategy: "avg_prob"
\ No newline at end of file
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment