Add robots.txt to avoid unnecessary calls to api.php

3 jobs for master
Status Job ID Name Coverage
  Test
canceled #40598
backend

canceled #40599
frontend

canceled #40600
stats