How to add a /robots.txt to DPM, LFC or DYNAFED for excluding HTTP bots
Configuring Apache so that it exposes a file names /robots.txt implies these two steps:
Add a line in the config file zlcgdm_dav.conf like the following:
# Robots.txt
Alias /robots.txt /usr/share/lcgdm-dav/robots.txt
Add a file named /usr/share/lcgdm-dav/robots.txt *readable by the user running the apache daemon with this content:
# Disallow all bots
User-agent: *
Disallow: /
The current trunk release 0.16 already has this. Pre-existing installations should do the manual thing anyway, as the config files cannot be overwritten.
Tags: NEWS