# www.robotstxt.org/ # Allow crawling of all content User-agent: * Sitemap: https://www.simbiat.eu/sitemap/index.xml Sitemap: https://www.simbiat.eu/sitemap/fftracker.xml # Forbid some folders Disallow: /cgi-bin/ Disallow: /data/ Disallow: /composer/ Disallow: /lib/ Disallow: /twig/ Disallow: /api/ Disallow: /uc/ user-agent: Google-Extended disallow: *