Add robots.txt#239
Conversation
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request adds a robots.txt layout file. The review comments point out two critical issues: first, using Disallow: / will completely block search engines from crawling the site; second, the sitemap URL concatenation will result in an invalid URL due to a missing trailing slash, and should instead use the absURL function.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| @@ -0,0 +1,4 @@ | |||
| User-agent: * | |||
| Disallow: / | |||
There was a problem hiding this comment.
Using Disallow: / will completely prevent search engines from crawling and indexing your website. If this is a public production site, you should allow crawling by leaving Disallow: empty.
Disallow:
| User-agent: * | ||
| Disallow: / | ||
|
|
||
| Sitemap: {{ .Site.BaseURL }}sitemap.xml |
There was a problem hiding this comment.
Since baseURL in config/_default/hugo.yaml is defined as https://clubalgoritmiaus.es (without a trailing slash), {{ .Site.BaseURL }}sitemap.xml will evaluate to https://clubalgoritmiaus.essitemap.xml. Using the absURL function is the idiomatic and safe way in Hugo to generate the absolute URL for the sitemap.
Sitemap: {{ "sitemap.xml" | absURL }}
There was a problem hiding this comment.
Pull request overview
Adds a custom robots.txt template to the Hugo site, overriding Hugo’s built-in robots generation (currently enabled via enableRobotsTXT: true).
Changes:
- Introduces
layouts/robots.txtto control crawler behavior. - Adds a sitemap directive pointing to
sitemap.xml.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| User-agent: * | ||
| Disallow: / |
| User-agent: * | ||
| Disallow: / | ||
|
|
||
| Sitemap: {{ .Site.BaseURL }}sitemap.xml |
|
Es gracioso porque las dos IAs coinciden en todo jajaja |
No description provided.