Sunday, June 29, 2025
2 changes · 18.0
Miscellaneous changes
Added the dirty hook `_get_additional_robots_rules_web` to allow modules to add extra rules to the robots.txt file whenever it's needed. Previously, it was blocking all the user-agents when we didn't have website installed. Part of https://github.com/odoo/enterprise/pull/85990 opw-4726371 Forward-Port-Of: odoo/odoo#216137 Forward-Port-Of: odoo/odoo#211734
Original PR description
Added the dirty hook `_get_additional_robots_rules_web` to allow modules to add extra rules to the robots.txt file whenever it's needed. Previously, it was blocking all the user-agents when we didn't have website installed. Part of https://github.com/odoo/enterprise/pull/85990 opw-4726371 Forward-Port-Of: odoo/odoo#216137 Forward-Port-Of: odoo/odoo#211734
Lately we have seen that when trying to upload to Instagram throught the media link, the request is blocked by the robots.txt file. This is because Facebook crawler is trying to access our image url and it is not being able to get the image in their side. This commit tries to add this user-agent to the robots.txt for both when we have and do not have website installed, without altering any of the existent templates. This will silently add it for website, so the user will be able to modify the
Original PR description
Lately we have seen that when trying to upload to Instagram throught the media link, the request is blocked by the robots.txt file. This is because Facebook crawler is trying to access our image url and it is not being able to get the image in their side. This commit tries to add this user-agent to the robots.txt for both when we have and do not have website installed, without altering any of the existent templates. This will silently add it for website, so the user will be able to modify the robots.txt file if needed, but won't be able ot remove it. Community side: https://github.com/odoo/odoo/pull/211734 opw-4726371 Forward-Port-Of: odoo/enterprise#88667 Forward-Port-Of: odoo/enterprise#85990