specify in your robots.txt which links google bot can follow
Stopping googlebot from following links in javascript
-
29-03-2022 - |
Question
While looking over my google bot errors one of them was a 403 error from a directory link I never put anywhere. After searching around in that page for a bit I found that google followed a link it found in some javascript code, this code in particular:
$.ajax({
type: 'post',
url: '/data/pages/' + r,
success: function(data) {
$('#mainbox').animate({"height": a}, 80, function() {
The google bot doesn't understand the + r part of it and keeps giving me errors about this. I know you can add a rel="nofollow" to anchors to tell googlebot not to follow them, but is there a way you can do this with script tags? (I searched around a lot and couldn't find a good answer)
Solution
OTHER TIPS
Matt Cutts just released a video on this very topic. In it he recommends using robots.txt for Javascript files. However, he states:
It turns out that as we are executing JavaScript, we do look at the attributes. So you can actually use Javascript and put, like, a nofollow attribute on individual URLs.
Its not clear to me what the syntax for putting a nofollow into javascript would be. I have asked about it here: https://webmasters.stackexchange.com/q/46659/14543