How to avoid Google indexing specific app version on GAE?

Question

I have a website deployed on gae. This resource has purchased a domain, but of course you can go to the site and a standard domain types app_id.appspot.com plus this can also go there and version_id.app_id.appspot.com. More than that if you enter abrakadabra.app_id.appspot.com get on Default version.

So google robot somehow found my version 1 and 2. For SEO is not very helpful :(. Plus all robots began to come to the site more often (increased load) quotas are spent quickly. Maybe someone has already encountered this problem, tell me the solution.

Answer 1

The best solution is to create filter on url /robots.txt and send for versions hosts text like this:

User-agent: *
Disallow: /* 

Google crawler no more come to versioned hosts! :)

Answer 2

To answer your question:

You might be able to specify a preferred domain in Google webmaster tools. See: http://support.google.com/webmasters/bin/answer.py?hl=en&safe=on&answer=44231

Also, perhaps you could use canonical URLs to tell Google (and other search engines) which version to index. See: http://support.google.com/webmasters/bin/answer.py?hl=en&answer=139394&ctx=cb&src=cb&cbid=gh96oax614pa&cbrank=0

(Note that there was a similar question on StackOverflow: appspot.com url shows up in google search results instead of custom domain name )

Answer 3

My solution for now is:

public class VersionFilter implements Filter {

...

    @Override
    public void doFilter(ServletRequest request, ServletResponse servletResponse, FilterChain filterChain) throws IOException, ServletException {
        String serverName = request.getServerName();

        if (serverName.contains("appspot.com") && !UserBean.isAdmin()) {
            HttpServletResponse httpResponse = (HttpServletResponse) servletResponse;
            if (request.getParameter("login") != null) {
                UserService userService = UserServiceFactory.getUserService();
                httpResponse.sendRedirect(userService.createLoginURL("/"));
            } else if (request.getParameter("logout") != null) {
                UserService userService = UserServiceFactory.getUserService();
                httpResponse.sendRedirect(userService.createLogoutURL("/"));
            } else {
                httpResponse.sendError(403);
            }
        }
        filterChain.doFilter(request, servletResponse);
    }

... }