• googlebot requests get cut off

    From kob81@1:0/0 to All on Sat Sep 7 16:58:20 2013
    Hello,

    I noticed that ANY http request to my server, with a phrase "googlebot" in its useragent string, is getting blocked on the network level (it's not getting logged by Apache, so it's not a matter of .htaccess nor robots.txt misconfiguration).

    And that happens only to successful "200" connections ("404" not found and "301" redirects are working fine).

    When I run:
    netstat -nat |grep my.ip.here
    the connection is only in TIME_WAIT state and then gone.

    I flushed firewall and it didn't help much.

    Other domains on the server are working fine...

    That's really bizarre.

    Do you have an idea what's causing it?

    I think the server might have been compromised by a malicious hacker who wanted to cut off googlebot.. does anyone know how to fix it?

    --- MBSE BBS v1.0.0 (GNU/Linux-i386)
    * Origin: The Kofo System II BBS telnet://fido2.kofobb
  • From Joe Beanfish@110:110/2002 to All on Mon Sep 9 13:33:12 2013
    On Sat, 07 Sep 2013 09:58:20 -0700, kob81 wrote:

    Hello,

    I noticed that ANY http request to my server, with a phrase "googlebot" in
    its useragent string, is getting blocked on the network level (it's not getting logged by Apache, so it's not a matter of .htaccess nor robots.txt misconfiguration).

    And that happens only to successful "200" connections ("404" not found and
    "301" redirects are working fine).

    When I run:
    netstat -nat |grep my.ip.here
    the connection is only in TIME_WAIT state and then gone.

    I flushed firewall and it didn't help much.

    Other domains on the server are working fine...

    That's really bizarre.

    Do you have an idea what's causing it?

    I think the server might have been compromised by a malicious hacker who
    wanted to cut off googlebot.. does anyone know how to fix it?

    Are you on a home network? Perhaps your ISP is blocking it. I know time warner severely restricts access to servers on home connections.

    --- MBSE BBS v1.0.0 (GNU/Linux-i386)
    * Origin: A noiseless patient Spider (110:110/2002@linuxnet)