# Block Baidu # http://help.baidu.com/question?prod_en=master&class=Baiduspider User-agent: Baiduspider Disallow: / # Block Yandex # https://yandex.com/support/webmaster/controlling-robot/robots-txt.xml User-agent: Yandex Disallow: / # Redbot # ttps://redbot.org/ User-agent: RED Allow: / # Generic bot rules User-agent: * Disallow: /backend Disallow: /blog/*/share.html Disallow: /blog/*/print.html #REWMOD Brian Hannay 08/22/2017 Listing pages aren't being indexed because they rely on a resource in /idx/inc/php/ajax. #Disallow: /idx/inc/php/ajax/ Disallow: /idx/?refine Disallow: /*/idx/?refine Disallow: /idx/search/ Disallow: /*/idx/search/ Disallow: /idx/search.html Disallow: /*/idx/search.html Disallow: /idx/*/print.html Disallow: /idx/*/map.html Disallow: /idx/*/friend.html Disallow: /idx/*/inquire.html Disallow: /listing/*/inquire/ Disallow: /listing/*/friend/ Disallow: /listing/*/brochure/ Disallow: /listing/*/mortgage/