Twitter's robots.txt question:
Twitter's robots.txt, It shows everything is disallowed, but surprisingly search engines are crawling and indexing everybody's profiles pages, Why?
Re: Twitter's robots.txt question:
i guess the robots.txt file isnt setup corectly?
|All times are GMT -5. The time now is 07:25 PM.|
Powered by vBulletin® Version 3.8.8 Beta 4
Copyright ©2000 - 2020, vBulletin Solutions, Inc.
Search Engine Friendly URLs by vBSEO 3.6.0