Forum FAQForum FAQSearchSearch MemberlistMemberlist Forum ignore listForum ignore list RegisterRegister ProfileProfile Log in to check your private messagesLog in to check your private messages Log inLog in
robots.txt

 
This forum is locked: you cannot post, reply to, or edit topics.   This topic is locked: you cannot edit posts or make replies.    mi3dot.org Forum Index -> SEO
View previous topic :: View next topic  
Author Message
Crazy Man



Joined: 02 Jun 2005
Posts: 68
Location: Croatia, Osijek

PostPosted: 04.05.2006 20:52    Post subject: robots.txt Add user to your forum ignore list Reply with quote

Ovako imam jedno pitanje vezano za robots.txt:

U page/ folderu se nalaze news.php, downloads.php...
Imam linkove ovoga oblika - index.php?page=news, index.php?page=downloads...
pomocu .htaccess sam napravio da mi ti linkovi budu kao - news/, downloads/
i sada u robots.txt moram napisati sta da trazilice tj crawleri ne gledaju i sada sta da stavim? index.php?page=nesto ili nesto/ ili mozda page/news.php. Ali opet nesto/ je folder pa opet sada neznam sta da stavim.. moze li me netko uputiti ?
Back to top
View user's profile Send private message Send e-mail Visit poster's website MSN Messenger
carr



Joined: 13 Nov 2003
Posts: 1397
Location: Karlovac

PostPosted: 04.05.2006 21:38    Post subject: Add user to your forum ignore list Reply with quote

bot ti se ponasa isto kao posjetitelj, dakle .htaccess i za njega rewritea urlove najnormalnije.

dakle stavi
news/
downloads/
pr0n/
itd.

_________________
Bolje biti malo lud, nego malo pametan.
Back to top
View user's profile Send private message Visit poster's website
lekke



Joined: 17 Jun 2004
Posts: 860
Location: 25th floor

PostPosted: 04.05.2006 21:44    Post subject: Add user to your forum ignore list Reply with quote

ne... u robots.txt stavi neki admin direktorij ako imas, direktorije u koje si trpao konfiguracijske fileove i sl... znas ono, ko kad radis sa dreamweaverom pa ti napravi Connections direktorij

_________________
You need more bass.
Back to top
View user's profile Send private message
Gaston



Joined: 14 Dec 2003
Posts: 619
Location: 10310

PostPosted: 04.05.2006 22:21    Post subject: Add user to your forum ignore list Reply with quote

hm, kad ste vec pokrenuli to pitanje, da li je ovo ok:
Code:
User-agent: *
Disallow: /cgi-bin/
Disallow: /admin/


znaci te direktorije sam zabranio

google ovdje kaze da, ako sam dobro shvatio, i u blokiranom direktoriju mogu odrediti file koji ce biti indexiran ?
Back to top
View user's profile Send private message
unLog!c



Joined: 27 May 2004
Posts: 185
Location: Osijek

PostPosted: 05.05.2006 07:22    Post subject: Add user to your forum ignore list Reply with quote

Ja pošto imam htaccess dozvolio sam pristup samo onim fileovima koji imaju neki sadržaj koji se nemože prije htaccessa includati, recimo slike, css, mp3, avi itd te admin panel i onda samo zabranim robotsima da mi prčkaju po tome..

Code:
User-agent: *
Disallow: /images/
Disallow: /cgi-bin/
Disallow: /bulletproof/

_________________
Why dont you go out and play hide and go fuck your self like all other NORMAL kids?
Back to top
View user's profile Send private message MSN Messenger
Crazy Man



Joined: 02 Jun 2005
Posts: 68
Location: Croatia, Osijek

PostPosted: 05.05.2006 07:30    Post subject: Add user to your forum ignore list Reply with quote

lekke wrote:
ne... u robots.txt stavi neki admin direktorij ako imas, direktorije u koje si trpao konfiguracijske fileove i sl... znas ono, ko kad radis sa dreamweaverom pa ti napravi Connections direktorij


Ne koristim drimviver Confused Znaci tražilice se prema robots.txt tj. prema tim fajlovima sto sam ispisao ponaša kao da su to folderi tj određeni fajlovi, znaci nisu vezani za .htaccess ? jer mi je samo to bitno.. znaci gledaju li .htaccess kod robota ili ne...
Back to top
View user's profile Send private message Send e-mail Visit poster's website MSN Messenger
Sulien



Joined: 04 Jan 2004
Posts: 2905
Location: Zagreb

PostPosted: 05.05.2006 13:35    Post subject: Add user to your forum ignore list Reply with quote

.htaccess pravila vrijede za SVE requestove, bili oni od tvoje mame, mene ili od robota

URL rewriting se može ponašati drugačije za neke robote ako mu ti to eksplicitno kažeš - što sigurno nisi

pomoglo bi ti u shvaćanju stvari kad bi znao kako serveri funkcioniraju
Back to top
View user's profile Send private message Send e-mail Visit poster's website
Crazy Man



Joined: 02 Jun 2005
Posts: 68
Location: Croatia, Osijek

PostPosted: 05.05.2006 13:37    Post subject: Add user to your forum ignore list Reply with quote

Puno ti hvala Wink

Quote:
pomoglo bi ti u shvaćanju stvari kad bi znao kako serveri funkcioniraju

Ali sta ces, neznam Confused
Back to top
View user's profile Send private message Send e-mail Visit poster's website MSN Messenger
Display posts from previous:   
This forum is locked: you cannot post, reply to, or edit topics.   This topic is locked: you cannot edit posts or make replies.    mi3dot.org Forum Index -> SEO All times are GMT + 1 Hour
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © 2001, 2005 phpBB Group