searches and protected subdirectories

So is there anything else I can do? Is this just a bad idea to begin
with?

Thanks,

Peter
Tips wrote:
> dp wrote:
>
> > If you don't have a publicly exposed link to the site, there's no way the
> > googlebot could find it.
>
> That is not entirely accurate.
>
> If you are working in your hidden directory and then navigate to another
> page, your browser may send a referrer string to another site. That site
> might be using a stat counter that publishes stats online (like some free
> trackers). That creates a link to your hidden directory.
>
> I've seen it happen before...
>
> Password protection for the entire directory will prevent search engine
> spiders from accessing it though.
 >> Stay informed about: searches and protected subdirectories 

If I don't click on links to other sites from within my protected
directory, will I be ok? I don't use any stattrackers that are
published online, just monstercontrols, if you have heard of that.

Thanks again,

Peter
Tips wrote:
> dp wrote:
>
> > If you don't have a publicly exposed link to the site, there's no way the
> > googlebot could find it.
>
> That is not entirely accurate.
>
> If you are working in your hidden directory and then navigate to another
> page, your browser may send a referrer string to another site. That site
> might be using a stat counter that publishes stats online (like some free
> trackers). That creates a link to your hidden directory.
>
> I've seen it happen before...
>
> Password protection for the entire directory will prevent search engine
> spiders from accessing it though.
 >> Stay informed about: searches and protected subdirectories 

peter wrote:
> I am very new to managing websites. I took over as webmaster at work a
> few months ago. I recently created a protected subdirectory and copied
> the entire website into it. I wanted to have a place to test new ideas
> without making the changes active on the real website. Does this cause
> any problems with search engines? Would a googlebot ignore these since
> they are protected? Are there any other reasons I should not have
> created this subdirectory?


It's not a bad idea at all, you want to work on an almost live site and
not provide access untill it works like you want it, with no password
the bots shouldn't be able to index the pages. If your PR is lower than
PR4, it almost wouldn't matter what the bots saw or didn't see.

What I do instead however, (usually) is I have a development system
with roughly the same server environement and I can work on sites from
the comfort of my desktop and don't need to upload the files to see if
they work, they either work or they don't. I then move the files to a
test directory, make sure they work there and move them to the live
area.

I use HTML kit which has a cool ftp client built in, which allows you
to basiacally edit the files in place, so I ftp to the site, open a
file, modify it and click save and cross my fingers. I much prefer
working on the desktop, but getting the same config, apache, mysql, ssl
working on a windows system isn't fun. I have it setup on XPpro and
Fedora 6, but prefer to develop from XPpro... I'de like to think I'm
getting smarter though as I am spending more time on linux lately.

Be good, have a great day
 >> Stay informed about: searches and protected subdirectories 

William Tasso wrote:
> Fleeing from the madness of the http://groups.google.com jungle
> peter <plaz987.TakeThisOut@yahoo.com> stumbled into news:alt.www.webmaster
> and said:
>
> > I am very new to managing websites. I took over as webmaster at work a
> > few months ago. I recently created a protected subdirectory and copied
> > the entire website into it. I wanted to have a place to test new ideas
> > without making the changes active on the real website. Does this cause
> > any problems with search engines? Would a googlebot ignore these since
> > they are protected?
>
> Aside from the other responses in this thread, the only other comment I
> have is make sure your password protection is set for everything,
> especially if you have the google toolbar (or similar) in your browser.

What do you mean by "set for everything"?

--
John Ross
 >> Stay informed about: searches and protected subdirectories 

peter wrote:

> If I don't click on links to other sites from within my protected
> directory, will I be ok? I don't use any stattrackers that are
> published online, just monstercontrols, if you have heard of that.


For privacy reasons, it is best to turn referrers off completely. You can
do that by using Firefox with the Web Developer Toolbar. The option to
turn off referrers is on the "Disable" menu.

The stat trackers would be located on other people's sites. Their stat
trackers can catch the referrering pages and publish links to them.

If your hidden directory is password protected, search engine spiders can't
access it.
 >> Stay informed about: searches and protected subdirectories 

peter wrote:

> So is there anything else I can do? Is this just a bad idea to begin
> with?

As long as it's password protected, it should be fine. You can reinforce it
to some extent by adding a noindex,nofollow meta tag on the pages in the
hidden directory. Just be sure to remove that meta tag before making those
pages live -- otherwise they will never get indexed.
 >> Stay informed about: searches and protected subdirectories 

peter wrote:
> I am very new to managing websites. I took over as webmaster at work a
> few months ago. I recently created a protected subdirectory and copied
> the entire website into it. I wanted to have a place to test new ideas
> without making the changes active on the real website. Does this cause
> any problems with search engines? Would a googlebot ignore these since
> they are protected? Are there any other reasons I should not have
> created this subdirectory?
>
> Thanks,
>
> Peter

If you don't have a publicly exposed link to the site, there's no way the
googlebot could find it.
If you do have a publicly exposed link to it, the googlebot doesn't know
your login and password.

--
dp
 >> Stay informed about: searches and protected subdirectories 

dp wrote:

> If you don't have a publicly exposed link to the site, there's no way the
> googlebot could find it.

That is not entirely accurate.

If you are working in your hidden directory and then navigate to another
page, your browser may send a referrer string to another site. That site
might be using a stat counter that publishes stats online (like some free
trackers). That creates a link to your hidden directory.

I've seen it happen before...

Password protection for the entire directory will prevent search engine
spiders from accessing it though.
 >> Stay informed about: searches and protected subdirectories 

William Tasso wrote:
> Fleeing from the madness of the http://groups.google.com jungle
> John Ross <pdr3535 RemoveThis @yahoo.com> stumbled into news:alt.www.webmaster
> and said:
>
> > William Tasso wrote:
> >> ...
> >> Aside from the other responses in this thread, the only other comment I
> >> have is make sure your password protection is set for everything,
> >> especially if you have the google toolbar (or similar) in your browser.
> >
> > What do you mean by "set for everything"?
>
> Well - I have no idea what technology the o/p is using to 'protect' access
> to the sub-directory. I simply mean ensure that the 'protection' applies
> to /all/ content, which usually means it is protected by the o/s or the
> web server rather than script.

OK, I was assuming htaccess, and was worried I was not making the file
correctly to "set for everything."

Good catch for not assuming I would never have thought he would use
a script, but maybe he is and needs to change that!

--
John Ross
 >> Stay informed about: searches and protected subdirectories 

Thanks to all for your replies. I actually used monstercontrols which
has a feature to protect subdirectories, but I think it just creates a
htaccess file. I'll have to wait to go to work tomorrow to see if the
protection is set for everything, but I'm thinking that it is.

Thanks,

Peter
John Ross wrote:
> William Tasso wrote:
> > Fleeing from the madness of the http://groups.google.com jungle
> > John Ross <pdr3535.DeleteThis@yahoo.com> stumbled into news:alt.www.webmaster
> > and said:
> >
> > > William Tasso wrote:
> > >> ...
> > >> Aside from the other responses in this thread, the only other comment I
> > >> have is make sure your password protection is set for everything,
> > >> especially if you have the google toolbar (or similar) in your browser.
> > >
> > > What do you mean by "set for everything"?
> >
> > Well - I have no idea what technology the o/p is using to 'protect' access
> > to the sub-directory. I simply mean ensure that the 'protection' applies
> > to /all/ content, which usually means it is protected by the o/s or the
> > web server rather than script.
>
> OK, I was assuming htaccess, and was worried I was not making the file
> correctly to "set for everything."
>
> Good catch for not assuming I would never have thought he would use
> a script, but maybe he is and needs to change that!
>
> --
> John Ross
 >> Stay informed about: searches and protected subdirectories 

Fleeing from the madness of the http://groups.google.com jungle
peter <plaz987.TakeThisOut@yahoo.com> stumbled into news:alt.www.webmaster
and said:

> I am very new to managing websites. I took over as webmaster at work a
> few months ago. I recently created a protected subdirectory and copied
> the entire website into it. I wanted to have a place to test new ideas
> without making the changes active on the real website. Does this cause
> any problems with search engines? Would a googlebot ignore these since
> they are protected?

Aside from the other responses in this thread, the only other comment I
have is make sure your password protection is set for everything,
especially if you have the google toolbar (or similar) in your browser.

> Are there any other reasons I should not have
> created this subdirectory?

well - it can mess with relative links. I've taken to using subdomains
for development.

--
William Tasso

http://williamtasso.com/words/what-is-usenet.asp
 >> Stay informed about: searches and protected subdirectories 

Fleeing from the madness of the http://groups.google.com jungle
John Ross <pdr3535 DeleteThis @yahoo.com> stumbled into news:alt.www.webmaster
and said:

> William Tasso wrote:
>> ...
>> Aside from the other responses in this thread, the only other comment I
>> have is make sure your password protection is set for everything,
>> especially if you have the google toolbar (or similar) in your browser.
>
> What do you mean by "set for everything"?

Well - I have no idea what technology the o/p is using to 'protect' access
to the sub-directory. I simply mean ensure that the 'protection' applies
to /all/ content, which usually means it is protected by the o/s or the
web server rather than script.

--
William Tasso

http://williamtasso.com/words/what-is-usenet.asp
 >> Stay informed about: searches and protected subdirectories 

Tips wrote:

> For privacy reasons, it is best to turn referrers off completely. You
> can do that by using Firefox with the Web Developer Toolbar. The option to
> turn off referrers is on the "Disable" menu.

Or the PrefBar. http://prefbar.mozdev.org/

Choose Customize, and place the HTTP_REFERER on the bar.

--
-bts
-This message has been tested for polonium-210 and found to be safe
 >> Stay informed about: searches and protected subdirectories