|
|
|
Next: Webmaster: Bandwidth vs. Users - 8.5mbit/s How Many Users?
|
| Author |
Message |
External
Since: Oct 31, 2007
Posts: 11
|
(Msg. 1) Posted: Wed Oct 31, 2007 3:53 pm
Post subject: 404 errors from Google's crawl engine
Archived from groups: alt>www>webmaster (more info?)
|
|
|
Hi there,
Our website is getting a lot of 404 errors from Google's crawl engine.
They all point to old pages (about 1-3 months old).
Funny thing is that we've taken all recommended steps to let Google
know that those pages are not there any more at the Remove URLs
section at Google WebMaster tools:
- Physically removed the files.
- Ensure requests for the page return an HTTP status code of either
404 or 410.
- Remove content from the Google index
- Disallow in robots.txt
Two of the Directory removals we requested appears as "Denied"
When we click on the "Learn more" link, we get:
"Why was my URL removal request denied?
Your request may have been denied because your content did not meet
the eligibility requirements for removal.
To block a page or image from your site, do one of the following, and
then submit your removal request:
* Make sure the content is no longer live on the web. Requests for
the page or image you want to remove must return an HTTP 404 (not
found) or 410 status code.
* Block the content using a meta noindex tag.
* Block the content using a robots.txt file."
But we have made sure the content is no longer live on the web, and
that the pages we want to remove do return an HTTP 404 (not found).
Additionally, we have locked the content using a robots.txt file.
And there's no point blocking the content using a meta noindex tag,
since no file in the site links to those URLs.
and now that we've done all that, we get even more errors กกก
Before we had 30 error pages and now we get 66 กกก
And our PageRank has dropped to NONE ?ก?
Anyone has an idea of what is going on? |
|
| Back to top |
|
 | |
External
Since: Jun 05, 2007
Posts: 69
|
(Msg. 2) Posted: Wed Oct 31, 2007 3:53 pm
Post subject: Re: 404 errors from Google's crawl engine [Login to view extended thread Info.]
Archived from groups: per prev. post (more info?)
|
|
|
|
|
| Back to top |
|
| |
External
Since: Oct 31, 2007
Posts: 11
|
(Msg. 3) Posted: Wed Oct 31, 2007 6:22 pm
Post subject: Re: 404 errors from Google's crawl engine [Login to view extended thread Info.]
Archived from groups: per prev. post (more info?)
|
|
|
On Oct 31, 6:15 pm, Andy Dingley <ding... DeleteThis @codesmiths.com> wrote:
> On 31 Oct, 15:53, 1001 Webs <1001w... DeleteThis @gmail.com> wrote:
>
> > Anyone has an idea of what is going on?
>
> People who know the relevant URLs might.
Are you talking about the same people that had the Washington Post,
Engadget and Forbes, for example, among the sites to have had their
Google rankings lowered recently? >> Stay informed about: 404 errors from Google's crawl engine |
|
| Back to top |
|
| |
External
Since: Oct 31, 2007
Posts: 11
|
(Msg. 4) Posted: Wed Oct 31, 2007 8:03 pm
Post subject: Re: 404 errors from Google's crawl engine [Login to view extended thread Info.]
Archived from groups: per prev. post (more info?)
|
|
|
On Oct 31, 6:15 pm, Andy Dingley <ding....TakeThisOut@codesmiths.com> wrote:
> On 31 Oct, 15:53, 1001 Webs <1001w....TakeThisOut@gmail.com> wrote:
>
> > Anyone has an idea of what is going on?
>
> People who know the relevant URLs might.
BTW,
I've just came across this:
"I don't understand why Google displays the bar anymore. It's purpose
was to get good public relations a long time ago. It's not needed now.
It only serves "text link brokers" and those who sell pagerank text
links. In other words, it's totally useless. It actually always has
been totally useless as a "rank" indicator." >> Stay informed about: 404 errors from Google's crawl engine |
|
| Back to top |
|
| |
External
Since: Jun 05, 2007
Posts: 69
|
(Msg. 5) Posted: Thu Nov 01, 2007 3:48 am
Post subject: Re: 404 errors from Google's crawl engine [Login to view extended thread Info.]
Archived from groups: per prev. post (more info?)
|
|
|
On 31 Oct, 18:22, 1001 Webs <1001w....DeleteThis@gmail.com> wrote:
> > > Anyone has an idea of what is going on?
> > People who know the relevant URLs might.
> Are you talking about the same people that
No, I'm talking about you, and about us.
Your OP said, "Google doesn't work as advertised, we've set our site
up correctly and Google still mis-logs it"
Now I can't comment on that unless I can see which pages you're
talking about (or your "dead" URLs to pages that are supposed to no
longer exist). With that information I can verify if indeed you have
"set your site up correctly" and it's thus Google's fault. Otherwise
I'm reduced to proof-by-authority, and I'd have to give Google the
benefit of the doubt. >> Stay informed about: 404 errors from Google's crawl engine |
|
| Back to top |
|
| |
External
Since: Oct 31, 2007
Posts: 11
|
(Msg. 6) Posted: Thu Nov 01, 2007 9:59 am
Post subject: Re: 404 errors from Google's crawl engine [Login to view extended thread Info.]
Archived from groups: per prev. post (more info?)
|
|
|
On Nov 1, 11:48 am, Andy Dingley <ding... RemoveThis @codesmiths.com> wrote:
> On 31 Oct, 18:22, 1001 Webs <1001w... RemoveThis @gmail.com> wrote:
>
> > > > Anyone has an idea of what is going on?
> > > People who know the relevant URLs might.
> > Are you talking about the same people that
>
> No, I'm talking about you, and about us.
>
> Your OP said, "Google doesn't work as advertised, we've set our site
> up correctly and Google still mis-logs it"
>
> Now I can't comment on that unless I can see which pages you're
> talking about (or your "dead" URLs to pages that are supposed to no
> longer exist). With that information I can verify if indeed you have
> "set your site up correctly" and it's thus Google's fault. Otherwise
> I'm reduced to proof-by-authority, and I'd have to give Google the
> benefit of the doubt.
And if I post links to those dead pages, Viper will hunt me back. No
thanks.
Never mind, I found a good explanation at the Google Webmaster Help
Group:
"Whenever you delete/move/rename pages in your site you should always
use http 301 permanent redirection to the nearest equivalent current
page. This is good for visitors and for search engine robots.
If you really don't care what happens to those "old" page URLs, then
just ignore the warning messages that you get.
You need to appreciate that when googlebot crawls a site, it adds to
its database a record of each URL. When the URL is requested on a
later crawl and your server replies 404 ("I don't have it") its not
very helpful because googlebot cannot tell if that is because it is
temporarily unavailable (eg by mistake) or because of permanent
change.
The last thing that you want is for google to immediately delete all
trace of a page, its indexing, its PR, its links etc etc just because
at the moment of crawling that page was not found, even though it was
the homepage or other important page of your ten-year-old site. "
I did 301 permanent redirection for some of the pages.
I suppose I have to do it for all of them. >> Stay informed about: 404 errors from Google's crawl engine |
|
| Back to top |
|
| |
External
Since: Jul 14, 2003
Posts: 1188
|
(Msg. 7) Posted: Thu Nov 01, 2007 9:59 am
Post subject: Re: 404 errors from Google's crawl engine [Login to view extended thread Info.]
Archived from groups: per prev. post (more info?)
|
|
|
1001 Webs wrote:
> On Nov 1, 11:48 am, Andy Dingley <ding... DeleteThis @codesmiths.com> wrote:
>> On 31 Oct, 18:22, 1001 Webs <1001w... DeleteThis @gmail.com> wrote:
>>
>>>>> Anyone has an idea of what is going on?
>>>> People who know the relevant URLs might.
>>> Are you talking about the same people that
>> No, I'm talking about you, and about us.
>>
>> Your OP said, "Google doesn't work as advertised, we've set our site
>> up correctly and Google still mis-logs it"
>>
>> Now I can't comment on that unless I can see which pages you're
>> talking about (or your "dead" URLs to pages that are supposed to no
>> longer exist). With that information I can verify if indeed you have
>> "set your site up correctly" and it's thus Google's fault. Otherwise
>> I'm reduced to proof-by-authority, and I'd have to give Google the
>> benefit of the doubt.
>
> And if I post links to those dead pages, Viper will hunt me back. No
> thanks.
>
> Never mind, I found a good explanation at the Google Webmaster Help
> Group:
>
> "Whenever you delete/move/rename pages in your site you should always
> use http 301 permanent redirection to the nearest equivalent current
> page. This is good for visitors and for search engine robots.
>
> If you really don't care what happens to those "old" page URLs, then
> just ignore the warning messages that you get.
>
> You need to appreciate that when googlebot crawls a site, it adds to
> its database a record of each URL. When the URL is requested on a
> later crawl and your server replies 404 ("I don't have it") its not
> very helpful because googlebot cannot tell if that is because it is
> temporarily unavailable (eg by mistake) or because of permanent
> change.
>
> The last thing that you want is for google to immediately delete all
> trace of a page, its indexing, its PR, its links etc etc just because
> at the moment of crawling that page was not found, even though it was
> the homepage or other important page of your ten-year-old site. "
>
> I did 301 permanent redirection for some of the pages.
> I suppose I have to do it for all of them.
>
>
No, Viper won't. Posting links to show a problem as part of a
conversation is different than posting an ad. And Viper understands
that, even though a couple of others here don't.
--
==================
Remove the "x" from my email address
Jerry Stuckle
JDS Computer Training Corp.
jstucklex DeleteThis @attglobal.net
================== >> Stay informed about: 404 errors from Google's crawl engine |
|
| Back to top |
|
| |
External
Since: Oct 31, 2007
Posts: 11
|
(Msg. 8) Posted: Thu Nov 01, 2007 12:35 pm
Post subject: Re: 404 errors from Google's crawl engine [Login to view extended thread Info.]
Archived from groups: per prev. post (more info?)
|
|
|
On Nov 1, 12:50 pm, Jerry Stuckle <jstuck....RemoveThis@attglobal.net> wrote:
> 1001 Webs wrote:
> > On Nov 1, 11:48 am, Andy Dingley <ding....RemoveThis@codesmiths.com> wrote:
> >> On 31 Oct, 18:22, 1001 Webs <1001w....RemoveThis@gmail.com> wrote:
>
> >>>>> Anyone has an idea of what is going on?
> >>>> People who know the relevant URLs might.
> >>> Are you talking about the same people that
> >> No, I'm talking about you, and about us.
>
> >> Your OP said, "Google doesn't work as advertised, we've set our site
> >> up correctly and Google still mis-logs it"
>
> >> Now I can't comment on that unless I can see which pages you're
> >> talking about (or your "dead" URLs to pages that are supposed to no
> >> longer exist). With that information I can verify if indeed you have
> >> "set your site up correctly" and it's thus Google's fault. Otherwise
> >> I'm reduced to proof-by-authority, and I'd have to give Google the
> >> benefit of the doubt.
>
> > And if I post links to those dead pages, Viper will hunt me back. No
> > thanks.
>
> > Never mind, I found a good explanation at the Google Webmaster Help
> > Group:
>
> > "Whenever you delete/move/rename pages in your site you should always
> > use http 301 permanent redirection to the nearest equivalent current
> > page. This is good for visitors and for search engine robots.
>
> > If you really don't care what happens to those "old" page URLs, then
> > just ignore the warning messages that you get.
>
> > You need to appreciate that when googlebot crawls a site, it adds to
> > its database a record of each URL. When the URL is requested on a
> > later crawl and your server replies 404 ("I don't have it") its not
> > very helpful because googlebot cannot tell if that is because it is
> > temporarily unavailable (eg by mistake) or because of permanent
> > change.
>
> > The last thing that you want is for google to immediately delete all
> > trace of a page, its indexing, its PR, its links etc etc just because
> > at the moment of crawling that page was not found, even though it was
> > the homepage or other important page of your ten-year-old site. "
>
> > I did 301 permanent redirection for some of the pages.
> > I suppose I have to do it for all of them.
>
> No, Viper won't. Posting links to show a problem as part of a
> conversation is different than posting an ad. And Viper understands
> that, even though a couple of others here don't.
Well, here you can see some then:
xxx.xxxxwebs.net/cms/drupal_en.htm 404 (Not found) [?] Oct 25,
2007
xxx.xxxxwebs.net/cms/joomla_en.htm 404 (Not found) [?] Oct 18,
2007
xxx.xxxxwebs.net/cms/mambo_en.htm 404 (Not found) [?] Oct 27, 2007
(Replace xxx.xxxx with http://www,1001, I don't want to add any more
external links to dead pages)
They correspond to the old structure of the site.
Now I've created a directory for every language, so those pages are
at:
http://www.1001webs.net/cms/en/drupal_en.htm
http://www.1001webs.net/cms/en/joomla_en.htm
http://www.1001webs.net/cms/en/mambo_en.htm
I am going to do 301 permanent redirection for all of them (66), since
it seems the right procedure.
I wonder whether you can redirect a whole directory or the pages have
to be individually redirected?
Thanks a lot. >> Stay informed about: 404 errors from Google's crawl engine |
|
| Back to top |
|
| |
External
Since: Jul 14, 2003
Posts: 1188
|
(Msg. 9) Posted: Thu Nov 01, 2007 12:35 pm
Post subject: Re: 404 errors from Google's crawl engine [Login to view extended thread Info.]
Archived from groups: per prev. post (more info?)
|
|
|
1001 Webs wrote:
> On Nov 1, 12:50 pm, Jerry Stuckle <jstuck... RemoveThis @attglobal.net> wrote:
>> 1001 Webs wrote:
>>> On Nov 1, 11:48 am, Andy Dingley <ding... RemoveThis @codesmiths.com> wrote:
>>>> On 31 Oct, 18:22, 1001 Webs <1001w... RemoveThis @gmail.com> wrote:
>>>>>>> Anyone has an idea of what is going on?
>>>>>> People who know the relevant URLs might.
>>>>> Are you talking about the same people that
>>>> No, I'm talking about you, and about us.
>>>> Your OP said, "Google doesn't work as advertised, we've set our site
>>>> up correctly and Google still mis-logs it"
>>>> Now I can't comment on that unless I can see which pages you're
>>>> talking about (or your "dead" URLs to pages that are supposed to no
>>>> longer exist). With that information I can verify if indeed you have
>>>> "set your site up correctly" and it's thus Google's fault. Otherwise
>>>> I'm reduced to proof-by-authority, and I'd have to give Google the
>>>> benefit of the doubt.
>>> And if I post links to those dead pages, Viper will hunt me back. No
>>> thanks.
>>> Never mind, I found a good explanation at the Google Webmaster Help
>>> Group:
>>> "Whenever you delete/move/rename pages in your site you should always
>>> use http 301 permanent redirection to the nearest equivalent current
>>> page. This is good for visitors and for search engine robots.
>>> If you really don't care what happens to those "old" page URLs, then
>>> just ignore the warning messages that you get.
>>> You need to appreciate that when googlebot crawls a site, it adds to
>>> its database a record of each URL. When the URL is requested on a
>>> later crawl and your server replies 404 ("I don't have it") its not
>>> very helpful because googlebot cannot tell if that is because it is
>>> temporarily unavailable (eg by mistake) or because of permanent
>>> change.
>>> The last thing that you want is for google to immediately delete all
>>> trace of a page, its indexing, its PR, its links etc etc just because
>>> at the moment of crawling that page was not found, even though it was
>>> the homepage or other important page of your ten-year-old site. "
>>> I did 301 permanent redirection for some of the pages.
>>> I suppose I have to do it for all of them.
>> No, Viper won't. Posting links to show a problem as part of a
>> conversation is different than posting an ad. And Viper understands
>> that, even though a couple of others here don't.
>
> Well, here you can see some then:
> xxx.xxxxwebs.net/cms/drupal_en.htm 404 (Not found) [?] Oct 25,
> 2007
> xxx.xxxxwebs.net/cms/joomla_en.htm 404 (Not found) [?] Oct 18,
> 2007
> xxx.xxxxwebs.net/cms/mambo_en.htm 404 (Not found) [?] Oct 27, 2007
> (Replace xxx.xxxx with http://www,1001, I don't want to add any more
> external links to dead pages)
>
> They correspond to the old structure of the site.
> Now I've created a directory for every language, so those pages are
> at:
>
> http://www.1001webs.net/cms/en/drupal_en.htm
> http://www.1001webs.net/cms/en/joomla_en.htm
> http://www.1001webs.net/cms/en/mambo_en.htm
>
> I am going to do 301 permanent redirection for all of them (66), since
> it seems the right procedure.
> I wonder whether you can redirect a whole directory or the pages have
> to be individually redirected?
>
> Thanks a lot.
>
>
Yep, that's the way to go.
You should be able to redirect everything in a directory. You might try
alt.apache.configuration - they've got some real experts over there 
--
==================
Remove the "x" from my email address
Jerry Stuckle
JDS Computer Training Corp.
jstucklex RemoveThis @attglobal.net
================== >> Stay informed about: 404 errors from Google's crawl engine |
|
| Back to top |
|
| |
External
Since: Oct 31, 2007
Posts: 11
|
(Msg. 10) Posted: Thu Nov 01, 2007 12:42 pm
Post subject: Re: 404 errors from Google's crawl engine [Login to view extended thread Info.]
Archived from groups: per prev. post (more info?)
|
|
|
On Nov 1, 12:50 pm, Jerry Stuckle <jstuck... DeleteThis @attglobal.net> wrote:
> 1001 Webs wrote:
> > On Nov 1, 11:48 am, Andy Dingley <ding... DeleteThis @codesmiths.com> wrote:
> >> On 31 Oct, 18:22, 1001 Webs <1001w... DeleteThis @gmail.com> wrote:
>
> >>>>> Anyone has an idea of what is going on?
> >>>> People who know the relevant URLs might.
> >>> Are you talking about the same people that
> >> No, I'm talking about you, and about us.
>
> >> Your OP said, "Google doesn't work as advertised, we've set our site
> >> up correctly and Google still mis-logs it"
>
> >> Now I can't comment on that unless I can see which pages you're
> >> talking about (or your "dead" URLs to pages that are supposed to no
> >> longer exist). With that information I can verify if indeed you have
> >> "set your site up correctly" and it's thus Google's fault. Otherwise
> >> I'm reduced to proof-by-authority, and I'd have to give Google the
> >> benefit of the doubt.
>
> > And if I post links to those dead pages, Viper will hunt me back. No
> > thanks.
>
> > Never mind, I found a good explanation at the Google Webmaster Help
> > Group:
>
> > "Whenever you delete/move/rename pages in your site you should always
> > use http 301 permanent redirection to the nearest equivalent current
> > page. This is good for visitors and for search engine robots.
>
> > If you really don't care what happens to those "old" page URLs, then
> > just ignore the warning messages that you get.
>
> > You need to appreciate that when googlebot crawls a site, it adds to
> > its database a record of each URL. When the URL is requested on a
> > later crawl and your server replies 404 ("I don't have it") its not
> > very helpful because googlebot cannot tell if that is because it is
> > temporarily unavailable (eg by mistake) or because of permanent
> > change.
>
> > The last thing that you want is for google to immediately delete all
> > trace of a page, its indexing, its PR, its links etc etc just because
> > at the moment of crawling that page was not found, even though it was
> > the homepage or other important page of your ten-year-old site. "
>
> > I did 301 permanent redirection for some of the pages.
> > I suppose I have to do it for all of them.
>
> No, Viper won't. Posting links to show a problem as part of a
> conversation is different than posting an ad. And Viper understands
> that, even though a couple of others here don't.
Well, here you can see some then:
xxx.xxxxwebs.net/cms/drupal_en.htm 404 (Not found) [?] Oct
25,
2007
xxx.xxxxwebs.net/cms/joomla_en.htm 404 (Not found) [?] Oct
18,
2007
xxx.xxxxwebs.net/cms/mambo_en.htm 404 (Not found) [?] Oct
27, 2007
(Replace xxx.xxxx with http://www,1001, I don't want to add any more
external links to dead pages)
They correspond to the old structure of the site.
Now I've created a directory for every language, so those pages are
at:
http://www.1001webs.net/en/cms/drupal_en.htm
http://www.1001webs.net/en/cms/joomla_en.htm
http://www.1001webs.net/en/cms/mambo_en.htm
I am going to do 301 permanent redirection for all of them (66), since
it seems the right procedure.
I wonder whether you can redirect a whole directory or the pages have
to be individually redirected?
And since we are at it.
How do you create customized 404 pages.
I did a few through cPanel, saved as 404.shtml and did appear for a
while, but now for some reason unknown to me, they are overridden by
system default 404 pages.
Do I have to save them as 404.html ?
Where? At root?
Thanks a lot. >> Stay informed about: 404 errors from Google's crawl engine |
|
| Back to top |
|
| |
External
Since: Jul 14, 2003
Posts: 1188
|
(Msg. 11) Posted: Thu Nov 01, 2007 12:42 pm
Post subject: Re: 404 errors from Google's crawl engine [Login to view extended thread Info.]
Archived from groups: per prev. post (more info?)
|
|
|
1001 Webs wrote:
> On Nov 1, 12:50 pm, Jerry Stuckle <jstuck... DeleteThis @attglobal.net> wrote:
>> 1001 Webs wrote:
>>> On Nov 1, 11:48 am, Andy Dingley <ding... DeleteThis @codesmiths.com> wrote:
>>>> On 31 Oct, 18:22, 1001 Webs <1001w... DeleteThis @gmail.com> wrote:
>>>>>>> Anyone has an idea of what is going on?
>>>>>> People who know the relevant URLs might.
>>>>> Are you talking about the same people that
>>>> No, I'm talking about you, and about us.
>>>> Your OP said, "Google doesn't work as advertised, we've set our site
>>>> up correctly and Google still mis-logs it"
>>>> Now I can't comment on that unless I can see which pages you're
>>>> talking about (or your "dead" URLs to pages that are supposed to no
>>>> longer exist). With that information I can verify if indeed you have
>>>> "set your site up correctly" and it's thus Google's fault. Otherwise
>>>> I'm reduced to proof-by-authority, and I'd have to give Google the
>>>> benefit of the doubt.
>>> And if I post links to those dead pages, Viper will hunt me back. No
>>> thanks.
>>> Never mind, I found a good explanation at the Google Webmaster Help
>>> Group:
>>> "Whenever you delete/move/rename pages in your site you should always
>>> use http 301 permanent redirection to the nearest equivalent current
>>> page. This is good for visitors and for search engine robots.
>>> If you really don't care what happens to those "old" page URLs, then
>>> just ignore the warning messages that you get.
>>> You need to appreciate that when googlebot crawls a site, it adds to
>>> its database a record of each URL. When the URL is requested on a
>>> later crawl and your server replies 404 ("I don't have it") its not
>>> very helpful because googlebot cannot tell if that is because it is
>>> temporarily unavailable (eg by mistake) or because of permanent
>>> change.
>>> The last thing that you want is for google to immediately delete all
>>> trace of a page, its indexing, its PR, its links etc etc just because
>>> at the moment of crawling that page was not found, even though it was
>>> the homepage or other important page of your ten-year-old site. "
>>> I did 301 permanent redirection for some of the pages.
>>> I suppose I have to do it for all of them.
>> No, Viper won't. Posting links to show a problem as part of a
>> conversation is different than posting an ad. And Viper understands
>> that, even though a couple of others here don't.
> Well, here you can see some then:
> xxx.xxxxwebs.net/cms/drupal_en.htm 404 (Not found) [?] Oct
> 25,
> 2007
> xxx.xxxxwebs.net/cms/joomla_en.htm 404 (Not found) [?] Oct
> 18,
> 2007
> xxx.xxxxwebs.net/cms/mambo_en.htm 404 (Not found) [?] Oct
> 27, 2007
> (Replace xxx.xxxx with http://www,1001, I don't want to add any more
> external links to dead pages)
>
> They correspond to the old structure of the site.
> Now I've created a directory for every language, so those pages are
> at:
>
> http://www.1001webs.net/en/cms/drupal_en.htm
> http://www.1001webs.net/en/cms/joomla_en.htm
> http://www.1001webs.net/en/cms/mambo_en.htm
>
> I am going to do 301 permanent redirection for all of them (66), since
> it seems the right procedure.
> I wonder whether you can redirect a whole directory or the pages have
> to be individually redirected?
>
> And since we are at it.
> How do you create customized 404 pages.
> I did a few through cPanel, saved as 404.shtml and did appear for a
> while, but now for some reason unknown to me, they are overridden by
> system default 404 pages.
> Do I have to save them as 404.html ?
> Where? At root?
>
> Thanks a lot.
>
>
No, it doesn't have to be 404.html - it can basically be any name. But
check your httpd.conf and .htaccess files - either one being incorrect
can cause this problem.
And when you look at .htaccess, remember you need to look not only the
one in the current directory, but all you can find in parent directories.
I had a problem one time when I changed a .htaccess in the root
directory and it screwed up a subdirectory .
--
==================
Remove the "x" from my email address
Jerry Stuckle
JDS Computer Training Corp.
jstucklex DeleteThis @attglobal.net
================== >> Stay informed about: 404 errors from Google's crawl engine |
|
| Back to top |
|
| |
External
Since: Oct 31, 2007
Posts: 11
|
(Msg. 12) Posted: Thu Nov 01, 2007 1:31 pm
Post subject: Re: 404 errors from Google's crawl engine [Login to view extended thread Info.]
Archived from groups: per prev. post (more info?)
|
|
|
On Nov 1, 1:57 pm, Jerry Stuckle <jstuck... DeleteThis @attglobal.net> wrote:
> 1001 Webs wrote:
> > On Nov 1, 12:50 pm, Jerry Stuckle <jstuck... DeleteThis @attglobal.net> wrote:
> >> 1001 Webs wrote:
> >>> On Nov 1, 11:48 am, Andy Dingley <ding... DeleteThis @codesmiths.com> wrote:
> >>>> On 31 Oct, 18:22, 1001 Webs <1001w... DeleteThis @gmail.com> wrote:
> >>>>>>> Anyone has an idea of what is going on?
> >>>>>> People who know the relevant URLs might.
> >>>>> Are you talking about the same people that
> >>>> No, I'm talking about you, and about us.
> >>>> Your OP said, "Google doesn't work as advertised, we've set our site
> >>>> up correctly and Google still mis-logs it"
> >>>> Now I can't comment on that unless I can see which pages you're
> >>>> talking about (or your "dead" URLs to pages that are supposed to no
> >>>> longer exist). With that information I can verify if indeed you have
> >>>> "set your site up correctly" and it's thus Google's fault. Otherwise
> >>>> I'm reduced to proof-by-authority, and I'd have to give Google the
> >>>> benefit of the doubt.
> >>> And if I post links to those dead pages, Viper will hunt me back. No
> >>> thanks.
> >>> Never mind, I found a good explanation at the Google Webmaster Help
> >>> Group:
> >>> "Whenever you delete/move/rename pages in your site you should always
> >>> use http 301 permanent redirection to the nearest equivalent current
> >>> page. This is good for visitors and for search engine robots.
> >>> If you really don't care what happens to those "old" page URLs, then
> >>> just ignore the warning messages that you get.
> >>> You need to appreciate that when googlebot crawls a site, it adds to
> >>> its database a record of each URL. When the URL is requested on a
> >>> later crawl and your server replies 404 ("I don't have it") its not
> >>> very helpful because googlebot cannot tell if that is because it is
> >>> temporarily unavailable (eg by mistake) or because of permanent
> >>> change.
> >>> The last thing that you want is for google to immediately delete all
> >>> trace of a page, its indexing, its PR, its links etc etc just because
> >>> at the moment of crawling that page was not found, even though it was
> >>> the homepage or other important page of your ten-year-old site. "
> >>> I did 301 permanent redirection for some of the pages.
> >>> I suppose I have to do it for all of them.
> >> No, Viper won't. Posting links to show a problem as part of a
> >> conversation is different than posting an ad. And Viper understands
> >> that, even though a couple of others here don't.
>
> > Well, here you can see some then:
> > xxx.xxxxwebs.net/cms/drupal_en.htm 404 (Not found) [?] Oct 25,
> > 2007
> > xxx.xxxxwebs.net/cms/joomla_en.htm 404 (Not found) [?] Oct 18,
> > 2007
> > xxx.xxxxwebs.net/cms/mambo_en.htm 404 (Not found) [?] Oct 27, 2007
> > (Replace xxx.xxxx withhttp://www,1001, I don't want to add any more
> > external links to dead pages)
>
> > They correspond to the old structure of the site.
> > Now I've created a directory for every language, so those pages are
> > at:
>
> >http://www.1001webs.net/cms/en/drupal_en.htm
> >http://www.1001webs.net/cms/en/joomla_en.htm
> >http://www.1001webs.net/cms/en/mambo_en.htm
>
> > I am going to do 301 permanent redirection for all of them (66), since
> > it seems the right procedure.
> > I wonder whether you can redirect a whole directory or the pages have
> > to be individually redirected?
>
> > Thanks a lot.
>
> Yep, that's the way to go.
>
> You should be able to redirect everything in a directory. You might try
> alt.apache.configuration - they've got some real experts over there
I googled for it but no one talks about 301 redirecting a whole
directory.
Maybe it's not possible.
I've posted at alt.apache.configuration, let's see ...
Thanks a lot Jerry. >> Stay informed about: 404 errors from Google's crawl engine |
|
| Back to top |
|
| |
|
FAQ
Search
Profile
Private Messages
Log in/Register/Password
