Have You Bing Crawled?

A post on the Live blog goes into discussion of the MSNBot and crawl delay.

My professional opinion leans toward not setting a crawl rate, at all. SEs will gauge the ‘crawlability’ of your site and will crawl at a frequency that is best suited for that site.

However, I do find it rather useful to have dynamic pages, such as news, blogs, and the like, to a more frequent setting so that the SEs are aware of and can find new, fresh content.

Crawl delay and the Bing crawler, MSNBot

Should you set a crawl delay?

Many factors affect the crawling of a site, including (but not limited to):

* The total number of pages on a site (is the site small, large, or somewhere in-between?)

* The size of the content (PDFs and Microsoft Office files are typically much larger than regular HTML files)

* The freshness of the content (how often is content added/removed/changed?)

* The number of allowed concurrent connections (a function of the web server infrastructure)

* The bandwidth of the site (a function of the host’s service provider; the lower the bandwidth, the lower the server’s capacity to serve page requests)

* How highly does the site rank (content judged as not relevant won’t be crawled as often as highly relevant content)

The rate at which a site is crawled is an amalgam of all of those factors and more.

via Bing – Crawl delay and the Bing crawler, MSNBot – Webmaster Blog – Bing Community.

1% SEO: The Little Things Add Up

While working for a client of mine with a large site, I was constantly expected to perform ‘miracles’ of SEO. Time and time, again, I tried to explain that achieving top rank for his home page was going to be difficult as long as dead links, broken code, bad layout, and other seemingly insignificant factors went unaddressed.

Needless to say, although I was able to optimize for commanding ranking across Bing, Dogpile, Ask, and Google on 2nd tier keyphrases, the top tier phrases that the client really wanted stayed just beyond reach. And never once did I complain. I just reiterated that once you have 90% of the SEO done, there are still 1% factors that need to be addressed – and when you’re going after firecely targeted keyphrases, the luxury of ‘but the competition doesn’t seem to have those problems’ as a point of insight is no longer relevant, really.

When minor tech issues are the only problems you can find, you’re much better off just correcting them.

So, even though Bruce seems to be reiterating obvious points that should be considered and optimized, they are points that all too many feel won’t matter in the long run…. They might be points that don’t make *the* difference, but they do have their place.

Chat Man

Bruceclay.com – SEO is in the Details: Bruce Busts the Boondoggle – SEO Blog.

HTML Constructs

It used to be that a site needed every Meta tag in order to rank. Now that’s not the case. No one element will cause a site to rank, but it’s crucial to remember that SEO is effective as a whole.

Make your site all that it can be by using HTML as it was intended.

XML Sitemaps

an XML file often causes pages to be added to the index, which is an important objective of search engine optimization. Also, consider the use of an XML Sitemap when redirecting one site to another.

Keywords in URLs and Image Links

Keyword-rich URLs add value to a site in more than one way.

Page Copy

There is no magic [keyword] density, but there are winning page footprints. density alone is nearly useless, but it does have its place.

ICANN to Shake-Up Entire Web

ICANN is considering adding new TLDs. The new TLDs are purported to be ‘more marketable,’ although I have no idea what that could mean outside of SEO.

ICANN: This proposal allows applicants for new names to self-select their domain name so that choices are most appropriate for their customers or potentially the most marketable. It is expected that applicants will apply for targeted community strings such as (the existing) .travel for the travel industry and .cat for the Catalan community (as well as generic strings like .brandname or .yournamehere). There are already interested consortiums wanting to establish city-based top level domain, like .nyc (for New York City), .berlin and .paris.

Well, well. Do you think this will have a genuine, long-term SEO impact, or will spammers get to it first and abuse it to the point that G, Y!, and Live reduce any weight of the new feature to near-zero power? Call me negative, but the spammers are going to own this one awfully quick, IMHO.

clipped from: www.mediapost.com

How New TLDs Could Impact Your SEO

Posted August 14th, 2008 by Andrew Hazen

---

 
(ICANN) announced June 26 that it had given preliminary approval to a recommendation to introduce a whole range of new Internet domain names, which would pave the way for a seismic increase in online real estate.

 

---

 
It should be noted, however, that ICANN will not be selling the new TLDs for some time. The recommendation must first receive final approval early next year, followed by a limited period in which any established entity can submit an application for evaluation.

 

---

 
ICANN reports that there will be a six-figure charge for registering these new domains. That is chump change for major domains — and even those that want to be

For the Love of Robots.txt

Top 5 reasons for bots to avoid your site like the plague!

clipped from www.free-seo-news.com Reason 1: Your robots.txt file is damaged or it contains a typo Reason 2: Your URLs contain too many variables Reason 3: You use session IDs in your URLs Reason 4: Your web pages contain too much code Reason 5: Your website navigation causes problems

SEO Programmers: Heed the Words of Savvy Redirects

Stephan, again, illustrates tech terms vs. common sense. 

I love the “Oh noooes” most!

clipped from www.stephanspencer.com Return a 404 When You’re Supposed To, Or Get Dinged by Google A friend’s website’s Google rankings have tanked after their redesign. And I think I know why. Have a look at the status codes their web server returns when you request a garbage URL (a page that couldn’t possibly exist)…   #lwp-request -S www.randomcompany.com/dafsadf GET http://www.randomcompany.com/dafsadf –> 301 Moved Permanently GET http://www.randomcompany.com/search?q=dafsadf%20 –> 200 OK   A 301 followed by a 200. Oh noooes!   That REALLY should be a 404 status code instead.   Make sure that garbage URLs like www.yourcompany.com/aadsfadsfdafs return a 404 status code. Googlebot is known to request garbage URLs and to see if you respond with a 404 like you’re supposed to. If you don’t, your quality score goes down the tubes. Posted by Stephan Spencer on 08/22/2008 | Permalink   Comments (0)| Comments RSS | Filed under: Search Engines � � � � � �

Redirect Savvy

Stephan goes into great detail concerning redirects and how best they should be used. I can especially appreciate his recommendation regarding retiring pages such as Holiday items: move the items, but keep the page/URL active for the next season to maintain the page/URL’s weight. That is some sage advice, indeed. 

Stephan also touches on one of my personal favorites: mod_rewrite.

New to SEO? Get ready to become a master of mod_rewrite and 301 redirects. Otherwise, things might get tough for ya!

clipped from searchengineland.com Redirects: Good, Bad & Conditional Programmers and sysadmins who are not SEO-savvy will likely default to using a “temporary redirect,” also known as a “302 redirect.” Just gently inform them that what they really need to be using is a “permanent redirect,” or a “301 redirect.” What would be some of the “use cases” for a 301 redirect? if any of your URLs are going to change if you are changing domain names (tiredoldbrand.com to newbrand.com) migrating to a new content management system (a.k.a. CMS) You’ll want to do it even if you are “retiring” certain pages to an archive URL (e.g., the current year’s Holiday Gift Guide once the holiday buying season is over—although I’d make the case that you should maintain such a page at a date-free URL forever and let the link juice accumulate at that URL for use in future years’ editions and NOT redirect at all).