Robots.txt(explicitly.me)
explicitly.me
Robots.txt
http://explicitly.me/robots.txt
140 comments
Keep up the good work Google. I hope this serves as encouragement for them to continue to screw over those who want to turn searching for relevant information into a war for who can cheat the system the best.
Exactly my thoughts.
Exactly what I thought, SEO should not have ever become a "thing" if Google did its job right.
"SEO should not have ever become a "thing" if Google did its job right."
SEO will always be a "thing" as long as people are not always interested in providing relevant information. It will always be an asymmetric battle between people who want to prioritize their noise over useful signal.
SEO will always be a "thing" as long as people are not always interested in providing relevant information. It will always be an asymmetric battle between people who want to prioritize their noise over useful signal.
>SEO will always be a "thing" as long as people are not always interested in providing relevant information. //
SEO is [also] about bridging the gap between machine-based abstraction of the data in your site and human searchers. Most of the on-page/on-site optimisations are simply ensuring that the SE can abstract the right information so that users searching for the content you're providing can find it.
SEO will always be a thing as long as Googlebot fails to perfectly abstract the data content of a site and perfectly rank sites according to user needs.
SEO is [also] about bridging the gap between machine-based abstraction of the data in your site and human searchers. Most of the on-page/on-site optimisations are simply ensuring that the SE can abstract the right information so that users searching for the content you're providing can find it.
SEO will always be a thing as long as Googlebot fails to perfectly abstract the data content of a site and perfectly rank sites according to user needs.
The discrepancy here is probably that "SEO" is mostly used as a pejorative versus the intentions you describe.
Amen brother.
There is still a case for consultants who understand how to make websites genuinely more useful and relevant. Then it's win-win. Their motives might be their rankings, traffic and profit, but if it improves the web for Google users, then that isn't necessarily a bad thing.
But the tone of this "article" makes this guy sound like he wears a black hat.
But the tone of this "article" makes this guy sound like he wears a black hat.
Not to mention helping sites avoid crawl suicide like massively duplicating content across sites, implementing faceted nav and pagination wrong, putting core content in AJAX, generating thin pages, making Google guess which page is the canonical one, and product managers who think "hey let's republish this database of millions of items on our site with generate millions of pages."
I don't really. I do believe there is value in learning the dark side of this profession though. There aren't enough genuine SEOs that really understand both sides of the equation.
I used to agree with this, but lately I feel like any learnings are going to have a short shelf life. With the EMD updates and on-going refinements to Penguin, how long can any truly effective hack really be expected to work?
I'm glad Google is cleaning house b/c it became annoying trying to counsel people to not violate the Webmaster Guidelines when they could point to successful competitors or some backwater infomercial claiming they could shortcut their way by spamming.
The best strategy I've found is just to not shoot yourself in the foot in terms of site structure and IA, and invest in long-term value through link building with PR and content that naturally attracts links.
I'm glad Google is cleaning house b/c it became annoying trying to counsel people to not violate the Webmaster Guidelines when they could point to successful competitors or some backwater infomercial claiming they could shortcut their way by spamming.
The best strategy I've found is just to not shoot yourself in the foot in terms of site structure and IA, and invest in long-term value through link building with PR and content that naturally attracts links.
I wish some of the "real" black hatters could come on here and speak. These guys are the ones putting together massive strategies/campaigns to rank across a wide variety of thousands of key terms. And there not writing about it. The techniques many people are talking about now were done maybe 2-3 years before this whole mess. They are so far ahead of what the "industry" is focused on it's not even funny.
Some black hatters make money, but for every 1 who does, 50 don't. Spend some time on WarriorForum or SEOBook and you'll see a very low signal to noise ratio. Lots of people are just preying on people who want to get rich quick with now work by selling them hope in the form of "info products".
And the black hatters who make money could probably make just as much if they did something legit.
The worst part about black hat is eventually the wheels fall off and you have to start over. So you're playing a game that keeps you on a constant treadmill. It's just as bad as consulting as a way to make a living.
If you take the high road, you build real value you can exit on eventually.
And the black hatters who make money could probably make just as much if they did something legit.
The worst part about black hat is eventually the wheels fall off and you have to start over. So you're playing a game that keeps you on a constant treadmill. It's just as bad as consulting as a way to make a living.
If you take the high road, you build real value you can exit on eventually.
You know there is a reason they don't participate on these forums... It's a waste of their time! They are so busy working their strategy, making money on the web, that any time spent on a discussion like this, while no money is made, is very low on their priorities.
I doubt they're making a ton on the hustle, but I agree that they're probably not the type to waste time discussing points with other, likely more hostile crowds.
What makes you doubt that? If there was no money to be made, these wouldn't be spam, Viagra Wordpress hacks or Phishing attempts.
You have no idea...
You have no idea...
I've seen what some of them do, and it's not as impressive as you'd think. Most of it breaks and stops working inside of 24 months. Then it's on to the next scam. Not a lot different from fly-by-night snake oil salesmen.
Just the existence of spam doesnt prove it's a big oppotunity, just that it's better for some people than their opportunity cost.
Just the existence of spam doesnt prove it's a big oppotunity, just that it's better for some people than their opportunity cost.
Some make &10,000 a DAY.
Ever looked at the payday SERP in the UK? Google can't control it. I know at least 3-4 people who make £100,000 a year with just basic level spamming.
If I scaled this technique http://explicitly.me/serp-sniffing-a-long-tail-keyword-strat... , I could make the same amount of money without breaking a sweat by dumping Adsense on there, and shopping affiliate links.
The point is I don't choose to.
Ever looked at the payday SERP in the UK? Google can't control it. I know at least 3-4 people who make £100,000 a year with just basic level spamming.
If I scaled this technique http://explicitly.me/serp-sniffing-a-long-tail-keyword-strat... , I could make the same amount of money without breaking a sweat by dumping Adsense on there, and shopping affiliate links.
The point is I don't choose to.
Thanks for posting that. It's funny-- your strategy is exactly the way I do lead gen for my customers. I wrote a bunch of software to sniff serps (great expression, btw) and find long tail keywords that will drive leads for the client.
Then we produce articles (we actually hire real journalists) to address the search intent behind the keywords. Then we post it on the client's site, and charge them for the leads we drive via organic search from that content.
The funny thing is, our customers just view it as "content marketing," not SEO. Search data (search vol, competition, etc.) guides our content strategy, but I don't view this part as either good or bad. Or even really "SEO". Why would anyone invest in content nobody is searching for?!?
If we spun content or used Markov chains to generate gibberish or hired incompetent writers via TextBroker for $10 a story, sure that's spam.
But my calculation is that spending $500+ on an article written by a journalist with domain expertise is going to win long term. And since the lead revenue makes the ROI calculation >1, why not invest in quality?
And we definitely don't make Google guess what keyword we think our article would help.
Then we produce articles (we actually hire real journalists) to address the search intent behind the keywords. Then we post it on the client's site, and charge them for the leads we drive via organic search from that content.
The funny thing is, our customers just view it as "content marketing," not SEO. Search data (search vol, competition, etc.) guides our content strategy, but I don't view this part as either good or bad. Or even really "SEO". Why would anyone invest in content nobody is searching for?!?
If we spun content or used Markov chains to generate gibberish or hired incompetent writers via TextBroker for $10 a story, sure that's spam.
But my calculation is that spending $500+ on an article written by a journalist with domain expertise is going to win long term. And since the lead revenue makes the ROI calculation >1, why not invest in quality?
And we definitely don't make Google guess what keyword we think our article would help.
thats the route I would take as well. But, I would use SEMrush to sniff out the rankings for Demand Media and other content farms, and then create quality content to target those keywords, trying to displace their crap hat content.
Totally with you. I was enjoying the amusing tone of the piece but he lost my sympathy at:
"Before, SEO was the game of really skilled people, you know like professional poker players. You had to know the game, know when to raise, and when to bluff."
Then with the "Maybe I should find another job?" I thought this has got to be satire. Well played, sir.
The way the piece is framed, the writing style, surely a work of humour?
"Before, SEO was the game of really skilled people, you know like professional poker players. You had to know the game, know when to raise, and when to bluff."
Then with the "Maybe I should find another job?" I thought this has got to be satire. Well played, sir.
The way the piece is framed, the writing style, surely a work of humour?
I agree; that last comment gives it away. I've been doing enterprise-level SEO for several years now as a day job and it's not as if algo updates are so frequent and far reaching as to make it difficult to keep up with. Most major updates from Google and the other engines are predicated on two major themes:
-Rewarding relevant, informative, keyword-rich content focused on specific topics and punishing sites that construct thinly constructed (UGC or otherwise) pages, particularly those who do it in huge numbers.
-Rewarding sites who links from similarly relevant, authoritative sources gained organically and punishing those who acquire links through artificial means (either through direct purchases, purchasing domain portfolios with thousands of backlinks etc.)
There are nuances that are market-specific that require slightly different approaches and concentrations (e.g. health care) but those are the main things folks should concern themselves with. There's no black magic behind it unless you're doing something shady to begin with.
-Rewarding relevant, informative, keyword-rich content focused on specific topics and punishing sites that construct thinly constructed (UGC or otherwise) pages, particularly those who do it in huge numbers.
-Rewarding sites who links from similarly relevant, authoritative sources gained organically and punishing those who acquire links through artificial means (either through direct purchases, purchasing domain portfolios with thousands of backlinks etc.)
There are nuances that are market-specific that require slightly different approaches and concentrations (e.g. health care) but those are the main things folks should concern themselves with. There's no black magic behind it unless you're doing something shady to begin with.
It was meant as humour, for other SEOs in reality. Kind of an in joke that took the hard part of our job and made it funny. Frankly, as in any industry, you have to adapt. (I am the Author of that site)
From his twitter postings, I don't believe this is fake.
Ah, thanks for this and your comment elsewhere in the thread. That Twitter link, for those also intrigued: https://twitter.com/rishil.
Plus one. SEO is absolutely annoying. If a site is built correctly and there is interest in the service or topic the site provides, then the product will be found. Stuffing keywords into title and heading tags along with other grey area SEO techniques really devalues the internet. I see SEO along the same lines as buying domains that you think may have value and sitting on them. Games like this hinder progress and real utilization of the resources at our disposal.
You clearly have no idea what SEO really is.
Of course, optimize your content so that it's easy to find and index. That's building your site correctly. All I am saying is that after market SEO is a little gimmicky because of the patchwork approach taken by many SEO specialists. As a longtime developer that has had lots of interaction with clients, I know they often spend their time worrying about SEO instead of posting good content. If a consumer isn't satisfied with search rankings and they think their site is poorly built, then they are better off contracting someone that will build it correctly the first time. Now obviously a dynamically served website has some SEO issues and needs some special attention, but this shouldn't be an after thought or third party component.
"if a site is built correctly" - right. That's far from a given.
I can show you dozens of examples of well-funded "A list" too-cool-for-SEO developers who have no idea whatsoever what "built correctly" entails from the POV of a search engine.
The role of SEO is a) Product Management to set requirements to optimize crawlability, indexing, and retrieval, or b) QA that arrives after the fact and highlights all the ways the site fails to be usable for bots.
My experience is if you design things right first, it costs much less than if you don't. Plus you see the upside of your efforts quickly.
But maybe some people take the attitude that "giving me traffic is Google's job" and then when they don't get organic traffic, they have to buy it. Hopefully they also enjoy raising money and giving up a big chunk of their company, too.
I can show you dozens of examples of well-funded "A list" too-cool-for-SEO developers who have no idea whatsoever what "built correctly" entails from the POV of a search engine.
The role of SEO is a) Product Management to set requirements to optimize crawlability, indexing, and retrieval, or b) QA that arrives after the fact and highlights all the ways the site fails to be usable for bots.
My experience is if you design things right first, it costs much less than if you don't. Plus you see the upside of your efforts quickly.
But maybe some people take the attitude that "giving me traffic is Google's job" and then when they don't get organic traffic, they have to buy it. Hopefully they also enjoy raising money and giving up a big chunk of their company, too.
"My experience is if you design things right first, it costs much less than if you don't. Plus you see the upside of your efforts quickly."
That's what I said.
That's what I said.
As long as search results are determined by algorithms, there will be search engine optimization. Why? Because every algorithm can be optimized for something. And as long as search engine optimization exists, engaging in it will be a competitive advantage. Why would you NOT want your site to appear at the top of the search results for as many keywords as possible? Your job as a business owner is to bring as many customers to your business as you can. To suggest anything else is absurd.
I'm happy that so many HN readers are anti-SEO, because that's less competition for me.
I'm happy that so many HN readers are anti-SEO, because that's less competition for me.
Google's ultimate goal is making "SEO" and "quality content generation" one-and-the-same.
>Your job as a business owner is to bring as many customers to your business as you can. To suggest anything else is absurd.
Technically your job is to generate as much profit as possible. Volume of people in the door is but a single coefficient in that equation. It's a common failure point, yes, but suggesting that it is the only one is disingenuous.
>Your job as a business owner is to bring as many customers to your business as you can. To suggest anything else is absurd.
Technically your job is to generate as much profit as possible. Volume of people in the door is but a single coefficient in that equation. It's a common failure point, yes, but suggesting that it is the only one is disingenuous.
Just because something is good for my business, doesn't mean it's good for society. SEO (beyond actually making your site more valuable) is a classic prisoner's dilemma.
I think this is where my views differ from many on HN. As a business owner, I will do whatever is best for my business, without regards to what is best for society. My job is not to worry about what is best for society -- that job belongs to the government. The government sets the rules and we play the game. As long as I am not breaking any laws, I will do whatever it takes to bring my business to the top. I have to do this because if I don't, somebody else will. Riding a moral high horse will only lead to my business being stampeded by others who are willing to engage in tactics I am not.
This is not to say that I actively work against what's best for society. My point is that I will do what's best for my business, with regard to nothing else. I think that is what makes a good business owner. Another trait of a good business owner is the belief that his business provides the best value in its market. If you truly believe that, then by engaging in SEO to rank your business at the top of its market, you ARE doing what's best for society. If you suggest that engaging in SEO tactics to rank your business at the top is BAD for society, then you are inherently suggesting that your business does not provide the best value to its market.
This is not to say that I actively work against what's best for society. My point is that I will do what's best for my business, with regard to nothing else. I think that is what makes a good business owner. Another trait of a good business owner is the belief that his business provides the best value in its market. If you truly believe that, then by engaging in SEO to rank your business at the top of its market, you ARE doing what's best for society. If you suggest that engaging in SEO tactics to rank your business at the top is BAD for society, then you are inherently suggesting that your business does not provide the best value to its market.
I sympathize, but take notice that this is part of the classic one-two strategy that capitalists, as a class, have historically used to gain a disproportionate amount of power in certain historical periods. Say, the 19th century, or the 80s/90s. Not with malice, or even consciously, of course, but it's worth to watch for it, lest we fall again, and again...
First, claim the reasonable principle that law should guard morality, and business should attend only to business. It's in principle correct, so it gets accepted by everyone. So, if child labor were allowed, business owners would be in no moral fault for hiring kids to mine ore -- to pick a extreme example.
Second, claim that economic progress is good for everyone. Again, very reasonable. Then, organize as a political class (say, in nowadays Republican party) to get some legal concessions that were morally abhorrent in the past legal framework, in more or less degree, but would bring economic progress (for some definition of it). Say, legalize child labor. Voilá.
Now, it's obvious that capitalists should be, of course, allowed to organize politically. But, I argue that we live in a world disproportionately controlled by capital (specially the US), for various reasons. It's disingenuous, thus, to be a capitalist and have your mindset. You, as member of a class, have certain privileges that other classes in our society would consider unfair. Our current legal framework is loaded in your favor, after all.
First, claim the reasonable principle that law should guard morality, and business should attend only to business. It's in principle correct, so it gets accepted by everyone. So, if child labor were allowed, business owners would be in no moral fault for hiring kids to mine ore -- to pick a extreme example.
Second, claim that economic progress is good for everyone. Again, very reasonable. Then, organize as a political class (say, in nowadays Republican party) to get some legal concessions that were morally abhorrent in the past legal framework, in more or less degree, but would bring economic progress (for some definition of it). Say, legalize child labor. Voilá.
Now, it's obvious that capitalists should be, of course, allowed to organize politically. But, I argue that we live in a world disproportionately controlled by capital (specially the US), for various reasons. It's disingenuous, thus, to be a capitalist and have your mindset. You, as member of a class, have certain privileges that other classes in our society would consider unfair. Our current legal framework is loaded in your favor, after all.
The "if you don't, then somebody else will" argument only applies if you're in a commodity market: A market with a lot of players selling the same product, or a market where there are few barriers to entry keeping out people who sell an interchangeable product.
Much of the world inhabited by HN'ers is not like this. If you have a typical software or web product, then a competitor would have to spend substantial resources developing code that duplicates your functionality, and then they have to worry about the head-start the first mover has in branding and network effects.
Much of the world inhabited by HN'ers is not like this. If you have a typical software or web product, then a competitor would have to spend substantial resources developing code that duplicates your functionality, and then they have to worry about the head-start the first mover has in branding and network effects.
its not that people are anti-SEO, its what you so-called "seo experts" think SEO is. As far as I'm concerned, optimizing for search engines means using best practices in designing the website. That includes, rich content, proper title and description meta, link text and title that describes the destination properly, alt/title for image and proper use of headings. SEO and accessibility go hand in hand. The marketing aspect of SEO, is integrating social media and targeted advertisement. If what your content is worth it, link building should come naturally.
If you go beyond these practices and try all these scams to build fake reputation for your worthless website then you truly do not deserve to be on the internet. Trying to scam your way up the SERPs is an awful thing to do and it just ends up pushing better qualified website down, eventually, ruining the online experience for millions of people who now have to take more time out of their lives to find the relevant information they need.
SEO is a must for every website, but if you try to scam your way up the ladder, you deserve to be thrown off it.
Am i the only one who thinks that SEO is more a plague then it is a real profession? Google making it hard for SEO is exactly the right thing to do. Making it impossible for SEO should be the ultimate _goal_ of a good search engine.
My opinion of google improved reading this article. I equate the quality of Google's search as the inverse of the happiness of "seo gurus".
Sometimes, that statement is fairly accurate, then in other times when google boosts its own commercial properties to increase their ad revenue, it wouldn't be.
I think this is a common view, and I share it: I really don't like SEO. The thing is, SEO does provide real value, but it provides value because the system is broken: your website should reach the top of search results based on merit, not based on some arcane black magic.
I think it's somewhat akin to lobbyists: I don't blame them for making money off and taking advantage off the political system, and they certainly provide something of value to their customers, but I dislike the fact that they can exist at all. But that's not the lobbyist's fault, it's the system. In this respect, I applaud Google for keeping people on their toes and making SEO less relevant. That's their job after all.
I think it's somewhat akin to lobbyists: I don't blame them for making money off and taking advantage off the political system, and they certainly provide something of value to their customers, but I dislike the fact that they can exist at all. But that's not the lobbyist's fault, it's the system. In this respect, I applaud Google for keeping people on their toes and making SEO less relevant. That's their job after all.
In short, don't hate the player hate the game?
I understand where you're coming from, but that's crap. Just because there is always someone willing to do ethically questionable things that fall within the letter of the law, doesn't mean that those types of people shouldn't be treated with disdain by the rest of us.
I understand where you're coming from, but that's crap. Just because there is always someone willing to do ethically questionable things that fall within the letter of the law, doesn't mean that those types of people shouldn't be treated with disdain by the rest of us.
don't hate the player hate the game?
Why choose? I'm already somewhat of a misanthrope, but the fact that SEO exists just makes me hate everyone and everything.
Why choose? I'm already somewhat of a misanthrope, but the fact that SEO exists just makes me hate everyone and everything.
I disagree. Let me quote Cutts: "I’ve said it before, but SEO is in many ways about change. The best SEOs recognize, adapt, and even flourish when changes happen."
There's a saying that goes something like "A computer will only do what you tell it to do". That said, merit is important, but you have to show Google merit with code. That's by on page content and structure and appropriate tags, etc or by off-page, with links. That's it. There's no other way. There's no black magic. Cutts speaks to SEOs but he calls he channel GoogleWebmasterHelp . That's the clue to website owners, but many times the owner is not a webmaster. So they can hire an SEO that knows a bit about ranking websites, or they can hire a web programmer. Not all web programmers have ideas on what to do. They likely will want to make a new site for the customer that is not even necessarily any better than the existing, as far as Google cares.
That got a bit preachy but it's not as black and white as you make it out to be. There's a niche for SEOs that involves improving existing websites that a web programmer might not be suited for, might not want to do, or might be too expensive for.
There's a saying that goes something like "A computer will only do what you tell it to do". That said, merit is important, but you have to show Google merit with code. That's by on page content and structure and appropriate tags, etc or by off-page, with links. That's it. There's no other way. There's no black magic. Cutts speaks to SEOs but he calls he channel GoogleWebmasterHelp . That's the clue to website owners, but many times the owner is not a webmaster. So they can hire an SEO that knows a bit about ranking websites, or they can hire a web programmer. Not all web programmers have ideas on what to do. They likely will want to make a new site for the customer that is not even necessarily any better than the existing, as far as Google cares.
That got a bit preachy but it's not as black and white as you make it out to be. There's a niche for SEOs that involves improving existing websites that a web programmer might not be suited for, might not want to do, or might be too expensive for.
The story really didn't dissuade me from thinking about the profession of SEO as the sorts of people looking for easy money at the expense of others.
I'm sure MLM operators and spammers didn't do well in school, were hostile to the people around them and love flashy clothes, big checks, and fast cars as well.
I'm sure MLM operators and spammers didn't do well in school, were hostile to the people around them and love flashy clothes, big checks, and fast cars as well.
Depends what you mean.
If you mean people whose goal it is to help site owners learn how to structure their data intelligently for dissection by machines, people who make companies think very hard about their content, people who encourage high-quality crosslinks and meta tags, and people who generally help sites manage data, then no, SEOs are great. They help users and companies alike.
If you mean the dirtball black hats, then yes, they are a parasite that deserves no mercy and should be eliminated at all costs.
If you mean people whose goal it is to help site owners learn how to structure their data intelligently for dissection by machines, people who make companies think very hard about their content, people who encourage high-quality crosslinks and meta tags, and people who generally help sites manage data, then no, SEOs are great. They help users and companies alike.
If you mean the dirtball black hats, then yes, they are a parasite that deserves no mercy and should be eliminated at all costs.
http://en.wikipedia.org/wiki/If-by-whiskey seems relevant.
Yes, SEO has clearly a negative notion for me. To deceive the search engine and make it less useful for me is practically the job description of SEO, in my opinion.
For all the "no-evil" stuff, it's just plainly the work of a website programmer. Properly formatting your HTML, making it accessible and easily "readable" for machines is part of your job when you create a website. Same for creating a robots.txt. If you say many people don't know how to do that, then those people are just bad in creating websites and they need help of a professional website programmer/creator. No need for a term like SEO here.
For all the "no-evil" stuff, it's just plainly the work of a website programmer. Properly formatting your HTML, making it accessible and easily "readable" for machines is part of your job when you create a website. Same for creating a robots.txt. If you say many people don't know how to do that, then those people are just bad in creating websites and they need help of a professional website programmer/creator. No need for a term like SEO here.
I can see what you are saying but I see SEO a bit differently than being simply good web programming. It's about content organization as well as information flow. Some of the time that comes naturally to a webmaster, but there are so many content rich sites with poor web programming and poor organization. SEO is a niche side of web programming that fixes that specific problem. The skills of an SEO sometimes don't even cross the mind of a "good" web programmer because of the unknown unknowns factor. They just haven't thought about even attempting to rank a website on Google. They aren't investigating the competition to see where they fit, and how to be better.
If you care about the underlying algorithms, I'd think it's pretty clear which kind of SEO we're talking about.
I think it would be reasonable to understand the underlying algorithms in order to optimise what white hat SEO to spend effort on. But I accept that this is probably a rare breed.
This is true. But there really are hundreds of thousands (probably millions and millions) of sites that are structured horribly where you can simply rely on knowing how the heck to structure a page correctly.
For instance...
I recently moved a corporate site for a major national law firm from Joomla to WordPress. They had 200+ pages with multiple H1 tags, nearly 300 pages missing descriptions, dozens and dozens of soft 404s, hundreds of pages with bad titles or URLs that are too long, dozens of pages with duplicate content because of bad sitemaps, and every single page was over 100 links because of the dummy that made the original nav toolbar put every subpage in there.
White hats can work for the next decade simply trying to save companies from themselves, their stupid CMS systems, and former well meaning SEO efforts that are now banned or deprecated.
Short term effect to fixing these issues? A nearly 20% increase in traffic since the switchover and slightly raised ranking for related firm content on OTHER sites.
For instance...
I recently moved a corporate site for a major national law firm from Joomla to WordPress. They had 200+ pages with multiple H1 tags, nearly 300 pages missing descriptions, dozens and dozens of soft 404s, hundreds of pages with bad titles or URLs that are too long, dozens of pages with duplicate content because of bad sitemaps, and every single page was over 100 links because of the dummy that made the original nav toolbar put every subpage in there.
White hats can work for the next decade simply trying to save companies from themselves, their stupid CMS systems, and former well meaning SEO efforts that are now banned or deprecated.
Short term effect to fixing these issues? A nearly 20% increase in traffic since the switchover and slightly raised ranking for related firm content on OTHER sites.
This is my stand. I will not retreat from it. I will not compromise.
Is the first group SEOs, or information architects? Good information navigation design will consider searchability and, more importantly, human accessibility (a large part of which will be searchability, but search serves the primary need of people to find information efficiently).
Good point, white hat SEO is simply applied IA.
Thanks for the resumé tip :)
Thanks for the resumé tip :)
Making your site easy to crawl and find can be spammy when taken too extremes, yes, but it's also about quality basics. Good titles, site structure, URL structure, relevant internal linking, etc.
As an analogy, making your restaurant easy to find (relevant name, standard names for things, signs out front, etc.) is important unless you're so well-branded that can get by on having a secret, unmarked door.
As an analogy, making your restaurant easy to find (relevant name, standard names for things, signs out front, etc.) is important unless you're so well-branded that can get by on having a secret, unmarked door.
> Am i the only one who thinks that SEO is more a plague then it is a real profession?
luckily, no. http://powazek.com/posts/2090
luckily, no. http://powazek.com/posts/2090
It seems that most people here would like it very much if SEO died a horrible death as quickly as possible. Let me try and paint a (probably somewhat sketchy) picture of why it should exist (I have nothing to do with SEO consulting btw):
1. You have a business. 2. Said business has a website. 3. Said website is bringing in no discernable revenue for your business. 4. Poking around for reasons, you discover that your website ranks really lowly on Google. 5. You have no idea why. 6. You pay an SEO consultant. They take a look at your web site, and your overall web presence, to try and find out what's going wrong. 7. They discover that your markup is terrible, the wrong things are enclosed in header tags, etc. Also, you have little to no visibility outside of your website. 8. They advise some sensible changes to your markup, and force you to start posting things on Google+, Twitter, Facebook, etc. 9. Your ranking improves. As does your revenue.
Which bit of that process is evil and wrong?
Of course, there will always be a shady side to SEO (people taking it too far, etc), but there's a valid reason reason it exists: many people simply do not know enough to get the basic components right. SEO consultants help them do that, and improve revenue as a result.
1. You have a business. 2. Said business has a website. 3. Said website is bringing in no discernable revenue for your business. 4. Poking around for reasons, you discover that your website ranks really lowly on Google. 5. You have no idea why. 6. You pay an SEO consultant. They take a look at your web site, and your overall web presence, to try and find out what's going wrong. 7. They discover that your markup is terrible, the wrong things are enclosed in header tags, etc. Also, you have little to no visibility outside of your website. 8. They advise some sensible changes to your markup, and force you to start posting things on Google+, Twitter, Facebook, etc. 9. Your ranking improves. As does your revenue.
Which bit of that process is evil and wrong?
Of course, there will always be a shady side to SEO (people taking it too far, etc), but there's a valid reason reason it exists: many people simply do not know enough to get the basic components right. SEO consultants help them do that, and improve revenue as a result.
I think the general contention is that the process you describe is just using someone who's going to write sensible HTML not SEO. The shady side of actually modifying code not for easy-to-read, browser-efficiency, etc. is SEO.
I have to disagree. I consider part of my job to be some SEO (not my primary task and neither the most important, but I take care of it at my company) and its definitely the first part of what you sketch. And trying to clean all the second part stupid stuff someone else may have done before.
I don't see it as a game against Google... I try to make our sites as useful for humans as possible, and hope Google agrees with a decent enough ranking so humans can visit it and enjoy the content. But when it doesn't, then the game is between me and the "other SEOs" who may not see the point of making something useful for readers, not machines.
I don't see it as a game against Google... I try to make our sites as useful for humans as possible, and hope Google agrees with a decent enough ranking so humans can visit it and enjoy the content. But when it doesn't, then the game is between me and the "other SEOs" who may not see the point of making something useful for readers, not machines.
Writing sensible HTML is optimising for machines. The Google algorithm runs on machines, so they'll like it and benefit from it. Browsers, screen-reader, future colleagues will also like it and benefit from it. Calling this search engine optimisation seems a bit weird, though it'll certainly help Google process your website properly.
If you read the OP, his mean concern is the tiny subtle changes in the algorithm that drive him mad. He calls optimising for them SEO. It is considered shady and evil by some, myself included. (Ultimately it's about the content, not about who best reverse-engineered Google's algorithm.)
If you read the OP, his mean concern is the tiny subtle changes in the algorithm that drive him mad. He calls optimising for them SEO. It is considered shady and evil by some, myself included. (Ultimately it's about the content, not about who best reverse-engineered Google's algorithm.)
An SEO consultant complaining about Google's frequent algo changes is like a mechanic complaining that cars break down too often.
If I was feeling uncharitable, I would say it's a bit like a burglar complaining that the homeowner keeps installing better locks.
And the house would be the White House (in the sense that SEOs aren't just working against Google but against its billion users).
More like an arsonist complaining about updates to building code.
Bingo
Actually you are more right than you think. Frankly I would rather do something else than SEO. Like real marketing. (I am the author of said site)
It's more like new cars being introduced without service manuals, forcing the mechanic to continually figure out how to fix them.
I don't think this is a good analogy. Google's job is to make sure good content is ranked high and crap is ranked low. You should just create content that is useful to others and let Google figure out how to discover it and rank it high (they are good at it).
To create useful content, you don't need to know Google internals.
To create useful content, you don't need to know Google internals.
And we would just do this if Google actually made good on their promises to do this, but I see an awful lot of crap still ranking in the SERPs especially because of hacked sites and link networks.
Full disclosure - I work in SEO.
Full disclosure - I work in SEO.
The solution to no-content black hattery is not to promote your no-content site with black hat techniques. Nobody here is criticizing site tuning and at the same time defending content farms. Why defend misbehavior as a justification for misbehavior?
Note that I am not saying that you yourself perform any of this in your role. Our anger is not at site tuning and placement but the techniques used to do so, which you appear to be unnecessarily defending as essential for any site's survival.
Note that I am not saying that you yourself perform any of this in your role. Our anger is not at site tuning and placement but the techniques used to do so, which you appear to be unnecessarily defending as essential for any site's survival.
Your comment doesn't make any sense. Of course the solution to something bad isn't promoting it with something bad. That's just ridiculous and I would never argue that.
You're automatically assuming that SEO is bad, or is all blackhat, or is just about promoting crap. I would argue vehemently that good SEOs only want to promote good content. I turn down many freelance gigs because the company is shit.
I think you're confusing legitimate SEO with crap affiliate marketing. Big difference there, buddy.
You're automatically assuming that SEO is bad, or is all blackhat, or is just about promoting crap. I would argue vehemently that good SEOs only want to promote good content. I turn down many freelance gigs because the company is shit.
I think you're confusing legitimate SEO with crap affiliate marketing. Big difference there, buddy.
You replied a person who said to just create content that was useful to others with suggesting that you would do that, only if there weren't hacked sites jumbling up the rankings. It ~seemed~ to suggest that you were less interested in original, interesting content than you were competing with the top sites regardless of how either of you got to that spot.
I said that we'd just do that if... The just implies that we do that plus other things.
If this were truly the case, we shouldn't expect quality sites to benefit from SEO. In practice, they do.
I'm not saying SEO is not effective, it certainly works, but it shouldn't, and it is Google job to make it happen.
On a high level good sites don't benefit from SEO, because it wastes time and money on something that should be granted: good sites should be ranked high, crap sites should be ranked low. SEO is a cargo cult (http://en.wikipedia.org/wiki/Cargo_cult_science) with no benefits to society.
On a high level good sites don't benefit from SEO, because it wastes time and money on something that should be granted: good sites should be ranked high, crap sites should be ranked low. SEO is a cargo cult (http://en.wikipedia.org/wiki/Cargo_cult_science) with no benefits to society.
If SEO is provably effective, it is by definition not a cargo cult. The defining feature of a cargo cult is that it's questionable at best whether their practices lead to the goal they want to achieve (e.g. making fake walkie talkies and yelling into them hoping that a plane will drop cargo because you saw someone with a real walkie talkie make that happen).
Except SEO isn't a typical use case. It like complaining that newer tires made for better grip in the wet aren't as good as cutting people off during rush hour.
Or whining about "too many computers in cars nowadays." Thanks but no thanks. I'll take the improved efficiency and reliability over your laziness to retrain.
A legit complaint would be the miniaturization making it difficult to work on your own car without a more complete disassembly, engine removal, etc, I rarely hear "too many computers" from actual mechanics or people who used to do self-repairs in the past. I tend to hear these complaints from less-knowledgeable consumers rationalizing why their cars are breaking down.
Not entirely - after all, to his clients an SEO consultant isn't just a black hole they throw money into. They do expect to see results in the form of higher search ranking, and Google's algorithm changes make it harder to deliver on that.
I think I'd mirror others' analogies. It's more that the complaint is futile - and kind of sad. One comes away with the impression that the author is naive enough about the nature of his profession to genuinely believe that, on this subject, he and Google should be buddies and not adversaries.
I think I'd mirror others' analogies. It's more that the complaint is futile - and kind of sad. One comes away with the impression that the author is naive enough about the nature of his profession to genuinely believe that, on this subject, he and Google should be buddies and not adversaries.
"to his clients an SEO consultant isn't just a black hole they throw money into. They do expect to see results in the form of higher search ranking, and Google's algorithm changes make it harder to deliver on that."
Wouldn't it then be harder to demonstrate objective returns if Google's algorithms are so obfuscated? If they're not afraid to game Google for their customer's benefit, they're certainly not going to be afraid to game customer expectations and fudge their results for a living.
Wouldn't it then be harder to demonstrate objective returns if Google's algorithms are so obfuscated? If they're not afraid to game Google for their customer's benefit, they're certainly not going to be afraid to game customer expectations and fudge their results for a living.
Not at all. It's like complaining they don't break down enough.
SEO has made a lot of damage to the Internet. 10 years ago links really expressed people preferences, and it was easy to get linked and ranked high when you've published useful stuff.
Today, almost all large sites that people use to share links to original content use nofollow links. Because of this, it is common that when you Google for some original content, you will find links to aggregators (Stack Overflow, Reddit, HN, etc.) but the original page will be very far on the result list.
Today, almost all large sites that people use to share links to original content use nofollow links. Because of this, it is common that when you Google for some original content, you will find links to aggregators (Stack Overflow, Reddit, HN, etc.) but the original page will be very far on the result list.
Well, actually Google did this first, because they started using links as "votes". Then everyone started hunting for links instead of trying build do good content.
I thought he was gonna ask for a job.
[deleted]
me too. in that case it would have been cool.
Ha! I should have... Maybe in the next version...
I still strongly feel that the best way to achieve excellent SEO is through building human-accessible pages. How is your site on a screen reader? If its easy to navigate by braille keyboard or voice output, then it probably is well enough that Google should be able to make sense of it.
Better yet, make it so that your CODE is readable and understandable easily by someone who doesn't have perfect vision.
If you achieve this and create great, relevant content, then Google will snap it up and the customers will appear. No other tricks needed.
Better yet, make it so that your CODE is readable and understandable easily by someone who doesn't have perfect vision.
If you achieve this and create great, relevant content, then Google will snap it up and the customers will appear. No other tricks needed.
10 people targeting the same keyword. Only 10 spots on page one of the SERPS, if the on site content and site is well built, then who gets the top spot?
The machine isn't clever enough yet to get the perfect answer, and till it is, humans and marketers will continue the manipulation of visibility in pursuit of traffic and revenue. It's sad, but it's a fact.
The machine isn't clever enough yet to get the perfect answer, and till it is, humans and marketers will continue the manipulation of visibility in pursuit of traffic and revenue. It's sad, but it's a fact.
I wonder how this post got to the top even though the sentiment in the comments is that this posts sucks.
I think you'll find only a very tiny fraction of HNers bother to comment, so it's not a representative view of people's opinions on a given article.
We can't downvote posts.
[deleted]
But we can flag them.
Yes, but why would we?
Right, I'd only do so if it was an outright scam.
[deleted]
I find the perspective of the SEO guy interesting, even if I find what he does kind of skeezy and annoying.
The honesty is a bit refreshing, I agree. I'd love to read a good, anonymous confessional of the daily life of a SEO that's more intimate and less of their usual humblebrag PR.
I upvoted it before I'd read all the way through.
I thought the guy was complaining that he had to jump through Google's hoops to get his product in front of his users.
Now that I know he's complaining that Google are making it harder for him to game the system, I'd retract and reverse my upvote if I could.
I thought the guy was complaining that he had to jump through Google's hoops to get his product in front of his users.
Now that I know he's complaining that Google are making it harder for him to game the system, I'd retract and reverse my upvote if I could.
What others have said about commenters/lurkers and not being able to downvote, but also I think the "sentiment" is more that SEO sucks than that this piece is garbage per se. I upvoted it because it's entertaining and it did, along with the discussion here, broaden my perspective on SEO.
Nobody else read this as a joke? I read it as a joke (and a slightly racist one at that).
Entitled SEOs tend to be more common than exceptional. While I didn't see anything so hamfisted that it had to be parody, but I can see what you mean.
From the topics on their default page and his other communications, it seems to be sincere.
https://twitter.com/rishil/status/299185628321566720
From the topics on their default page and his other communications, it seems to be sincere.
https://twitter.com/rishil/status/299185628321566720
I read it as a joke by the end but, worryingly, didn't automatically recontextualise it as racist in light of that. Which, I think I agree with you, it is, if it's a joke and all the East Africa stuff is made up for "comedy value".
....? where did the idea of "The East Africa stuff" being made up come from?
Nowhere. It was speculation on my part extrapolated from the premise that the piece was written as a joke and nothing in it could be taken at face value. A little research on the guy suggests I was reading too much into it. Here he is http://www.linkedin.com/in/rishilakhani and his languages are listed as "Hindi, Swahili, Gujarati, Punjabi" - we don't need much more than that to connect the dots.
I thought he was genuinely from Kenya or Ethiopia or somewhere.
I am an Indian born in East Africa ;)
I assure you I am not racist, being an Indian raised in Kenya.
I think SEO is practically dead, and for good. The Social Web is supposedly lot more democratic and apparently a better choice.
I think any good social marketers (as they've now rebranded themselves) are fully involved in subverting The Social Web. Any site that reaches a certain size (Digg, Reddit, etc.) will become a ripe target for less than organic posts.
I can give plenty of counter examples, but the one that immediately jumps to mind is orgs like Pissed Consumer, who are essentially using subdomains to hijack other companies' Google juice. However, it's also hard for me to suggest that those links are not relevant to queries if you look for something like "_____ reviews"
SEO Won't be any more dead than health fads (which really started in the late 1800's). As long as Snake Oil exists, people will buy it.
They'll probably take on a different guise now, like calling it "Social Media Promotion" or some other malarkey. Plenty of companies want to be the talk of the town on FB and Twitter.
They'll probably take on a different guise now, like calling it "Social Media Promotion" or some other malarkey. Plenty of companies want to be the talk of the town on FB and Twitter.
These days I'd argue to a client to spend their money hiring PR people. PR people obtain high quality backlinks the legit, organic way.
"I think SEO is practically dead, and for good. The Social Web is supposedly lot more democratic and apparently a better choice."
Subverting "The Social Web" occurs to feed users to their advertising or other attempts at monetization, they don't get paid to Tweet.
Subverting "The Social Web" occurs to feed users to their advertising or other attempts at monetization, they don't get paid to Tweet.
Well, there's a negative as well, hence I used keywords like supposedly, apparently etc. But wouldn't you rather your insurance-selling uncle visit you for yourself, instead of an intent to sell you the insurance? I think those days will be here eventually.
Say people eventually buy everything or get some form of advertising kickbacks from "the Social", do you think that there won't be ways to game those income streams?
That robots.txt is so big that even with comments maybe google will drop it after trying to parse it for a while....
I would hope that google, of all companies, would be capable of writing a crawler that ignores robots.txt comments.
Other than that the guy only has one rule.
Other than that the guy only has one rule.
robots this size aren't that unusual http://www.bbc.co.uk/robots.txt
I think you just proved the point here: SEO can be very valuable. Any good SEO would point out to the BBC that having similar rules for User-agent: Googlebot-Mobile and User-agent: * is not very efficient.
I bet they are fairly liberal with their size limits:
http://www.amazon.com/robots.txt http://aws.amazon.com/robots.txt http://www.google.com/robots.txt http://store.apple.com/robots.txt
http://www.amazon.com/robots.txt http://aws.amazon.com/robots.txt http://www.google.com/robots.txt http://store.apple.com/robots.txt
Apple should use git instead of subversion :)
Anybody know how big robots.txt can be and if anyone has ever given a spider grief by screwing with the size or download rate of robots.txt?
The robots.txt limit in Google's Webmasterpanelthing is 100,000 characters. Googlebot has been said to stop parsing robots.txt after 500,000 characters. I don't know if that's bytes or characters.
500kb is the hard upper limit in the crawler itself. Limits that are exposed elsewhere (Webmaster Tools, etc) are irrelevant.
I always thought SEO was supposed to be a cat/mouse game. The search engine tries to balance the search results fairly. The SEO consultant tries to game the system and get certain links to appear ranked higher, whether fair or not. Isn't that just how this works?
gaming the system is blackhat SEO. There's also supposed to be the SEO business where you actually improve the site's quality, indexability and speed.
Very cool stuff but I have to disagree with the general idea.
I've been doing SEO for 8 years and I have to say that I came for the fun and money but I stayed for the challenge and algo changes.
I remember how, 8 years ago, it was all about "5% density" and link stuffing. Today is content quality, social signals, link profile (not just quantity), contextual co-relations and so much much more...
I think that I only truly fallen in love with SEO when Google started using rel=canonical. This is when it really became a game of poker and not a "hungy hungy hippos" with links in it.
I've been doing SEO for 8 years and I have to say that I came for the fun and money but I stayed for the challenge and algo changes.
I remember how, 8 years ago, it was all about "5% density" and link stuffing. Today is content quality, social signals, link profile (not just quantity), contextual co-relations and so much much more...
I think that I only truly fallen in love with SEO when Google started using rel=canonical. This is when it really became a game of poker and not a "hungy hungy hippos" with links in it.
You should use `humans.txt`. :)
http://humanstxt.org/
http://humanstxt.org/
[deleted]
Bmac27, what kind of enterprise seo toolset do you recommend ? I am currently evaluating Skycara, Brightedge and SEOcatalyst but wanted something else to try and look at. I'm runnign a site with 15,000 pages, 10,000 errors and about 20k keywords
[deleted]