SEO Guide. Pic by Dinukshan Kuruppu

SEO: A Complete Guide to Optimizing for Search

Search Engine Optimization (SEO) is important because it can mean the difference between capturing high page rankings in Google, for relevant keywords, and remaining undiscovered and anonymous.

Web traffic is the lifeblood of any blog or business that relies on the Internet to generate leads and make sales. Organic traffic from search engines is generally the largest (and most valuable), single source of traffic making it a vital component of any successful online venture.

SEO is such an incredibly broad, far-reaching topic that can be difficult to define properly. Here's our definition:

SEO uses a diverse range of strategies and techniques, covering a broad spectrum of disciplines, in order to help Web properties capture relevant, high page rankings in the SERPs (Search Engine Results Pages).

Because SEO is a relatively new industry there are a lot of misconceptions, myths, half-truths, cons, and pitfalls waiting to trip up the unsuspecting webmaster. That's why we have compiled this comprehensive guide that will cover:

Regardless of whether you are a first time webmaster wanting to understand what search optimization is all about, or a professional SEO or SEO agency looking for in-depth information and insight, this guide should provide an all-in-one resource that helps you achieve your goals.

SEO Basics

Optimizing a website for search is not about following any single formula. If it was, that formula would be widely known and everyone would be able to get to the top of Google's search results. Clearly that's impossible because there are only 7 - 10 results on the first page and there are sometimes hundreds of millions of possible results to chose from.

With so much to consider, I think a good place to start is with an SEO introduction from a Googler. They should know a bit about the topic, right?

In its most basic form,

SEO is about creating a visible, unique and interesting Web resource that offers value and a great user experience.

In order to do this you will need:

  1. Use a high quality Web platform with baked in SEO features
  2. Set goals
  3. Research and understand your target audience
  4. Research and understand search keywords
  5. Produce high quality content that meets (and exceeds) the demands of the target audience
  6. Share & promote content to build up organic backlinks

That's the bird's eye overview. Probably a bit vague, right? Let's take a look at each section in a bit more depth.

1. Choose a Good Web Platform for SEO

While I started my career writing books about programming and Web development, I've come to realize that what businesses actually need is a great Web solution that doesn't need a huge investment in terms of time and effort. The best way to create a fantastic, responsive website (and quickly) is to use one of the leading website builders that offers the functionality your business needs - regardless of whether it is a blog or eCommerce store.

Why?

Website builders offer a graphical user interface that makes designing webpages quick and easy. They offer cost effective, reliable hosting and site managements all for one low monthly cost. Add to that teams of top people working round the clock to keep their platform up-to-date with the latest technology and trends.

They also tend to have responsive or mobile-first design built in.

Websites must support mobile devices (ideally using responsive design) in order to capture high page rankings in Google's mobile search index.

Not to mention the fact that these big platforms have entire communities of people creating plug-in modules and 3rd party integrations that can streamline everything you do and generally make being a webmaster a pleasure.

By comparison, building your own website with a custom design means expensive development - plus ongoing maintenance and upgrades ( that take time and cost money). Rather avoid the hassle of managing the technical aspects of a website and put it into the hands of professionals who do it for a living.

Here's a table of the best site builders ranked by their SEO features.

RankPlatformFeaturesFlexibilityImplementation
1Bigcommerce (eCommerce)AllExcellentGood
2Shopify (eCommerce)AllGoodGood
3WeeblyAllGoodGood
4WixMostAverageGood

Using a good site builder also means that a lot of the underlying technical SEO issues you would normally have to integrate into a custom design is handled for you. Things like META descriptions, page layouts, headings, server response codes, etc are all handled behind the scenes - freeing you up to focus on content and marketing.

Talking of which...

2. Set Goals

Its important to have some sort of goal before you set about creating new content. Sitting down and writing about whatever pops into your head may be fun, but it's the equivalent of closing your eyes and firing blindly as fast as possible. Its much better to take careful aim at a specific target and hit it the first time.

Any time you sit down to write something you should be thinking of what you want to get out. Maybe its tonnes of backlinks, or maybe its to boost your website's PageRank above a competitor's for a certain topic. Whatever it is, know what you want before you get down to business.

Write with a goal in mind; not for the sake of creating content.

Understand that you have limited manhours with which to create content. Therefore, writing stuff arbitrarily can be wasteful and unproductive.

3. Understand the Target Audience

There’s absolutely no point in writing anything unless you know who you want to reach. Content that is not relevant becomes an annoyance to the reader who now has to look elsewhere for what they’re after.

Remember that there are literally millions of pieces of new content being added to the Web's background noise every day. It is in your own interests to target a specific audience because it means you are far less likely to waste their time, and far more likely to engage with them.

At the start, when you don't yet have an audience, this can be a tricky task to manage. But, as time goes by you'll be able to watch traffic patterns on your new pages (using Google Analytics).

Make it a point to regularly monitor visitor behaviour in order to better understand what is happening. Are people bouncing. Are they searching for something else? Are they converting, browsing away or doing something else completely unexpected?

Think about the page’s performance in terms of how well it meets the average user’s intent. The single most important SEO tweak you can make to any content is to,

Ensure content is pitched perfectly to meet the target audience's intent.

Understanding user intent is extremely important, and quite tricky. Google often gets it wrong and sends mismatched traffic, which may mean updating your content to do a better job for the visitors you're getting - otherwise conversions may suffer.

4. Research Keywords

Selecting the right keywords can help your content dominate Google search results and using highly focused content containing specific keywords is the good way to indicate relevance to people searching on the Web.

In order to do any research, you'll first need to find a tool that allows you to do so. Google Adwords' Keyword tool is the best option if you don't currently use professional SEO software.

Make sure you have the "Keyword ideas" results tab (at the bottom of the Adwords' Keyword tool page) selected for the purpose of this article.

1. Select one or two "Exact match" keyphrases

An article should only ever focus on one or two exact phrases. More simply can't be stuffed in while still having content that humans will read.

In addition, it is critical that you do initial keyword research using the "[Exact]" match type. Select this from the left hand column in the Adwords' Keyword tool.

Exact match gives traffic estimates for specific phrases and this can be wildly different from the default broad match data. Exact match traffic volume estimates are therefore much lower than broad searches.

To give you an example, search for "SEO guide" using the "Broad" and "Exact" options simultaneously. You'll see that broad returns approximately 9 times the estimated traffic volumes, which can be misleading, if your looking to target specific phrases (which you are).

2. Use "Broad match" research to find additional keywords

After isolating one or two exact match keywords, it's time to use the broad search to help guide you away from using ambiguous or unpopular keyword variations and synonyms.

Content writing should be natural and easy to read for humans. But, at the same time, there is no harm in selecting for higher traffic phrases.

For example, the phrases "appearing higher in Google search results" and "ranking higher in Google search results" are both easy to read, and people will interpret them in exactly the same way.

The difference is that "ranking higher in Google" has thousands of times more traffic so it makes sense to always use this phrase.

3. Target keywords with appropriate traffic volumes

Be realistic about the keywords you go after. Look at the authority and PageRank of your blog or site and decide whether or not you think Google will place your article above some of the biggest websites in your niche.

There is far more benefit to be had by going after phrases that you can "win".

Look at it like this: If each article you write gets to the top result for a term that has only 500 hits a month, then each article your write should garner around (60% of 500 searches = 350) 350 additional visits per month.

Even writing ten articles a month like this will increase your organic traffic by 3500 visits per month. You'll quickly be making big bucks at that rate.

4. Filter keyword research by location

Local SEO is hugely important. Even if your blog or business does not have geographical limitations, it pays to do regional keyword research.

South Africans search more for "rugby", while Canadians search more for "hockey" (even though it's Ice hockey, not actual hockey ;).

Regional keyword research gives you an additional opportunity to get to the top of search results for specific demographics.

5. Keyword density

Ok, so this is not technically research. But, after doing all the hard work in finding exactly the right keywords to get your content in front of the people you want, don't throw it all away with "keyword stuffing".

Google doesn't like the look of unnatural content. It knows that SEOs regularly try to over-optimize their content and it will penalize you by dropping your content off the top page results if it thinks this is what you are doing.

As a general rule of thumb (that works for me), I use keywords once in the title, once in the URL, and once in the opening sentence or paragraph (and this is then automatically included once in the META description). After that I only use those keywords where it is natural to do so within the context of the article.

5. Create Quality Content

I've always found this concept of "quality" content quite annoying. What does quality actually mean? It's not something that is quantifiable by obvious metrics, like English grammar, article length, use of visual media, etc. In fact, many sites that do very well seem to care about one thing only.

Quality content offers value to its target audience.

Every webmaster has to find and understand the precise mix of qualities that will work for their own, unique audience. But there are at least a few general features of good quality content to build into your content creation processes.

1. Highly focused

Cover very specific topics.

The reality of SEO today is that unless Google knows an article is precisely focused on a specific search term or phrase, it is unlikely it will rank highly for that exact search phrase. There is simply so much content being generated that there will always be many tens, hundreds or thousands of articles devoted to each and every possible SEO keyword or phrase.

2. Authoritative

Create content that is both well written and comprehensive. This allows it to be used as a reference source by many people - improving the chances that is will be quoted and linked to.

3. Fresh

Keep your content regularly updated by adding new articles and improving old ones.

Google tends to favor content that is fresh for what it calls QDF (Query Deserves Freshness) terms. It's no longer sufficient to add content once and expect organic search traffic continue to roll in indefinitely.

4. Great linking patterns

Don't be shy to link. Wherever it is appropriate, link internally (i.e. to other resources on your own site) and externally. Internal links help readers find or research related content, lowering your bounce rate and keeping visitors engaged. External links give you an opportunity to reach out to other publishers with the good news that you've linked to them (this could be the start of a mutually beneficial relationship with an important influencer in your niche).

5. Great document structure

It's imperative that content is well organized and easy to follow. Document structure is important to Google and is a good indicator of quality content. Always ensure that you utilize a single <H1> title tag for each article, and break up chunks of content into logical categories headed by H2, H3 and smaller heading tags.

In addition, make use of formatting to highlight important phrases and break up dense text into easily consumable chunks - preferable with quotable quotes that can encourage social media sharing.

Always structure content to be easy for humans to understand and utilize.

Ultimately Google wants to rank content that is believes offers a great user experience for humans. It doesn't want to rank content that has been created to fool it into thinking it would be great for users.

5. Get Referrals & Backlinks

Building backlinks is hard. Google frowns upon any strategy that seeks to artificially manipulate its search engine rankings that relies heavily on backlinks. But without backlinks your site simply won't build up the trust and authority required to rank well.

The best way to encourage natural, organic backlinks from high quality, relevant sites is to build relationships with people in your niche industry.

Growth in valuable organic backlinks is proportional to the number and quality of your relationships with other influencers.

SEO, and marketing in general, has already begun a shift toward influencers. This trend is only going to gather momentum because it offers a far more cost effective way to gain long term, sustainable reach and coverage (i.e. quality backlinks) than any other marketing strategy.

The reason?

Online influencers hold a lot of authority and trust, and Google continues to place emphasis on high quality inbound links (i.e. ones that appear naturally within editorial content) from these influencers.

Building relationships with influencers (and by extension, building backlinks for SEO) is really about implementing a strategic blogger outreach campaign with a view to creating a genuine friendship based on mutual interests and collaboration – usually oriented around sharing and linking to fantastic content.

Technical SEO

Technical SEO is about improving performance and speed, as well as ensuring your Web server is doing the right things under the hood to help Google access and index the right content. This can be tricky because a lot of communication with Google is not immediately visible to humans.

The best place to start is with Google Analytics.

Google Analytics

Google analytics provides a wealth of valuable SEO data and it is important to use it to its full potential to help create better content, drive more traffic and convert it more effectively.

Google analytics showing Web traffic volumes

While there are plenty of metrics to explore, here are a few great tips to get you started.

1. Segment Analytics Data for Today

By default, Google analytics displays a week's worth of data. This is great for a brief overview of how your site is performing, but you can derive deeper SEO insights by splitting the data into segments - day, week, month, year, historical.

I always take a look at my current day's analytics information. It's useful to:

  • Determine the immediate impact of content as soon as it appears in Google's index: Use it to measure the impact of trending, topical content. The article entitled "Google's Panda and Penguin updates are great for bloggers, marketers and business appeared immediately in the number one spot for "Google panda penguin", just after the Penguin update went live. It continues to drive plenty of highly engaged traffic.
  • Isolate localized spikes in referral traffic: In the grand scheme of things, a referrer who sends a hundred hits in one afternoon won't make much of an impact. However, if you are watching on the day it happens, you can leverage this knowledge. Find out who said what about you, and react appropriately. It may lead to a new social connection or business partner.

2. Exclude Erroneous Data

Every now and then a social media site, or news site, will send a thousand visits in a couple of hours and then go silent again for months, years or forever. On occasion, referral spammers also weasel their way into Google's analytical data. These traffic spikes, if they're large enough, can warp the data and skew your search strategy as a result.

To remove referral traffic from a specific source in Google analytics:

  1. Click the "advanced" link
  2. Select "Exclude"
  3. Select "Source/Medium" from "Dimensions"
  4. Enter the name of the source (i.e. spammerA) and click Apply

How to exclude source data from Google analytics

Remember that you can add as many advanced rules to your data as possible. This helps to focus on specific aspects of your site's SEO and analytical data.

3. Use Content Navigation to Analyze Traffic Patterns

It can be difficult to "guide" visitors to a conversion. It helps to understand exactly how people are navigating your site in order to make intelligent decisions about how to improve the conversion rate.

The content navigation summary can shed light on what people are looking for when they browse a certain page. Often the results can be surprising and lead you to a new understanding of what visitors want.

To use the content summary:

  1. Click the "Behavior" tab in the left sidebar
  2. Select "All Pages" under "Site Content"
  3. Select the "Navigation Summary" tab at the top of the page
  4. Select the page you want to analyze from the "Current Selection" drop down

The Google analytics navigation summary can provide valuable SEO insight

The navigation summary is particularly useful for gaining insight into what people are looking for. Do they go to your services? Do they read your blog? Do they bounce?

4. Use Secondary Dimensions

By default, Google analytics displays the primary data dimension only. However, the secondary dimension can help provide drill-down information about the primary dimension's data.

For example, looking at the "Audience >> Location" data map for traffic from the U.S on my site, it's possible to see where my visitors come from. This is interesting and all, but I also want to know how they arrived at my site. This can tell me if a localized surge in traffic occurred because of a specific event, marketing campaign, etc.

The Google analytics 'secondary dimension' facility provides powerful SEO intelligence

To display a secondary dimension in your data, use the "Secondary dimension" drop down and choose the pertinent option.

5. Monitor Segmented Behavior Flow

Looking at lists of statistical SEO data is one thing, but being able to actually visualize the flow of traffic into and around your website is a very powerful feature of Google analytics.

Google analytics makes it easy to visualize this flow for different segments of your market - great for precision SEO intelligence gathering. It's hugely valuable, for example, if you want to understand what new visitors are doing, as opposed to regular visitors who might already understand what you do and what you offer.

The Google Analytics BehaviorFlow for analyzing segmented traffic patterns

To access the visitor flow visualization, select "Behavior >> Behavior Flow" from the left sidebar. You can then segment the data according to visitor type - new, returning, paid, non-paid, etc. In addition, it is also possible to filter the results by one of the default dimensions, so you can limit the traffic you are looking at by location, source, content and system.

Google Search Console

The Search Console provides information that Google knows about webpages, site structure, errors and content on your site(s).

Google Search Console Dashboard

In particular, the following metrics are very useful:

  • Crawl Errors under Crawl can indicate if Google is accessing your site correctly
  • Security Issues can alert you to the presence of malicious, foreign content or code
  • Search Analytics under Search Traffic provides insight into what search keywords Google ranks your blog or site on
  • Index Status under Google Index can alert you to potential indexing issues on your site
  • HTML Improvements under Search Appearance highlights many of the most common webpage structural problems that can affect the user experience

If you are new to the search console you will need to add a sitemap in order to help Google crawl and index your site.

A sitemap is an XML file that lists webpages on a site in order to aid crawling and indexing by search engines.

Most good Web platforms will automatically generate a sitemap and update it regularly to reflect new or removed pages. Once Google has processes the sitemap it will display information comparing what's in your sitemap to what it has in the index.

Search Console Sitemap

While the search console can help detect issues with your site, not everything can be solved from here. For example, a server might return an incorrect HTTP response code even though a human visitor can view the page as normal, or, Google may inadvertently be blocked from indexing certain pages by a faulty setting in your robots.txt file.

Let's take a quick look at two important ways of controlling access and indexing of your site taken from our article entitled When to use robots.txt and 301 redirects for Panda SEO recovery.

robots.txt

The robots.txt file can be used to control which directories and pages search engines have access to. While the robots file cannot enforce the rules it lays out, most of the important search engines (like Google) will obey robots.txt.

Here's an example of how to prevent Google from crawling a page entitled affiliate_links.html:

User-agent: *
Disallow: /affiliate_links.html

You aren't limited to listing each and every file that you wish to be Disallowed.

In fact, there is quite a bit to learn about how to use robots.txt, and every webmaster should have a basic knowledge of how to use this file. I recommend you visit the Web Robots Pages for more information.

It's also easy to test a robots.txt file using search console's robots.txt Tester.

Search console robots tester

.htaccess

301 redirects are extremely important for SEO purposes. A 301 response code tells Google that a page has been permanently moved to a different address. This is obviously important when a page name or path changes, or a domain name change occurs.

Without redirection, Google will treat the same content at a new address as completely new content. And what's worse, it will most likely treat this as duplicate content since the old address is likely still indexed.

301 redirects can be implemented from an .htaccess file - depending on the permissions granted to you by your hosting company. People using hosted website solutions may not be able to edit .htaccess directly, although leading website builders should provide other ways to implement 301 redirects.

A typical redirect from one page to another looks like this:

Redirect 301 /oldpage.html /newpage.html

URL redirection is actually a lot more powerful and flexible than this quick example, and it is possible to use regular expressions to redirect entire domains, directories, or pretty much anything you like. Wikipedia has a pretty good URL redirection write-up, if you want to learn more.

Aside from these two main issues that you will most ilkely need to handle manually, there are a few very useful, freely available tools to help you analyze and fix any other technical SEO issues. With that said, it's important to remember that no tool is perfect. They can only follow the rules built in to them. They can't make decisions that apply to unique circumstances so you will need to use your best judgement or reach out for help in online forums if unsure.

Performance

Page load time is one of the many factors that determine where in the SERPs Google places your site.

What is your average page load time?

If you don't know, then the chances are you're losing customers and revenue without even realizing it.

Speeding up a site will lower its bounce rate and increase conversions while improving the chances of ranking better.

If you want to make sure your site's SEO is in tip-top condition then you need to look at ways to make your blog or site as fast as possible. Most people these days simply do not have the patience to wait for slow loading sites... they'll simply go to a competitor for what they need. You're left wondering why your site's bounce rate is so high.

A slow loading page can undo all the good work you put into your content, marketing, SEO and community. What's even worse is that, most likely, no one will bother report it to you - they'll simply leave.

There are a wide array of techniques that developers, programmers and webmasters use in order to speed up their webpages. Let's take a look at a few of them here (Note: most of these technical issues will be handled for you, behind the scenes, if you're using a good website builder)

CDN

A CDN (Content Delivery Network) makes static files available via a distributed network of server nodes allowing webpages to serve resources cheaply and quickly. In particular, heavy bandwidth resources like aggregated code files and images can be served from a CDN (instead of your own server) to speed up a website and reduce load on the webserver.

Amazon Cloudfront offers fairly cheap way to speed up file serving from their global network of servers and are definitely worth checking out. In fact, clicking f12 and reloading this page, while viewing the network downloads, will show how cloudfront serves images, static CSS and JS files for this website.

Cloudfront CDN serving static files

Caching

Caching a page, or any elements of a page, means that a "copy" of that element is creating once and served again and again until it needs to be refreshed. This helps your server because it doesn't have to rebuild each page from scratch on each request. The trade off is that any cached content is not updated until the cache refreshes.

Offhand, I can't think of a single instance in which a site does not require some form of caching. Every site should be caching whatever it can. Look at SME Pals. It caches almost all the blocks around the main content of the page.

This information only needs to be updated when there is a new course, book or service available, so there's no need to make the server rebuild all this information on every request, reducing the load on the server and increasing page load time.

Image optimization

Images that go into the design of your site should absolutely be optimized. You can decolorize them, use neat CSS tricks to cause them to repeat instead of using a large image and so on. You'll often find that images make up the bulk of the page size, and therefore contribute most to the page load time.

I've uploaded images directly to a site during development without first reducing their size. Once it has come time to optimize, you can save massive amounts off the size of the page in images alone.

Let's assume you save 50K by optimizing all your images. This is a huge saving if you consider that after 10 000 visits you have saved 500 Meg of transfer.

CSS aggregation

This is an important one. Most modern sites have a number of CSS files. Each file requires a server request, which slows down the page load time because there is always going to be some network latency.

By aggregating CSS files into one large file, you cut out all those return trips to the server and speed things up significantly.

Script aggregation

This is the same principle as CSS aggregation. However, not all JavaScript files can be aggregated into one. For example, the Google tracking urchin must go at the bottom of the page because you don't want to stall your own site if the urchin is not loading properly (which happens from time to time).

Everything else that appears in the HEAD section of your page, should be aggregated.

Page compression

Please tell me you are not sending plain HTML from your server to the browser? HTML text, as well as CSS and JavaScript compresses very well. Uncompressed pages could be several hundred KB, but the same, compressed file, might only take 30 - 40 K of transfer giving you huge speed improvements.

Throttling

Throttling is a little more complex. Often different parts of a page, especially supporting data that appears in sidebars or blocks, require resource intensive processing. When the site is busy these non-core elements can slow your server down. When you throttle these, you are basically telling them, under certain conditions, to stop chewing resources.

This means the data in these blocks may not be up-to-date, but at least the main content (which is the most important part) can be served in a timely manner.

If you are using a good website platform then you should be able to implement a fairly sound throttling policy. If not, time to look for a better platform, or speak to someone who can advise and develop this for you.

Server resources

I am continually amazed at people who purchase hosting that is "the fastest in the world" for next to nothing, and wonder why their site is so slow. The trick that many hosting companies use is to have a very fast server with an incredibly quick connection, which they then use to advertise to customers.

What they don't tell you is that in order to make it profitable they have to cram so many people onto that awesome server that you end up only getting a trickle of bandwidth anyway.

Make sure you find a solid hosting service and don't be afraid to pay a little for it. Chances are you will get better reliability as well as better performance, and support is always nice.

Google Mobile Friendly Test

There is now more mobile traffic than traditional PC traffic on the Web. The Internet has become a mobile technology and Google is taking this seriously. It won't rank your pages in the mobile index if they are not mobile friendly (i.e. responsive). Updating or migrating a website to a mobile friendly version is not always an easy task so Google makes it easy to test your pages online with it's Mobile friendly test.

Google's online mobile friendly test

Google Page Speed Insights

Page speed insights is a great free service that measures your page speed performance and provides a suite of suggestions for improvement (categorized by priority).

Google's pagespeed insights tool

Remember that not every suggestion this tool makes will be possible to implement. Half the time it complains over 3rd party integrations such as Google Ads. However, it is a great way to find low hanging fruit when it comes to performance and you can often make huge gains in speed for relatively little effort.

Google Chrome Audits

Google's Chrome browser comes with a brilliant feature - you can perform a full analysis of any page at any time using their Audits tool - hit f12when using Chrome to bring up the built-in Web developer tools. This not only includes audits, but also a complete suite of design and development tools, including network information (useful for inspecting download times for various page elements).

To use Google Chrome's Audits:

  1. Browse to any page you would like to test
  2. Right click anywhere on the page and select "Inspect element" to bring up the Developer tools pane (Hint: you can also do this from the Tools menu)
  3. Select the Audits tab and click Run

Audits doesn't provide a score. Instead, it brings up a huge array of results that are categorized by importance (how much goodness you can gain by implementing fixes). The fewer results you see, the less there is to improve.

Google Chrome's Audits

Woorank

WooRank is a great online SEO audit tool. It tests a wide range of attributes of your site, and gives you an overall grading to measure the performance (the average being 50). Much of the data comes from freely available sources, but the value of this service is the fact that there is a diverse range of metrics gathered in one place.

Woorank SEO audit score 65

Test out your site and then comb through each section of the results, making improvements where possible. Admittedly, many of the suggestions for improvement that you'll see are not trivial things - like get more followers on social media, etc.

Advanced SEO

Pretty much every other aspect of search optimization we have discussed focuses almost entirely on your own site. But, like most aspects of business, there is no need to work in a vacuum. With the appropriate SEO software and tools, it is easy to learn about your niche (i.e. competing websites) and how you fit into it.

By taking a step back and focusing on your niche as a whole (as opposed to only your own site) it is possible to learn from other webmasters, find out what other sites are doing to drive traffic, learn about potential links and referrals that can help improve your backlink profile, and much more.

While Google Search Console does a good job of monitoring our own sites, it can't really help us learn about other sites and what's happening in and around a broader niche. For that we need a professional SEO tool like SEMRush.

SEMRush

SEMRush offers in-depth SEO analytics for a huge range of reports and you can use it (on a limited basis) for free. Type in your domain below and check out some of their most useful reports, live.

Initially, you see an overview of your site's main SEO metrics on their dashboard.

SEMRush dashboard

I particularly like the positions reports that shows which keywords your site ranks for and also offers a useful traffic estimate.

SEMRush Keyword Positions

It is also possible to view day to day keyword position changes to see which new keywords you have won, those you have lost as well as whether or not your site has improved or declined in rankings.

SEMRush Keyword position changes

Another handy report shows off a selection of close competitors along with a summary of their keywords and traffic estimates.

SEMRush competitors

You can also ge a comprehensive overview of your site's backlinks - including what type they are, whether or not they are followed or nofollowed, their TLDs (Top Level Domains - i.e. .edu, .com, .gov, etc).

SEMRush backlinks report

For me, the real power of this tool is not being able to view your own stats (although that is pretty addictive, I'll admit), it's being able to analyze competing websites. Unlike Google's Search Console, you are not limited to viewing your own site in SEMRush. You can learn everything you need to know about any site and use that information to inform your search and marketing campaigns.

It is possible to look at specific competing sites, including their individual pages, and discover what keywords they are winning and where they are being linked from. This is super useful information because it helps you to build up a list of blogs and sites that might also help you compete better for the keywords you want to ultimately rank for.

Here's a list of some of the juicy search related data you can get your hands on:

  1. Discover organic competitors
  2. Monitor changes in keyword volumes and positions
  3. Uncover competitors' Ad strategies and budgets
  4. Conduct deep link analysis
  5. View referring domains' authority
  6. Backlink geolocations
  7. Video advertising research
  8. Find long tail keywords
  9. Domain vs. domain comparison
  10. Site audits
  11. Track social media campaigns
  12. Brand monitoring
  13. SEO ideas & recommendations
  14. Backlink audits
  15. ...

SEMRush essentially offers enterprise level SEO tools and reports that professional SEO services and agencies make use of in order to help their clients rank better. So learning how to use it yourself essentially cuts out the middleman and gives you the information and insights you would otherwise pay a service to do on your behalf.

There are so many aspects of the service to explore that we can't cover all of them here. However, they have plenty of great resources that will help you explore and understand their more powerful features:

Online Resources

SEO is an ever-changing, fast evolving industry, which makes it vital to keep learning. Here's a list of the best resources to check on regularly to keep yourself up-to-date with everything from news to strategies and insights and find help if necessary.

Algoroo

Algoroo
Algoroo tracks changes in the Google search results in order to monitor changes in the algorithm that may affect your site. While there are a number of these algorithms trackers, I find algoroo to be more accurate than the others more often than not.

It's worthwhile checking in occasionally in order to see if there are have been any major changes. However, SEMRush will also provide notifications of major ranking changes so it's easier to check there if you are using that tool.

Open Site Explorer

Moz open site explorer
Moz's Open site exploreris a link research tool that provides some pretty handy information on inbound links, recently discovered links and also offers a few useful metrics like Domain Authority (DA) and Page Authority,/em> (PA).

Both of these metrics can be thought of as a way of quantifying Google's trust in a page and domain. This can help you to understand which backlinks are likely to be more valuable and which might even be toxic or harmful.

BackLinko Blog

Backlinko SEO blog
Brian Dean's Backlinko blog is arguably the best resource for learning SEO. He writes really in-depth articles that are jam packed with really useful tips and tricks, strategies and insight and data that is often collected on a staggering scale.

The only downside, in my opinion, is that he tends to publish once or twice a month at most.

SERoundtable

SERoundtable SEO & SEM News
Barry Schwartz' Search Engine Roundtable is a news blog that keeps webmasters up to date with the latest goings on in the world of SEO, SEM, Google, Bing and other search related issues. He usually posts between three and five news pieces a day, and often leverages his network of contacts within Google to squeeze a bit of useful information out of the people in the know that might not otherwise filter out to the broader webmaster community.

Google Webmaster Blog

Google Webmaster Blog
Google's Webmaster Central Blog is worth keeping tabs on because they post information about new updates and technologies that they are releasing or putting emphasis on. Changes to their search guidelines and other SEO related documents and guidelines are also mentioned here.

Google Webmaster Forum

Google Webmaster Forum
Google's Webmaster Central Forum is an online resource that is open to anyone who needs a bit of human help. There are plenty of knowledgeable people hanging out there who might take a look at a specific issue you post and offer some advice.

Bear in mind that there are a few well-known trolls there too, so you don't always receive a warm welcome. I kind of understand this attitude from the people there since the vast majority of people who arrive there have essentially been caught spamming red-handed. This often means you will automatically be painted with the same brush. But, if you need help, this is still your best bet.

Google Penalties & Traffic Drops

Every second request for help I see from business owners wondering why Google has dropped their site from the search results (and killing their Web traffic) is how to fix the damage done by the SEO company they hired. It's not uncommon for genuine businesses to have to lay off staff or even shut their doors entirely as a result of a Google penalty cutting off a significant portion of their income (derived from organic search traffic).

Basically, this is what every webmaster dreads waking up to see.

SEMRush traffic chart showing traffic drop

A precipitous drop in Web traffic can be devastating, but it does happen so it's important to be prepared. Sometimes, if you're lucky, you'll see a recovery.

SEMRush traffic chart showing recovery in traffic

Early Warning Signs

There are often tell-tale signs of an impending penalty or traffic drop. These may be the result of other problems affecting your site in some way (i.e. you've been hacked and spammers are injecting bad content and links into your site). With something like 200 factors that go into Google's algorithm it's almost impossible to predict with any certainty when a site will experience a catastrophic drop in traffic.

Google tries to help Webmasters keep their sites clean and healthy but the fact of the matter is that you can't passively rely on them to notify you. It's important to be both security conscious an take a proactive approach to avoiding penalties.

1. Unusual Traffic Patterns

It seems to me that when a site has been flagged for abuse or something spammy, it's not penalized straight away.

Google seems to keep it ranking while it runs diagnostics to confirm their suspicions. This can mean that traffic stays unusually flat, or decreases instead of increasing, or comes from different places, etc.

2. Low Percentage of Indexed Pages

Check the health of your site's index using the site: operator in search.

If you have 1000 webpages, and Google only returns 500 results in the site index, then it's a good indication that it isn't happy with 50% of your content. Panda isn't going to be far behind.

3. Unexpected Keywords

Look at the primary keywords in the Search Console. Are there words there that shouldn't be there? Are some keywords way too common? Finding keywords that don't belong could be an indication you've been hacked. Removal from the index will follow soon after.

Alternatively, overusing keywords (keyword stuffing) can also lead to drop in trust and authority and a corresponding loss of organic traffic.

4. Non-existent, Missing or Incorrect Pages in the Index

Sometimes pages that don't exist appear in the index. This is a sure sign that something, usually of a technical nature, has gone wrong.

Make sure your CMS or Web platform is operating correctly and that META information like canonical URLs are set.

A favorite trick for webmasters of any experience level is to make a mistake in their .htaccess or robots.txt file, preventing the normal behavior and access of the googlebot.

5. Spikes or Drops in Backlinks

Google likes to allay fears that negative SEO attacks are common or work. But, the Internet is essentially lawless and anything goes.

Through no action of your own, thousands of low quality backlinks can appear or disappear overnight.

Although, before you assume you're innocent, make sure you never paid an SEO company to "boost your SEO". Chances are they built inbound spammy links that are now hurting you.

Since Penguin started penalizing spammy linking practices, an entire link extortion industry has sprung up.

Basically, nasty people use bots to post links to your site over thousands of poor quality domains. Your traffic crashes, and when you attempt to have those links removed, they extort payment.

6. Unusual Crawling Behavior

If Google stops crawling your site for an extended period the chances are something is up. If they suddenly start crawling like mad, it may also indicate a change is coming.

Don't worry excessively over crawling patterns though. Often there is an innocent explanation, such as an impending update or data refresh that is not necessarily targeting you in particular.

7. Spikes in 404s

While 404 errors aren't a problem in themselves, they may be an indicator that there is a problem. For example, if you delete a lot of thin, poor quality pages, Google will pick up on these and report the 404s. That's perfectly natural and no cause for concern.

But, if a developer accidentally deletes part of your database and half your site disappears that is a problem. If your server starts returning a 404 response code even if the pages are being served - that's also a problem.

If 404s are returned for long enough the affected pages will be de-indexed and traffic to them will die out. Not what you want if those pages are supposed to be there.

8. Swings in Bounce Rate

Another indication that traffic patterns may change is a volatile bounce rate. High quality traffic should have a stable, lower bounce rate than traffic arriving from unusual or unrelated search terms.

Sometimes spikes in foreign traffic occur and the bounce rate goes through the roof because visitors don't even speak the same language.

9. Warnings & Messages in Search Console

Ok, so this one is pretty obvious. If Google makes a special effort to reach out and inform you of a problem (i.e. your site has been hacked, or has potential security risks), it's important to deal with these issues as quickly as possible.

These signs and symptoms aside, if your site has lost traffic there are the steps you should take to analyze the problem (bearing in mind that some penalties, like Penguin, may last for years even after the issues that caused the problem have been resolved).

Manual or Algorithmic Penalty or Neither?

The first thing to do is determine if there is a penalty and whether or not the penalty is manual or algorithmic. This is easy to do because a manual penalty will be accompanied by a message in Google Search Console. Click on Search Traffic >> Manual Actions to see if there is a notification there.

Bear in mind,

A huge drop in search traffic does not necessarily imply a penalty.

Google regularly tweaks its algorithms, changes its focus, changes how it values sites and links, and hundreds of other little things that may affect your rankings drastically leaving you with little recourse but to continue trying to improve your site or give up. This is the cold hard truth.

Dealing with a manual penalty

You can learn more about manual actions at Google's webmaster support site.

In order to receive a manual penalty, someone from Google has looked at your site and decided that you've been breaking their rules. The only way to lift this penalty is to convince them you've done everything possible to clean things up. But, before you submit a reconsideration request, it's important to really work hard at fixing the problem. So, for example, if you've been hit with an 'unnatural links to your site' penalty, it means you have to:

  1. generate a list of all incoming links
  2. determine which ones are natural and which are toxic
  3. approach all the webmasters of all the sites with toxic backlinks and request they be removed
  4. document all your actions
  5. disavow any domains and links that can't be removed

Once all that is done, you can submit a reconsideration request, including all your documented efforts.

I should point out that even if you are successful and the manual action is removed, it may well be that Web traffic does not recover. If the spammy links were leading to higher traffic levels then removing them will return your site to its normal levels, which might be a lot lower than you expect.

Dealing with algorithmic penalties

Assuming there are no manual penalty notifications, it may be that you have an algorithmic penalty.

Many webmasters enjoy 5 - 10 years at the top of the results, only to be unceremoniously dumped overnight. This may or may not be a penalty. Newer, better content may have simply taken over. However, there are a few steps you can take to find out if your site was hit by a specific penalty...

1. Match drops in Web traffic to known algorithm updates

It's likely a site has been hit by an algorithmic penalty when a sharp drop in traffic coincides with a known algorithm update.

Use Google analytics (or whatever analytics software you have) to compare when traffic dropped to a list of known algorithm release dates. If the drop coincides with a documented release date, then you have the first clue about what went wrong.

The next step is the hardest...

2. Be honest about the quality of your site

While I believe much of the blame of the current state of the Web lies with Google, and more specifically their Penguin algorithm, the fact of the matter is they get things right far more often than they get it wrong.

In other words, a Panda penalty is, most likely, and indication of poor quality content on the site. Most webmasters appear to be unable to objectively assess the quality of their content. Most of the time the content quality is poor and Google has correctly dropped it from their results.

Read Google's Webmaster quality guidelines carefully to see how your site compares to their list of best practices.

3. Take action

Once you have an understanding of the major contributing factors, it is time to start dealing with them. Draw up a plan of action and work through it systematically - i.e. clean up links, rewrite or remove bad content, reduce ads, etc.

Once everything is spotless and clean, and the site and content are as good as you can make it... hope for the best. There is no mechanism to inform Google of the improvements. You have to wait for Google to recrawl, reindex, and refresh it's algorithms. This can take time.

Incidentally, if you've been hit by Penguin, the steps to fix it are the same as outlined above in the section entitled Dealing with a manual penalty, as both these cases deal with cleaning up spammy links. It's also worth noting that different penalties can take differing amounts of time to be lifted. A Panda penalty may last only a month because this algorithm is now run in real-time and acts as fast as the time taken to re-index your pages.

Like manual penalties, fixing the problems that led to an algorithmic penalty doesn't automatically mean that traffic levels will return to their original volumes. In fact, it may be that having a penalty lifted leads to no discernible increase in traffic at all - depending on what the reason for the penalty was in the first place.

Hopefully this overview on how to go about fixing penalties has given you some solid practical advice to follow, but also realistic expectations on the chances of a full recovery. Don't be disheartened if post penalty traffic volumes are low. At least you can create great, high quality content and hope to climb the rankings in the near future, free of any penalties (site-wide or partial) dragging your site down.

Tips & Tricks

For all the strategies and techniques we've covered in this guide, there are still plenty of great little tips and tricks that can, potentially, offer huge benefits for very little effort. This section will be regularly updated as we come across more helpful hints, so check back here regularly.

Go Local

If you are a site covering issues relevant to your neighborhood, city, or state then it makes a lot of sense to focus your keywords on that geographic region. A plumber in Manhattan, for example, probably doesn't wantto compete for a search term like "plumber" because while it might be a high traffic term there is almost no discernible intent behind it. Is the searcher looking for a plumber, or are they looking for a definition of the word plumber?

While local SEOmight seem like it is restricting the potential volumes of traffic, what it actually does is bring focus.

It is far more valuable to be on the first page of search results for localized searches, than on the second page for one popular, generic search term.

Some of the ways you can optimize for local search include:

  • Adding an accurate physical address to your contact page
  • Talk about local events
  • List on local websites & directories
  • Claim your Google business listing
  • Research local keywords and phrases

Use the Long Tail

Did you know that around 70% of all organic search traffic originates from the long tail. Long tail terms are any keywords or phrases that do not drive as much traffic as highly competitive, more popular search phrases, but they tend to convert better because they are more focused.

Long tail keywords account for 70% of all search traffic
Chart by ahrefs

For example, the phrase "Panda penalty" is pretty competitive. But, many people wonder whether content syndication might incur a Panda penalty too. The long tail phrase Panda penalty content syndication might not drive as much traffic, but it is more focused and this allows you to monetize the traffic generated from it much more effectively.

Instead of competing directly for highly competitive terms, why not give your content the best chance of dominating more long tail searches by including what I call "qualifiers". A qualifier is information about your content that also contributes to the focus and relevance of the content.

For example, the phrase "business ideas" is hyper competitive. But, "business ideas for rural Ohio" is a phrase that might still be up for grabs. In addition, the inclusion of the qualifier "rural Ohio" serves to focus the content better. It tells people more about what they can expect from that search result.

Implement Image SEO

Organic search traffic arising from Google image searches is increasing all the time. Search traffic from images is just as valid as search traffic coming from web searches - i.e. you can make just as much money from those visitors. It makes sense then to get ahead of the curve and ensure that your images are properly SEO'd.

Here's what Google looks at when it crawls and indexes images:

  • Image file name
  • ALT tag
  • TITLE tag
  • Context

As a result, you need to ensure that highly relevant phrases are used in the ALT, TITLE and filename of the image. The terms you include in the image optimization should be the same or similar to those used within the body of the article, so that Google understands that the image is closely related to the context in which it is shown.

Build Backlinks & Referrals with HARO

HARO (Help A Reporter Out) is a platform for journalists and other publishers to get expert commentary and insights from industry experts (i.e. you and me).

Help a Reporter Out

Anyone can sign up and ask questions that get emailed out in several categories three times a day. What makes this really useful is that not only does it provide an opportunity to show off your expertise and get an attribution link (not always guaranteed) back to your site, it also helps you build a network of relationships with the media.

Meeting lots of influential writers and bloggers via HARO is a great way to build up relevant, quality backlinks on an ongoing basis as you create mutually beneficial relationships with other people in your industry.

Remove "Junk" URL Space

Search engines, including Google search, only return a certain number of URL characters in their search results. It is important to squeeze as much SEO juice into every part of a search result, so why waste parts of the URL?

Unless you have a good reason for doing so, don't include www in your site's URL. Instead, edit your .htaccess file to redirect to the non www version. Those three characters could be used for valuable keywords and phrases.

Here's an example of how to redirect requests from www to non-www in .htaccess (Note that the rewrite rules must be placed after the RewriteEngine on command):

RewriteEngine on

RewriteCond %{HTTP_HOST} ^www.mydomain.com$ [NC]
RewriteRule ^(.*)$ http://mydomain.com/$1 [L,R=301]

Replace mydomain in the example above with the domain of your own site, and away you go.

In the hyper-competitive world of SEO, three characters can give you the edge you need to out-compete business rivals online.

Build on Existing Content

From an SEO perspective, I never treat a piece of content as ‘complete’. There is always room for improvement with updates, additional information, and various other things that can make it more useful to visitors. At the same time, once the content has been published it is important to monitor the keywords that Google ranks it for. Often this can help direct your efforts at improving the content and capturing better rankings for better keywords.

It is far better to invest time and effort into fewer high ranking pages than it is to constantly churn out once-off articles.

A good example comes from one of our forum answers about which are the best website builders for SEO. Initially this started off as a quick recap of an earlier article we had written about website builders. But, after watching the organic search traffic coming from Google we decided to update it… several times.

Now it boasts tables of data ranking website builders across a range of SEO related metrics, it talks about on-page features, content creation, links to additional SEO resources, how to get started creating a websites, what great tools there are to test their site’s SEO, example of fantastic SEO design, and more.

As the page has grown, so has the rankings it captures in Google – considerably faster than fresh content would.

Recover Link Juice with Search Console

Google's Search Console has a Crawl Errors dialog that appears under Crawl in the main menu. This shows a variety of URL errors that may (or may not) be part of the normal operation of your site.

Of real interest is the Not found tab that shows all the pages Google crawled that returned a 404 page not found error. These errors don't necessarily affect your site and are often intended (i.e. deleting out of date or irrelevant pages).

Search console errors

Click on any page listed on this page and you'll see a list of URLs that link to that page. If there is a link from a valuable website that is potentially driving Web traffic and PageRank then 301 redirect that page to a relevant equivalent page so that the link juice can boost an existing page and direct visitors to something they'll find useful.

Provide useful 404 pages

User experience is an integral part of SEO because Google wants to return helpful, user friendly sites in its results. From a visitors' perspective, there’s nothing more enraging than landing on a “page not found” error that becomes a dead-end. More often than they will simply hit the back button on their browser and look elsewhere.

Unhelpful 404 pages can drive valuable traffic away.

Instead, get 404 pages to do a search for the keywords in the URL, so that “page not found” errors return suggestions for related products and pages. For example, I have a complete beginner’s guide on how to make a website. Let’s assume that someone linking to that page missed out the final letter in the URL, leading to this broken link:

http://smepals.com/create-website/how-create-website-guide-complete-begi...

Instead of a basic 404 not found error, the user will see the correct article in the search results allowing them to go straight to what they’re after - providing a much better user experience.

Case Study: Content into Referrals for SEO

Referral traffic (traffic that originates from links to your site from other sites) is arguably one of the most stable and valuable of all types of traffic – provided the links you obtain are from high quality blogs and sites, and highly relevant. Not only will great backlinks send engaged visitors your way, they will also serve as a strong signal to Google that maybe, just maybe, it should be returning your site on page one for some high competition keywords.

Here's an example of how to go about it.

1. Find a Story

Is there a particular blogger you have been trying in vain to talk about your blog, or even a product in your store? How about a specific journalist who could potentially give you game changing coverage? Whoever you’re going after, there is something you can do that will almost compel them to talk about you - talk about them.

In particular, if you can find something noteworthy (read: newsworthy) that they have done – it could be anything that will show off their particular talents and work.

For example, I regularly share fantastic small business ideas to help aspiring entrepreneurs, but there are some fantastic startup ideas coming out of US entrepreneurial colleges all the time. Having a strong relationship with these important early phase entrepreneurial influencers would be extremely valuable for me.

I decided that sharing a list of the business ideas & startups would be a fantastic way to show off the creativity and innovation coming out of American colleges would make for a great story. The takeaway here is,

Put together a list of all the people, businesses, blogs, and industry influencers you would like to get referrals and recommendations from at some stage.

2. Create Fantastic Content

It’s not enough to reach out to influencers and ask for a backlink, or even ask them to look at something you’ve done (that doesn’t involve or interest them in any way). Everyone who is worth reaching out to is constantly bombarded with spammy outreach emails that ask for much but offer them nothing in return.

Do something that is interesting and engaging for an influencer before reaching out to them.

What’s the best way to do this? Create a fantastic piece of content that talks about them in a positive light.

I scoured all the winners from university business competitions and compiled a list of the top 10 new small business ideas from university entrepreneurs. In particular, this list highlighted the contribution that entrepreneurial and business colleges make in supporting and nurturing young business talent – and that is incredibly valuable for them.

But that value is locked away in your content unless you can get it out to the world.

3. Share & Promote

With an interesting story in hand it’s time to promote it to larger publishers who have a chance to generate some genuine buzz and interest.

Easier said than done, I’ll admit.

But with a bit of perseverance and some in-depth research into finding and reaching out to exactly the right people (who would be interested in your story) you will eventually hit paydirt.

The reason you want to media coverage is because it gives you credibility in the eyes of the target influencers who you really want to build a relationship with. In my case, I was able to get national media coverage on USAToday.

National media coverage on USA Today

With this bit of success, I was now ready to move to the final phase of the content into referrals for SEO workflow.

4. Share Success

Media coverage about one of your stories is great. It’s a backlink from a (hopefully) highly trusted and respected site, and is pretty much a PR victory within itself. Lots of companies pay PR firms a lot of money to get coverage like this.

But it’s not the end of our journey because the whole point of what we’re trying to accomplish is to build a relationship with targeted influencers in order to generate valuable referrals and backlinks from them. It’s these highly relevant, highly valuable backlinks from within your niche industry that will boost page rankings and search traffic volumes.

Instead, reach out to the influencer with news of the good work you have done that has benefited them greatly. Nothing rouses people out of their apathy towards unsolicited outreach emails than a message that goes something like this:

You just got national media coverage on XXXXX because of our story about how awesome you are.

You’ve got to admit that’s gonna work a lot better than:

I’d love you to link to my new article.

With such a compelling outreach message it’s not surprising that influencers would want to talk about your story, mention you in social media, write about you in their news feeds, or even link to your post in their content. This is exactly what happened in my campaign. Many top universities shared the story in their college news, like this mention from UCDavis:

Referrals & backlinks from valuable influencers, like UC Davis

Now I appreciate that this seems like a long and roundabout way to get backlinks and referrals. But remember that it’s not only about the links. It’s about the relationship. A good relationship is far more valuable than any one link because it can often lead to plenty of great links as well as plenty of great opportunities that might not otherwise present themselves.

From a single article I managed to get national news coverage, and links, referrals and recommendations from top US colleges that have been driving stacks of traffic from my primary target audience (aspiring entrepreneurs and online business owners) to my site. Plus, many of the marketing and communications people from those universities regularly reach out with more stories they’d like us to cover.

Case Study: Disrupting Google Search

Have you ever spent time wondering how established companies maintain their online dominance in Google search results, appear regularly in leading blogs and websites, and generally enjoy visibility and reach you can only dream of?

Industry experts spend plenty of time telling us to create fantastic content that people will want to share. That's great and all, but the only articles that seem to get shared around are precisely those produced by established industry players.

If you feel that despite publishing fantastic, useful, engaging, and informative content you simply aren't getting the desired returns, you're not alone. But, it's also not necessarily your fault either. As we'll see in a minute, the entire Internet ecosystem is lopsided and bias in favour of established and/or well connected players.

Fortunately there is a way to beat them at their own game - or at least call it a draw (better than getting beaten hands down).

Web traffic & success

When we talk about growth online, what we really mean is traffic. The more Web traffic you can generate, the more people are being exposed to what you're saying, and the more revenue and profits stream in.

It's all about traffic. Any business that wants to succeed online needs it.

Now, there are a few major sources of traffic, like:

  • Google search
  • Referrals
  • Social media

On the surface of things, these traffic channels might all seem unrelated and independent. But they're not. They are, in fact, tightly integrated with each other.

How all Web traffic is related

It's a mistake to think that we can pigeonhole each type of content and target each one individually. Sure, you can employ specific strategies for each channel, but losing sight of how they all relate to each other won't help anything.

Let's take a look at a quick example of how everything is connected behind the scenes:

  1. A renowned university decides to recommend establishedwebsite.com on their website (i.e. a referral)
  2. In addition to sending referral traffic to establishedwebsite, Google indexes this link and starts ranking it in their search results
  3. In addition to sending organic search traffic, human searchers discover establishedwebsite and start mentioning it on their blogs/sites
  4. In addition to sending more referral traffic to establishedwebsite, publishers promote their content (along with references to establishedwebsite) on social media
  5. In addition to sending social media traffic, Google picks up on these additional blog links and starts ranking establishedwebsite even higher in search, leading to even more referrals, traffic and social buzz

... and so it goes on.

Where does the status quo come in?

Referrals (i.e. backlinks) are what Google uses to establish whether a domain and its content are trustworthy. The more trust it has in domain the better chance it stands of ranking pages in search results.

The problem is that established cycles of referrals, mentions, and increased page rankings tends to persist and build up over time.

If you look at a top 10 roundup of anything across any of your industry's big sites, they tend to mention the same things.

Why?

Well, one reason is that those are the top 10 in their niche. But another, perhaps less obvious, reason is that most people don't dig too deeply. Instead, they search on Google. And here's where it gets interesting, because...

Where does Google get its information?

From links given by earlier referrals.

The net effect being that people recycle older content because their research on Google shows up results that are linked from older pages. Those sites become more and more 'established' because as more people research them on Google and write about them, they receive more backlinks, better rankings, and more traffic - leading to more referrals.

This cycle can perpetuate almost indefinitely.

"But how pronounced can this effect really be?", you might ask.

Well, it's probably worse than you think. A quick search on Google will highlight the issue quite nicely - you can search for anything you like, in any industry or niche that interests you. It took me all of 5 minutes to find an example, which likely means the problem is pretty widespread.

I did a search for best small business resources - something a student or aspiring entrepreneur might type into Google to kick off their research:

Google search for small business resources

The fourth result on this page gives you this:

Google SERP showing blank SCORE page

Why on earth would Google return a blank page containing a single link to another page right at the top of a really important search? It's obviously not a useful result despite the fact that there are probably millions of pretty good pages to choose from.

The answer is that Google trusts the SCORE website (to the extent that it doesn't even check if there's content on the page).

Why?

Because SCORE is a non-profit association supported by the U.S government. This means it automatically gets recommendations and links from a host of government and educational websites giving it (in Google's eyes) a backlink profile that is simply unbeatable.

In fact, many small business and state funded universities won't even consider linking out to sites that are not also state funded (because there are strict rules governing the relationships between state funded entities and private ones). This means that sites like SCORE (and many others) enjoy an unfair advantage over... well, everyone.

To the extent where Google would rather return a blank page from SCORE ahead of anything by 497 million other results it has in its index.

Please note: I'm not criticising SCORE - I'm sure it wasn't their intention to rank in Google search for this page. But, this does demonstrate how things can get skewed in favor of established sites.

Becoming a Disruptor

On virtually any type of platform - whether it be organic search results, social media networks, or online communities - the vast majority of traffic gets sucked up by a very small percentage of established players.

For example, about 92% of all organic search traffic goes to page one results in Google. If you're not on page one, you're not going to get enough traffic to survive.

The challenge is therefore how to get to the point where you are established enough to squeeze into that small percentage of influencers who enjoy a hugely disproportionate share of the traffic. Well, either that or try to change human psychology.

It's not easy. But it is possible.

1. Create a Quality Platform

As much as I bought into Google's line that creating fantastic content will ultimately lead to success, I'm afraid this is only partly true. The whole truth is that fantastic content has to be marketed and promoted aggressively in order to overcome bias in the system - as demonstrated by Google's search results above.

Don't be misled into pouring all your energy into content. Instead:

Create small quantities of the extremely high quality content and spend far more time spreading it around and having it linked and referred to.

But, don't do this aimlessly either. It's important to know where to direct your efforts for maximum return.

2. Analyze the Source of Established Players' Influence

There are plenty of SEO tools that will analyze the links of competing domains - see our advanced SEO section for more on how to do this with SEMRush. Usually you can use them on a limited or trial basis for free, but paid accounts can be quite pricey.

I tend to use Google - since Google SERPs are ultimately what it's all about.

There are a few tricks you can use to narrow down search results to find the type of information you're after. For example, let's say we wanted to find resource pages on university sites that link out to external websites. Something like this would be a reasonable start:

site:.edu resources useful links

The important thing here is the use of the site: operator that filters Google's results to only educational (.edu) sites. There's a reference page on how use Google search operators as well as punctuation and symbols for more advanced searches.

It can take a bit of trial and error to learn how to surface the best, most relevant results. But, once you do, build a huge list of all the most important domains that link out to established competitors - including all pertinent information like content info, and specific pages that might be appropriate for link building.

3. Develop an Insertion Strategy

Knowing where there are links to be had, and actually convincing those webmasters to insert a link onto their site/pages are completely different things. And, each organization may have different reasons for linking out, or have different requirements, personality traits, and so on.

This is where things get tricky. There is absolutely no point in spamming everyone on the list because all you'll do is ruin your reputation with all the most important potential partners in the industry.

Instead, dedicate some time and effort to ensure that you have a site and content worth linking to, for everyone on the list. This may mean producing additional content to fill in any gaps in your own coverage, or, more importantly, in their coverage (so that they can plug their own gaps with your content).

4. Reach Out, Convince & Convert

There's no set way to establish a connection with someone online. Each person is different. However, there is one fundamental prerequisite for success - namely:

Offer something of value BEFORE asking for something in return.

It's tempting to reach out and ask for a link. But remember, almost anyone who is in charge of a high value page has been approached by thousands of people asking for the same thing. 99% of those people have little in the way of quality content to offer, or worse, are downright spammers.

Against this backdrop, it's not realistic to expect people to take much time over an outright request for a backlink. Instead look for somewhere you can add value. It could be anything. Maybe they accept guest articles, maybe they have broken links you can alert them to, maybe they'd appreciate an unsolicited review of something they have recently published.

Here's a few articles that can flesh out some outreach best practices:

5. Analyze, Refine & Repeat

As with any new business/marketing venture it's important to constantly asses your performance in order to identify weak points and make improvements to increase conversion rates.

Links tend to come slowly at first. People are hesitant to connect with others they haven't heard of. But, hopefully, you will find that there at least one or two little successes.

The thing about success is that, no matter how humble it is to start with, you can leverage it to build more success.

Once at least one site has linked out to you, use it. Tell the next person that others are linking to you. Demonstrate that other influencers trust you. Endorsements from others players (especially ones that are known and trusted) are extremely powerful.

If you find that people simply aren't willing to recommend your site or content at all, then it may be time to do some cold, hard soul-searching. Is what you have created of sufficient quality to compete with the best in that niche? It's important to be honest with yourself because while it's easy to fool yourself, it isn't possible to fool everyone else.

Changing the Status Quo

Over time, as more and more links accumulate, you should notice the following, positive patterns of change:

  1. Improved backlink profile from high authority, trusted sites leads to improved rankings in Google search
  2. Improved rankings in Google search lead to more traffic, which in turn leads to an increase in backlinks from other bloggers/sites
  3. In addition, having a presence on a wide range of trusted domains leads to additional backlinks as more people discover your site
  4. More and more backlinks are generated without requiring active effort on your part and backlink growth becomes (almost) self-sustaining

With any luck, your domain will eventually become part of the status quo. You haven't changed the system, but it has been disrupted to make a bit of space for you. It's not easy, but the low hanging fruit is all gone and nowadays you have to promote and market aggressively to secure a place in the sun.

Common SEO Questions

While this guide has hopefully given you plenty of high impact SEO insights, advice and strategies to work on, no doubt you still have questions. Feel free to ask a question in the comments below and we'll answer it and add it to this section to help others too.

Is it Possible to Recover from a Penguin Penalty?

It is not possible to recover all the traffic you had before the penalty was applied - at least not straight away. Let me explain...

Let's say that your site had 10 000 backlinks that were helping drive your content up the search result rankings. Because of this you had lots of organic search traffic - for argument's sake, let's say 5 000 visits per day. Next, Google's Penguin algorithm came along and slapped your site with a penalty due to spammy backlinks it believes were intended to manipulate the page rankings.

At this point the traffic coming from Google search may be next to nothing.

In theory, recovering from Penguin is easy. Go find all the bad links that are pointing to your site (usually as a result of hiring a cheap SEO company) and have them removed.

Google Search Console provides a big list of external links pointing to your site in the Links to Your Site report, under Search Traffic .

Remove links by approaching the webmasters or domains in charge and request them to be removed. If it is not possible to get those links removed, disavow them (but only as a last resort). Once the bad links have been dealt with you have to wait for Google to re-index everything and then re-run the Penguin algorithm.

This can take many years.

Assuming you have done everything right and your site's backlink profile is no longer spammy, contains high quality editorial backlinks, and meets all applicable guidelines, you can expect the penalty to be lifted.

Having a Penguin penalty lifted won't bring all your Web traffic back.

It will bring some of it back because your domain is no longer penalized, but the backlinks that were used to artificially inflate page rankings are gone. In other words, once the penalty is lifted your site will rank naturally - as it would have done if those links weren't boosting it in the first place.

This effect is the reason that many webmasters believe that a full recovery from Penguin is impossible. When in fact, a recovery from Penguin is possible, but a full recovery of traffic is not.

If you think that there are too many toxic backlinks pointing to your domain, it may well be easier to migrate any quality content to a new domain and start again. It may takes months of hard work to clean up backlinks, months again to reindex, and months again 'til Penguin refreshes.

Moving to a fresh domain will lift the penalty immediately but only if you don't 301 redirect anything. If you 301 redirect old pages to the new one, the penalty will also be transmitted to the new domain.

Your two options are as follows:

  1. Work hard to remove the Penguin penalty and potentially wait a long time for recovery
  2. Move content to a new domain without 301 redirects, but loose any quality backlinks and other positive factors associated with the old domain

Which option you choose is up to you, but if you've been hit by Penguin there is a lot of hard work ahead either way.

Why Does Google Hate my Site?

Over time, for any number of reasons, your site might simply have fallen out of favour - even if you did nothing wrong. Remember that for any type of competitive term there are likely going to be hundreds of millions of potential results, so to expect to rank in the top ten out of a hundred million is not really realistic for tens of millions of people.

The point here is that it is really hard to compete for valuable search terms, and just because Google likes you one day is no guarantee that it will like you the next. It may be that your site is actually ranking where it should now, whereas previously it was ranking higher than normal (for whatever reason).

This is a common issue among webmasters who mistakenly believe that their high search traffic volumes are something they have a right to. In fact, Google can do anything it likes.

The answer to why Google hates you may therefore be nothing more than the fact that your site doesn't stand out amongst the millions of other sites sufficiently to make it to the top page. I know this is probably a really unsatisfying answer because it means that there is no tangible course of action to take to "fix" things (from your perspective).

That's the risk you take when creating an online business that relies on Google search traffic as a primary source of revenue. It's generally best to treat organic search traffic as a bonus and build up other strong, sustainable and alternative sources of Web traffic.

Why is it so Hard to Get Decent Backlinks?

Google uses natural backlinks as a way to determine the "quality of a web page" so that it can determine where in the search engine results it will appear. They even tell you to get people to talk about your content (blog posts) so that as many inbound links are generated as possible.

Given the value of inbound links, how come it is so hard to get people to link back to your content - even if you offer links back to them?

People who have high authority websites, popular blogs, etc have often had to struggle for several years to build up their popularity. They understand the value of a high quality backlink, so they don't often give them away for free. More often than not, the most widely accepted way to get backlinks from a high quality blog or site is to write a great guest article that they can publish (with attribution - a backlink to your own blog/site).

The problem is that writing guest blogs is time consuming and has been abused as an SEO practice to the extent that Google really frowns on the practice of guest blogging for links.

Ultimately, your time is probably best served creating great content that regularly talks about or promotes influential people in your niche. This strategy works well because you are giving them exposure first (as opposed to asking them to spend time reading your content before deciding to publish or not).

Will my Blog Content Always Rank Badly in Google Search Results?

Webpage ranking in Google are pretty dynamic. They move up and down on a daily basis. Content rankings in search can improve, but it takes time.

When Google talks about your ranking in Webmaster Tools, it is referring to the average rank. This is because the rankings swop around all the time as new content is added, algorithms are updated and refreshed, and so on.

There will always be substantial fluctuations in traffic originating from organic search in Google. There's not much point in worrying about minor changes to page rank, search result rankings and traffic on a day to day timescale.

However, there are signs that major changes have occurred when significant changes in PageRank, rankings in search results, and traffic occur. These changes can often be substantial and instantaneous - for example, when an algorithm refresh penalizes your site.

If you notice a big change in traffic volumes via Google analytics over the course of a day then it is important to investigate the causes - regardless of whether the changes are positive or negative.

Understanding why changes have occurred help you to avoid future mistakes, or capitalize on helpful developments.

In general, provided you continually add content and improve your SEO (Search Engine Optimization), organic search traffic volumes should increase steadily with time.

If all goes well, this gradual increase should also be punctuated by periods of quick growth in traffic as your site establishes trust and authority that is recognized by Google's search ranking algorithms. This is when older content that didn't rank well initially, may start ranking higher in the SERPs (Search Engine Results Pages).

Found This Useful?

Share it on your own site using this quick link (and tell us about it in the comments so we can read your article too).

Does Google suck, or do you? Pic by Dean Hochman

Would you love to get completely irrelevant traffic from Google's organic search results? How about absolutely no traffic at all? Or zombie traffic that doesn't do anything?

Get rid of that pesky, high converting traffic that floods in via relevant keywords and drives revenue, builds authority and trust, and generally makes life easier. I can even show you how to have ad placeholder pages outrank your own site for a search on your own domain.