EMPOWERING ORDINARY MARKETERS
TO BUILD EXTRAORDINARY LINKS.

Introducing LinkMiner – Broken Link Checker on Steroids

by Jon Cooper
166 Flares 166 Flares ×

We’ve created a more sophisticated broken link checker for SEOs that allows you to make quicker insights on the links you’re checking. You can download the Chrome extension for free here:

The rest of this post will contain documentation on what it does & how to use it.

This is only the initial version, and I already have a list of additional improvements that I will be making, which are listed at the end of this post.

Before going any further, I want to give a HUGE shout out to Paul Livingstone & Check My Links, the extension this was built off of. Under the MIT License, he’s allowed anyone to improve upon the original extension, which is what we’ve done.

Why This Extension Is Noteworthy

Checking a page for broken links on-the-go is not new. The first to make it practical & efficient was Check My Links, an extension that my agency has used heavily up until this point. It’s simple: click a button, and see which links are broken, just like the below.

Screenshot 2015-03-10 22.30.52

What it’s missing is what we do next. We want to take this link and find out more about it, which you can now do in one click with LinkMiner, as shown below.

Screenshot 2015-03-10 22.33.26

You can also display link data on the page itself, right next to the status code of the link, as shown below.

Screenshot 2015-03-10 22.34.40

You can even export all of that information about those links as a CSV, instantly.

Screenshot 2015-03-10 22.37.30

There are a few other features, and there are far more improvements to come, but this extension will take your page link analysis to the next level. With that said, let’s get into the details!

Documentation

Before we get into anything, let’s first get setup.

Setup

First, visit the following link in the Chrome Store to add the extension to your browser:

Once added, you will now be able to locate it on your Extensions page, as shown below.

Screenshot 2015-03-10 22.46.49

Before clicking on the ‘Options’ link and playing around with the settings, let’s first run the extension on a test page to authenticate Ahrefs, which is what the link data is being pulled from. If you don’t have an Ahrefs account, then you will be unable to use some of the main additional features. You can sign up for an account here.

Note: we plan on integrating this with Majestic in the future as well (so you can choose), given we don’t run into any development roadblocks.

So to start, visit a test page like this: http://pointblankseo.com/link-building-resources

Then hit the extension’s icon, which should now be showing up in your extension’s list, as shown below.

00000939

Ignore the number in red for now; simply click on the icon, and the page will start to be checked for broken links, like the below.

YqPP2TgYJ8

As you can see, it scans the page & checks if any links are no longer working, and if they aren’t, they’ll be highlighted in red. If it’s a 404, it’s in dark red. If it’s any other status code, it’s in light red (as they can be hit or miss in terms of whether they’re actually broken).

Next, click on the status code that is displayed next to the broken link, which will trigger a popup, as shown below.

joanzLyxpx

Next, click ‘Authorize’, and you will be taken to Ahrefs.com to login & allow the extension to authenticate, as shown below.

Screenshot 2015-03-10 23.03.22

Once you authenticate, you will be given this dummy message. You’ll then return to the page you were checking to see if it’s now working.

Screenshot 2015-03-10 23.04.08

Going back to the page, click on the status code again, and you’ll now see working data in the extension.

qqjNlKwHIA

From here, let’s now talk about the Options/Settings that allow you to unlock additional features with a few clicks.

Options

You can locate the Options page in two ways: via the gear icon on the graphic that shows up when the extension is activated on a page, or via a link on the extension’s listing on your browser’s Extensions page. Both are shown below.

00000940

00000941

Once clicked, you will be shown the following page.

Screenshot 2015-03-10 23.15.46

Let’s run through what each option is asking:

Status code. Display the status code next to working links, so you can also view more data on those links too.

Ahrefs metrics. Display Ahrefs link data on the page next to each link as you would with the status code. You can display Backlinks, Referring Domains, or both. You can also only ask the extension to generate this data if it’s an external link.

Caution: this can eat up a lot of your OpenApp API calls, so be cautious!

Top links. Display the top 5 links to a given URL you’re investigating at the bottom of the popup window, below the link & social data that is displayed.

Number displayed in front of the extension’s icon. Have the number be either the total links on the page, or just the number of external links.

A few tactical notes about the above features:

Ahrefs metrics. If you enable these, when you export the links from a page (will address later), this data that you’re pulling will be exported with it.

Screenshot 2015-03-11 01.26.12

Top Links. Next to the URL & page title of the top 5 links is the Ahrefs Rank metric. This is only used to figure out which 5 we should be pulling (the 5 highest); personally, I don’t find this metric valuable for anything else other than this feature.

Number displayed on icon. If you select ‘External Links’, you can quickly tell if the page you’re viewing links out, which is handy for quickly assessing link opportunities, regardless if you’re checking it for broken links.

With that out of the way, let’s talk about a few things you’ll notice when using the extension.

Additional Features

Here is a list of the additional features that aren’t addressed on the Options page:

Exporting. You can export the list of links on the page, as well as their status codes, whether they’re internal or external, and they’re Ahrefs data (if setting is enabled) as a CSV file. To do this, click the icon pointed out below.

00000942

Ahrefs View More links. This idea isn’t original; we stole it from ShareMetric (highly recommend). You can quickly view more information on different aspects of that URL/Domain, courtesy of the dashboard on Ahrefs.com, which you’d have an account on if you got this far.

Screenshot 2015-03-10 23.29.15

With that said, let’s see if I can answer as many of your questions in the below list of FAQs.

Frequently Asked Questions

Your worst nightmare came true and somehow it’s not working for everyone. Which brand of pitchfork would you recommend? If this actually happens despite all my testing, then I promise I’ll be working through it non-stop until it’s solved.

Why does this permissions popup make me so uncomfortable? Trust me, I’m with you on this one. This just seems to be permissions needed by my developer to make the extension work properly. Turns out, all extensions ask for permissions like these.

Screenshot 2015-03-10 23.40.28

Where are you getting the core functionality? We’re using the same code that was used to make Check My Links. We’ll be further improving on this code in future updates to prevent false positives & improve detection of pages that should be marked as broken.

Why does it only integrate through Ahrefs? We simply just haven’t gotten to Majestic yet, and we plan to seriously look into it for version 2.0. We are aware that Open Site Explorer might be worth including as well, but I can’t promise we’ll get to it.

What are my Ahrefs API credit limitations? As of this writing, here are the limits for each plan:

  • Lite – 50k rows
  • Standard – 300k rows
  • Plus – 1M rows
  • Premium – 5M rows

Please double check this with the source. In terms of what those numbers mean to you:

  • 1 row for each link checked when displaying Ahrefs metrics on the page itself
  • 2 rows for each link that you investigate by clicking its status code

Therefore, if you’re looking to display Ahrefs metrics on the page itself, note the number in red on your icon in case you happened to be on a page that was absolutely loaded with links, as it would eat up a lot of API calls.

What exactly are those numbers being pulled from Ahrefs? We’re looking at links to the URL (not the domain or prefix).

The data on the popup window is loading slowly. Therefore, your extension must be broken. I guess it’s not a question, but if this happens on occasion, give it time. I’m still not completely sure whose fault this is when it happens (on your side, Ahrefs side, or our side), and if it’s fixable, but if it is, we’ll get to it in future updates.

So I bet you’ll charge for this in the future, right? Nope. I have no reason to. This is more so for the (gobs?) of free PR all you guys will hopefully give us. The only thing I foresee being paid is if we integrate with other services that are paid. So i.e. you might need to pay for them (like Ahrefs) in order to use those additional features.

Are you open to feedback? VERY. You’d be a massive help simply by letting me know any bugs or any feature requests you have. I’d pay you in links if I could.

List of Feature Requests

To hold myself accountable, I’m going to list feature requests right here on the post, and I’ll even throw in a shout out to who ever gave it (if it’s serious enough to be considered; i.e. not: “Make it play Friday by Rebecca Black whenever I click the button”).

So here’s the list I’ve already put together myself:

  • Be able to prevent false positives (issue duplicated from Check My Links)
  • Be able to detect if URL is on parked domain (should be flagged)
  • Be able to detect if URL is on domain with malware (should be flagged)
  • Integrate with Majestic and/or Open Site Explorer
  • Allow you to view social data without having to integrate with Ahrefs
  • Allow you to view social data on page-load, just like you can with Ahrefs metrics
  • Allow you to export only external links
  • When calculating the number of external links to show in front of the icon, don’t count links to a certain set of domains (i.e. facebook.com, twitter.com, etc.) so you don’t get thrown off by social profile links for assessing if they link out.
  • Show # of external/internal links next to each Google search result
  • Option to display domain or prefix (not URL) level metrics
  • Add a ‘View More’ link to the Wayback Machine

I can’t promise I’ll add all of the above, but I’ll be looking into the possibility of each.

With that said, here are the feature requests & bugs found from you guys. You can send me feature requests via the Comments on this post, or by tweeting at me @PointBlankSEO.

  • Ability to automatically scan page for broken links and display a red notification on top if there is one present. (via @anthonydnelson)
  • An option to only check external links (via Baptiste Place)
  • The icon in Chrome can be a bit slow to respond (via Alex H)
  • 301s are showing as 405 or 408 Status Codes in the export
  • Allow authorization of Ahrefs on the Settings page as well (confuses new users who don’t read this guide) (via Patrick Hathaway)
  • Add the source URL & info at the top of each export (via Patrick Hathaway)
  • Let us know exactly what those permissions are for & be able for them to be verified (via Darren Shaw)
  • Make a version for Firefox (via Matthew)
  • Make a color blind-friendly version (via Jordan)

Regardless, I’d love to hear your thoughts in the comments below, whether you think the extension is cool or if your opinion is just flat out wrong (just kidding!). Overall though, the more critical you are, the better!

This post was written by...

Jon Cooper – who has written 128 posts on Point Blank SEO.

Jon Cooper is a link builder based out of Gainesville, FL. For more information on him and Point Blank SEO, visit the about page. Follow him on Twitter @PointBlankSEO.

NEED LINKS?
Relax - I send out free emails full of
cutting edge link building tips.
38 Comments
  1. Alex H says:

    I was using Check My Links but this beats it with ease. The ability to export links is useful.

    Some feedback:
    When I click the icon in Chrome it can be a bit slow to respond
    301s are showing as 405 or 408 Status Codes in the export

  2. Benjamin says:

    Jon this is awesome!

    Thanks for the animated gifs, they are really helpful.

    Just gave it a test drive and it is so helpful!

    Thanks again

  3. marc says:

    wow. i’m kicking myself that i didnt think of this 🙂

    something to think about for future versions. you should pull a thumbnail screenshot of the broken page because a lot of times these tools say a link is a 404 but it’s really not. So you have to click through to verify but a thumbnail would be helpful to skip that step as well.

    cheers,
    marc mcdermott

    • Jon Cooper says:

      Thanks Marc! Really like that idea as a way around false positives. We’ve got a few ideas for making sure they don’t get marked as red in the first place, but if we can’t crack that nut, this is a solid workaround. Really appreciate the tip.

  4. This new extension rocks! I love being able to see the backlinks from 404 pages without leaving the page.

    There are soooo many steps to SEO and link building, eliminating a few steps helps us all!

    Thanks, Jon

  5. Matthew says:

    This sounds incredible. Do you plan on making the link checker available for Firefox users?

    • Jon Cooper says:

      Hi Matthew! That would be awesome, wouldn’t it? I’ve got to look into how much of an option it is to take a chrome extension and code it in Firefox, but I’ve got a hunch it won’t be easy. Unfortunately might be out of our scope 🙁

  6. Gopal says:

    Hi Jon,

    It was good experience to use this tools, Compare to all the other software for find broken links.
    But I agree with alex’s point, little bit slow to response in Google Chrome.

    But overall its good.
    Thanks

  7. HenryV says:

    Nice one. This will be really useful.

    Thanks

  8. Jon, great work. Off to test this baby out now.

    Edmund

  9. Ranjit says:

    Can you plz let me know, which type of links its consider as a broken?? what are the criteria or header responce code you have set… thx 🙂

  10. Hammad says:

    Hey jon thanks for sharing this extension, but it’s working slow like a tortoise on my PC, I’m prevously using “Check my links” chrome extension and it’s faster than any other and let me tell you one thing that this LinkMiner seems a clone of “Check my links” in terms of design and interface. Have you used “Check my links”?

  11. I have been using Check My Links also, but I will give LinkMiner a go.
    Thanks.

  12. Amazing improvement over Check My Links. Was using it and ahrefs so it will definitely be a go 🙂

    thanks Jon !

  13. Grind says:

    Awesome job. Now how about a non chrome version? 😉

  14. Paul Livingstone says:

    Glad you found the code useful. Thanks for the attribution too!

    • Jon Cooper says:

      The legend! Thanks Paul for your support. Can’t thank you enough for CML, and the fact you put the MIT license on the code. You deserve all the credit for this.

  15. Jay says:

    Hi, this tool is way better than other tools for checking links. Just wondering if this extension will be available for firefox users?

  16. Simeon says:

    Hi, i installed and test this app with my site, and after that i uninstall it. But now i see my site blocked for 1-2 sec when i click any button on it and it rendering slowly only this site. When try with IE all is fine. Reinstall of Chrome not helped me. Can you help me ?

  17. Hi Jon,

    Usually I use “Check My Links” extension of Chrome to find out the 404 pages on websites. I think you should have to check this as well. Also I am installing your extension as well for future use.

  18. Boni Satani says:

    Hey Jon,

    Here is one issue:
    got to this link – https://blog.kissmetrics.com/50-resources-for-getting-the-most-out-of-google-analytics/

    Run the tool & check the broken links.

    Tool shows 4 links as 404, however after opening them, I learned that they are working well.

    Great, if you can address this , if not it still the awesomest 404 tool.

    Cheers!
    Boni

  19. JValenzuela says:

    Hey Jon,

    First of all, thank you for this post. It’s a very clear review of this extension, but, do you think this extension can be better than Xenu for checking bad links?

    Somet

  20. sunil says:

    hi jon,
    You must be happy to receive so many thanks, take mine as well. Well broken link checking has to be a priority and it is a must to do thing to avoid complications. For sure your quality article will inspire new webmaster’s to use extension.

    Thanks
    sunil

  21. Riya Arora says:

    Thanks Jon, for such a great information. This broken link checking tool is really helpful for me.

  22. Adam says:

    This is a really neat and wonderful article. I would like to thank the author for sharing about this extension. The broken link problem is solved now.

  23. Bruno Pinna says:

    Hi Jon Cooper i understand your words but you think that the broken links will influence in rank?

  24. Harekrishna says:

    Nice article Jon,
    Thanks for sharing the process.
    It was nice tool, Compare to other software for find broken links.

Introducing LinkMiner – Broken Link Checker on Steroids - Point Blank SEO
166 Flares Twitter 0 Facebook 39 Google+ 77 LinkedIn 50 166 Flares ×