After nearly twenty years online, I’m very happy to report that the early-days attitude of caring and sharing and helping complete strangers with their tech issues via online discussions and forums has not ended.
Not even close.
Besides an amazing weekend at WordCamp Vancouver 2012 and BuddyCamp Vancouver, with volunteer experts coming from around the globe to share their knowledge and expertise and do a little team hacking to improve both WordPress and BuddyPress, I had the pleasure of stumbling across a very helpful post by an anonymous stranger who called him- or herself Tezgno that solved a thorny issue I’d been picking at for over a month.
All of a sudden, I could not access one of my accounts at my webhost. I could not get my email, access webmail, get into my website adminstration panels, log in to my CPanel, nor, to my astonishment, even view any of my websites hosted on that one particular server. Worse, everyone else on my network couldn’t access their email etc provided via this one server, either! No problem with any of the other accounts/servers I have with that webhost, just the one. No problem with any other websites that had nothing to do with me. The device didn’t matter: smartphone, tablet, laptop, desktop, Windows-based, Mac-based…nothing worked.
Horrors. How do I look after my clients’ sites? How do I see my email? How about everyone else? What do they do? It’s my job to solve it.
WHAT I TRIED:
Start with the end of the line: my webhost. Since I have no other problems with getting onto the internet, and not even any problems with my other accounts, there must be a problem at my webhost’s end. Three or four hours of support chats online and phone support calls later, we finally figured out that everything appeared to be fine with Bluehost, especially since everything was accessible via other internet access points. Must be my internet service provider (ISP).
Two, three, four, I don’t remember anymore, calls to the support department at Shaw, my ISP, and test after test followed by reboot after reboot, plus problem-solving ideas one after another, and we finally determined that if we went through the WiFi system, we had a problem. If I hard-wired by computer to the internet via an ethernet cable, my computer had no issues gaining access. That doesn’t work for a whole bunch of people on the network, and it doesn’t even work for me as I have multiple devices and don’t want to be chained to my desktop just to read my afternoon email! Hmm… Must be my Apple Airport Extreme Base Station (AEBS), my WiFi router of choice.
Except that didn’t make sense, either. Again, test after test, it passed with flying colours. Everything was working perfectly. No problems accessing anything at all on the internet; only the stuff that mattered the most! Talked to each of Bluehost and Shaw again, pointing out this important flaw in their theories, and started getting the brush-off. They didn’t know the answer any more than I did. “Blame the AEBS” seemed the easy (and expensive to me) answer. For some reason, I didn’t buy it.
The other people on this network who were being affected were very patient with me while I struggled to find an answer. However, after more than a month of no resolution despite my hours and hours of trying, they were tired of using our workarounds and were starting to get (understandably) frustrated. When was I going to solve this, they wondered? I wish I knew… Seemingly endless conversations with support techs, both online and by (international long distance) telephone, and I was no further ahead than when this all started.
WHAT I TRIED NEXT:
In the afterglow of a great immersion in the world of people who code because they love to do it and then share it with the world because they feel like it, I decided to dig harder into the world of online support forums. This time I decided to try the “blame the AEBS” approach that everyone seemed to favour. I typed “Reset Apple Extreme” into Google, and followed all the steps to restore it to the factory default setting. Again. After setting up the network – also again – from scratch, the issue was still there. But another searcher’s question caught my eye as I travelled around a few Mac/Airport discussions. Someone asked a question about the possibility of their AEBS blocking their own IP address from accessing their own server, and just their own IP address trying to get to just their own server.
That sounds a lot like my point of pain! No answer there, but a new tack to try.
Back to Google. Searching landed me on “Can Airport Extreme block IP addresses?” Still not the right direction, but I sense I might be onto something. At last.
Now Google won’t let me in. Eh? OK. Over to Bing. I found this: One certain site will not load! Help!! That led to this list of related questions. Wow, who knew how many people were having variations of the same issue as me? Big hint came through as I scanned these various posts: lots of people were having similar issues, regardless of which webhost they were using and regardless of which brand of router they were using! OK. Now what? Hmm… What is my own IP address? And is it different whether I’m accessing the net via my AEBS WiFi versus hardwired to my modem versus hopping on via cell service on my iPad?
USEFUL LINK NUMBER ONE:
Did you know you can check your own IP address? (This is how Penelope Garcia in Criminal Minds tracks down the location of bad guys doing dastardly deeds on the internet, by the way. Now, we, too, can be just like her!) Just go to WhatIsMyIPAddress.com, and violà, there is your IP address! Try it with a different internet connection, like I did, and you’ll get a different IP address.
Armed with this confirmed evidence that only devices connected through my WiFi’s IP address couldn’t connect to my webhost – and only one account at my webhost – I kept hunting. I found this thread. It’s huge. Pages and pages of posts. I decided to read each one, to see if there were any more clues that might help, even though the thread was not marked as “resolved.” Of course, there were the obligatory Apple Haters on there who have nothing useful to say, skim past them and keep going. Stop at the occasional post where someone says they tossed their AEBS and got a LinkSys router, problem solved. Hmm. That’s not a Hater, that’s a Solution Seeker. In spite of what I’d read earlier about this issue not being confined to one brand of router or one particular webhost, I found myself (in a weak moment) considering following suit… until a few posts later when someone else said they had done exactly that, only to have it the issue return again after only a couple of months! Oh, my. Then someone else said they had tried switching to a NetGear router instead, also to have the problem come back a little while later.
USEFUL LINK NUMBER TWO:
Then I came across this amazing post, on Page 7, by a user named Tezgno (if you click on the link, scroll down that page until you see it).
For those of you who are having this problem, I thought that I would chime in explain what is going on here. First, and foremost, I’m a network engineer who has seen this issue countless times and, surprisingly, the problem is not what a lot of people think it is (although it may seem like it is the case). I’ll try to explain this in laymans terms.
First, the issue isn’t necessarily the AEBS, although it is apart of the problem. What is actually happening here is that you are being blocked by the site, not the other way around. Specifically, your IP address is being “banned” by the site in question (in some cases, temporarily, in others, more permanently). Issues like these are hard to diagnose because most people troubleshoot the issue by unplugging their router, plugging their computer straight into the modem, and then realizing that the issue is gone. Likewise, many people take their routers (in this case, AEBS) to Apple, plug it in, and it starts to work as well. So, let’s break this into two parts and address the problem first and why it is happening.
As stated above, the issue is happening because of your IP address being banned by the site you are trying to visit. The reason why you are being banned is a combination of a couple of things on both the sites’ side and your side, creating a “perfect storm” if you will. From the sites side of the house, around 2008, many linux-based servers started receiving updates that included security products that would detect rapid or repeated connection attempts (products such as fail2ban and csf). When these security products detect rapid connection attempts or repeated failures, they either temporarily or permanently ban the IP address in question. By the end of 2009, many popular web hosting services and control panel companies included these products as well (control panel companies such as CPanel). The problem is that many people do not know that these products are installed on their servers, let alone that they are running (larger entities do, but the smaller shops and individual site owners rarely do). From the user side of the house, around 2008, many browsers started to include HTML5 capabilities through optional software (such as Google Gears). By 2009, most browsers in some way, shape, or form, started to support HTML5 with its ability to do intelligent caching and DB storage in the browser itself. And, this is where we have the problem. Modern day browsers, in an attempt to make our browsing experience better, tend to open multiple connections to servers so that we can browse the site. For sites that have multimedia, photos, tons of links, etc, (such as blogs, video or photo sharing sites, etc) this results in your browser opening hundreds of connections at once. The problem with this is that if the server you are accessing has systems such as csf installed and you attempt to open a ton of threads to the page, the result is that the software mistakenly thinks that you are a security problem and bans your IP address from accessing the site. For most software (such as CSF), the ban is temporary, but increases in time the more you are banned. For other software, the ban could be permanent.
Why Changing Routers or Plugging in Directly Works:
The reason why changing routers, plugging in your computer directly to your modem, or even taking in your router back to Apple works is because the minute you do any of those things, your IP address changes. This is because most ISPs link the IP address that you are given to your MAC address. Once the MAC address changes (you changing routers, plugging in directly, etc), your ISP will give you a new IP address. When you plug back in your original router, your ISP will return the IP address that you previously had (in most cases. Sometimes, it will give you a completely new address again).
Why this issue is more common with the AEBS than other routers:
In general, this issue is more common with the AEBS (along with cheaper routers) is because the AEBS (along with cheaper routers) generally do not contain QoS (quality of service) controls. Now, for those of you who are technical, you may note that QoS’s purpose isn’t about preventing you from being banned. And, that is true. For those of you who are not technical, QoS is a process by which your internet bandwidth is regulated by your router so that higher priority items (such as a VoIP phone or a video game) can have more bandwidth available to them when multiple devices are trying to access the Internet at the same time. The reasons why routers with QoS do not experience this issue as often (if ever) is because one of the things that they do is rate and thread limiting. Because your throughput rate and threads are distributed more evenly (rather than a free for all), chances are you will not experience the issue since products like csf work by detecting uneven or rapid connections.
So, in conclusion, 99% of the time, you are being blocked on the other end. To solve the issue, you generally will have to contact the site owner to have your IP address unblocked. If you are using AEBS, the best thing to do would be to disconnect your AEBS from your internet connection for 24 hours and then plug it back in (most ISPs will assign you a new IP if you are offline for 24 hours or more). Also, try to limit the number of tabs that you open to the same site. If you are being blocked by your own site (and, your site uses CPanel, which it likely does), go into your CPanel settings (WHM settings) (from another connection, of course), go to CSF, remove the ban on your IP address, and add it to the exclusion list.
CALL ME “STUBBORN”: I TRIED SUPPORT CHAT AT MY WEBHOST AGAIN:
So, I hopped into the queue at my webhost’s support chat service again, and discovered I had a long wait for the 18 people ahead of me to run through their issues. To pass the time without getting so distracted I’d miss seeing my turn come up, I continued to scroll through the rest of the discussion, partly to see if anyone reported any relief from following Tezgno’s advice.
I was grateful for the long wait, for once; reading the entire conversations that Tezgno had with a number of participants in this thread was quite enlightening. He was helpful, polite, consistent, and persistent in helping a bunch of strangers. I learned that CSF stands for “ConfigServer & Firewall.” I didn’t figure out what WHM stood for, but I did try going in to my server and looking for /csf in my /etc/ folder, as he suggested. Nothing that I could see. Maybe the support tech could see it. This post, by “BiggusDikkus” (where do people come up with these names???), gave me hope that support could dig deeper than me, and find the issue:
In this case, Vodahost (Texas) was the host, and my router could not access my domains, my emails and other domains on that same hosting account.
So, the problem was definitely the host/server. But the problem seems to have been embedded deeper than Cpanel, and not correctable by user.
Adam, my unfortunate support tech, came on, and read through all the information from Tezgno that I pasted to him. He looked for CSF too, but it turned out he couldn’t look any deeper than I could. While he tried his thing at his end, I found this post by a fellow Bluehost customer, Vision, posted on Page 9 [sic]:
Tezgno, on page 7.
This was the Solution for me. Thank You very much for your post. I initially talked to bluehost.com and let them know that my airport extreme was being banned from connecting to a specific domian. They told me that it was not banned. After chasing my tail around, dealing with tech support with ISP, and running my own ping tests. I took this post back to the domain host (bluehost) and they put me through to tier 3 support. I was quickly removed from the ban list..
Your post was extremely helpfull, and gave me the infomation that i needed to go back to bluehost and get to the root of the problem.
I have submitted a support ticket with them with hopes that this will save many other from running out to replace an airport extreme that isn’t broken.
CALL ME STUBBORN AGAIN: I SUBMITTED A SUPPORT TICKET:
After Adam tried everything he could at his level of access, spurred on by the success experienced by Vision, he finally reached the end of his options, and suggested I submit a support ticket to higher-level technicians. In my issue description, I spared no words. I copy-pasted every helpful thing on the topic I could find, mostly from Tezgno but also outlining everything I had already tried with Bluehost Chat Support, my ISP’s support, and Apple. Of course, I included Vision’s proof of success with Tier 3 Support from Bluehost.
And then I waited.
That was yesterday. Usually these Bluehost guys are really quick, but I guess this was a little trickier than your average question. No response overnight. No response all morning.
IP BLACKLIST PROBLEM FINALLY SOLVED!
Suddenly, this afternoon, this wonderful message popped onto my screen, newly arrived from the Bluehost Support Team:
Seems the IP itself was in fact blacklisted, I’ve just gotten a report back from the admins that it has in fact been cleared. You shouldn’t have any further problems. Let us know however if you do. If so, it’s best to reply to this very email so as to keep all the info in one place and not let the issue get more confusing than it needs be.
I hopped onto the internet via the affected IP address (through my AEBS WiFi), and everything suddenly worked again, just like it had for years! Yippee!
Tezgno was right.
Better yet, he spent all that time explaining all this network-level information to a bunch of strangers on an anonymous discussion list more than a year ago! Who knows where in the world he is? Who knows if he is really a she? Who knows why he decided to share his hard-won knowledge with us? And how in the world could I ever thank him? Were it not for him, I don’t think I could have found the correct language to explain the issue effectively enough to get someone to look deep enough into the system to find the blacklist that was blocking my network from reaching my server. I was ready to do one of two things — or maybe both: get another new router, and/or switch webhosts. Neither option sounded very appealing, but they were next on the list of ideas. I had to solve this issue.
GRATITUDE AND PAYING IT FORWARD:
I’m so grateful for this person’s help. How do I pay this forward? All I can think of is to share my long, sad story with a happy ending online, where I hope it will prove to be helpful to others who are experiencing the same painful struggle!
Yes, blog posts are supposed to be 500 words long, and this is six times that, but this issue and my gratitude to a complete stranger is worth it. I just hope it helps you, too (seeing as you read this long, you must be struggling with the same thing!).
I ask only one thing: Once you solve your issue, too, pay it forward, please.