Bad bots can cause some real damage to your website’s performance and your business overall. By now you would have good understanding of what a bot is. The difference between good and bad bots as well as general idea about the impact on your website. If you have not had chance to look into our previous article about introduction to bots, you can check it out here.
Here are a few things you need to consider to tackle bad bots.
#1 Monitor overall website performance
Have you noticed that your overall website performance has declined?
A few ways you can tell include your website would take longer to load. In some cases, it may not even load at all.
What could be the possible reason you might be thinking?
It is possible that your competitor is trying to crush your website or business before it even takes off. It may be that a digital marketing agency or SEO training institute is showing their students on how to evaluate a competitor’s website.
You must be thinking what this all has got to do with you?
Some of the common and cheap tactics companies and individuals will use include send fake traffic, or pay someone to create as many clicks as possible to have your website run out of resources. Look out for another article to get detailed idea on what are resources and how to prevent them from reaching maximum capacity.
#2 Website Speed issues caused by fake bots
It could be that some or all pages on your website are loading slow or not loading at all for the following reasons.
- Your website is not designed by a professional
- The website does not have god foundation or platform (Designed by newbee, not properly scripted)
- The website has a basic hosting plan that runs out of resources, i.e. cpu usage, ram, memory, storage or bandwidth.
- You have got too many plugins either they are not compatible with your version of website or they are just chewing up your website speed by running cron jobs. Learn more about cron jobs in another article.
- Server crash by using maximum resources (Learn more about server and resources in next article)
- More downtime means users can’t access information or make purchase on your website
Your website will be more and more offline than online. The visitors will not be able to access the necessary information. This means loss of your business before your can start to make profit.
#3 Competitor may use these tactics (Who would attack my website a hacker or competitor)
Competitors whether online or offline will always have the grudge against their competition. They will then delve into paying other individuals and companies to ruin your website performance and business altogether. However sometimes very rarely you may come across healthy competitor. They will not only guide you on how to improve but also play huge part in your online and business success. Simply because they believe in healthy competition.
#4 Bad bots send fake traffic that causes bounce rate (Check in Google Analytics)
You will need Google Analytics tools to monitor and analyse the data. As a professional or website owner you should be able to study your visitor behaviour to grow your business. Google algorithms are good to exclude fake traffic. However, should only be considered the only option to prevent other issues that come along. Constant bearing of fake traffic starts to stack up your server resources and eventually crash your website for few minutes to few hours.
#5 How to identify bad bots
We have really taken care of quite a few bots that can cause issues for your website. Bad bots have got some common names and you can download a list of 500+ bots. You will need to either ask a professional to add the list to your robots.txt file or do it yourself. However, you can identify from their behaviour in Google Analytics as well as through raw log of your server. (Do not attempt to access the file or make any amends, if you are beginner). You should familiarise yourself with some basics about robots file to avoid negative impact on your website ranking.
#6 Google Analytics
A free tool or snippet of code generates by Google to help you and Google at the same time learn about your visitor behaviour. You will need to have a Google account and you can create one easily for free. Once you have the account simply go to your web browser (Google Chrome, Firefox or internet explorer) and type the following link as it is https://analytics.google.com. Google will ask you to log in. Once you have logged in follow the steps outlined below.
Note: Steps outlined below are for users with intermediate to professional level knowledge. Do not tamper with files in cPanel, if you are a beginner. Instead follow the instructions after step 12.
Step 1) Go to admin panel
Step 2) Create a new property if you don’t see one already (A property is simply put your website name i.e. https://koaas.com)
Step 3) Generate code snippet
Step 4) Go to your cPanel on your hosting providers website.
Step 5) Look for your theme files
Step 6) Find header.html or header.php file
Step 7) Paste the code generated in Google Analytics in header file right before “ </head> “ tag.
Step 8) Test if the analytics are working
Step 9) Open your favourite browser and go to your website.
Step 10) Keep the analytics page open on the other tab to see if you can see 1 live visitor being yourself.
Step 11) If you can see live visitor then you have setup all correctly, if not repeat steps 1-10.
Step 12) All Done.
You can simply install a Google Analytics plugin and connect to your Google Analytics account by clicking yes to authorize the plugin to access data and personal information. Once installed you will be able to see the results on your WordPress Dashboard or under analytics plugin.
#7 Look for fake comments
Always look for fake comments, you can recognize them fairly easily. A few examples would include things like “Live Casinos”, “Website link or Website name in the name section instead of persons name”, “A generic comment”, “Repetitive comments with different names”
#8 Look for fake membership names
Have you noticed a sudden hike in lots of users signing up? Don’t be fooled by the number of users. Always look for the sign-up information carefully. Set up to allow the users to activate manually by your website administrator or you if you are managing yourself. Most of the times the provided data i.e. name, email address, contact details would seem generic and not validated through verification link sent to users in email after signing up.
#9 Need SEO professional to handle some delicate tasks
The following tasks are quite delicate and must be handled under supervision of a professional. If you are a beginner try to avoid until you have achieved better understanding of these tasks in depth. They can potentially make and break your website and in turn your business could be in jeopardy.
Update robots.txt file through your cPanel to stop bad bots
Before updating the robots file make sure that you understand it fully. Ask for professional guidance and do plenty research as this will require a bit of technical knowledge. If you do it properly it can do wonders for your website otherwise your website could be in a deep hole.
Google Re-captcha/Aksmit for signup, password recovery page, comments,
Google re-captcha plugin is the most easily available option for free. Once installed you will see a box appearing similar to the one in image below.
You may have noticed yourself that almost every website has this option to prevent companies using bots to cause harm to other website owners. You should use it for the following pages to prevent spam. These pages include signup, login, password recovery, comments in your articles and feedback or contact us pages.
Aksmit is used similarly, however does not have free version like Google. You should look into the benefits that are being offered by both plugins. Once compared based on your own judgement choose the one that suits you best.
Feedback forms are good way to get your visitors input about anything that they find suspicious on your website. There might be something that you are not aware of and could be causing your website to slow down. i.e. a broken link, image or poorly formatted text (Websites functions differently depending on the browser or device you are using), bad comment (if you have auto approved option enabled)
You must find the best security plugin that you possible can. There are plenty available and most of them would offer some basic protection. You can easily get by with those options if you are just starting up a website/blog. However, if you are in for serious business this is one area you should invest in some money. This will provide a greater piece of mind knowing that there is something protecting your website when you are asleep or away. You can easily find plugins for very minimal monthly cost.
Raw access log on your server
Access the raw data log file from server and read it carefully. You may be able to apply some filters and look for common occurrences. Most of the bots would normally have the word bot or crawler in the end, middle or beginning. They are generally part of hyperlink as shown in example below.
#10 Overall use of plugins (Uninstall any unused plugins)
Uninstall any unused plugins to free some space on the server as well as helps speed up your website. There are quite a few plugins that use automated cron jobs every minute, hour, daily, weekly and so on. Understanding the purpose of cron jobs can not only enhance your website performance but also chew up lesser resources. It means your website will be up and running more often.
Warning: Do not attempt to amend cron jobs unless you are professional or have sound understanding of their functionality in depth.
There are fair few things we have mentioned in this article for you to be able to tackle bad bots ruining your website and business. Having said that do not limit yourself to these practices only. You should put every possible effort to go beyond to achieve success for your business.
Tell us in your comments, if you have come across a similar situation and if you were able to eradicate the problem or at least minimise the impact. We would love to hear your feedback about this article.