Why Does Site or Server Load Increase
There's only one good reason why the load on your hosting server starts increasing–the rest are bad. I'll look at how and why they all happen.
I was often contacted by site owners who had a problem with high server load. This common condition is first noticed when an owner gets a warning message from their hosting company. Such messages can be precursors to the blocking of the site, and it can happen to almost any site owner or webmaster.
This article covers the different reasons why the load on a site or server might be increasing, and what can be done about it.
Webmasters usually find out about excess load from their hosters. Hosting companies regulate and control resource usage for each hosting plan. If they run operating systems such as CloudLinux OS, they can set numbers for allowable CPU and memory usage for each account, using figures such as the percentage of total CPU power, or elapsed CPU time.
Examples of charts used to detect patterns in CPU (left) and I/O (right).
The hoster usually wants to distribute CPU resources evenly across all client sites on a server. (If someone's hosting account uses 90% of the system's resources, that leaves only 10% for other clients.) In these cases, the account owner will get a warning, notifying them of their overuse and a reminder of their quotas. Systematic violations may cause the account to be blocked, to avoid service interruptions for other users on the same server.
Let's be clear. This is not an attempt to upgrade clients to more expensive tariffs, as some webmasters tend to think. It is hosting companies doing their best to make sure that all clients can run their businesses smoothly, without the interruptions caused by a few renegade sites.
I'll now go through what can cause excessive server load patterns, considering them from the perspective of external versus internal factors.
External factors
This means anything that does not depend on the configuration of the hosting platform, the operation of scripts or the administration of the site. In other words, external requests to the site from various services, bots or other sites. There are multiple possibilities.
Intensive site queries consume a lot of CPU resources, especially if they are using the POST method. Scanning a site with an external scanner can
If an attacker finds a vulnerability or a variant for hacking while scanning, they are more likely to download malicious
When scanning stops, the site will continue to work as normal, and system load will return to its normal value. That is, until the next scan.
2.3. Mass registration of users or bulk sending of spam via unprotected feedback forms
The hosting platform CPU gets loaded and starts spamming. The hosting company disconnects the email service or blocks the site completely.
Almost all simple defense mechanisms are easily circumvented by modern bots. You must use something more effective such as Google's reCAPTCHA or the CAPTCHA built into the Imunify360 firewall.
4. Indexing a site by search botsSometimes, with a sufficiently large search index (e.g. when many pages are accessed by a Google search index), the re-indexing process can take a long time and create a heavy load on the server. Even if there are only a dozen pages on your site, you can also face a similar problem, such as, for example, the site was hacked and there are 50,000 pages of search results on it. It can also happen when the search index was spammed by a competitor that took advantage of errors in the work of the scripts of your site. There are many possibilities.
5. Grabbing and scraping content (site cloning)Owners of unique content should be concerned with the problem of their content being downloaded from their site (so-called 'scraping and grabbing'). This can be done with special bots that load the pages of the site, copy all text and pictures, then create clone sites. If the scanning of your site is continuous, and there are many pages on the site, a significant load will be suffered by the hosting server.
6. Importing data (feeds, downloading of e-commerce data)Often, e-commerce resources use a mechanism for exchanging data with external services. For example, a list of items can be downloaded from online stores, and news sites can regularly export news feeds, etc. If the content is dynamic, each such request will create a high load on the server.
7. Using pictures or links to your siteOne of the less obvious causes of server load is through linking to or using an image on a much more popular site. This is the so-called
Another situation is when someone posts a picture on a blog (for example, in the comments box). This is downloaded by each visitor and creates a load on your hosting server. Serious problems occur if the picture is generated through the action of a script (for example, when being scaled with
Sites that contain vulnerabilities are often used by hackers to conduct attacks on other resources. Sometimes, an attacker doesn't even need to hack the site. Owners of the latest versions of WordPress, for example, may see this problem when the xmlrpc.php file is compromised. In this case, your site will act as an intermediate link, and the work of the site's scripts creates a heavy load on the server.
9. DDoS attackIf a DDoS (Distributed Denial of Service) attack happens, the site will fail unless special technical approaches are used, such as using a CDN or a DDoS traffic-cleaning service.
Detecting a DDoS attack is difficult. Because of the large load on the server, each hoster may act differently, offering DDoS attack protection services, moving the site to another server, or completely blocking and disconnecting the site. To protect against such an attack, you must have a prearranged solution, so that when a problem arises, it can be quickly solved.
10. Increase inThe best reason for the growth of server load is a natural, organic increase in visitor numbers. This tells you it's time to scale-up the site and think about optimizing your scripts for increased traffic.
To find the cause of such an externally-caused load, you can analyze the logs of the web server, with special applications or a combination of commands in the SSH console.
A good list of the things to start looking for is:
- top 20 requests that use POST in the results of the analysis;
- top 20 requests by the method GET/HEAD;
- top 20 IP addresses by
number of hits; - top 20 referring pages by
number of hits.
Together, these will reveal the source and type of traffic, the entry points to the site, and the scripts that are called most often, as most likely they are causing a high load.
To reduce the load for external attacks or intensive requests, in most cases it is enough to enable HTTP flood protection (for example, a classic "splash screen"). Or you can connect the site to DDoS traffic cleaning services that will block dangerous or especially active requests, skipping legitimate ones. In addition, static content, such as pictures, scripts
You can try to connect a caching plugin in the CMS, or a caching service on the hosting, but in the case of external factors affecting the load, this may not help
Internal Factors
Internal factors include everything that affects site performance at the level of scripts and settings, i.e., anything that can be controlled by the webmaster or site owner.
- Suboptimal scripts and a growing database
It is possible that a visit to a start page, or just showing search results, can seriously load a server. This might be because of a badly designed web application architecture, or bad programming by inexperienced developers. The growth in the size of any underlying database with each site update, for example, because of an increase in e-commerce store items, will slow it down more and more, increasing the load on the hosting server. Individual pages of a site, each page with many content blocks, can send dozens of requests to the database, repeatedly performing the same operations on files, and sometimes even blocking the work of other elements of the site. This problem is common in online stores running old versions of Joomla with the VirtueMart plugin. - Infecting a site with viruses
Hacking and infecting a site with malicious scripts is a common cause of load growth. It is caused by virus activity, itself the result of malicious code fragments inserted into legitimate scripts. The load time and operation of processes suffer, and connections between scripts to external resources can time-out. - External legitimate connections to third-party resources
Few system administrators remember to take into account the load created by connections to external sources of information. Examples of these are widgets, weather and currency rate tickers, and news feeds. Data downloaded from elsewhere is rarely cached, so every time the page is opened, a script connects and downloads content from another server.If, for some reason, the external source stops responding, the download speed of the main site can be affected. - Errors in the operation of scripts
When running scripts, errors can occur that are invisible tovisitors, but are visible inweb server or PHP logs. If the site has high visitor numbers, or there are many errors, the load on the server can increase. Errors appear when the site is switched to a more recent version of PHP, one incompatible with existing scripts. Conflicts can arise between the new CMS core and old versions of the plugins, especially when only some, not all, site components are updated.
To analyze a high-load problem caused by internal factors, it is necessary to scan the site for malicious code. If none is detected, script profiling can be performed using the
High load caused by malicious code activity should be rectified with a site
If the cause of problems is the website architecture, the site must be optimized with the help of an experienced web developer. An alternative solution is to install a caching plugin, which can reduce the consumption of CPU resources (i.e. the load on hosting) and speed-up the site.
Conclusion
There's one more property of processor load I would like to mention, and that is its duration.