27Feb

WordPress website page not loading

27 Feb 2017 TMS

Q: Our wordpress website hosted on Linux + Apache + MySQL. Here the issue is page not loading at all. Have did restart the apache server. Then it worked well for few seconds. Again the page stopped loading. All of my clients have seen server unreachable error. There was no configuration change been done in wordpress site. All of us sudden it happened. How to fix it quickly and make my website available to users?

Cause

Start debugging by go through the apache access logs. The location of log vary depends upon Linux distribution. Search for log file using this command.

#find / -name access.log

From access log found suspicious activity. The server has getting same kind of POST request from same IP. Also 5-10 request per second. This is abnormal and server starving for resources to process more request. These are the sample log entries.

191.96.249.54 - - [19/Feb/2017:07:52:32 +0000] "POST /xmlrpc.php HTTP/1.0" 200 370
191.96.249.54 - - [19/Feb/2017:07:52:37 +0000] "POST /xmlrpc.php HTTP/1.0" 200 370

From apache error log, can be seen that PHP server process getting failed due to server overload.

[Sun Feb 19 21:49:54.039906 2017] [mpm_event:error] [pid 8236:tid 140484269891888] AH00484: server reached MaxRequestWorkers setting, consider raising the MaxRequestWorkers setting

[Sun Feb 19 21:49:55.756567 2017] [pagespeed:warn] [pid 20908:tid 140483430867184] [mod_pagespeed 1.9.32.14-0 @20118] Fetch timed out: http://test.com/wp-content/plugins/google-captcha/css/gglcptch.css?ver=1.26 (connecting to:x.x.x.x) (1) waiting for 50 ms

If you see above symptoms, then your server under DDoS and Brute force attack by accessing xmlrpc.php file.

Solution

Follow these two steps to come out of page not loading issue. This makes your website available immediately. Once get your breath back work on preventing permanently.

  1. Using OS or Network firewall block the source IP address. The source IP can be get it from Apache access logs (first column). It will stop receiving any further request from that particular malicious user. Restart the apache service to drop existing connection. Further new connection will be blocked by firewall. I did this, immediately server resumed working normally.

If the server hosted in AWS, it is very simple. Navigate to NACL of VPC. Place the entry at top of line i.e with least value of rule number.

AWS NACL block IP

2. Disable access to xmlrpc.php file through httpd.conf.

Add the below lines between the <VirtualHost> tags

<VirtualHost>
.....
    <files xmlrpc.php>
      order allow,deny
      deny from all
    </files>
</VirtualHost>

About permanent fix, there were many articles already available in internet for blocking xmlrpc attacks. Have listed few of the links below.

  1. https://www.digitalocean.com/community/tutorials/how-to-protect-wordpress-from-xml-rpc-attacks-on-ubuntu-14-04
  2. https://www.saotn.org/huge-increase-wordpress-xmlrpc-php-post-requests/
  3. https://wordpress.org/plugins/wp-fail2ban/

I feel “fail2ban” should work well. Since it can dynamically identify and block suspicious IP address.

To make our life easier, have created simple bash script to monitor and report the xmlrpc attack events in near real time. Get the script from here.

Was the information useful? Please leave your comments below.