Require wp-load.php from subdomain redirects to 404 page

Question

I face a very complicated situation, which I will try my best to describe as accurately as possible.

• First of all my website is behind Cloudflare.
• I wrote an external .php script that loads wp-load.php, fetches some data from an API, processes the data, and then stores the processed data in WP to some custom post types.
• As execution of the script usually takes longer than Cloudflare's 100 seconds time-out, in order to evade the 524 time-out, I used what's called a grey-clouded subdomain, as suggested everywhere on the internet.

The only difference was that I had to use an absolute path to my wp-load.php file, like so:

 require_once '/home/cityp2020/public_html/wp-load.php';

Everything was working fine, until I used W3 Total Cache plugin to further enhance the website's performance (it's a very heavy and crowdy site). So, as soon as I setup W3 Total Cache to use memcached method, all hell broke loose regarding my custom script...

Now, whenever I try to execute the script by visiting its URL sub.domain.com/script.php the site redirects to https://domain.com/script.php (the subdomain part is gone) and of course I get a 404 error... What's more, I get the following error in my subdomain's error log:

[04-Apr-2021 19:21:25 UTC] PHP Warning: Cannot modify header information - headers already sent by (output started at /home/cityp2020/sub.domain.com/script.php:13) in /home/cityp2020/public_html/wp-includes/pluggable.php on line 1299

where line 1299 reads:

$x_redirect_by = apply_filters( 'x_redirect_by', $x_redirect_by, $status, $location );
if ( is_string( $x_redirect_by ) ) {
    header( "X-Redirect-By: $x_redirect_by" );
}

header( "Location: $location", true, $status ); // <- This is line 1299 of course

return true;

So I wonder, in what way does memcached affect my site, making WP get confused, and what exactly fires that redirect from subdomain to mainsite?

After honestly a lot of research the closest solution I found was to cancel the redirect by adding add_filter( 'x_redirect_by', '__return_false' ); to my functions.php, but it didn't change anything...

Any help would be greatly appreciated. This one's puzzling me for a week already and I'm getting desperate...

If you need any further clarification, please let me know. TIA.

Answer 1

OMG, after a week of searching and searching, I was finally able to debug this - almost coincidentally...

First I had the idea to write a function instead of the __return_false in add_filter( 'x_redirect_by', '__return_false' ); to log and find what is calling the redirect.

(Un)fortunately, I made a typo (forgot a param actually in the function's declaration), and found some interesting info in the error log (see image below)...

So after googling the term W3TC\PgCache_Plugin->redirect_on_foreign_domain I finally found this post by the plugin's author, suggesting to someone who was facing a similar issue to add an alias domain in the plugin's settings... In my case it wasn't an alias domain; it was a subdomain... Nevertheless, I gave it a try...

Et voila, the redirect is magically gone and the script and everything is working like a charm! :D