Odd Issue Discovered via Logs

[Guest]

I'm coming across an odd issue with Featurific that only seems to affect certain crawlers, not normal browsers, which I discovered via my monitoring service Pingdom.

Every ten minutes, Pingdom returns a 500 internal server on my home page where Featurifc is, yet when I visit my site the page loads fine. I've noticed 500 internal errors were also being returned for another monitoring service and for a couple of other crawlers as well.

The only correlation I could find is that it only occurs when it's set to show the most recent posts. Also, if I set the update in Featurific to update every 60 minutes instead, then the 500 errors only show up every hour. If I set it to update every minute, then the 500 errors show up every minute, but only with Pingdom (and a few crawlers).

Normally I wouldn't mind but since it seems to affect certain feed crawlers as well I'm worried some of my content isn't being properly updated as a result.

I'm running WP 2.7.1 latest version of Featurific on a Litespeed server if that helps.

Regards,

Lincoln

[Guest]

Update to previous post:

After much testing the above issue seemed to be indicative of a larger problem. Apparently the plugin has issues when it is set to show either the most recent posts or most commented posts. If I try to set it to either and hit save, the entire page vanishes except for a line that says "Options Saved." However when this occurs the data.xml file is not automatically generated. If I choose it to show the most popular posts according to wordpress.com stats and hit save, then the page refreshes normally. The internal errors are being caused by a a failure to regenerate the data.xml successfully when set to show recent posts or most commented posts.

I've deactivated every plugin and yet the issue still occurs, including two out of memory errors that show every now and then when I attempt to adjust the settings or reactivate the plugin:

Fatal error: Allowed memory size of 67108864 bytes exhausted (tried to allocate 64 bytes) in /home/xxxxx/public_html/wp-includes/wp-db.php on line 814

Fatal error: Out of memory (allocated 48234496) (tried to allocate 78 bytes) in /home/xxxx/public_html/wp-includes/functions.php on line 2913


I've tried raising the PHP memory limit all the way to 512M and these errors still occur.

Not sure what else to do, but the only way the plugin can work without issue is by setting it only to show popular posts. Any help in resolving this would be appreciated.

Lincoln

[rinogo]

Hi Lincoln,

Thanks for your thorough bug reports! You've certainly done a lot of homework on this issue.

The 500 errors you're getting occur on your main page, correct? (Not on an admin page) And they're full blown HTTP 500 errors (i.e. no content is returned - just an ugly 500 Server Error page?)

The "Options saved." issue that you report is very strange indeed, however, this would explain the 500 Server Errors upon XML regeneration. If you've looked at the code, you might have (correctly) deduced that Featurific is failing on line 964:

PHP Code:

        //Force XML generation
        featurific_do_cron(true); 


The challenge that faces us now is finding out the reason why the call to featurific_do_cron() is failing. One possible way to track down the error message is to perform the steps outlined by me (rinogo) in this message:

http://featurific.com/support/thread-11.html

I'm guessing that it's related to the memory errors that you've been receiving. Can you post the URL of your blog, as well as a screenshot (or saved web archive) of your Featurific options page? (Feel free to PM them to me if you're not comfortable posting them here).

It's particularly strange that you only experience this problem in certain scenarios (e.g. when certain crawlers visit your site). I'm not entirely sure what the problem is, but hopefully we can get to the bottom of it soon.

Talk to you soon,
-Rich

P.S. If you create an account on this forum, you can subscribe to this thread and be notified every time I respond to one of your posts. Just thought it might help!

[Lincoln]

Created an account since it seems like I'm working full time debugging anyway.

Yes the 500 internals only occur on the main page, and oddly enough only get triggered by crawlers and monitoring services like Pingdom.

I've done the suggestion recommended in the other thread, which is how I was able to discover the out of memory errors. These are the only errors that show, but since my server disables error logs it's not clear if they're the only errors that occur.

I've discovered another clue though. Apparently these errors don't occur if I leave Featurific alone for a while. I've set it to regenerate the data.xml every 12 hours now, and so far no errors have been logged after 14 hours. if I constantly update my settings or keep hitting save repeatedly the out of memory errors show up almost immediately. I think what's happening is that the memory footprint is not getting flushed, so every time it regenerates it takes up more and more memory until it hits the limit and an out of memory issue occurs. This explains why the first few times my data.xml file was updated (after every ten minutes) there were no issues, and then the 500 errors would just start popping up maybe an hour or so later.

This all occurs behind the scenes, had I not been using a monitoring service I never would have discovered the issue, and ironically when I post a new entry, it correctly picks up the latest post once a successful regeneration is made, so others may be experiencing the same issue and just haven't realized it. Ultimately I think this is a memory leak issue of some sort.

[rinogo]

Very strange. With the frequency set to 12 hours, have you successfully eliminated the problem?

If not, can you post the URL of your blog, as well as a screenshot (or saved web archive) of your Featurific options page? (Feel free to PM them to me if you're not comfortable posting them here).

-Rich

[Lincoln]

No 500 errors since then! I did have a few 503 errors but I'm not sure if they're related. I'll have to keep monitoring. I did notice something else in the logs which kept showing the following:

2009-04-14 22:17:58.352 [INFO] Auto Index is disabled for [/home/xxxx/public_html/wp-content/plugins/featurific-for-wordpress/], access denied

I have no idea if this is related to the 500 internal errors but I thought it was worth mentioning. I had to change my htaccess file to enable auto-index so the errors would go away. So far so good.

[Lincoln]

Update: The 500 internal errors are still occurring even after regeneration updates had been set to 24 hours apart. It also failed to update the gallery with my latest post as well. I'm just about ready to rip this plugin out, stomp all over it, and set it on fire. Ay yi yi....

The url to my site is http://www.habitationofjustice.com

A screenshot of the settings have been attached: as well.

[Lincoln]

Update: After MUCH troubleshooting I''ve determined that the issue is being caused by the WPG2 plugin. http://wpg2.galleryembedded.com/index.ph...=Main_Page

The plugin interfered with Featurific before, but I thought deactivating it while activating Featurific would solve the issue, but it apparently hasn't. With both plugins enabled Featurific gives an out of memory error every time the data.xml is regenerated. The only way I could make it go away was by disabling WPG2. This is a really nasty plugin conflict and Im not sure what about WPG2 would cause the out of memory errors. If I set Featurific to shows posts based on Wordpress.com stats instead I can manage to avoid a conflict meltdown. It's a shame since the WPG2 plugin is built into my theme and I really can't do without it until a major redesign is done.

At least the mystery is finally solved though. :X

---

Is there another way to update data.xml that I could try that might avoid triggering a conflict? Maybe setting up a manual cron job? That might be an alternative solution other than setting it to pull posts based on wordpress.com stats.

[rinogo]

Hi again, Lincoln!

Great work on discovering the conflict between WPG2 and Featurific. Just out of curiosity, how did you identify the conflict?

Also, have you tried using manual post specification (instead of wordpress.com stats)? This approach might work as well, but requires a bit more manual attention.

As far as setting up a cron job, you could try creating a new php file with the contents:

PHP Code:

<?php
featurific_do_cron(true);
?>

Then, you could set up a cron job to execute this script directly via the php interpreter, or access the page via HTTP so that it executes with Apache's permissions (use wget or curl). (Also, of course, set Featurific's regeneration frequency to something huge like 999999999 minutes)

If that still triggers the conflict, you might try:

PHP Code:

<?php
featurific_data_xml_housekeeping();
?>

The second approach is somewhat of a hack, while the first one is the 'appropriate' way of doing things. However, feel free to try both. It's hard to know how to avoid the conflict since I'm not sure why the conflict is occurring. :/ Let me know if these manual workarounds work for you!

-Rich

[Lincoln]

Hi rinogo,

I tried to give the php file a try but got a fatal error: Fatal error: Call to undefined function featurific_do_cron()

I identified the conflict simply by disabling all the plugins except featurific, the reactivating one plugin at a time and regenerating the data.xml file by saving settings (without changing anything). there were no issues until I hit the WPG2 plugin, in which case when I try to save settings in featurific, it results in a blank page. When checking the data.xml files I can see they were not updated either.

I tried using the manual posts before but that results in the same issues. Only the feature by wordpress.com stats option results in no conflicts.