i installed the akismet anti-comment-spam plugin for wordpress a few days ago. so far it is working GREAT. i was getting pretty sick of tagging every spam entry in my moderation queue (there were a LOT), all while keeping an eye on legitimate comments.

highly recommended!

i spent most of tuesday messing around with various freeBSD and lighttpd settings. even though netstat -m didn't report insufficient buffers, i was getting no buffer space errors in my log files. so, i upped kern.ipc.nmbclusters from 6000-ish to 16000 and rebooted.

the errors continued. however, netstat -an counted more connections before dying than before. hmmm…

netstat also revealed that many, many, many of those connections were hanging around in the TIME_WAIT state. i googled that and found some articles about MSL and socket connections… to be honest i didn't understand it all, but it seemed that it was possible to reduce the amount of time that sockets remained in the TIME_WAIT state by changing the value of net.inet.tcp.msl. on freeBSD 4.10 the default value is 30000 (30 seconds), so i reduced it to 7500 (7.5 seconds). i understand that lowering it too much can have other consequences, like having old data from an old connection getting mixed up in a current stream?? i admit i am a TCP/IP newbie.

anyway, i changed that value, and restarted the server. so far so good… the server seems to be handling the load quite nicely now, with no connection failed/buffer space errors. i don't know if increasing nmbclusters was necessary, but since things are working ok now i am just going to leave it there for now

lighttpd is great (serves a lot, quickly), but under very heavy load it seems to act rather erratically. i'm not sure where the fault lies, though: in lighttpd, in fast-CGI (which is how lighttpd runs PHP), or in freeBSD, or some combination.

the symptoms are error messages like this:

2005-12-19 10:11:52: (mod_fastcgi.c.1561) connect failed: 242 Connection refused 61 0 /tmp/php-fastcgi.socket-0
2005-12-19 10:11:52: (mod_fastcgi.c.2663) socket failed: No buffer space available 263 4086
2005-12-19 10:11:52: (mod_fastcgi.c.2663) socket failed: No buffer space available 263 4086
2005-12-19 10:11:52: (mod_fastcgi.c.2663) socket failed: No buffer space available 263 4086

"no buffer space available" is supposed to suggest that the system is running out of network buffers. running netstat -m should reveal that buffer memory was exhausted, but the results of netstat show:

158/1552/18176 mbufs in use (current/peak/max):
        127 mbufs allocated to data
        31 mbufs allocated to packet headers
74/496/4544 mbuf clusters in use (current/peak/max)
1380 Kbytes allocated to network (10% of mb_map in use)
0 requests for memory denied
0 requests for memory delayed
0 calls to protocol drain routines

which says that there's plenty of buffer space…

then there's this ticket on the lighttpd bug tracker… the bug is old, since lighttpd no longer consumes 100% cpu, but it still becomes unresponsive and hangs under the same conditions.

all in all i probably just need to add another server. i need to compile some statistics on exactly how many pets etc i serve up, but i stopped running a stats package on the image/swf servers because the log files were too large to analyze with the amount of RAM i've got. i guess i can just grep -c something appropriate, or heck, even just count the # of lines with wc -l.

i've started to look at imagemagick and the magickwand API for PHP. the docs are a little bit opaque, IMHO, but i did find the sample/demo PHP file, which is supposed to be included with the magickwand PHP distribution, but isn't.

in any case, the demo file helps a lot.

i'm surprised i don't see more animated-GIF generators online (maybe i'm just not looking hard enough?). i may start generating animated GIFs instead of SWF files for things that don't require interaction… for example, the non-interactive versions of the pets, for sites that don't allow you to embed flash.

[UPDATED: 2006-05-09: the link to the sample code is gone now. grr! and the README file in the magickwand for PHP archive refers to sample directories that don't exist. anyway, i have found another location of the sample code. i think i should make a local copy available here for future reference, in case that disappears too…]