i spent most of tuesday messing around with various freeBSD and lighttpd settings. even though
netstat -m didn't report insufficient buffers, i was getting
no buffer space errors in my log files. so, i upped
kern.ipc.nmbclusters from 6000-ish to 16000 and rebooted.
the errors continued. however,
netstat -an counted more connections before dying than before. hmmm…
netstat also revealed that many, many, many of those connections were hanging around in the
TIME_WAIT state. i googled that and found some articles about MSL and socket connections… to be honest i didn't understand it all, but it seemed that it was possible to reduce the amount of time that sockets remained in the TIME_WAIT state by changing the value of
net.inet.tcp.msl. on freeBSD 4.10 the default value is 30000 (30 seconds), so i reduced it to 7500 (7.5 seconds). i understand that lowering it too much can have other consequences, like having old data from an old connection getting mixed up in a current stream?? i admit i am a TCP/IP newbie.
anyway, i changed that value, and restarted the server. so far so good… the server seems to be handling the load quite nicely now, with no connection failed/buffer space errors. i don't know if increasing nmbclusters was necessary, but since things are working ok now i am just going to leave it there for now