Posted by Anil Madhavapeddy
Thu, 28 Dec 2006 15:04:00 GMT
The conversion of the Recoil web services to external FastCGI pinned our Trac installation at Melange as the source of the CPU hogging. It turned out the Google crawler was indexing the entire source tree via Trac, causing it to go ballistic.
I then stumbled on the latest cool Googlism: the Google Webmaster Tool, which lets you register your sites and displays options, diagnostics and statistics about how the Google crawler views your website.
I turned down the frequency at which Google hits the Trac installation (as well as installing a suitable robots.txt file). This solved the immediate problem, but some of the search statistics were fun to check out as well.
It turns out the gallery is pretty highly ranked for image searches. My trips to Japan seems to have made it big, with popular searches including "Shibuya", "tokyo at night", and "japanese roof". My random pictures of indian buffaloes, smoggy skylines and fried ice-cream seem especially popular as well. It's a wierd old Internet eh?
The gallery has fallen a bit by the wayside in recent months. I'll update it when I get back to Cambridge!
Posted in net, recoil, travel | 2 comments
Posted by Anil Madhavapeddy
Wed, 27 Dec 2006 20:59:00 GMT
Our lighttpd setup has been very unstable in recent months, probably brought on by the load of the large Mercurial repositories hosted on Recoil since the Google Summer of Code mentoring.
The source of the instability was really hard to track down, but it seems to be the automatic spawning of FastCGI processes by the web-server, and lighttpd failing to handle a SIGCHLD somewhere when a child process crashes. To sort this out, I just converted all the Ruby on Rails setups (this blog and Nick's) to use an external spawn.
This only leaves our Mercurial vhost hg.recoil.org to switch to using FastCGI, and I couldn't find a module for this anywhere and so lashed up some Python glue to do the job.
You can download the small distribution for Mercurial 0.9 (hg-fcgi-0.9.tar.gz). It has a FastCGI library written by someone else, the Python files to glue the Mercurial and FastCGI libraries together, and a simple rc script to launch the external web process.
Read more...
Posted in hacking, net | no comments
Posted by Anil Madhavapeddy
Wed, 27 Dec 2006 00:01:00 GMT
The switch to qpsmtpd does seem to have reduced my spam intake somewhat, so out of curiousity I looked at the statistics from 2 years of procmail logs to see what's been happening in terms of filtering effectiveness.
A quick import and bug-fix of Log::Procmail into OpenBSD, and some lashed up Perl and gnuplot later, the graph on the right showed up. The red and green are ham and spam respectively, as classified by SpamAssassin.
The large amount of ham in 2004 was not actually real mail, but mostly postmaster bounces from forged spam; I am currently forced to destroy all domain bounces without even reading them due to the sheer volume. This is something that Sender Permitted From promises to help solve once we determine if any our users send @recoil.org mail from sources other than our mail server.
Since the turn of this year the amount of spam has jumped, but more concerningly, SpamAssassin has been missing increasing amounts, and it's been flowing through straight to my Inbox (despite sa-update running daily). I'm going to do these graphs again in a few months and see just how much the switch to the new paranoid SMTP has helped.
Posted in net, recoil | no comments
Posted by Anil Madhavapeddy
Mon, 25 Dec 2006 23:35:00 GMT
It's Christmas Day, I've eaten far too much, and am lounging around doing the now-traditional Annual Recoil Cleanup as the year's todo list has grown ever larger. I've been meaning to switch from our venerable qmail-smtpd for some years now, and finally made the move over to qpsmtpd.
qpsmtpd is a drop-in replacement for the SMTP portion of qmail, and is written in Perl with a number of plug-ins which lets us increase our paranoia levels considerably. It's a pity we have to do this, but the policy of 'accept anything' has been under increasing stress for the last few years, and when I looked at my e-mail stats last night, I realised over 99.99% of my incoming e-mail was some kind of virus or spam. Even a 1% miss rate on SpamAssassin is enough to chuck 100s of mails into my inbox!
So now the new e-mail setup at Recoil includes virus scanning via the wonderful clamav, reverse DNS RBL looksup via rfc-ignorant.org, and even early-chatter detection of viruses which blindly blast messages before the initial SMTP greeting has completed. I'm hoping to enable global SpamAssassin checking soon if all else is stable and I don't get bleating about missing mail from our users.
I played with Greylisting as well to see if it had improved from my earlier experiments a couple of years ago. Unfortunately, it still looks as if there are many broken MTAs out there which don't cope well with rejection, and manual whitelists are required, which sounds a bit unreliable for setups like ours which sometimes don't get looked at for years on end (ahem).
So it's with a tear in my eye that I wave goodbye to qmail-smtpd, the first ever network-facing service deployed on Recoil back in 1998, and incredibly, the only one I've never had to upgrade in the 8 years since.
Posted in recoil | no comments
Posted by Anil Madhavapeddy
Wed, 30 Aug 2006 21:35:00 GMT
An oft-cited criticism of virtualisation is that 3D hardware acceleration doesn't work, preventing you from enjoying your hard-earnt game of Quake 3. Rumours abound that Parallels is developing it for its software, and that VMware is doing something in this area as well.
However, thanks to the Google SoC, the power of open-source itching, and the talented Andrés Lagar-Cavilla, Xen now has support for 3D acceleration as well! Check out the xen-gl web-page with screenshots, or just clone xen-gl.hg and get hacking!
Rather than getting down and dirty with foreign grant mappings, PCI pass-through and all that malarky, Andres adopted for the more pragmatic approach of packetising OpenGL using the Chromium project, and creating an x.org module to correctly position the resulting OpenGL. End result: hardware rendering in a guest domain, without requiring any extra hardware privileges. Awesome to the max!
Posted in research, xen | no comments | no trackbacks
Posted by Anil Madhavapeddy
Mon, 28 Aug 2006 22:12:00 GMT
I noticed on the OCaml mailing list that Practical OCaml is due to be released in the US quite soon. Could this be the first real competition that Jon Harrop will face for his excellent but pricey Objective Caml for Scientists ?!
Things are heating up on the OCaml book front at last, and Jon has been getting some rave reviews of his book! We have a long-running joke about how much I hate the colour syntax highlighting he used in his book, and it amused me greatly to note that someone else in the comments for the rave review of his book also shared my heretical opinion, ha ha!
Incidentally, I just pushed the beginnings of an OCaml config file parsing library to the Melange source tree. The code in there is almost beginning to look useable...
Posted in books, hacking | 5 comments | no trackbacks
Posted by Anil Madhavapeddy
Mon, 21 Aug 2006 17:08:00 GMT
I've been mentoring a few Xen projects as part of the Google Summer of Code program. One of the most fun is the OpenBSD/Xen porting effort which Christoph Egger has been hacking on. As of a few days ago, if you clone the Mercurial repository for the project, openbsd-xen-sys.hg, and build the kernel with with the i386 bsd.rd, you get the thrill of the following boot log:
[avsm@kremlin ~]$ sudo xm create -c openbsd
Using config file "/etc/xen/openbsd".
Started domain OpenBSD
[ using 187532 bytes of bsd ELF symbol table ]
Copyright (c) 1982, 1986, 1989, 1991, 1993
The Regents of the University of California. All rights reserved.
Copyright (c) 1995-2006 OpenBSD. All rights reserved. http://www.OpenBSD.org
OpenBSD 4.0-beta (RAMDISK_XENU) #0: Mon Aug 21 14:59:09 BST 2006
root@mortar.cl.cam.ac.uk:/usr/src/sys/arch/xen/compile/ RAMDISK_XENU
cpu0: Genuine Intel(R) CPU 3.00GHz ("GenuineIntel" 686-class) 3 GHz
cpu0: FPU,V86,DE,TSC,MSR,PAE,MCE,CX8,APIC,MCA,CMOV,
PAT,PSE36,CFLUSH,ACPI,MMX,SSE,SSE2,SS,HTT,TM,SBF,PNI,
EST,CNXT-ID
cpu0: EST: unknown system bus clock
real mem = 62554112 (61088K)
avail mem = 55603200 (54300K)
using 789 buffers containing 3231744 bytes (3156K) of memory
mainbus0 (root)
cpu0 at mainbus0
hypervisor0 at mainbus0
debug virtual interrupt using event channel 3
xenbus0 at hypervisor0: Xen Virtual Bus Interface
xencons0 at hypervisor0: Xen Virtual Console Driver
xencons0: console major 86, unit 0
xencons0: using event channel 2
npx0 at hypervisor0: using exception 16
Xen clock: using event channel 4
rd0: fixed, 3800 blocks
xenbus0: using event channel 1
xennet0 at xenbus0 id 0: Xen Virtual Network Interface
xennet0: MAC address 00:16:3e:05:83:11
xennet0: using event channel 5
root on rd0a
rootdev=0x1100 rrootdev=0x2f00 rawdev=0x2f02
erase ^?, werase ^W, kill ^U, intr ^C, status ^T
(I)nstall, (U)pgrade or (S)hell?
Christoph has done a superb job of porting the NetBSD/Xen code over to OpenBSD. There are still a few bugs to be worked out in the networking driver, the virtual block driver to finish up, and the rather more messy job of getting the user-land tools to run if we want an OpenBSD dom0. But this initial booting is fantastic to see!
Read more...
Posted in openbsd, xen | 10 comments | no trackbacks
| 
