No Content, No Fuss: Category net

Deens, welcome to the Internet!

anil@recoil.org (Anil Madhavapeddy) — Sat, 30 Dec 2006 01:11:00 +0000

Inspired by finishing my PhD corrections (!) today, I decided to hook up the DNS server from our Melange project up to the Internet. The authoritative server is called deens (since the co-author is one Tim Deegan, geddit?), and is written in pure OCaml.

This is all rather experimental, to put it mildly, but I stuck in the zone file below, hooked it up as a delegate to our main name-servers, checked it against the DNS Report, and it all seems to be working!

$ORIGIN deens.recoil.org. ;
$TTL    240
deens.recoil.org. 604800 IN SOA  (
    deens.recoil.org. anil.recoil.org.
    2006122401 3600 1800 3024000 1800
)
        IN  NS     ns1.deens.recoil.org.
        IN  NS     deensns.recoil.org.
ns1     IN  A      194.70.3.132
dynamic IN  CNAME  dynamic.recoil.org.
static  IN  CNAME  static.recoil.org.
anil    IN  CNAME  dynamic
stats   IN  CNAME  dynamic

I also modified stats.recoil.org to be an alias to stats.deens.recoil.org, so all the requests for that domain will go via the deens setup. You actually need a user/pass to access the site, but that doesn't matter; if it gets that far, the DNS bit has worked.

There's still an awful lot of tedious work to get the server into a production-ready state, such as proper logging, more error handling and recovery, etc., but I really hope to find the time in 2007 to polish this up somewhat. Performance is excellent already; faster than BIND by quite a lot, and it can optionally use more memory to cache responses to shoot up to crazy levels.

Incidentally, the dig replacement utility also seems to be working fairly well, and David Scott has been messing around with a Bonjour implementation that will get finished sometime in 2007 as well (honest!).

Google Webmaster tools

anil@recoil.org (Anil Madhavapeddy) — Thu, 28 Dec 2006 15:04:00 +0000

The conversion of the Recoil web services to external FastCGI pinned our Trac installation at Melange as the source of the CPU hogging. It turned out the Google crawler was indexing the entire source tree via Trac, causing it to go ballistic.

I then stumbled on the latest cool Googlism: the Google Webmaster Tool, which lets you register your sites and displays options, diagnostics and statistics about how the Google crawler views your website. I turned down the frequency at which Google hits the Trac installation (as well as installing a suitable robots.txt file). This solved the immediate problem, but some of the search statistics were fun to check out as well.

It turns out the gallery is pretty highly ranked for image searches. My trips to Japan seems to have made it big, with popular searches including "Shibuya", "tokyo at night", and "japanese roof". My random pictures of indian buffaloes, smoggy skylines and fried ice-cream seem especially popular as well. It's a wierd old Internet eh?

The gallery has fallen a bit by the wayside in recent months. I'll update it when I get back to Cambridge!

Mercurial FastCGI module

anil@recoil.org (Anil Madhavapeddy) — Wed, 27 Dec 2006 20:59:00 +0000

Our lighttpd setup has been very unstable in recent months, probably brought on by the load of the large Mercurial repositories hosted on Recoil since the Google Summer of Code mentoring.

The source of the instability was really hard to track down, but it seems to be the automatic spawning of FastCGI processes by the web-server, and lighttpd failing to handle a SIGCHLD somewhere when a child process crashes. To sort this out, I just converted all the Ruby on Rails setups (this blog and Nick's) to use an external spawn.

This only leaves our Mercurial vhost hg.recoil.org to switch to using FastCGI, and I couldn't find a module for this anywhere and so lashed up some Python glue to do the job.

You can download the small distribution for Mercurial 0.9 (hg-fcgi-0.9.tar.gz). It has a FastCGI library written by someone else, the Python files to glue the Mercurial and FastCGI libraries together, and a simple rc script to launch the external web process.

Instructions are for lighttpd... install the Python files somewhere, modify them to point to the Mercurial directory, run the rc script to start the daemon, and then add something similar to the following to your lighttpd config file:

fastcgi.server = (
  ".fcgi" => ( "localhost" =>
    ( "socket" => "/var/cache/fcgi/sites/hg.recoil.org/dirsock" )),
  ".hg" => ( "localhost" =>
    ( "socket" => "/var/cache/fcgi/sites/hg.recoil.org/sock" )),
)

Also add "index.fcgi" to index-file.names in the config file, and touch it in the vhost directory to create an empty file (this is to avoid getting a 404 error and instead pass it through to the FastCGI process). Similarly, touch a .hg file for every repository you want to serve. You could do this differently by passing through a URL prefix and modifying the Python appropriately, but I prefer finer control over what we're serving.

Hope this is useful; I won't bother submitting it back to the Mercurial list as it looks like the official hg repo has a different code layout; I'll check it out later on when I have a bit more time and integrate properly.

I have no idea whether or not this will actually improve our stability, but it's at least easier to move onto a different web-server now that everything is FastCGI. All I need now is an OpenBSD/php5-fastcgi port, which doesn't seem to exist (yet).

Looking my Spam statistics

anil@recoil.org (Anil Madhavapeddy) — Wed, 27 Dec 2006 00:01:00 +0000

The switch to qpsmtpd does seem to have reduced my spam intake somewhat, so out of curiousity I looked at the statistics from 2 years of procmail logs to see what's been happening in terms of filtering effectiveness.

A quick import and bug-fix of Log::Procmail into OpenBSD, and some lashed up Perl and gnuplot later, the graph on the right showed up. The red and green are ham and spam respectively, as classified by SpamAssassin.

The large amount of ham in 2004 was not actually real mail, but mostly postmaster bounces from forged spam; I am currently forced to destroy all domain bounces without even reading them due to the sheer volume. This is something that Sender Permitted From promises to help solve once we determine if any our users send @recoil.org mail from sources other than our mail server.

Since the turn of this year the amount of spam has jumped, but more concerningly, SpamAssassin has been missing increasing amounts, and it's been flowing through straight to my Inbox (despite sa-update running daily). I'm going to do these graphs again in a few months and see just how much the switch to the new paranoid SMTP has helped.

I balance, I weave, I dodge, I frolic, and my bills are all paid

avsm — Fri, 08 Oct 2004 13:49:45 +0100

While assisting various graduating PhD students (lucky for some) with their resumes, I ran across the sublime urban legend that is Hugh Graham's "College Essay". I'm now inspired for future job applications! Here is a taster of the first paragraph:

I am a dynamic figure, often seen scaling walls and crushing ice. I have been known to remodel train stations on my lunch breaks, making them more efficient in the area of heat retention. I translate ethnic slurs for Cuban refugees, I write award-winning operas, I manage time efficiently. Occasionally, I tread water for three days in a row.
Read on...

New clustering search engine

avsm — Thu, 07 Oct 2004 18:14:05 +0100

Just stumbled across a beta of Clusty, which is a pretty good search engine in its own right (not as minimal as google, but still usable). The novel thing about Clusty is that it automatically clusters searches into groups to help narrow down the search. So searching for my name brings up a bar on the left with categories such as "OpenBSD", "High Energy Magic", etc. Not bad!

It's a pity that I'm kind of locked into google now, just by virtue of the Safari toolbar not having an easy option to remap the search engine to use. It does appear people have started hacking Safari though, so perhaps a Clusty bar isn't too far off!

Playing with spammers

avsm — Thu, 29 Jul 2004 09:53:48 +0100

The amount of spam sent to Recoil accounts has dramatically sprung up over the last few years, sending the machine loads skyrocketing accordingly. Luckily, we're running OpenBSD, which added a fun tool called spamd(8) a couple of releases ago.

It's activated by tracking IP addresses of known spammers from blacklists like Spamhaus, and redirecting them to the spam daemon via pf rules. Once the mail reaches spamd, it "tarpits" it by dropping its TCP send and receive buffers to a very small value, encouraging the spammers and virii to (slowly) send their malware on. If they ever do reach the end of their data, it then rejects it with a temporary failure - costing the spammers more resourcs if they decide to retransmit it.

The load has dropped quite a bit since I activated this filtering; it seems to help against some of the latest worms quite a lot, which just connect to port 25, spew off a buffer-overflow attempt, and repeat this once every few seconds. Since spamd, things take a bit longer though!

quick spamd: 221.2.232.138: connected (9/9), lists: spamhaus
quick spamd: 221.2.232.138: disconnected after 431 seconds. lists: spamhaus

Very satisfying. I did play with the greylisting mode of spamd as well, but it wasn't quite as successful as some valid mail sites such as EDAS (bless its underwhelming soul) take five days to send conference paper rejections into a greylisted system. Public whitelists do exist, but I think I'll wait a while and see if things mature a little more first.

Friendster

avsm — Fri, 08 Aug 2003 00:22:00 +0100

In what could be a silly move, I stumbled across the Friendster web-site, and joined up. It works on the famous six degrees of separation principle, which means that it takes very few hops to know someone else in the world.
Imagine my amusement when it turns out that Nick is already a member ... I should have guessed :-) If you join up, do invite me - I'm really curious to see how this network works out. They seem to be "good guys" and promise not to use the information they collect for nefarious purposes.