Chasing Nuts

The Slony-I development team announced Slony-I 1.1.0, a major new release of the most advanced replication system for the most advanced “open source” database system.

There have been a great deal of changes since version 1.0.5,
including:

• Greatly expanded documentation
• Logs optionally include date stamps
• Slonik now supports define/include (similar to CPP)
• Automatic generation of “Listen Paths” (formerly an error-prone manual process)
• slon daemons are greatly more configurable and can use config files
• Log shipping allows creating offline replicas
• Numerous improvements to automate maintenance
• Build environment no longer requires a full PostgreSQL source tree
• This enormously eases the use of Slony-I on systems using prepackaged binaries.
• Numerous bug fixes

Head on over to the Slony-I homepage for more details.

Reading arslinux today I noticed whiprush mad a comment about Brandon wanting some feedback on e-mail stuff. I decided this is a good time to plug what I think is the best thing since square watermelons!

Every mail system I get my hands on a mail server I set up DBMail. From the dbmail site:

DBMail is made up of several components. A normal MTA (Postfix, SendMail, QMail, Exim) is used for accepting messages. The MTA hands the messages over to dbmail-smtp, using a pipe interface, or dbmail-lmtpd, using LMTP (Local Mail Transport Protocol). These programs take care of delivering the message into the database. Messages can be retreived from the database using dbmail-pop3d, using the POP3 protocol, and dbmail-imapd, using the IMAP4Rev1 protocol.

The whole email is stored in the database. That includes attachments. The DBMail programs do not have to touch the filesystem to retreive or insert emails. User information is also stored in the database, so users do not need an account on the machines DBMail is running on.

All I can say is this system is hella fast and hella scalable. Ever try deleting 10,000 messages off an IMAP account? DBMail does not even break a sweat. Need to add 1000 users? No problem just write a fancy INSERT statement. Want a custom user management web page? Just make some simple Perl pages with a pinch of SQL. The best part of all is if you use PostgreSQL you can put triggers into the database to do anything to an e-mail when it gets inserted.

I was reminded last night why I like packages so much. I am doing some work trying to track down a little problem we are having with an application. The problem seems to be with taint checks failing in an unexpected way under load. To read more about the problem you can take a peek at the Debian bug entry http://bugs.debian.org/303308

To see if the bug was still around in some future versions of Perl I thought I would go ahead and build Perl 5.9.2. Now because I like doing things the hard way I thought it would be nice just to build a deb. This way after my testing if there was a problem I could just downgrade the deb. I started out bye pulling the src for 5.8.7 from testing “apt-get -t testing source perl”
I then pulled the source down for 5.9.2 and pulled the debian dir out from 5.8.7 and stuck it into 5.9.2. A couple changes to changelog a tweek or two and I was ready for dpkg-buildpackage. I fired it off and waited….

This was the point when I knew why I am WAY to impatient for Gentoo. I went got some coffee. Then I went to the bathroom. Then I finaly said screw it and went to the bar. I come in this morning to find this

Can’t open Perl script “debian/build/perl/usr/bin/pod2man”: No such file or directory
make: *** [install-stamp] Error 2

Well I guess I have to do some work and to find out more you will have to tune in tomorrow. Did I mention that I hate compiling stuff!

When people talk about the size of their database it is amazing how they like to throw around that they have a table with one million rows. I find it funny that for some database admins there is a direct correlation to how many tuples a table has and how long their e-penis is. If one more person runs up to me and tell me how many rows a table has I might just have to hit them. Now that I got that out of my system lets take a look at just how big a TABLE is.

In most databases the length of a table is not that important. More curious to me is how much space is that database object taking up on the disk. The disk IO being the main bottle neck of a database you should be mind full to try and keep disk IO down. One way to do this is keep an eye on how big those INDEXES and TABLES are and try to keep them down in size. So, how big is that table? Fortunately in postgres we can easily look

SELECT relname, relpages*8*1024 as Bytes, relpages*8 as KB, (relpages*8)/1024 as MB
FROM pg_class
ORDER BY relpages DESC;

The above will show you how much space you database object are taking up. The next question is, what to you with the information? I think I answered that above. You can run around and tell folks how big your TABLE really is :-p

I have to make a few comments today about some things that have happened yesterday that I really thought would not happen anytime soon.

First for you Debian folks out there Debian 3.1 (Sarge) was Released. For those who watch the Debian project we all know that it moves pretty glacial and for some time I did not know if Sarge was ever going to come out. So Chears and thanks to all the Debian guys! Now I have to go and apt-get update ; apt-get dist-upgrade a whole lot of servers.

The second piece of news that there was a LOT of speculation about and I found more shocking is Apple is going to be moving to x86. Not only that, they hope to complete the transition in 2 years. Now if any company can pull this off it is Apple but I will be watching closely to see how this plays out. I will also be watching to see how easy it will be to get OSX running on my x86 machines. I know there was a lot of speculation and people like Devorak had been predicting it for years. People had pointed out that why would Apple start making Darwin architecture specific if it was only to be on one architecture. Even with the facts I admit that seeing the Intel people on stage with Jobs was like a crazy dream.