haruki zaemon

As part of some other work I’m currently undertaking, I’ve just extracted a VERY new Rails plugin called, you guessed it, Conventional Validations.

As the README says, Conventional Validations is a plugin that attempts to apply validation based on the naming of column. Specifically, the plugin searches for validation methods such that a columnnamed foo or ending in _foo would be validated using a method named validates_foo.

For example, by defining a class method as:

def self.validates_phone(*attr_names)
  validates_format_of attr_names, :with => /[0-9]+(\.[0-9]+)*/, :allow_nil => true
end

Any columns named phone or ending in _phone would be automatically validated.

I’ve only just added it (it’s only available in trunk) and the matching rules are VERY simplistic but if you have company-wide or even project-wide validation and you’re strict about your naming conventions (as I am), consider extracting them into a module and mixing them into ActiveRecord::Base and letting the plugin apply them for you.

There are already some existing plugins such as (off the top of my head) “validates_email” that would most likely just work out of the box.

Enjoy.

WOW! It’s been a while since I found something non-politcal to have a good old rant about but dear god what’s with facebook!? After being harassed for months with invitations by people I usually respect, I finally caved and signed up. What a load of bollocks!

So the idea is we want people to find one another. OK I got that. Then you only really want to link to friends. OK that sounds great. Then once we’re connected, we can…send each other messages via a web page. Holy cow! What an idea! I wish I’d thought of that. Even better, when I receive one of these master pieces I get…wait for it…an email telling me there’s a message waiting to be read. Fan-bloody-tastic.

My wife tells me I don’t appreciate it because I’m not “as social as other people” I suspect it’s more because I’m not bored out of my brains at work with nothing better to do with my time or my life for that matter!

You’ve moved to another country. Used the opportunity to take some time off work and find a place to live. Settle in and get to know your surrounds.html). Indulge a little in the local food, culture and just a few of the local beverages. Your wife get’s bored so you get connected to the internet but resist the temptation to do anything really useful.

Finally, after much um’ing and ahh’ing and hollow excuses as to why it’s still not the right time, you decide you really had better start earning a living again. So, you make sure the VPN still allows you to connect to your customer’s servers, switch to the directory of the project you were working all those weeks (ok months) ago and type svn up. A dozen or so filenames whizz by and then the screen appears to freeze. 5 minutes go by. Then 10. Then an hour. Then two. What the…?!

After a few emails back and forth between the customer’s technical people it becomes apparent that the freeze is a result of downloading a 33M jar file from the repository coupled with the client’s internet connection speed being at least 1/10th if not 1/100th that of my 50M/s ADSL. So what do you do? There’s no studio audenice; it doesn’t make sense to go 50/50; so, I guess I’ll have to phone-a-friend.

A cunning plan emerges. James would update his checked-out version onto his laptop over the LAN at work – resulting in a whopping 1.14G of source material:

james> svn up project

Even zipped this would still be enormous but remembering that good-old SVN keeps two copies (one as a backup) we then deleted all files except those in the .svn (ie backup) directories:

james> find project -type f -not -path '*.svn*' -delete

This was then zipped up resulting in a modest 150M:

james> tar -cvzf project.tgz project

And sent to a server in the US – being a whole lot easier than tunnelling through firewalls and NAT’d routers – from his home connection – being a whole lot faster than the one in the office:

james> scp project.tgz server:

I then downloaded it:

simon> scp server:project.tgz .

Unzipped it:

simon> tar -xvzf project.tgz

Switched the user:

simon> cd projectsimon> svn switch --relocate svn+ssh://james@repos svn+ssh://simon@repos

Restored all files from their backup copies:

simon> svn revert -R .

And finally did an update to be sure:

simon> svn up

Which all makes me wonder if were it not for all these nifty command-line tools with which to play, might there have been an easier solution? Was this perhaps just a little too much technical masturbation? M’eh who cares. It was fun and it worked!

Spoiler: JDK 1.5 – and near as I can tell 1.6 even though it’s supposed to be fixed – have a pretty fatal memory leak when using java.util.concurrent libraries.

How do I know?

Well, besides finding the earlier link via our good friends at Google, we recently switched from using the backport to the standard JDK libraries and the throughput of our application under load rapidly approached zero.

After spending some time creating a a quick-and-dirty load test on my own machine, I found I could easily replicate the problem with the added bonus of java.lang.OutOfMemoryErrors in under a minute. Running it under a profiler confirmed as much with the culprit being java.util.concurrent.locks.AbstractQueuedSynchronizer$Node.

“Never” cries the guy we work for. “Doug Lee’s stuff has been around for a long time and I’ll bet his stuff works and you guys have screwed something up!” he continues.

We all agreed and decided to replace the locking code in the class I’d just created (and presumably screwed up) with some good old-fashioned synchronisation.

Nope. The problem persists but it’s not as bad this time. We clearly “fixed” something but not everything. Hmmm.

On further investigation we realised there was still some code in the call chain using the concurrent stuff so we decided, just for laughs, to revert back to using the concurrent stuff but use the backport libraries instead.

Voila! Suddenly the load test exhibited exactly the expected behaviour: no out-of-memory errors; no grinding to a halt; nice see-saw memory usage; and using only 2MB of heap instead of, well, as much as it could get (>64M in our case).

Stunned, we went back to the default JDK version. Yup! Problem is back and not just in our own locking code either, we noticed the problem in much of the concurrent classes including the collection implementations.

The good news is that using backport we now have a build that works.

The scary part is that I can only presume some Sun engineer decided he knew better and screwed the pooch right royally on this one.

Once and for all could we all just get over pair programming as something novel and thus something about which to be suspicious and fearful. The idea that it’s some new fandangled practice is utter bollocks! It’s called COLLABORATION! and believe it or not, some of us have been pair-programming collaborating with much success for many years now and with no ill effects. You’d be amazed!

Getting people to collaborate takes more than simply mandating pair-programming. If you don’t know how to do it, how about occasionally getting up off your bum and sitting with another developer. Perhaps once in a while you might like to try taking those goddam earphones out, turning around and asking for someone to come and watch what you’re doing. Talk to other members of the team about your work. Involve other people in what you do. Fundamentally that’s what pair programming is about. Yes, there are added benefits such as having your pair catch little mistakes here and there but more than anything it’s the collaboration that comes from sitting with another human being and discussing what you’re doing that makes the biggest difference.

Where I work now we all sit in close proximity, we all listen (whether we know it or not) to what everyone else is doing and all chime in if we believe (rightly or wrongly) we have something to contribute or even just to ask questions because something sounded interesting. Snobs, cowboys, incompetents and the socially-challenged are among those that might fear such behaviour; precisely the people we don’t want on our project.

I highly recommend visiting Jason Yip’s blog (among others) for ways in which you might get your team to collaborate more effectively and yes, pair-programming may well be one.

I can take it no longer. If I hear or read about how Getting Things Done (GTD) saved yet another person’s “life” I’m going to scream. This fascination the world (and geeks in particular) have with doing more and more stuff is frightening.

When you feel there is just so much to do in your life that you can’t seem to make it through the day without stressing about something, that your life is spiralling out of control and there seems to be nothing you can do about it, there is an alternative course of action: DLS - Do Less Stuff™

Yes, hard to believe but there is actually no particular need to devise yet more processes in order to deal with the stuff that is ultimately controlling you. Instead, you can actually take control of your life and say “I’m going to do less stuff” and tell those that would foist their agendas upon you to go jam it.

I choose not to reduce my life (and the enjoyment thereof) to a series of prioritised index cards pinned to a wall.

If you already have a continuous integration (ie build) server (such as Pulse, Bamboo, Hudson, or even the venerable old CruiseControl) and you’ve been trying to include some of your Rails applications then your life just got a whole lot easier.

Enter CI::Reporter. To quote the documentation:

…an add-on to Test::Unit and RSpec that allows you to generate XML reports of your test and/or spec runs. The resulting files can be read by a continuous integration system that understands Ant‘s JUnit report XML format, thus allowing your CI system to track test/spec successes and failures.

So, I simply installed the gem on the build-server and the plugin into my plugins project, configured the build to slurp up the generated XML files and voila! I now get build notifications letting me know how many tests passed and which ones failed.

Thanks Nick!

After much installation pain and anguish experienced by some users of our plugins (due mostly to outages with RubyForge SVN servers and the inability of some users to access SVN from behind firewalls), I’ve finally managed to spend some time getting a plugin installer-friendly HTTP mirror up and running.

The root of the mirror can be found at http://www.redhillonrails.org/svn from which you can browse the entire repository.

If you live on the edge, you’ll find them at http://www.redhillonrails.org/svn/trunk/vendor/plugins.

If you’re after are the Rails 1.2 stable compatible plugins, you’ll find them at http://www.redhillonrails.org/svn/branches/stable-1.2/vendor/plugins.

If you’re after the older Rails 1.1.6 versions, they’re available at http://www.redhillonrails.org/svn/tags/release-1.1.6/vendor/plugins.

The mirror is presently refreshed once a day which should be enough for most people.

As always, let me know if I’ve right royally screwed something up. I’ve manually tested all of the above but “it works on my machine” definitely applies in the case I’m sure.

So, you found what you think is a bug in a bit of software and you need it fixed desperately, so desperately that you take advantage of the fact that you have direct contact with the developer(s) and ask for help. Allow me to give you some handy hints as to what kind of information may be useful to said developer(s) when diagnosing a problem experienced intermittently by your lone yak herder in outer-Mongolia:

• Have you done a quick search on Google? If not, go to jail; do not pass go; do not collect 200 dollars
• What version of the software are you using?
• What are the steps necessary to re-create the problem?
• Is there a stacktrace?
• Have you changed anything recently? And I mean ANYTHING? You don’t get to decide if it’s important enough to warrant a mention
• What other software/add-ons are you running at the same time?
• Which operating system/s and what version/s are you using?
• Is there a database involved? If so, what platform is it running on and what version are you using?
• Does this happen locally or only when run remotely?
• Does it happen in development/test/production/etc?
• …

In fact, the more information you provide the quicker the problem (if in fact there even is one) is likely to be solved because quite frankly, and perhaps contrary to popular belief, not only does “You’re software sucks and we might stop using it if you don’t fix it” not really carry that much currency, it doesn’t really help much either.

UPDATE: As fate would have it, comments seem to be barfing. Feel free to send me your comments via email and I’ll add them manually until the problem is resolved.

So today, after I don’t know how many years of doing Java development, I find out there is a command to list all the running Java processes. It’s available on all platforms I could find and it’s called jps. On most platforms it’ll display something like:

1170 Jps -lm1162 org.apache.tools.ant.launch.Launcher -cp  dist

Except under Mac OS X where you’ll get something more like:

1170 Jps -lm1162 -- process information unavailable

Pretty bloody useless!

So, I wrote a little ruby script to simulate the desired behaviour as much as possible (by using the built-in jps to collect the process ids; then filtering the output from ps accordingly):

#!/usr/bin/env rubyrequire 'set'pids = Set.new`#{ENV['JAVA_HOME']}/bin/jps`.each_line do |line|pids.add($1) if line =~ /^(\d+) /end`ps -x -w -w -o pid,command`.each_line do |line|print line if line =~ /^ *(\d+) / && pids.include?($1)end

Which, when run, produces output similar to (line continuation not included):

1162 /System/Library/Frameworks/JavaVM.framework/Versions/CurrentJDK/Home/bin/java \-classpath /opt/local/share/java/apache-ant/lib/ant-launcher.jar \-Dant.home=/opt/local/share/java/apache-ant \-Dant.library.dir=/opt/local/share/java/apache-ant/lib \org.apache.tools.ant.launch.Launcher -cp  dist

Granted it does produce lots more output than standard jps but it does at least produce something useful to look at when monitoring Java processes. Best of all, it means we can automate the monitoring in a manner that works across most platforms.

If anyone has a Better Way™, I’m all ears.

UPDATE: After a bit of googling I discovered that it’s a bug which is fixed in Java 6 (Mustang) and relates to, wait for it, user names containing an underscore. Go figure!

Possibly the only thing I like about MySQL is when performing a search, the values 'SIMON' and 'sImOn' are considered equal – case-insensitive searching. PostgreSQL on the other hand considers them to be different – case-sensitive searching. Now I don’t know about you but for %99.999~ of the applications I’ve ever written, I’d rather 'Australia' and 'AuStRaLiA' weren’t considered different countries.

The “standard” approach to solving this problem is to change a query from this:

SELECT * FROM countries WHERE name = ?;

To something like this:

SELECT * FROM countries WHERE **LOWER(**name**)** = **LOWER(**?**)**;

Thereby forcing the database to perform a pseudo case-insensitive search. The only problem is that all those nice indexes you’ve created to ensure fast, efficient searching are totally ignored. (Who can spell full-table scan?) Performance issues aside (I mean after all we know that premature optimisation is the root of all evil right?) what’s just as annoying is that I can’t actually guarantee uniqueness, which is pretty much the whole point! No, even with a unique index on countries.name, the database will still quite happily allow me to:

INSERT INTO countries (name) VALUES ('Australia');INSERT INTO countries (name) VALUES ('AUSTRALIA');INSERT INTO countries (name) VALUES ('aUsTrAlIa');

So when I perform a case-insensitive search as previously discussed, I’ll end up with three (count ’em 3) records. Thankfully, there is a solution (of sorts): expression indexes.

PostgreSQL allows you to create indexes based on expressions, say for example LOWER(name), allowing us to create a unique, case-insensitive index as simply as:

CREATE UNIQUE INDEX index_countries_on_name ON countries (**LOWER(**name**)**);

Ok, so perhaps you knew this already and you’re wondering what all this has to do with Rails? Well I’m glad you asked.

Rails (as of 1.2) has a new option for validates_uniqueness_of named, oddly enough, case_sensitive. This is assumed to be true by default (meaning all searches are case-sensitive). Set it to false however and you’ll magically get validation queries that look like:

SELECT * FROM countries WHERE (**LOWER(**countries.name**)** = 'australia') LIMIT 1;

To compliment this feature, I’ve recently enhanced the RedHill on Rails Plugins in two interesting (and hopefully useful) ways.

The first is in the core and supports the creation of case-insensitive indexes during schema migration:

add_index :countries, [:name], :unique => true, **:case_sensitive => false**

The second is in schema validations and causes case_sensitive => false to be passed as an option to validates_uniqueness_of whenever a case-insensitive index is detected.

(I also looked at the possibility of automagically surrounding query parameters, etc. with LOWER() inside find methods but given the myriad forms queries can take, it seems altogether too difficult for my feeble mind at this point.)

The upshot of all this is that at the very least, it should now be possible to add case-insensitivity to your queries and be assured that (bugs not withstanding) the performance of your application won’t suddenly plummet as a consequence.

With the rising popularity of video podcasts as training material, I thought we needed a suitable acronym:

What You See Is What You Learn.

If only Google weren’t around to prove that I wasn’t the first to think of it. D’oh!

On the positive side, it’s apparently a method with "…guaranteed results."

I was chatting with my brother today (somewhat of a professional student with degrees in neuroscience, physics and maths) about software development. I was explaining how yesterday had a been a rather unpleasant day working out how to integrate with Crystal Reports. You see, what I wanted was an interface that looked something like this:

CrystalReport report = new CrystalReport("report.rpt");
report.setParameter("Posting Year", "2007");
report.setParameter("Account Number", "5678");
ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
report.writeTo(outputStream);
outputStream.close();

Only instead what I got was this:

ReportClientDocument document = new ReportClientDocument();
document.open(_reportName, OpenReportOptions._openAsReadOnly);
DataDefController dataDefController = document.getDataDefController();
ParameterFieldController parameterFieldController = dataDefController.getParameterFieldController();
parameterFieldController.setCurrentValue("", "Posting Year", "2007");
parameterFieldController.setCurrentValue("", "Account Number", "5678");
InputStream inputStream = document.getPrintOutputController().export(ReportExportFormat.PDF);
int b;
while ((b = inputStream.read()) != -1) {
    outputStream.write(b);
}
inputStream.close();
document.close();

The first is nice and semantic; it’s pretty obvious what the code is doing. The second requires you to read, very carefully, each line in order to work out what is going on. Talk about leaky abstractions. Apparently my Document’s connected to my, DataDefController; my DataDefController’s connected to my, ParamaterFieldController; …

My brother drew an analogy with explaining to someone how a telephone works. In the first case, we’ve gone through a very simple explanation with just enough information to allow someone to have a go themselves; in the second example, we’re now explaining how the spin of each electron determines the probability of it going down the wire and thus contributing to the current that ultimately makes the phone call possible.

At first I was concerned: Having just recently ranted (for the umpty umpth time) that developers don’t seem to understand even the most fundamental principles of software development, here I was lamenting the fact that most APIs weren’t simple enough. My brother then asserted that perhaps the reason software is (in general) so poorly written is precisely because the APIs we are forced to use are so primitive. That because we are forced to follow so many steps in achieving something that is conceptually so simple (such as producing a report) the likely-hood of failure is much greater.

The scary thing is, this is certainly not an isolated example. Have you ever tried using the javax.mail packages? JNDI anyone?

In the end, I wrote a class named, unsurprisingly, CrystalReport with an interface exactly as in the first example and implemented almost exactly as in the second. But I seem to need to do this quite a lot when dealing in the “Enterprise” Java world.

My mate Marty Andrews is running a Design Improvement Workshop on March 24th here in Melbourne as part of Cogent’s Easy Access Training (EAT). The content is in part, a follow on to the TDD Workshop with emphasis on identifying and removing code smells using refactoring tools and techniques.

One interesting point about these courses is that rather than being a fixed price, places are allocated by auction with the final price determined by the lowest bid! So, if you think some first-class developer training is worth $100, get in quick before you’re outbid.

James has a home-grown GTD system which he’ll hopefully write about if not release to the world one day. It’s totally command-line driven using a combination of bash, perl and ruby to manipulate plain text files and, from what I can tell, it’s not only blindingly fast, it seems to work seriously well.

One of the daily tasks that James performs is to prioritise the things he wants to (attempt) to get done and what better time to do that than on the 45 minute train trip to work. As you can well imagine, prioritising 40 items takes a fair amount of time not only due to the sheer number but also, being vision-impaired, reading (and re-reading) takes a bit of concentration. So, we came up with a nifty solution in two parts.

The first part was quite simple: use binary insertion. That way, instead of O(n^2) comparisons, we’d get O(n log n). So far so good. But the that was pretty obvious. The next bit was the real doozy!

Rather than have each pair printed out so that James could read them and make a decision, we instead chose to use some Text-To-Speech (TTS) to literally speak each pair; something along the lines of:

Is calling your boss to discuss staff pay reviews more important than send wife a bunch of flowers for her birthday?

In addition, we actually run the TTS in another thread so that we can easily interrupt it, primarily because James: has already chosen a suitable answer; needs to repeat the question; or wants to cancel the whole thing.

About the only problem we encountered was that, on his Powerbook, the say command (built-in to Mac OS X) ran a little too slowly for our liking so we simply aliased it to use the much faster swift command that comes as part of the commercial Cepstral William voice he uses for all his Text-To-Speech.

Now James can listen to the minimal set of questions, making the appropriate decision for each by way of a simple keystroke, and have it all done within 15 minutes all the while free to sit and enjoy the view (such that it is on a suburban train during peak hour) rather than staring at the screen.

When I used to run FreeBSD, I loved the ports system that comes as the standard mechanism for installing most freely available (and some commercial) software. When switched to the Mac, I immediately starting using DarwinPortsMacPorts as my main method for obtaining and installing all my “geekware”: PostgreSQL; Subversion; Ruby; and even Ant to name but a few.

Installing a new bit of software using MacPorts is usually pretty trivial. Something like:

sudo port install subversion

is all that’s needed to install Subversion and all its dependencies.

Sure, there are Mac-specific GUI-based installers for some software that I use but I generally find it easier to use MacPorts for managing dependencies, versioning, updates, etc.

There are however, other command-line based software such as Pulse that, although relatively simple to install, still take some time and thinking not only to install but to install in such a way as to not turn my Mac into a glorified Windoze machine with all those lovely security holes. If only there was a way to create a MacPort installation for these too.

Well, there is: Create one yourself. So this morning I decided to see how difficult it would be to do just that.

Reading the quick-start guide, I soon learned that writing a Portfile – Literally a TCL script named Portfile – is often all you need to do. Unfortunately, the “quick” in “quick-start” is pretty much just that, leaving quite a lot to the imagination.

So, after reading a few existing Portfiles and following a bit of experimentation, I managed to get a relatively clean (and importantly, working) Portfile for installing Pulse which I figured I’d not only share with you but I would use as a guide to creating your own.

Now, before we continue, if you haven’t already done so, I highly recommend reading the quick-start guide. It’s ok, I’ll wait…

The start of a Portfile looks something like this:

PortSystem 1.0name              pulseversion           1.2.14categories        javamaintainers       [email protected]       Pulse automated build serverlong_description  Pulse is an automated build or continuous integration server. \Pulse regularly checks out your project's source code from your \SCM, builds the project and reports on the results. A project \build typically involves compiling the source code and running \tests to ensure the quality of the code. By automating this \process, pulse allows you to constantly monitor the health of \your project.homepage          http://www.zutubi.com/products/pulse/master_sites      http://www.zutubi.com/download/

All pretty self explanatory really: The name and version of the software; what exactly it is and why I’d want to use it; who to blame when something goes wrong; and where to download the installation (in this case .tar.gz) file from.

Following on, we have:

checksums         md5 dcebf03b7a7099a476371b8142ba7624depends_lib       bin:java:kaffe

These specify the MD5 checksum for the installation file to ensue we download the correct one, and any dependencies; in this case we need a Java runtime which comes pre-installed on most Macs anyway but I figured it doesn’t hurt to make sure.

Next, we set up some variables for use later on in the script, again all pretty self explanatory:

set pulseuser     pulseset pulsegroup    pulseset home          ${prefix}/share/java/${name}set bin           ${home}/binset executable    ${bin}/pulseset dbdir         ${prefix}/var/db/${name}

(${prefix} is pre-defined by the port system as the path to the base of the installation – usually /opt/local.)

Now for something kinda cool: launchctl integration. If you don’t know, launchctl is the preferred method for starting, stopping and otherwise controlling server processes under OS X. It can be a little tricky to get right with various XML configuration files, etc. however MacPorts comes to the rescue. With a few simple TCL commands, the installer will magically do all the heavy-lifting for us:

startupitem.create  yesstartupitem.init    "PULSE_HOME=${home}"startupitem.start   "su ${pulseuser} -c \"${executable} start\""startupitem.stop    "su ${pulseuser} -c \"${executable} shutdown\""

Neat huh? Now when we install the port, we’ll automagically have scripts generated in all the right places so the pulse server can be started and stopped by the system.

We’re almost up to the installation part but before we get to that, we need to stub out a few things.

You see, the whole installation process caters not only for unpacking and deploying of files but in the case of C/C++, etc. also configuring, patching and compiling, etc. In our case though, not only are these latter steps unnecessary, they’re not even possible – we’re dealing with a binary distribution – so we need to override them to do nothing:

configure {}build {}

At last we get to the actual deployment of the files, creation of users, setting of permissions, etc.:

destroot {# Create the Pulse useraddgroup ${pulsegroup}set gid [existsgroup ${pulsegroup}]adduser ${pulseuser} shell=/bin/sh gid=${gid} home=${dbdir} realname=Pulse\ Server# Ensure we have the needed directoriesxinstall -m 755 -d ${destroot}${home}# Copy the filessystem "cp -R ${worksrcpath}/ ${destroot}${home}"# Keep empty directoriesdestroot.keepdirs-append ${destroot}${home}/logs ${destroot}${home}/versions# Fix ownership of some directories pulse really needs to write tosystem "chown -R ${osuser}:${osgroup} ${destroot}${home}/logs"system "chown -R ${osuser}:${osgroup} ${destroot}${home}/versions"# Add a symlink from bin directory to the pulse scriptsystem "ln -fs ${executable} ${destroot}${prefix}/bin/pulse"}

Once again, pretty straight forward except for that odd looking ${destroot}. Why is it being used in some cases and not in others you may ask.

This actually got me for a while to until I realised that what the destroot step actually does is create an “image” of the installed software in a working directory – ${destroot} – after which the predefined install step copies the staged files into their final destination. Not a bad idea really when you think about it. Certainly saves screwing up the production environment when something goes horribly wrong – like when you’re playing around with things to see what happens ;-).

Last but not least, a little bit of post-installation goodness to instruct the user with any manual (or perhaps optional) processes, or even just a nice message to say “why thanks for using our software”:

post-install {ui_msg "#"ui_msg "# The script ${executable} has been installed to facilitate starting and"ui_msg "# stopping ${name} as a true daemon process. It must be run as ${osuser}."ui_msg "# For example:"ui_msg "#"ui_msg "#   sudo su pulse -c "${executable} start"ui_msg "#"ui_msg "# This script assumes it is run from ${home}. To run from outside this"ui_msg "# directory, you must set the value of PULSE_HOME to the absolute path"ui_msg "# of this directory. For example:"ui_msg "#"ui_msg "#   PULSE_HOME=${home} sudo su pulse -c "${executable} start"ui_msg "#"ui_msg "# You will also need To create the directory ${dbdir} if it does not"ui_msg "# already exist:"ui_msg "#"ui_msg "#   sudo mkdir -p ${dbdir}"ui_msg "#   sudo chown ${osuser}:${osgroup} ${dbdir}"ui_msg "#"}

And there you have it. Not too bad really and although we didn’t cover quite a number of the other features the port system provides, I think you will have seen enough to get you started and on your way to integrating your favourite software into the ports system.

It sure beats rememberingforgetting a lot of manual steps each time you need to install/re-install software.

A customer walks into a bank and, after a waiting in the queue, reaches the teller:

Teller: How may I help you? Customer: I’d like to transfer a million, billion, squintillion dollars from Jo Blogg’s account into mine please. Teller: Certainly but first I’ll need some proof of identification. Customer: enters account number and PIN and clicks enter._ Teller: Ok, now I’ll need to authorise the transfer, please wait a moment. Customer: Oh, that’s ok, Joe gave me permission. Teller: I’ll get to it immediately then.

As ridiculous (and somewhat contrived) as this scenario seems, it’s a pretty apt reflection of how many large companies implement security in their web applications. Indeed many VERY large companies. Companies that, if you knew who they were, would certainly have a lot of explaining to do.

The problem as I see it is that developers in general don’t understand security. Actually, no, the problem is a little more fundamental than that. Developers in general don’t understand the technology stack with which they work day in, day out.

I can appreciate how difficult it might be to accept that you’re forking out in excess of $80k a year for a developer that doesn’t seem to understand that just because the user can’t see a hidden field on a form doesn’t mean they can’t look at the page source; that just because a user typically interacts with the server using a browser, doesn’t preclude the use of something as simple as a telnet client. In fact, I’m willing to bet that if you asked something along the lines of “have you tried seeing if you can connect to the server using telnet” you’ll get blank stares all around.

The sad truth is that using hidden form fields, magic request parameters, default credentials, and even sending permissions to the client (even if that means web server as a client of EJB) on login and then trusting the client to send those same permissions back with each request in order to, I don’t know, perhaps “save database lookups” or something, is disturbingly common in practise.

Heck, I don’t even claim to be a security expert but I’m pretty sure that not using SQL bind parameters is generally considered a bad idea. Then again, perhaps it’s not so much of a problem when all you’re doing is reading clear-text passwords out of a database?!

Oh, you haven’t heard? Well then, let me be the first to tell you: I’m getting married on April 14th to the loverly Jesseca ;-). We’re then heading to Japan (possibly via Borneo) in early June to spend 12 months in a little country town where we will hopefully spend our time telecommuting for work, travelling and partying. Oh and of course all I can eat Aikido training. Woohoo!

So if anyone knows of any conferences, geeky get-togethers or the like I’d love to get a little taste of the software development community in Japan.

By default, my current toy (Pulse) stores its meta-data in an HSQLDB database. Which is fine (it’s a darn good database) but I already have plenty of infrastructure around periodically dumping all PostgreSQL databases on my system for backups. Wouldn’t it be nice if Pulse ran on PostgreSQL.

Thankfully, the fellas at Zutubi (purveyors of the afore-mentioned toy) have written up a quick HOWTO on migrating to PostgreSQL and it worked like a charm.

About the only downside is that each time I upgrade, I need to remember to add the postgres.jar to the lib directory. Hmmm. Perhaps I need to add it to the classpath before starting the server…?

If, like me, you need to read in various bits and pieces of meta-data in PostgreSQL, you’ve probably found it rather tiresome navigating your way around the various pg_catalog tables in order to determine where the one piece of information you need lives.

After a bit of googling, I discovered a nifty option you can set in psql: \set ECHO_HIDDEN 'yes'

Then whenever you describe a table, index, etc. using \d and friends, you’ll also get the SQL that was used!