Sorcerer's Tower

Welcome to the Tower, my programming and web development blog. Here you will find articles relating primarily to CFML and regex, plus assorted other technologies, techniques, and ideas.

If you are interested in more than just programming, you can visit my aggregated blog, where you will also find entries on my photography and other interests.

accessibility Beehive Forum CFML ColdFusion conferences Eclipse events frameworks fun Fusebox Java JavaScript Lucee news Open BlueDragon projects qpscanner Railo regex reviews security semantics software SQL Web Development

Why Test-Driven Development?

Test-Driven Development - TDD - is a great way to catch bugs before they go live, to ensure fixes stay fixed, and to prove the functionality of software. This stability alone is reason enough to ensure tests exist for as much of every application as is feasible, but is not the only benefit.

When you write the all the key tests for a piece of functionality in advance of writing the rest of the code, you are defining when that functionality will be complete, how much progress has been made, and specifically what tasks are next to be done.

For anyone practising Agile development, the parallels should be blatant. There is similarity in the benefits too, particularly with respect to keeping focused on the task being worked on and knowing when you're done and ready to move on.

With Agile development, each piece of functionality is broken down in a couple of ways - as business-focused acceptance criteria, and development-focused tasks, which is (in part) about making large pieces of work less daunting. Likewise, TDD provides a way for an otherwise overwhelming piece of work to be described step-by-step by someone who understands the matter at hand, and (with appropriate software structure and functionality) can then be divided up amongst multiple developers to increase overall velocity.

Incomplete functionality can be more easily passed from one developer to another, simplifying the explanations of what has been done and still needs to be done; the tests do the talking.

TDD means all newly written code has tests to describe what it should do, which helps to build trust in the application - a full set of passing tests proves a particular change has not altered the tested functionality, and allows you to deploy it with far more confidence than otherwise.

TDD means writing the tests happens first - before making the changes that the tests apply to, not afterwards. Writing tests afterwards still has advantages compared to no tests at all, but it does not guarantee that all code has tests - indeed, it makes it more likely that tests will be rushed or dropped if time is limited; precisely the situation when bugs are more likely to be introduced - and thus can give false confidence in what the result of a set of changes may be.

Test-Driven Development increases productivity. As with most things, TDD has a learning curve - it's not something a programmer will instantly be completely efficient with - but some programmers will still complain (erroneously) that writing automated tests slows them down.

It's obvious that writing tests is additional work compared to not doing so, but this additional upfront cost comes with the saving of not having to manually step through performing repetitive tasks with minor variations each time - each test is written once, and is executed easily each time the code progresses. When automated, tests are not forgotton - the full test suite can be run at any time to check for regressions or unexpected side-effects. Automated testing doesn't remove the need to use an application as a user, but it far reduces the time it takes to check that changes are working as intended, and helps reduce the space for human error.

Of course, practising TDD does mean that pesky bugs are identified at the time of development - not days, weeks, or months down the line - and this can also appear to make things slower; in reality identifying bugs sooner generally makes the issue easier to fix.

Finally, with TDD you know when you're done - when all the tests pass. Without tests written up-front, there may not be a clear finishing point. It's easy for developers to become side-tracked, or to unconciously introduce scope creep, and thus slow down the completion of a task which an appropriate set of test cases would have identified as already implemented.

This article hopefully explains how Test-Driven Development is not just about code quality, reducing application errors and increasing stability. It enables developers to work smarter and faster, give clearer progress updates, know what they're doing next and not become disheartened on bigger chunks of work.

TDD is a critical methodology which any competent professional developer should want to use.

Introduction to Password Security

Earlier this year I gave a talk on Password Security at Digital Croydon #5, and didn't want to simply put up the slides without any commentary, so I've written this article to accompany them. It is primarily for the benefit of those that attended - by which I mean it wont explain every slide or go into detail on everything the talk covered, but will explain the key points, provide links, etc.

Similarly, as mentioned in the talk, it didn't cover everything there is to know, and this article wont do that either - it's an introduction, not a comprehensive guide.

(At some point I do hope to write up more detailed articles, both on the points covered and on related subjects - if/when that happens I will add the relevant links to this article.)


Introducing Lucee on Jetty

Lucee on Jetty bundles the Jetty server with the Lucee CFML engine, cleanly configured using the home/base functionality in Jetty 9, and extensively documented.

The main aim is to provide a Lucee package that is a simple unzip-and-run to get started, whilst providing a fully functional and capable web server, and also making it clear how everything works to allow it to be adapted as needed.

This first release is v0.5 because I don't consider it sufficiently complete yet - it all works fine, but doesn't yet contain everything I feel it should - for example, HTTPS has not been configured and documented, and whilst Jetty itself does support HTTPS and there's nothing stopping anyone consulting the Jetty docs and setting it up, this project is about reducing that work.

So for development use it's fine, if you don't need HTTPS or are willing to configure it yourself, go ahead - otherwise I hope to get what I consider a complete v1.0 ready as soon as time permits, but didn't want to delay releasing what I've done so far.

Downloads are available from the Lucee on Jetty project page; there's a documentation wiki at GitHub, and the template for building bundles in the GitHub repo .

As ever, I welcome any feedback or questions you might have - please use the issue tracker.


Lucee is the best CFML engine.

Nine years ago I began a blog article with a similar claim, and set out to explain why Railo was at that time the best CFML engine. Well the simplest proof of Lucee taking the crown is that Lucee is a fork of Railo by its original creator, Michael Offner.

The thing that made Railo great and that Lucee will be taking further is in being a CFML engine written for developers. That is to say, with features added through developers saying "I need feature X to do my job better" - and specifically not via vague ideas decided on by product managers getting feedback from non-technical clients who say "we need to do mobile" and then having a bunch of disconnected non-programmers come up with a horrendously buggy and useless mess called cfclient. Eugh!

Good programmers already know what tools they need to achieve certain tasks, and if those tools don't exist or aren't good enough, they need the ability to create/improve them - that is what Micha gave us with Railo, and Lucee promises to take this further - to make it even easier for the developer community to adapt it to their needs.

Bering a fork, Lucee continues the versioning from Railo, launching tonight with Lucee 4.5 available already, and an excellent Lucee 5 just around the corner.

Why Not Railo?

Many will be wondering why fork Railo, instead of working on what was there, and the best way to answer that is simply to refer to what Brad Wood has already written on the Railo mailing list:!msg/railo/B_1S3WzVPXY/hlIeZDE1u98J

To re-iterate the key points: this is the original Railo developer, taking the Railo source code, and refreshing the project. Don't mistake for division what is actually an inclusive evolution, and importantly: a sign of exciting things to come!

Lucee 5

With the next release, Lucee will bring incredible flexibility to CFML and JVM developers through a couple of key technologies.

OSGi is a modular platform for the JVM which allows only the necessary libraries to be loaded. So if, for example, you don't use Hibernate, it doesn't get included and wont add any overhead. Railo was already lightweight, and Lucee with OSGi will take this even further.

JSR223 or "Scripting for the Java Platform" is a standard for embedding different languages on the JVM, and what this means is being able to use Lucee to write CFML in far more places than before. A good example is Ant build scripts - doing certain things with Ant can be awkward and convoluted and Lucee 5 will allow embedding CFML which makes those same tasks trivial.

Together these bring some great opportunies, and this is only the beginning...

More Info

I've tried to avoid simply parrotting what others have already written, so to get further details on Lucee's launch and future you should definitely check out Mark Drew's blog post, Adam Cameron's blog post, the thread started by Igal on the Railo list, and of course the official Lucee website:

QueryParam Scanner 0.8 Release Candidate

The release candidate for the next version of QueryParam Scanner is available on GitHub.

So what's changed?

Well it now runs on FW/1 rather than Fusebox, and the UI has a new theme - the previous gold/beige is gone, and in its place is a theme based on a "new" logo which I've actually had sitting around for several years. There's CSS used that will require a modern browser - FF4 and IE9 both work, but no guarantees for anything older.

Functionality-wise there's a couple of fixes: an error is now thrown when a directory doesn't exist (previous behaviour was to return 0 matches in 0 files), and the IDs returned in data structures are now content-based hashes (previously they were ever-changing UUIDs). Oh, and the IDs are now displayed with the HTML results, in preparation for future functionality that'll potentially use them.

A new experimental (i.e. buggy) feature has been added to seperate the query code into SELECT/FROM/WHERE/etc parts, when returning data structures. This may help with post-processing the data, but has known flaws so use with care. (The existing ORDER BY functionality has also been marked as experimental to similarly indicate that it's not perfect.)

There's a minor change in that relative paths are officially not supported - the UI always stated absolute paths or mappings were required, but there was ugly code in place to try and make relative paths work too - that code has been removed. If you used relative paths before, you need to resolve them before passing to qpscanner.

In summary:

Changed:  Switched to FW/1 and removed unnecessary files.
Changed:  New logo and front-end UI.
Removed:  Dropped unofficial relative path support.
Added:    Experimental ability to separate query code into segments
Fixed:    IDs now use content-based SHA hashes, not random UUIDs.
Fixed:    Throw error when path does not exist, instead of zero results.
Supports: ColdFusion 9/10 and Railo 3.3/4.0/4.1

That's it for now. There are several new features planned to make qpscanner faster, more flexible and more useful, but you'll have to wait for a future release for those.

As ever, if you have any feedback, feature requests, or find any bugs, then please go ahead and get in touch via the GitHub issue tracker.