Design

Requirements

1. Manage large numbers of scripts
2. Enforce dependencies between scripts
3. Able to re-start at point of failure
4. Maintain an audit trail of scripts executed
5. Execute SQL scripts (DML, DDL and procedural SQL)
6. Execute OS scripts (Unix and Windows)
7. Perform SQL variable binding and string parameter substitution
8. Multi-threaded execution
9. Cross-platform
10. Capable of non-interactive execution

Hierarchy

Organizing a large number of files is a commonplace occurrence: we use directories. There is a descriptive language which nicely parallels the directory hierarchy: XML. By describing each Task, either a script to execute or a decision to make, in XML, a Task object hierarchy can be built up as well.
This is the basic strategy of the ScriptRunner. In each directory, an XML document describes how the contents of that directory should be processed, including sub-directories. The sub-directories can be included by simply including their XML file (using W3 standard XInclude).

Persistence

Task re-start and the audit trail require persistence of task state. On the one hand, it is best to leave each task to decide what needs to be persisted; on the other hand, a common mechanism should be used. We satisfy these constraints by passing an ExecutionContext into each Task when it is processing.
Scripts running in the database are offered finer grained control by a procedural SQL package, to be used in conjunction with an ExecutionContext backed by the same database. A simple file-based ExecutionContext is also provided.

In a word…

Task is a Composite, responsible for processing any child Tasks or its own script. The critical ExecutionContext dependency is injected when a Task is called to process. (For validation, the ValidationContext is a Visitor to all Tasks.)

Tasks

Most other requirements can be satisfied by specific Task implementations. A group of control tasks provide the glue for work tasks. In general, the XML describes the dependencies and the control Tasks enforce them. In particular:

• Parallel and Spawn tasks run tasks in separate threads. Parallel tasks wait for all child tasks to finish running and fail if any child fails.
• The Spawn task runs its child in a new thread but returns successfully immediately; the result of the spawned child will have no effect on the result of the run as a whole.
• The Serial task runs its children in order, failing immediately if any child does.
• SQLTask and descendents execute SQL scripts, including handling SQL bind variables..
• ProcessTask executes operating system processes, including shell scripts.
• AbstractTask provides parameter substitution.

Final choices

Java is used as the programming language, Java 1.6, in fact, in the absence of any reason to use an earlier version. The excellent JDBC is the best reason for this choice and helps satisfy the cross-platform requirement.
Secondly, any UI but the command line is forsaken. This adds versatility, especially considering that we wish to use ScriptRunner on production machines without windowing environments and remotely. And it is easy to run a command line tool in background.
The XOM library is used for handling XML, as we like its philosophy of correctness, simplicity, and then performance. It supports XPath, XSLT and XInclude.