Design

Requirements

  1. Manage large numbers of scripts
  2. Enforce dependencies between scripts
  3. Able to re-start at point of failure
  4. Maintain an audit trail of scripts executed
  5. Execute SQL scripts (DML, DDL and procedural SQL)
  6. Execute OS scripts (Unix and Windows)
  7. Perform SQL variable binding and string parameter substitution
  8. Multi-threaded execution
  9. Cross-platform
  10. Capable of non-interactive execution

Hierarchy

Organizing a large number of files is a commonplace occurrence: we use directories. There is a descriptive language which nicely parallels the directory hierarchy: XML. By describing each Task, either a script to execute or a decision to make, in XML, a Task object hierarchy can be built up as well.
This is the basic strategy of the ScriptRunner. In each directory, an XML document describes how the contents of that directory should be processed, including sub-directories. The sub-directories can be included by simply including their XML file (using W3 standard XInclude).

Persistence

Task re-start and the audit trail require persistence of task state. On the one hand, it is best to leave each task to decide what needs to be persisted; on the other hand, a common mechanism should be used. We satisfy these constraints by passing an ExecutionContext into each Task when it is processing.
Scripts running in the database are offered finer grained control by a procedural SQL package, to be used in conjunction with an ExecutionContext backed by the same database. A simple file-based ExecutionContext is also provided.

In a word…

Task is a Composite, responsible for processing any child Tasks or its own script. The critical ExecutionContext dependency is injected when a Task is called to process. (For validation, the ValidationContext is a Visitor to all Tasks.)

Tasks

Most other requirements can be satisfied by specific Task implementations. A group of control tasks provide the glue for work tasks. In general, the XML describes the dependencies and the control Tasks enforce them. In particular:

Final choices

Java is used as the programming language, Java 1.6, in fact, in the absence of any reason to use an earlier version. The excellent JDBC is the best reason for this choice and helps satisfy the cross-platform requirement.
Secondly, any UI but the command line is forsaken. This adds versatility, especially considering that we wish to use ScriptRunner on production machines without windowing environments and remotely. And it is easy to run a command line tool in background.
The XOM library is used for handling XML, as we like its philosophy of correctness, simplicity, and then performance. It supports XPath, XSLT and XInclude.