Once the SchemaTree
for a model has been updated as required, the next stage
in the process is for transforms to be applied. These convert the Schema
Tree
into different formats suitable for subsequent deployment. Transforms
are all implemented in XSLT.
All transforms are done using a tool called schematic
that is included with a
Theon installation. This tool builds a single XML-Schema document for a
SchemaTree
and can optionally apply a transform to that file. It can also
optionally validate the SchemaTree
.
Since all the results of the transforms are re-generatable (by re-applying the
transforms to the same version of the SchemaTree
) they do not need to be
retained beyond testing/usage. The results included for a packaged model in a
Theon installation are primarily there for convenience (performance) as they
can all be re-generated on demand from the actual SchemaTree
(also included
in the installation). However, as the SchemaTree
for any model is altered
then so must all these transformations be re-applied for consistency across all
applications that use the results.
The DDL transforms build a set of SQL DDL commands (much like the basic output
of pg_dump -s
) to create a physical database that will be the realisation of
a model held in an SchemaTree
. Ultimately, this is how a database managed in
Theon is installed. The result of these transforms, a directory of sql
files, is written into the current working directory by default and the primary
file (ddl.sql
) is directly applied to psql
to create the database. The
generated files are included in a packaged ModelLibrary
and used in the
installation and bootstrap process.
A feature of schematic
that is only used in the DDL transforms is
differencing. Here schematic
is given two versions of a SchemaTree
.
Normally this would be two different revisions in the version control
repository, but it could also just be a tree at two different file locations.
Then the generated DDL SQL will comprise of only those necessary statements to
alter the structure of a physical database from that of one SchemaTree
to
the other. This robustly automates the upgrade of a database structure to a
new version without a data unload, destroy and data reload step (which in
itself is not robust depending on the nature of the changes).
Implementation of differencing for all possible schema changes in PostgreSQL would be extremely complex (particularly when moving beyond purely additive changes) and so Theon at the moment only provides the framework. Transitive rules need to be manually written to cater for each specific change to support.
The DDLDoc transforms build a tree of web pages that act as the documentation
for the physical database aspects (Dictionary cluster) of a model held in a
SchemaTree
. Assuming all the relevant elements in the model have been fully
annotated then the output of this transform can act as the sole documentation
for a physical database schema. These transforms also automatically build SVG
diagrams of entities, their attributes and their relationships. The result of
these transforms - a directory of HTML web pages, index pages and SVG files - is
written into the current working directory by default, and a local browser can
simply be pointed at the root to access the content. The generated
documentation tree is included in a packaged ModelLibrary
.
A variation of these transforms is also used internally to build Docbook
formatted output also included in a packaged ModelLibrary
.
The UI transforms build datasets of JSON and SHTML files that are used by the
TheonUI as metadata to present and manipulate table content in the physical
database described by the model, as well as to control the function of desktops
and panels designed for business workflows associated with the model. The
result of these transforms - a directory of structured metadata files - is
written into the current working directory by default, and the TheonUI server
is configured to point at this directory so that its functions will be
based around that particular dataset. The generated dataset directory is
included in a packaged ModelLibrary
and TheonUI is configured to operate
out of this directory by default.
The UIDoc transforms build a tree of web pages that act as the documentation
for the TheonUI aspects (Reflection cluster) of a model held in an XML-Schema
tree. Assuming all the relevant elements in the model have been fully
annotated then the output of this transform can act as the sole documentation
for TheonUI desktops and panels. These transforms also automatically build
SVG diagrams of desktops and panels structure (entities, links and connectors).
The result of these transforms - a directory of HTML web pages, index pages and
SVG files - is written into the current working directory by default, and a
local browser can simply be pointed at the root to access the content. The
generated documentation tree is included in a packaged ModelLibrary
.
A variation of these transforms is also used internally to build Docbook
formatted output also included in a packaged ModelLibrary
.
The Coupler transforms build scripts used as the executor for processing data
in streams. These scripts effectively format the external data in a stream and
load it into the physical database described by the model. They then trigger
the processing of individual Couples defined against the stream. The result of
these transforms, a directory of Python scripts, is written into the current
working directory by default. The TheonCoupler server would be configured to
point at this directory so that its functions would be based around that
particular set of scripts. The generated coupler configuration is included in a
packaged ModelLibrary
and TheonCoupler is configured to operate out of this
directory by default.
The CouplerDoc transforms build a tree of web pages that act as the
documentation for the TheonCoupler aspects (Streams and Couples) of a model
held in an XML-Schema tree. Assuming all the relevant elements in the model
have been fully annotated then the output of this transform can act as the sole
documentation for TheonCoupler. These transforms also automatically build SVG
diagrams of the Coupler structures (source/target entities and their "sync"
relationships). The result of these transforms - a directory of HTML web
pages, index pages and SVG files - is written into the current working
directory by default, and a local browser can simply be pointed at the root to
access the content. The generated documentation tree is included in a packaged
ModelLibrary
.
A variation of these transforms is also used internally to build Docbook
formatted output also included in a packaged ModelLibrary
.
The Model transforms are used internally to Theon, and build PostgreSQL
COPY
datasets for each element in the XSDDB. These transforms are what
facilitate the content of XSDDB being loaded from a SchemaTree
. The result
of these transforms, an SQL file per-XSDDB element containing a COPY statement
and a set of records, is written into the current working directory by default,
and other commands then use it to update XSDDB. The generated element load
files are included in a packaged ModelLibrary
and used by Theon for
initial bootstrap of XSDDB where the user has not specified an alternative.