Follow this process after making changes to the self model, the model representing the XSDDB itself. These changes could be directly to the model content held in XSDDB, or changes to the factory generator and templates in the ModelLibrary. It could also include changes to XSLT outside of the self model itself. Due to the relationship between all of these it is important to test through a full cycle through to make sure nothing has been missed out of any stage.

This process assumes working from within the self ModelLibrary of Theon (library/self in a working copy of the repository). It also assumes an existing XSDDB exists containing any modifications (if any).

(1) First create a new alpha testing live profile:

rm -fr /dev/shm/alpha
ttkm launch upon alpha into /dev/shm/alpha using database=alphaxsddb

Above uses the new profile name alpha from a new model library held at /tmp/alpha and defines the database name to use (alphaxsddb).

Note that "launch" requires "xsddb" to exist to run, although no actual content needs to be in it.

(2) Then export the revised schema held in xsddb:

ttkm export

(3) Then derive the installation SQL for self from the exported schema tree:

ttkm derive ddl

(4) Then install it as alphaxsddb:

ttkm install upon alpha aok

If the install fails at this point then an incorrect DDL is being produced. So this may require alteration of the content in XSDDB, or changes to the XSLT transforms producing the DDL (if the changes to XSDDB are incompatible). In both cases iterate around (2) to (4) until the DDL installs. The DDL being applied is in the derived/model/ddl directory. A single install file in the correct order can be created by simply doing cat derived/model/ddl/*. When the XSLT has been modified the derive action needs to be pointed at it, e.g.

THEON_MODEL_XSLT_PATH=../../xslt ttkm derive ddl

if in the library/self directory of the repository and have modified the repository version of the XSLT.

(5) Now derive the reload transform for self from the exported schema tree:

ttkm derive ddlxsl

By default above will include references to XSLT in the default path, if these included references have been modified then above needs to be rebuilt relative to those modified files, so in the same way as above do:

THEON_MODEL_XSLT_PATH=-/CPY/ASM/theon/xslt ttkm derive ddlxsl

But do not use a relative path otherwise the included references won’t be found, it must be an absolute path. The output is in derived/xslt/ddldata.xsl.

(6) Now derive the reload SQL for self from the reload transform:

ttkm derive ddldata

Generated content will be in derived/model/reload. A single reload file in the correct order can be created by simply doing cat derived/model/reload/*.

(7) Use the derived reload SQL for self to load the new self model into alphaxsddb:

ttkm reload upon alpha

If this fails there is likely to have been structural change to XSDDB which is incompatible with the reload XSLT transforms. They will need to be amended and iterate back through steps (5)/(6) to (7) until the reload is successful.

(8) Export the schema tree for self that was reloaded into alpha

ttkm export at /dev/shm/alpha using self=alpha

Generated content will be in /dev/shm/alpha/schemat.

(9) Compare exported schema with original schema.

diff -qr schemat /dev/shm/alpha/schemat

They should be identical, if not iterate back (1) through (9) to correct. The problem could be in the changes, the new structure, XSLT, gather, reload, etc.

Note that /dev/shm/alpha/schemat will additinally contain a live directory (created during launch), this difference can be ignored.

(10) Re-gather into reloaded self model on alphaxsddb

ttkm -q gather using self=alpha

If this fails check the compatibility of the forge feeds with the revised structure in alphaxsddb as changes may have caused inconsistency. The -q option makes errors more obvious by suppressing all other output. Correct and re-test by iterating back to (2) potentially. Ensure gather uses any modified factory generators outside of the ModelLibrary for self, i.e. in theon/factory/generator, e.g.

THEON_MODEL_FACTORY_PATH=-/CPY/ASM/theon/factory ttkm gather using self=alpha

or even for self if an installed package exists, e.g.

THEON_MODEL_FACTORY_PATH=-/CPY/ASM/theon/factory:-/CPY/ASM/theon/library/self/factory ttkm gather using self=alpha

(11) Export the schema tree for self again that was first reloaded into alpha and then had a gather run applied to it.

rm -fr /dev/shm/alpha/schemat
ttkm export at /dev/shm/alpha using self=alpha

Generated content will be in /dev/shm/alpha/schemat.

(12) Compare exported schema with original schema.

diff -qr schemat /dev/shm/alpha/schemat

They should be identical, if not iterate back (1) through (12) to correct. The problem will specifically be related to the gather now.

(13) Force a coupler re-run, to make sure that it works and to propogate changes.

ttkm derive coupler
ttkm stream upon alpha agent coupler using self=alpha

(14) Export the schema tree for self again that was first reloaded into alpha, and then had a gather run applied to it and now has a coupler process applied.

rm -fr /dev/shm/alpha/schemat
ttkm export into /dev/shm/alpha using self=alpha

Generated content will be in /dev/shm/alpha/schemat.

(15) Compare exported schema with original schema.

diff -qr schemat /dev/shm/alpha/schemat

They should be identical, if not iterate back (1) through (15) to correct. The problem will specifically be related to the coupler now.

(16) Import installed "alphaxsddb" to test the PGCAT import process.

# for some reason we don't need to specify /disk/scratch explicitly until now, not sure why:
ttkm launch upon alpha into /dev/shm/alpha using database=alphaxsddb host=/disk/scratch
#
#
ttkm import upon alpha using self=alpha

(17) Export the schema tree for self again that was first reloaded into alpha, and then had a gather run applied to it, then a coupler process applied and now has been re-imported from its physical realisation.

rm -fr /dev/shm/alpha/schemat
ttkm export at /dev/shm/alpha using self=alpha

Generated content will be in /dev/shm/alpha/schemat.

(18) Compare exported schema with original schema.

diff -qr schemat /dev/shm/alpha/schemat

They should be identical, if not iterate back (1) through (15) to correct. The problem will specifically be related to the import process now.

(19) Re-launch a ModelLibrary for beta from the alpha output directory, say Yes when prompted.

ttkm launch upon beta into /dev/shm/alpha using database=betaxsddb

(20) Derive DDL from the alpha export.

ttkm derive at /dev/shm/alpha ddl

or if using modified XSLT:

  THEON_MODEL_XSLT_PATH=-/CPY/ASM/theon/xslt ttkm derive at /dev/shm/alpha ddl

(note need an absolute path above now as the origin has shifted into the /dev/shm/alpha directory)

(21) Compare derived DDL with original.

diff -qr derived/model/ddl /dev/shm/alpha/derived/model/ddl

They should be identical, if not iterate back (5) through (12) to correct.

(22) Derive UI Metadata from the alpha export.

  ttkm derive at /dev/shm/alpha ui

or if using modified XSLT:

  THEON_MODEL_XSLT_PATH=-/CPY/ASM/ui/xslt ttkm derive at /dev/shm/alpha ui

(note need an absolute path above now as the origin has shifted into the /dev/shm/alpha directory)

(23) Compare derived UI Metadata with original.

diff -qr derived/ui /dev/shm/alpha/derived/ui

They should be identical, if not iterate back (5) through (14) to correct.

(24) Repeat steps (22) and (23) for ALL the other derived data sets (excluding ddlxsl/reload which are addressed below), e.g. ddlbook, other docs, coupler config, etc. They should be identical, if not iterate back (5) through (24) to correct.

(25) Dump the schema created structure for comparison with reloaded one

pg_dump95 -s alphaxsddb > /tmp/alphaxsddb.sql

(26) Install betaxsddb from the alpha export

ttkm install at /dev/shm/alpha upon beta aok

(27) Dump the installed betaxsddb for comparison

pg_dump95 -s betaxsddb > /tmp/betaxsddb.sql

(28) Compare reloaded export beta with orginal export alpha

diff /tmp/alphaxsddb.sql /tmp/betaxsddb.sql

They should be identical, if not iterate back as necessary.

(29) Dump the original working XSDDB

pg_dump95 -s xsddb > /tmp/xsddb.sql

(30) Compare original working XSDDB with reloaded export beta and orginal export alpha

diff /tmp/betaxsddb.sql /tmp/xsddb.sql

They may or may not be identical depending on the types of changes made, however where there are changes they should all reflect the intended changes made to the self model within the working XSDDB (either directly or via factory and any modified XSD/XSLT).

(31) Loop through alpha export → reload beta XSDDB, then export schema to compare.

THEON_MODEL_XSLT_PATH=-/CPY/ASM/theon/xslt ttkm derive at /dev/shm/alpha ddlxsl
ttkm derive at /dev/shm/alpha ddldata
ttkm reload at /dev/shm/alpha upon beta
mkdir /dev/shm/beta
# gather below uses original factory again as this does not get put into "alpha"
# except via ModelLibrary commit/checkout cycle, in practice it would be identical
# so this makes no odds
THEON_MODEL_FACTORY_PATH=-/CPY/ASM/theon/factory ttkm gather using self=beta
ttkm stream agent coupler using self=beta
ttkm export at /dev/shm/alpha into /dev/shm/beta using self=beta
diff -qr /dev/shm/alpha/schemat /dev/shm/beta/schemat

They should of course be identical, if not iterate back as necessary …

Note that /dev/shm/alpha/schemat will contain live as it was launched as a ModelLibrary but /dev/shm/beta/schemat won’t, so to really test also need to launch beta then diff, i.e.

ttkm launch upon beta into /dev/shm/beta using database=betaxsddb
diff -qr /dev/shm/alpha/schemat /dev/shm/beta/schemat