Chapter 32. Importing an Existing Database and Building a Desktop

This recipe is for importing a pre-existing physical database into a ModelLibrary in Theon and then using this to create a new Desktop to interact with it. We will use "ordershost" as an example.

This recipe can be run from anywhere on a machine with the necessary packages installed and suitable configuration of PostgreSQL (see configuration recipe).

(1) Get the Database Schema. This assumes no direct connection is available. In which case we need to get on the server, do a schema dump and transfer that to our own host first and then instantiate a local database from that.

ssh ordershost "pg_dump -Upostgres -s orders" > /tmp/orders.sql
createdb orders
psql orders </tmp/orders.sql

You may have to manually create any roles.

(2) Create a directory for the ModelLibrary, initialise it and change into it.

ttkm orders launch at /tmp using database=orders host=/disk/scratch
cd /tmp/orders

(3) Import the database.

ttkm import

(4) Drop the original database and install from our SchemaTree.

dropdb orders
ttkm export
ttkm derive ddl
ttkm install aok

If the install fails you may have to make adjustments to the SchemaTree (then re-derive) or original database (then re-import, export and derive). Use "-d" option for more install information.

(5) Now fetch and load the content.

ssh ordershost "pg_dump -Upostgres -a orders" > /tmp/orders.dat
psql orders < /tmp/orders.dat

Sometimes it may be more robust to use "--column-inserts", particularly if you need to adjust anything in the data to be consistent with changes made during the install. However for large data sets this will be a lot slower.

(6) Test the UI.

THEON_MODEL_XSLT_PATH=/usr/share/theon/ui/xslt ttkm derive ui
ttkm server ui

Should be able to navigate a locally running browser to "localhost:8000" and choose the "Browser" desktop, "All Clusters" and open each of the defined tables and query/update the content.

(7) Make a Desktop. First have to load the model into the management database.

THEON_MODEL_XSLT_PATH=/usr/share/theon/self/xslt ttkm derive ddldata
ttkm create aok
ttkm reload

(8) Now fire up the server and use the management UI to create a desktop.

ttkm self server ui

Open the "Reflection" desktop and the "orders" model should be shown. Follow steps to create a desktop. Once done CONTROL-C the server.

(9) Test the desktop.

ttkm export
THEON_MODEL_XSLT_PATH=/usr/share/theon/ui/xslt ttkm derive ui
ttkm server ui

If it doesn’t work iterate through steps (8) and (9) until it does.

FIXME Note: desktop and panel tags must be lowercase. Name[Info] on desktop and panel is mandatory. XSD generation with no connectors seems to be broken, odd. Looks like fields need "Label" as well.

If a lot of iteration is needed its faster to set up a parallel server for the management database on a different port so it can be left running.

ttkm self server ui on 8001

(10) Commit and deploy the ModelLibrary. Optionally commit the SchemaTree, create a release and build a package so it can be deployed and run on a central server.

git add schemat
git commit
ttkm release
ttkm package
[timc@babel orders]$ ls -l package/
total 116
-rw-r--r-- 1 timc people 18067 Mar 29 14:12 orders-1.1.0-1.src.rpm
-rw-r--r-- 1 timc people 19600 Mar 29 14:12 orders-theon-library-1.1.0-1.noarch.rpm
-rw-r--r-- 1 timc people 46320 Mar 29 14:12 orders-theon-library-devel-1.1.0-1.noarch.rpm
-rw-r--r-- 1 timc people 28400 Mar 29 14:12 orders-theon-library-ui-1.1.0-1.noarch.rpm
  1. do a test install of rpm here, then installing/running from system installation …