Chapter 21. Stream and Couple Definition

The configuration of TheonCoupler is done within Theon using XSDDB to manipulate the content of the SchemaTree. Ultimately the configuration is pushed out to be accessable to TheonCoupler itself by a special derivation. Other changes directly affect the physical database schema and are pushed out via the DDL derivation process.

A stream configuration defines what format the stream data is in and how to load this into the Stream Source Table (if loading is necessary at all). In order to add new streams and update existing streams use the Streaming desktop. Start by querying or selecting an appropriate model on the initially opened Choose Model panel.

Basic stream details can be entered by adding a row to the Streams box. Where appropriate the stream data table can be defined via the entity drop-down. One mandatory field is Origin. This is an identification string used in the couple meta data columns added to the target table. It should be unique across the model. It separates each couple process (particularly needed when multiple processes are updating rows or columns of the same table). Save changes. Additional stream details can be entered by opening the Stream panel and querying for or selecting the relevant stream.

A couple configuration defines the Couple Source Table, Target Table, columns that make up the common Handle and columns that are maintained by the synchronisation process. The Couple Source Table is often the Stream Source Table, but it need not be, often it is a view constructed from this and other tables. It can also be a function (that returns a table of data) which can be parameterized (additional non-handle options to the couple process calls are passed down to the function). Basic couple details can be entered by adding a row to the Couples box on the Stream panel. The Source Entity, Target Entity and Position are mandatory. Save changes. Additional couple details, in particular with/without options, can be entered by opening the Couple panel and querying for or selecting the relevant couple. The Source Origin Value is mandatory, and much like the stream origin, is used as an identifcation string for the stream specific to that couple - often it is constructed dynamically - it must be an SQL literal expression.

A couple has at least one bond. A couple must have one or more bonds of type Handle. All these bonds together define the single synchronisation key that will tie records in the source and target to each other. A couple can have zero or more bonds of type Column. These define the groups of columns that are kept in synchronized by the couple process (it may be all columns in the Target Table but generally its only a few). Each column group is either "straight" (1:1 mapping) or "lookup". A lookup group is used to indirectly provide the values for columns via another table. This is used when foreign keys need to be maintained by the couple but the stream data does not provide the actual key values, rather tags that can be used to lookup the actual key values. Lookups can be mandatory or optional (if nulls are allowed in the foreign key). Basic bond details can be entered by adding a row to the Bonds box on the Couple panel. The Position and Type are mandatory. Additional bond details can be entered by opening the Bond panel and querying for or selecting the relevant bond.

A bond must have at least one pair. A pair defines the tie between one column in the Couple Source Table and the corresponding column in the Target Table (and optionally an indirection column in a lookup table). A pair can also supply literal values (basic SQL expression with actual column value substitution where necessary) which can be used to achieve simple mappings where source data is not quite directly correlated with the target table types and/or values. For more complex mappings the Couple Source Table needs to be (or is better) defined as a view. Pair details can be entered by adding a row to the Pairs box on the Bond panel. Specify either: a Source Column (with optional modifying Source Value) and Target Column; just a Source Value and Target Column; either of the previous with no Target Column but either a Lookup Column (with optional modifying Lookup Value) or just a Lookup Value (possibly modifying a Source Column if specified).

In order to add new couples to a stream and update existing couples use the Streaming desktop. After querying for an appropriate model if necessary on the Choose Model panel choose the Couple panel. Create an entry for each couple. There are some mandatory fields. Select the source entity and target entity using the drop-down fields. It may be necessary to enter a couple position - this is needed when there are dependencies in target tables with data maintained by couples and consequently the couples must be initiated in a specific order. Save the changes and then re-query for the couple just added. Using the Bond box add each bond. There must be one and only one Handle. Additionally to this define each Column group bonding. For lookups use the entity and relationship drop-downs to define the intermediary table and foreign key value to return. Save changes (you cannot at the moment go on to define the pairs for each bond until the bonds themselves have been saved). Finally for each bond define the pairs. Use the drop-downs to specify the appropriate column from the source table, target table and optionally lookup table to be paired (kept in sync). For the source and lookup a literal SQL expression value can be used in place of actual column (or if a column is specified as well it can be substituted within the literal SQL expression value by use of the @ character). Also set the drop-down for the bond. Save changes.