Table of Contents
To get an instance of SymmetricDS running, it needs to be given an identity and know how to connect to the database it will manage. A basic way to specify this is to place properties in the symmetric.properties file. After connecting to the database, the Node reads its configuration and current status. If the configuration tables are missing, they are created automatically, unless that feature is disabled. A basic configuration describes the following:
Node Groups - each Node belongs to a group
Node Group Links - two Nodes Groups are linked together for synchronization
Nodes - each instance of Symmetric has an identity
Channels - data is categorized to synchronize independently
Triggers - specify which changes in the database are captured
During initialization, the Triggers are verified against the database, and database triggers are installed on tables that require data changes to be captured. The PullJob and PushJob begin running as required to synchronize changes with other Nodes.
As of 1.4 most system parameters are accessed using the ParameterService. The
ParameterService looks for parameters in a hierarchy of properties files and
in a database table. Parameters are re-queried from their source at a configured interval and can
also be refreshed on demand by using the JMX API. The following table defines the way parameters may be defined.
| Location | Required | Description |
|---|---|---|
| symmetric-default.properties | Y | Packaged inside symmetric-ds.jar file. This file has all the default settings along with descriptions. |
| symmetric.properties | N | Provided by the end user on the classpath. The first symmetric.properties found on the classpath will be used. |
| symmetric.properties | N | Provided by the end user in the current system user's user.home directory. |
| named properties file 1 | N |
Provided by the end user as a Java system property (i.e.
-Dsymmetric.override.properties.file.1=file://my.properties) or in
the constructor of a
SymmetricEngine
.
|
| named properties file 2 | N |
Provided by the end user as a Java system property (i.e.
-Dsymmetric.override.properties.file.2=classpath://my.properties) or
in the constructor of a
SymmetricEngine
.
|
| Java System Properties | N | Any SymmetricDS property can be passed in as a -D property to the runtime. It will take precedence over any properties file property. |
| Parameter table | N | A table which contains SymmetricDS parameters. Parameters can be targeted at a specific node group and even at a specific external id. These settings will take precedence over all of the above. |
| IParameterFilter | N | An extension point which allows parameters to be sourced from another location or customized. These settings will take precedence over all of the above. |
Table 5.1. Parameter Locations
Also see the appendix on Startup Parameters to see all the possible
properties and their defaults.
Each Node requires properties that will connect it to the database and register it with a parent Node. To give a Node its identity, the following properties are used:
The Node Group that this Node is a member of. Synchronization is specified between Node Groups, which means you only need to specify it once for multiple Nodes in the same group. For example, you might have groups of "STORE", "REGION", and "CENTRAL" that synchronize.
The External ID for this Node has meaning to the user and provides integration into the system where it is deployed. For example, it might be a retail store number or a region number. The External ID can be used in expressions for conditional and subset data synchronization. Behind the scenes, each Node has a unique sequence number for tracking synchronization events. That makes it possible to assign the same External ID to multiple Nodes, if desired.
The URL where this Node can be contacted for synchronization. At startup and during each heartbeat, the Node updates its entry in the database with this URL.
When a new Node is first started, it is has no information about synchronizing. It contacts the registration server in order to join the network and receive its configuration. The configuration for all Nodes is stored on the registration server, and the URL must be specified in the following property:
The URL where this Node can connect for registration to receive its configuration. The registration server is part of SymmetricDS and is enabled as part of the deployment.
For a deployment to an application server, it is common for database connection pools to be found in the Java naming directory (JNDI). In this case, set the following property:
The name of the database connection pool to use, which is registered in the JNDI directory tree of the application server. It is recommended that this DataSource is NOT transactional, because SymmetricDS will handle its own transactions.
For a deployment where the database connection pool should be created using a JDBC driver, set the following properties:
The class name of the JDBC driver.
The JDBC URL used to connect to the database.
The database username, which is used to login, create, and update SymmetricDS tables.
The password for the database user.
The name of a Spring bean to use as the DataSource. If you want to use a different DataSource other than the provided DBCP version that SymmetricDS uses out of the box, you may set this to be the Spring bean name of your DataSource.
Each Node must belong to a Node Group, a collection of one or more Nodes. A common use of Node Groups is to describe a level in a hierarchy of data synchronization. For example, at a retail store chain, there might be a few Nodes that belong to "corp", which sync with hundreds of Nodes that belong to "store", which sync with thousands of Nodes that belong to "register".
The following SQL statements would create Node Groups for "corp" and "store".
insert into SYM_NODE_GROUP
(node_group_id, description)
values
('store', 'A retail store node');
insert into SYM_NODE_GROUP
(node_group_id, description)
values
('corp', 'A corporate node');
To establish synchronization between Nodes, two Node Groups are linked together. The direction of synchronization is determined by specifying a source and target Node Group. If synchronization should occur in both directions, then two links are created in opposite directions. The target Node Group receives data changes by either push or pull methods. A push method causes the source Node Group to connect to the target, while a pull method causes it to wait for the target to connect to it.
The following SQL statements links the "corp" and "store" Node Groups for synchronization. It configures the "store" Nodes to push their data changes to the "corp" Nodes, and the "corp" Nodes to send changes to "store" Nodes by waiting for a pull.
insert into SYM_NODE_GROUP_LINK
(source_node_group, target_node_group, data_event_action)
values
('store', 'corp', 'P');
insert into SYM_NODE_GROUP_LINK
(source_node_group, target_node_group, data_event_action)
values
('corp', 'store', 'W');
Each instance of SymmetricDS is a Node that can be uniquely identified. The Node has a unique identifier used by the system, and the user provides an external identifier for context in the local system. For most common use, the two identifiers are the same. The registration process generates and sends the identity and password to the Node, along with its synchronization configuration. The top-level registration server must have its identity provided by the user since it has no parent to contact.
The following SQL statements setup a top-level registration server as a Node identified as "00000" in the "corp" Node Group.
insert into SYM_NODE
(node_id, node_group_id, external_id, sync_enabled, symmetric_version)
values
('00000', 'corp', '00000', 1, '1.4.0');
insert into SYM_NODE_IDENTITY values ('00000');
Data changes in the database are captured in the order that they occur, which is preserved when synchronizing to other Nodes. Some data may need priority for synchronization despite the normal order of events. Channels provide a higher-level processing order of data, a limit on the amount of data, and isolation from errors in other channels. By categorizing data into channels and assigning them to Triggers, the user gains more control and visibility into the flow of data.
The following SQL statements setup channels for a retail store. An "item" channel includes data for items and their prices, while a "sale_transaction" channel includes data for ringing sales at a register.
insert into SYM_CHANNEL
(channel_id, processing_order, max_batch_size, enabled, description)
values
('item', 10, 100000, 1, 'Item and pricing data');
insert into SYM_CHANNEL
(channel_id, processing_order, max_batch_size, enabled, description)
values
('sale_transaction', 1, 100000, 1, 'retail sale transactions from register');
At the heart of SymmetricDS are Triggers that define what data to capture. Nodes in the source Node Group will capture changes for a table and send them to a target Node Group. Changes can include inserts, updates, or deletes to the table, and it is even possible to filter data by a conditional expression. An entry in Trigger results in a database trigger being installed on the table. Whenever the Trigger entry is updated, the last_updated_time should be updated to indicate that the database trigger should also be updated.
The following SQL statement defines a Trigger that will capture data for a table named "item" whenever data is inserted, updated, or deleted. Data will be captured on Nodes in the "corp" Node Group and sent to Nodes in the "store" Node Group.
insert into SYM_TRIGGER
(source_table_name, source_node_group_id, target_node_group_id, channel_id,
sync_on_insert, sync_on_update, sync_on_delete,
initial_load_order, last_updated_by, last_updated_time, create_time)
values
('item', 'corp', 'store', 'item',
1, 1, 1,
105, 'demo', current_timestamp, current_timestamp);