xSQL Data Comparison and Synchronization SDK 3.0

xSQL Data Compare Command Line (xSQLDataCmd.exe)

Overview

The xSQL Data Compare Command Line utility allows for comparing the content of SQL Server databases via the command line. With the xSQL Data Compare Command Line you can:


Data compare options are specified via an xml configuration file. Additional command line switches are used for settings such as encoding, file location etc.

Runtime files

The following files are required to run the Command Line:



Command line argument and options

The command line options control the general behavior of xSQL Data Compare Command line utility such as the mode of operation (quiet vs. verbose), file encoding, the location of the log and error file. Each option has a short and a long form. For example /q and /quiet specify the “quiet” mode which suppresses all comparison messages from being displayed on the screen or saved to the output file. The value should be separated from the option using a colon, not a space. For example /l:c:\compare.log saves all comparison messages to the output file c:\compare.log.

The data comparison options and preferences on the other hand are specified via the xml configuration file. When the same command line option exists for an xml-specified parameter, the command line value has priority over the one specified in the xml file.


The xSQL Data Compare Command Line utility supports the following argument and options:

Argument Description
Xml File The xml configuration file that contains the comparison arguments and options.
Option Description
? Displays the help screen.
/v[alidate] Performs validation of the Xml configuration file. No data comparison is performed.
/l[ogfile]:filename Specifies the log file where the comparison messages should be saved.
/t /logtype:File|WindowsLog Indicates the type of logging for data comparison errors. Supported options are disk file (File) or Windows log (WindowsLog).
/e[rror]:filename Specified the name of the error log. The Command Line can log the errors on disk files or Windows logs depending on the /t option.
  • If the /t:WindowsLog is specified, the /e option indicates a Windows log. If the specified log is one of the Windows built-in logs, the Command Line uses that log for all the comparison errors; otherwise it creates a new one.
    The following example logs comparison errors to Windows Application log:

    /t:WindowsLog /e:Application

  • If the /f:File is specified, the /e option indicates a disk file.
Note: Use /t:WindowsLog carefully. The Command Line creates a new Windows log if the log name specified with /e option does not exist.
/o[verride] Overrides all the output files.
/q[uiet] Indicates the quite mode, which suppresses all comparison messages from being displayed on the screen or saved to the log file.
/en[coding] Defines the encoding that will be used for all output files. Specify a UTF7, UTF8, ASCII, Unicode or ANSI encoding with this option. The default encoding is ANSI.


Xml configuration file


Overview

xSQL Data Compare Command Line utility uses an Xml configuration file for all comparison options and preferences.

The Xml configuration file allows you to:

The xml file groups the comparison parameters and options into the following sections:

Section Description
Database
   <LeftDatabase>
   <RightDatabase>
Contains the databases that will be compared. The Command Line requires two sections, one for each database. These are referred to as the “left” and the “right” database.
Data Compare Options
   <DataOptionTypes>
Contains data comparison options.
Pairs of Tables
   <TablePairs>
Defines the pairs-of-tables used in the comparison. The database tables are normally paired before the comparison based on their names, but any other custom pairing method can be used as well.
Command Line Settings
   <CommandLineSettings>
Specifies additional settings such as the file name of the data synchronization script, synchronization log, error log, file encoding.

A simple Xml data configuration file has the following structure:

<?xml version="1.0" encoding="utf-8" ?>
<CommandLineParameters xmlns="http://www.xsqlsoftware.com/schemas/xSQLDataCompareCmdSchema.xsd">
        <LeftDatabase>
                <SqlServer>(local)</SqlServer>
                <DatabaseName>DB_Prod</DatabaseName>
                <TrustedConnection>true</TrustedConnection>
        </LeftDatabase>
        <RightDatabase>
                <SqlServer>(local)</SqlServer>
                <DatabaseName>DB_Staging</DatabaseName>
                <TrustedConnection>true</TrustedConnection>
        </RightDatabase>
        <DataOptions>
                <Option>DefaultOptions</Option>
        </DataOptions>
        <CommandLineSettings>             
                <DataScriptFile>DataScript.sql</DataScriptFile>
                <DataLogFile>DataLog.txt</DataLogFile>
                <ErrorLogName>DataEror.txt</ErrorLogName>
                <CompareData Direction="RightAsLeft"  Synchronize="false"></CompareData>
        </CommandLineSettings>
</CommandLineParameters>
      
The <LeftDatabase> and <RightDatabase> elements define the database on the “left” and the “right” side of the comparison respectively.
In this example these are the databases DB_Prod and DB_Staging located on the (local) SQL Server. The connection to SQL Server is Trusted Connection.
<DataOptionTypes> element specifies the data comparison options. The value DefaultOptions indicates the default data compare options.
<CommandLineSettings> element contains:

<ErrorFile>: The name of the file that will store the data comparison errors if any such errors occur.
<DataScriptFile>: The name of the file that will store the data synchronization script.
<DataLogFile>: The name of the file that will store the data synchronization log. The data log describes the actions performed by the data synchronization script.
<CompareData>: This element contains additional attributes that specify how the data comparison is performed. The Direction attribute determines the direction of the comparison as LeftToRight or RightToLeft. In a left-to-right comparison, the Command Line makes the content of the <LeftDatabase> the same as the content of <RightDatabase> database. The Synchronize attribute determines whether the command line should stop with the generation of the script or whether it should execute the script to synchronize databases. The false value indicates that script should only be generated and should not be executed.

The following command invokes the xSQL Data Compare Command Line with this Xml configuration file (assuming that the Xml file is named simpleXml.xml)

        xSQLDataCmd simpleXml.xml
      

Customizing data comparison

Data comparison uses the concept of pair-of-tables to perform database content comparison and synchronization. A pair-of-table is an entity that maps together two tables identified as comparable from the data point along with two unique keys, one for each table. The tables are mapped based on their name equality; for example the table A on left database is paired with table A on the right database. The unique key on each of the mapped tables is selected by evaluating the primary key first, then unique constraints and unique indexes, whichever is found first. Keys are selected in such a way that they are compatible. Data compare requires that each pair of table has a selected key; if not the pair cannot be compared.


The Command Line utility allows for customization of the data comparison by performing the following:


The <TablePairs> section of the Xml configuration file provides the necessary elements and attributes to achieve these customizations. The <TablePairs> section contains one or multiple <TablePair> elements. Normally you would have one <TablePair> element for every pair-of-table entity that requires customization.

Depending on the value of the TableMappingSchemaTypes element defined under the CommandLineSettings section, the Command Line chooses to perform one of the following actions when encounters custom-defined pairs:


A <TablePairs> defined in the Xml file might contain the following elements:

Element Description
LeftTable
RightTable
Defines the left and right table of a <TablePairs> element. Tables mapped via this element are not required to have the same name, but should be selected so that they comparable.
WhereClause A Sql server “where” clause that is applied to both tables in a pair. This can be any valid Sql server expression used in the “where” part of the T-Sql without the keyword “where”. It acts as a filter and allows for comparison of the subset of data.
Action It is an attribute that determines whether a pair of tables is included or excluded from the data comparison. It can have the Include or Exclude value.
Columns It is a collection of <ColumnPair> elements. Each <ColumnPair> maps a column from the left table with one from the right table. The pair contains an attribute of the type Include/Exclude. This allows for certain columns to be excluded from the comparison or even custom column mapping between the tables in a pair.
Key It is a <DataKey> element that defines a custom key. The custom key is a construct used by the data compare engine to perform comparison between tables that either don’t have any unique key in the database or when the database key is not suitable. Normally, the absence of the database unique key would make such tables non-comparable. To overcome this limitation, xSQL Data Compare Command Line uses custom keys. A custom key is similar to a unique index in the database, except that it is not created on the database. It only exists for the purpose of the data comparison; it has a name and a set of participating columns.
Note: The Command Line always performs pair validation before comparing the data. The validation guarantees that the pairs meet the criteria required for comparing them such as the data compatibly between columns, the existence of a unique key etc. An invalid pair, whether created automatically or custom-defined, is always excluded from the comparison.

The following examples illustrate some common customizations that can be performed with pair-of tables.

1. Excluding a pair-of-table from the data comparison
This Xml fragment excludes the pair that contains Author table from the data comparison. Both the left and the right table are required to be specified even when they have the same name:

<TablePair Action="Exclude">
<LeftTable Name="dbo.Authors"></LeftTable>
<RightTable Name="dbo.Authors"></RightTable>
</TablePair>
      
2. Creating a custom key
This Xml fragment creates a custom key for the pair that contains the table Customer. The key is named CompanyAndCity and contains two columns: CompanyName and City. The key is specified for each table in the pair.
<TablePair Action="Include">
    <LeftTable Name="dbo.Customers">
        <Key Name="CompanyAndCity">
            <Column>CompanyName</Column>
            <Column>City</Column>
        </Key>
    </LeftTable>
    <RightTable Name="dbo.Customers">
        <Key Name="CompanyAndCity">
            <Column>CompanyName</Column>
            <Column>City</Column>
        </Key>
    </RightTable>
</TablePair>
      
3. Filtering the data with the “where” clause
This Xml fragment uses a where clause to filter that data that will compared between two tables in a pair. Remember NOT to include the keyword “where” as part of the clause.
<TablePair Action="Include">
<LeftTable Name="dbo.Customers"></LeftTable>
    <RightTable Name="dbo.Customers"></RightTable>
    <WhereClause>Country='USA'</WhereClause>
</TablePair>        
      
4. Excluding columns
The following Xml fragment excludes the Country column from the pair that contains Customer table. The other columns mapped between the two tables have been listed here for illustration only; by default they are included.
<TablePair Action="Include">
    <LeftTable Name="dbo.Customers"></LeftTable>
    <RightTable Name="dbo.Customers"></RightTable>
        <Columns>
            <ColumnPair Action="Exclude">
                <LeftColumn>Country</LeftColumn>
                <RightColumn>Country</RightColumn>
            </ColumnPair>
            <ColumnPair>
                <LeftColumn>CustomerID</LeftColumn>
                <RightColumn>CustomerID</RightColumn>
            </ColumnPair>
            <ColumnPair>
                <LeftColumn>CompanyName</LeftColumn>
                <RightColumn>CompanyName</RightColumn>
            </ColumnPair>
            <ColumnPair>
                <LeftColumn>City</LeftColumn>
                <RightColumn>City</RightColumn>
            </ColumnPair>
            <ColumnPair>
                <LeftColumn>Region</LeftColumn>
                <RightColumn>Region</RightColumn>
            </ColumnPair>
        </Columns>
    </TablePair>
</TablePairs>       
      

Examples


xSQL Data Compare Command Line sample Xml files are located in the \Samples\CommandLines\DataCmd subfolder under the main installation folder of xSQL Bundle.



Xml Configuration File Schema


<CommandLineParameters>
    <LeftDatabase>
        <SqlServer>
        <DatabaseName>
        <Username>
        <Password>
        <TrustedConnection>
        <ConnectionTimeout>
        <CommandTimeout>
        <SnapshotFile>
        <CreateSnapshot>
    </LeftDatabase>
    <RightDatabase>
        <SqlServer>
        <DatabaseName>
        <Username>
        <Password>
        <TrustedConnection>
        <ConnectionTimeout>
        <CommandTimeout>
        <SnapshotFile>
        <CreateSnapshot>
    </RightDatabase>
    <DataOptionTypes>
        <Option>
    </DataOptionTypes>
    <TablePairs>
        <TablePair Action>
            <LeftTable Name>
                <Key Name>
                    <Column>
                </Key>
            </LeftTable>
            <RightTable Name>
                <Key Name>
                    <Column>
                </Key>
            </RightTable>
            <WhereClause>
            <Columns>
                <ColumnPair Action>
                    <LeftColumn>
                    <RightColumn>
                </ColumnPair>
            </Columns>
        </TablePair>
    </TablePairs> 
    <CommandLineSettings>
        <LogFile>
        <ErrorFile>
        <DataScriptFile>
        <DataLogFile>
        <FileEncoding>
        <CompareData Direction Synchronize>
    </CommandLineSettings>
</ CommandLineParameters >  
      

Element Description
CommandLineParameters Root element.
LeftDatabase, RightDatabase The elements that specify the left and the right database that participate in the data comparison.
SqlServer The Sql Server that contains the database.
DatabaseName The name of database.
Username The Sql Server login name.
Password The Sql Server password.
TrustedConnection The Sql server authentication type. true if the Windows authentication (trusted connection) should be used to connect to Sql Server; otherwise false. If not specified the default authentication method is Windows authentication.
ConnectionTimeout The connection timeout in seconds, which is the amount of time that Command Line waits to establish a connection with the Sql server before generating an error. The default timeout is 60 seconds.
CommandTimeout The command timeout in seconds, which is the amount of time that Command Line waits to execute a Sql statement before generating an error. The default timeout is 20 minutes.
DataOptionTypes Starts the section that defines the data options.
Option A data comparison option. Specify one of the followings:
DefaultOptions

Equal
Different
LeftOnly
RightOnly
DisableFKConstraints
DisableTriggers
GenerateDetailedLog
SynchronizeTextDataAsChar
SynchronizeWithBulkInsert
TrimSpaces
RoundApproximateNumber
TablePairs Starts the section that customizes the pair-of-tables.
TablePair Specifies a pair-of-table entity.
Action The attribute that defines whether a pair is included in or excluded from the data comparison. It can have the values Include or Exclude.
LeftTable
RightTable
The left or the right table that participates in a pair-of-table mapping.
Key The data custom key. The key is specific to each table in a pair.
Name The name of the custom key.
Column The name of a column that participates in the custom key. The key can contains one or multiple columns.
WhereClause Defines a “where” clause that will be applied to both tables in the pair.
Columns Starts the section that defines the columns that will be excluded from the comparison.
ColumnPair A pair-of-column to be excluded.
Action The attribute that defines whether the pair of columns is included or excluded. It accepts the values Include or Exclude.
LeftColumn
RightColumn
The columns in a pair-of-column.
CommandLineSettings Contains additional settings used by the command line utility.
LogFile Logs the data comparison messages to the specified file.
ErrorLogType Determines the type of logging of the data comparison errors. This can be File or WindowsLog.
ErrorLogName Logs the errors that might occur during the comparison or synchronization to the specified file or Windows log.
DataScriptFile Saves the data synchronization script to the specified file.
DataLogFile Saves the data synchronization logs to the specified file.
FileEncoding Specifies the encoding of all output files. This element can have one of these values: Unicode, UTF7, UTF8, ASCII, ANSI. The default encoding is ANSI.
CompareSchema Contains the following attributes:
  • Direction:  Indicates whether the content of the left database should be synchronized with the content of the right database or vice versa. Specofy LeftAsRight or RightAsLeft values.
  • Synchronize:  If true the data synchronization script is executed at the end of the comparison; otherwise the script is only generated.
TableMappingSchemaTypes Determines the behavior of the Command Line against the custom pairs. It can have one of the following values:
  • Mix:  custom-defined pairs override the pairs created automatically. All pairs are compared.
  • CustomOnly:  only the custom-defined pairs are compared. No automatic paring takes place.


Error Codes

The Command Line utility returns the following error codes:

Error Code Description
0 No error
1 Invalid Command Line options
2 Invalid Xml file
4 The path of the output files is invalid
8 Failed to compare databases.
16 Failed to execute the synchronization script.