NME Internal Columns

Transaction ID Column	Date Column	Event	DB Columns
eng_creation_tid	eng_creation_date	Record creation. Always non-null.	eng_ctid/eng_ctdt
eng_deletion_tid	eng_deletion_date	Record deletion. Null for existing records. Reasonable in internal plans and delta exports.	eng_dtid/eng_dtdt
eng_last_update_tid	eng_last_update_date	Last update of the record. Always non-null.	eng_mtid/eng_mtdt
eng_last_source_update_tid	eng_last_source_update_date	Used only for instance records. This is the last update of the record in delta detection phase (excludes internal calculations). Always non-null. For example, the last time the record was sent by source system and change, it was detected in the delta detection phase.	eng_smtid/eng_smtdt
eng_activation_tid	eng_activation_date	(Last) record activation. Always non-null.	eng_atid/eng_atdt
eng_deactivation_tid	eng_deactivation_date	(Last) record deactivation. Null for active records.	eng_itid/eng_itdt

NME Database Columns

List of internal columns, which is a part of each instance or master table maintained by the NME engine. Mentioned column names (both database columns and pseudocolumns) cannot be renamed or used for a different purpose than specified. Values cannot be changed directly, but only by the NME processes or features.

These columns are not a part of the NME Model (nme-model.xml) definition, but all of them are added by the NME engine automatically. Some of the columns mentioned below are also available in the related iWay DQS plans, serving as transitions in NME under either the database column name (DB) or audit pseudocolumn name (AP) - see Available in a plan as DB/AP column in the table below. There are also NME Audit Pseudocolumns, which are not related to any database column (usually), but are available in iWay DQS plans only.

Default Columns

There are no database types shown in following table, because they are dependent on the particular DB used.

Database Column

Instance Table

Master Table

Description

Available in a Plan as DB/AP

Related Audit Pseudocolumn Name

iWay DQS Data Type

id

x

Unique id automatically assigned to each incoming instance record by the NME engine

Serves as PK when DB persistence is used

DB

N/A

long_int***

id

x

Mapped in merge plan manually from master_id as a result of representative creator step.

Serves as PK when DB persistence is used.

DB

N/A

long_int***

source_id*

x

Unique ID of source system record. Mapped in load plan/process delta service.

DB

N/A

string

master_id

x

Used to store the result of the default matching, filled in match plan as a result of the unification step.

Serves as a foreign key to the corresponding master entity.

DB

N/A

long_int***

origin_id

x

Stores the origin ID of data origin, defined in [prefix]_ORIGIN table, CODE column (number).

Related AP contains textual representation of the origin_id. Equal to origin mapped in a load plan/process delta service.

AP

eng_origin

string

eng_active

x

x

AP

eng_active

boolean

eng_existing

x

In general, when a record is deleted, it has eng_existing=false. These records are not deleted physically because they are used for the purposes of exporting.

Determines if the record is reported from the source system as existing (when using sourceDeletionStaregy=DELETE). If the record is deleted, eng_existing = false.

N/A

N/A

boolean

eng_existing

x

In general, when a record is deleted, it has eng_existing=false.

These records are not deleted physically, because they have to be there for export purpose.

Determines if the record is reported as deleted. It happens when two groups are merged into the one and eng_existing=false is set for the master record which was terminated.

N/A

N/A

boolean

eng_external_record

x

Flag identifying a record sent by the Identify Service.

AP

N/A

boolean

eng_system

x

AP

eng_source_system

string

eng_ctid

x

x

AP**

eng_creation_tid

long_int***

eng_ctdt

x

x

AP

eng_creation_date

datetime

eng_dtid

x

x

AP**

eng_deletion_tid

long_int***

eng_dtdt

x

x

AP

eng_deletion_date

datetime

eng_mtid

x

x

AP**

eng_last_update_tid

long_int***

eng_mtdt

x

x

AP

eng_last_update_date

datetime

eng_smtid

x

AP**

eng_last_source_update_tid

long_int***

eng_smtdt

x

AP

eng_last_source_update_date

datetime

eng_atid

x

x

AP**

eng_activation_tid

long_int***

eng_atdt

x

x

AP

eng_activation_date

datetime

eng_itid

x

x

AP**

eng_deactivation_tid

long_int***

eng_itdt

x

x

AP

eng_deactivation_date

datetime

Database Column	Instance Table	Master Table	Description	Available in a Plan as DB/AP	Related Audit Pseudocolumn Name	iWay DQS Data Type
id	x		Unique id automatically assigned to each incoming instance record by the NME engine Serves as PK when DB persistence is used	DB	N/A	long_int***
id		x	Mapped in merge plan manually from master_id as a result of representative creator step. Serves as PK when DB persistence is used.	DB	N/A	long_int***
source_id*	x		Unique ID of source system record. Mapped in load plan/process delta service.	DB	N/A	string
master_id	x		Used to store the result of the default matching, filled in match plan as a result of the unification step. Serves as a foreign key to the corresponding master entity.	DB	N/A	long_int***
origin_id	x		Stores the origin ID of data origin, defined in [prefix]_ORIGIN table, CODE column (number). Related AP contains textual representation of the origin_id. Equal to origin mapped in a load plan/process delta service.	AP	eng_origin	string
eng_active	x	x		AP	eng_active	boolean
eng_existing	x		In general, when a record is deleted, it has eng_existing=false. These records are not deleted physically because they are used for the purposes of exporting. Determines if the record is reported from the source system as existing (when using sourceDeletionStaregy=DELETE). If the record is deleted, eng_existing = false.	N/A	N/A	boolean
eng_existing		x	In general, when a record is deleted, it has eng_existing=false. These records are not deleted physically, because they have to be there for export purpose. Determines if the record is reported as deleted. It happens when two groups are merged into the one and eng_existing=false is set for the master record which was terminated.	N/A	N/A	boolean
eng_external_record	x		Flag identifying a record sent by the Identify Service.	AP	N/A	boolean
eng_system	x			AP	eng_source_system	string
eng_ctid	x	x		AP**	eng_creation_tid	long_int***
eng_ctdt	x	x		AP	eng_creation_date	datetime
eng_dtid	x	x		AP**	eng_deletion_tid	long_int***
eng_dtdt	x	x		AP	eng_deletion_date	datetime
eng_mtid	x	x		AP**	eng_last_update_tid	long_int***
eng_mtdt	x	x		AP	eng_last_update_date	datetime
eng_smtid	x			AP**	eng_last_source_update_tid	long_int***
eng_smtdt	x			AP	eng_last_source_update_date	datetime
eng_atid	x	x		AP**	eng_activation_tid	long_int***
eng_atdt	x	x		AP	eng_activation_date	datetime
eng_itid	x	x		AP**	eng_deactivation_tid	long_int***
eng_itdt	x	x		AP	eng_deactivation_date	datetime

* means that it can be defined as a string with size 1000, mapped to DB as varchar(2000), and respectively varch2(2000) for Oracle (2000 because of UTF). This is coded in the NME source code. Maximum allowed content is 1000 chars.

** means that the column can be available in an iWay DQS plan, but is unreasonable. In general, various TID/DATE appearances in iWay DQS plans depend on a particular MMD definition.

*** currently, it is used only in iWay MDS. It refers to a data type longer then an iWay DQS integer and shorter then iWay DQS long. For example, its size is limited to 8 bytes because of performance reasons (max value is 264, roughly 18 extra bytes).

Optional Columns

Database Column

Instance Table

Master Table

Description

Available in a Plan

Related Pseudocolumn Name

iWay DQS Data Type

xid

x

x

Primary key of each table.

N/A

N/A

long_int***

xctid

x

x

Internal technical column.

N/A

N/A

long_int***

Database Column	Instance Table	Master Table	Description	Available in a Plan	Related Pseudocolumn Name	iWay DQS Data Type
xid	x	x	Primary key of each table.	N/A	N/A	long_int***
xctid	x	x	Internal technical column.	N/A	N/A	long_int***

Parameter	Description
eng_active (boolean)	Instance records. Determines if the record is reported from source system as existing (when using sourceDeletionStaregy=DEACTIVATE). Master records. Determines if at least one instance record corresponding to this master has eng_active=true.
eng_origin (string)	Instance records only. Contains the textual representation of the record origin. It is stored as origin_id column in a database.
eng_source_system (string)	Instance records only. Contains the textual representation of the system corresponding to the record origin. Database column name is eng_system.

Parameter

Description

eng_active (boolean)

Instance records. Determines if the record is reported from source system as existing (when using sourceDeletionStaregy=DEACTIVATE).
Master records. Determines if at least one instance record corresponding to this master has eng_active=true.

eng_origin (string)

Instance records only. Contains the textual representation of the record origin. It is stored as origin_id column in a database.

eng_source_system (string)

Instance records only. Contains the textual representation of the system corresponding to the record origin. Database column name is eng_system.