There are two types of entities:

• Instance Entities. Defines entities and their behavior on the instance layer. An entity is defined by columns, relationships, and operations.

  <entities>
  	<entity name="party">
  		<columns>
  			..
  		</columns>
  		<relationships/>
  	<cleansingOperation>
  	..
  		<cleansingOperation>
  	<matchingOperation>
  	..
  	</matchingOperation>
  	</entity>
  	..
  	</entity>
  </entities>

  Parameters:

  • <name>. Name of the entity.
  • usage. Usage of the entity.
    • DATA_ENTITY. Default, standard data entity.
    • DICTIONARY.Entity with relatively small number of records used for storing dictionary values, no indices will be generated on relationships from data entity to dictionary.
• Master Entities. Defines entities and their behavior on master layer.

  <entities>
  	<entity instanceEntity="party" name="party"
  class="com.ataccama.nme.engine.model.PersistentMasterEntity" groupingColumn="master_id">
  		<columns>
  		..
  		</columns>
  		<relationships/>
  	<mergingOperation>
  	..
  	</mergingOperation>
  	</entity>
  		<entity instanceEntity="party" name="party_instance"
  class="com.ataccama.nme.engine.model.VirtualInstanceEntity">
  		<relationships>
  	..
  	</relationships>
  	</entity>
  </entities>

  where:

  PersistentMasterEntity

  Defines a physical table to be stored in a database.

  VirtualInstanceEntity

  Defines a virtual entity, which is required to create master to instance relationship.

  name

  Is the name of the entity.

  instanceEntity

  Is the corresponding entity on the instance layer.

  groupingColumn

  Specifies the grouping column used to create a golden record. Column has to be of long_int data type (only for PersistentMasterEntity). Default column name = master_id.

  Usage of the PersistentMasterEntity:

  • DATA_ENTITY. Default, standard data entity.
  • DICTIONARY. Entity with relatively small number of records used for storing dictionary values. No indices will be generated on relationships from data entity to dictionary.

There are two kinds of relationships:

• Instance Relationship. Relationship definition is always a part of a child entity definition, and it points to a parent entity.

  <relationships>
  	<rel reverseName="addresses" parentEntity="party" parentColumn="source_id" name="party" foreignKeyColumn="pty_source_id">
  		<type class="com.ataccama.nme.engine.model.SameSystemRelationship"/>
  	</rel>
  </relationships>

  where:

  name

  Is the relationship name. It can be used as a reference name for copy columns.

  reverseName

  Is the reverse relationship name. It can be used as a reference name for copy columns.

  parentEntity

  Is the name of the parent entity.

  parentColumn

  Is the primary key column of the parent entity.

  foreignKeyColumn

  Is the foreign key column of the child entity.

  <type>

  Defines the type of relationship.

  SameSystemRelationship

  Is the relationship created only between IDs having the same origin. For example, within a single system. If it is not defined, sameSystemRelationship is used by default.

  CrossSystemRelationship

  Enables you to create a relationship between two entities regardless of the origin. Useful for code books and dictionaries.

  This is the same definition as the instance relationship, except <type> is not defined because there is no longer a source system reference. Therefore, the behavior is the same as described for sameSystemRelationship above.

• Master Relationship. The Master relationship uses the same logic as the Instance relationship. However, there is a difference in <type> behavior:
  • CrossSystemRelationship. This has to be used for a master-to-master relationship and master-to-instance relationship. It can be used between VirtualInstanceEntity entities especially for code books and dictionaries. If <type> is not defined, then CrossSystemRelationship becomes the default behavior of the relationship where one of the entities is PersistentMasterEntity.
  • SameSystemRelationship. Can be used only between VirtualInstanceEntity entities. If <type> is not defined, SameSystemRelationship becomes the default behavior of the relationship between VirtualInstanceEntity entities.

<relationships>
	<rel reverseName="addresses" parentEntity="party" parentColumn="id" name="party" foreignKeyColumn="pty_id">
	<type class="com.ataccama.nme.engine.model.CrossSystemRelationship"/>
	</rel>
</relationships>

The following syntax shows the column definition per each entity.

-- instance columns definition
<columns>
	<column name="src_name" origin="source" type="string" size="100"/>
	<column name="std_name" origin="clean" type="string" size="100"/>
	<column name="src_ssn" origin="source" type="string" size="100"/>
	<column name="std_ssn" origin="clean" type="string" size="100"/><column
	 name="uni_can_id" origin="match" type="long" size="20"/>
</columns>
-- master columns definition
<columns>
	<column name="cmo_first_name" type="string" size="100"/>
	<column name="cmo_last_name" type="string" size="100"/>
</columns>

where:

name

Is the column name.

origin

Defines a place in a plan, where the column appears (column definition context - instance columns only).

“source”

Is the source value and appears in the cleanse plan input.

“clean”

Is cleansed and appears as a cleanse plan output.

“match”

Is the match value and appears as a match plan output.

type

Is the DQS data type.

size

Is the column length (used as a DB column size).

Copy Columns

The copy column functionality is necessary for copying an attribute from one entity to another. To define a copy column, a relationship definition must exist between the entities. After it is defined, you can define the direction of copy columns: parent to child or child to parent.

<column name="mrg_addresses" origin="COPY_CLEAN" type="STRING" size="100">
	<valueSource relationshipName="addresses" columnName="std_value">
	<aggregation class="com.ataccama.nme.engine.model.ConcatenateDistinct" separator=";"maxLength="100"/>
	<filterExpression>source.std_address_type = "R"</filterExpression>
	</valueSource>
</column>

where:

name

Is the name of the column.

type

Is the type of the column.

size

Is the size of the column.

origin

In the copy column context, it defines which columns are taken, and the place where the copied columns appear.

“copy_source”

Is the source column added to a cleansing plan output.

“copy_clean”

Is the cleansed result column added to a match plan input.

“copy_match”

Is the matched result column added to a merge plan input.

<valueSource>

Contains the columnName or the relationshipName.

columnName

Is the source column for the copy.

relationshipName

Is the relationship definition and direction to be used for the copy.

aggregation class

Contains the different values and concatenations.

ConcatenateDistinct

Is used to perform child to parent copy column. Because of the 1:M relationship, there is a concatenate distinct functionality to store all the values related to a single attribute on the parent entity.

separator

Is the values separator definition for concatenation.

maxLength

Is the length of concatenated outcome.

MinValue

Is used to perform child to parent copy columns. It takes minimum value from all related entities (because of 1:M relationship).

MaxValue

Is used to perform child to parent copy columns. It takes maximum value from all related entities (because of 1:M relationship).

FirstValue

Is used to perform parent to child copy columns. It can be used for child to parent copy columns, but there is no deterministic selection.

<filterExpression>

Is (boolean) optional. It provides an ability determine when column copying should be performed using a filter expression. There are two pseudo inputs available: source (origin of copy) and target to construct the expression.

History Collector

Enables you to keep historical values of a given column and stores concatenated values to a newly defined column. Such functionality can be useful in a match process.

<column name="std_uir_adr_id_h" origin="CLEAN" type="STRING" size="40000">
<historyCollector columnName="std_uir_adr_id" separator="^~" maxCount="20"/>
</column>

where:

columnName

Is the source column.

separator

Is the values separator definition for concatenation.

maxCount

Is the number of values kept in history.

OldValueProvider

OldValueProvider is a way to access old value of the instanceEntity column in the transition plan.

<column name="std_expiration_date" origin="clean" type="datetime" size="100">
	<oldValueProvider columnName="std_expiration_date_old" />
</column>

where:

columnName

Defines target column with oldValue, which is available in transition plans.

The following table shows how oldValueProvider can be defined for columns with their origins.

Origin of Column	oldValue Column Availability in Transition Plan
SOURCE	clean, match, merge
COPY_SOURCE	clean, match, merge
CLEAN	clean, match, merge
COPY_CLEAN	match, merge
MATCH	match, merge
COPY_MATCH	merge

Note: Do not use oldValue based expressions after the Extended Unification step in the match plan. Some additional records can be expanded from the unification repository during the match process, and the result will not be exact. Instead, move this computation to the merge plan, where all the relevant data and their oldValues are available.

This section describes the operations involved for the NME configuration.

Cleanse Operation

Defines a cleanse logic for a given entity, which is represented by an iWay DQS plan.

<cleansingOperation class="com.ataccama.nme.dqc.operations.CleansingPlan"
planFileName="../trans/party/party_clean.comp"/>

Match Operation

Defines a match logic for a given entity, which is represented by an iWay DQS plan.

<matchingOperation enableIdentify="false" class="com.ataccama.nme.dqc.operations.MatchingPlan"
planFileName="../trans/party/party_match.comp"/>

where:

enableIdentify

Is boolean and defines whether the match plan is used to perform identify services or not. It is also the performance optimization (default is true).

Custom Identify Plan Definition

The NME engine allows you to define a special matching plan to perform the identifyService using the identifyPlanFileName attribute.

<matchingOperation enableIdentify="false" class="com.ataccama.nme.dqc.operations.MatchingPlan"
planFileName="../trans/party/party_match.comp"
identifyPlanFileName="../trans/party/identify_party_match.comp"/>

Merge Operation

The operation defines a link between NME plan representing a merge operation and master entity definition.

<mergingOperation customActivityTracking="false"
class="com.ataccama.nme.dqc.operations.MergingPlan"
planFileName="../trans/party/Master_party_merge.comp"/>
	<recordFilterExpression>eng_active=true</recordFilterExpression>
</mergingOperation>

where:

customActivityTracking

Is boolean and uses custom rules for activity calculation, for example, overwriting the eng_active value in the merge plan. The setting is false by default.

<recordFilterExpression>

Is boolean and optional, and enables you to apply a filter to the merge plan record input.

Defines a list of source systems, which are connected to the NME engine.

<sourceSystems>
	<system name="crm">
		<instanceMappings>
			<originId name="crm_customer#Party" entity="party"/>
			<originId name="crm_address#Address" entity="address"/>
		</instanceMappings>
	</system>
	<system name="life">
		<instanceMappings>
			<originId name="life_contract#Party" entity="party"/>
			<originId name="life_contract#Address" entity="address"/>
		</instanceMappings>
	</system>
</sourceSystems>

where:

<system name>

Is the source system name. It is available in the eng_source_system column in the DQS plans.

<instanceMappings>

Is the descriptive name, which provides text information about the record origin. It is available as the eng_origin attribute in DQS plans.

• <source_system>_<source_entity>#<target_iWayMDS_entity>, for example, life_contract#Party
• <source_company>##<source_system>##<source_entity>, for example, ATACCAMA##LIFE##contract

entity

Is the name of the instance entity.