How to use the Slowly Changing Dimension stage

The Slowly Changing Dimension stage was added in the 8.0 release of InfoSphere Information Server DataStage. It is designed specifically to support the types of activities required to populate and maintain records in star schema data models, specifically dimension table data.
The Slowly Changing Dimension stage encapsulates all of the dimension maintenance logic —

•  Finding existing records
•  Generating surrogate keys
•  Checking for changes  and 
• What action to take when changes occur. 
• In addition, you can associate dimension record surrogate key values with source records, which eliminates the need for additional lookups in later processing.      

Define the lookup condition and purpose codes
The first task on this page is to define what the various columns of the dimension table are used for. This information is used in a number of ways in the SCD processing. The choices for purpose codes are:

• Surrogate Key— This column is the primary key of the dimension table and is populated with a surrogate key value.
• Business Key— This column is the identifier of the business objects that the dimension table is representing, but is not the primary key of the dimension table. This column is typically used as a lookup column and corresponds to a key or some other field of the source data that identifies the associated business object. The lookup is used to find the dimension table row that corresponds to a source data row.
• Type 2— Check this column for a change in value. If the value has changed, perform a history tracking change to the dimension table.
• Type 1— Check this column for a change in value. If the value has changed, perform an overwriting change to the dimension table.
• Current Indicator— This column is used as a flag to indicate whether it is the most current record for a particular business key.
• Effective Date— This column is used to specify when a record first became the most current record, that is, when it became the active record.
• Expiration Date— This column is used to specify the ending date of when a record was the active record. For currently active records, this value is typically a future date or NULL.
• SK Chain— This column is used to store the surrogate key of the previous or next record in the history for a particular business key.
• (blank) — This column is not used for anything with respect to SCD processing. Data for this field is inserted into the table when a new row is inserted, but this column will not be checked for changes against the source data.

Set purpose codes for the columns as shown below in Figure 7. Because this dimension table is tracking history, it contains columns to track whether a row is current and the date range for when it was current.
Click on the ProdSKU source field and drag it to the SKU dimension column to create the lookup condition.



 Although this tab looks similar to a mapping tab, it is actually defining the lookup keys from the source record to the dimension record. Any source column can be associated with any one dimension column. This creates an equality lookup condition between those columns. If more than one source column is associated with a dimension column, then those equality conditions are AND'ed together. In this manner, multi-column lookup keys can be used.