Skip to content

Hierarchical Data

Introduction & IMan Hierarchical Data Handling

Hierarchical data is organised with headers above and details beneath them, in repeating parent-child relationships.

IMan supports hierarchical data throughout. Every Reader can read hierarchical data. Every transform and task handles hierarchical data. The Flatten and Hierarchy transforms convert data into and out of a hierarchical structure. See Destination Defined Data Structure to find out why you need this.

Example

A sales order where the header record contains customer information, the first child (order details) contains the items, and a third level contains lot or serial number data.

Repeating parent-child relationships: a Sales Order with Order Details beneath it, Lot / Serials beneath that, and Pay Details back at the Order Details level; then a second Sales Order repeating the same shape

IMan handles any number of levels. It can process simple header-detail data, such as invoices, and complex structures such as a Project Setup in a Project or Job Costing system, a Multi-Level Bill Of Materials, or Purchase Order Receipts with Landed Costs.

Reader Support For Hierarchical Data

Each of IMan's readers can read hierarchical data. The type of support depends on the data format:

Naturally Hierarchical

This data has explicit relationships between parent and child.

XML and JSON support this style.

A connector reader always provides its data in hierarchical form.

Keyed Hierarchy

In a keyed hierarchy there is no explicit relationship. Instead, records in the dataset are related through common field values.

The Key Fields identify the fields that express the relationship. Records whose key values match are related.

The CSV, Database and Excel Readers support keyed hierarchies. The Hierarchy transform uses the same key concept.

Key Fields

IMan uses key fields to build the hierarchical dataset.

When you set up a hierarchy, you mark the fields that define the relationship between records as keys by giving them incrementing numbers. A record can have one key field or several.

IMan inserts a child record into the dataset under the parent whose key field values match its own.

Because IMan matches children to parents by these values, a child does not have to follow immediately after its parent. A child must not appear in the data before its parent.

Each child transaction must have at least one more key field than its parent. Otherwise an error occurs.

How keys build a hierarchy, in two rows. Above, a reader field mapping with Field 1 marked Key 1: the four data rows collapse into two Level 1 groups, A and B, A carrying its three rows. Below, a second level is added by marking Field 2 as Key 2 on Level 2: the same data now nests as Level 1 A with Level 2 groups A C and A D beneath it, and Level 1 B with Level 2 B E

Ordered Hierarchy

In an ordered hierarchy, the order of the records in the data defines the relationship.

The ordered hierarchy uses a 'Record Identifier' field to determine each record's type.

When reading the data, IMan works out the relationships between records from the Record Identifier field and the order of the records in the file.

Unlike a keyed hierarchy, an ordered hierarchy needs the data in logical order. Otherwise an error occurs.

The CSV, Excel and Fixed Width Readers support ordered hierarchies.

Ordered data. A logical structure of Top Parent, Child A and Child B sits beside a flat list of records tagged Order, Item, Lot, Lot, Item, Order in that sequence, and then the hierarchy IMan builds from it, where each record falls beneath whichever parent preceded it. A caption reads that IMan uses the order of data to determine the logical arrangement

Transform Support for Hierarchical Data

Each transform carries every hierarchical 'level' across from the previous transform. The field mapping grid lists the fields of one level at a time. Change the Transaction Type drop-down to see the fields of another level.

The Hierarchy and Flatten transforms convert data into and out of hierarchical form.

Task Support for Hierarchical Data

Most tasks work with hierarchical data through Expando fields. The IMan dataset supplies the 'expanded' list of values.