Skip to content

Reading Data - Introduction

These cookbook articles are step-by-step recipes for getting data into an integration. Each one is a scenario a Reader is asked to handle that its setup screen does not make obvious.

They are worked examples, not reference. The Reader transforms themselves — every field on every tab — are described in the User Guide, and there are further reader examples in the training Homework and in the training manuals.

CSV Reader

  • Reading Hierarchical Data Files
    • A single file holding header, detail and charge records distinguished by a record type column, with the key fields that let its rows arrive interleaved.

Excel Reader

  • Reading Excel Worksheets
    • Selecting one worksheet out of a multi-sheet workbook, and the fact that the Worksheet Id is zero-based whatever the on-screen help says.

Fixed Width Reader

  • Reading Fixed Width Files
    • Header, detail and trailer records in a file with no delimiters: record type length, skipping banner and totals lines, and defining every field's position by hand.

Xml Reader

  • Reading Xml Documents
    • The entry point, the top record type and its children, namespaces and the prefix you have to invent for the default one, and per-field XPaths onto attributes. The last section covers referencing a repeating node by its own value.

Database Reader

  • Databases Named by the Data
    • A parameterised connection, so each row is read from — and written to — the database that row itself names. Also builds a hierarchical reader out of a UNION.

Email

  • Reading Data from Email
    • A CSV Reader whose source is an IMAP mailbox: the mailbox record, the folder and filters, the EML.* fields, and the Mark Email Read and Move Email tasks that stop a run reading the same email twice. Ends with what changes on POP3.

JSON Reader

JSON reading is documented in the Webservices section rather than here. That is deliberate: IMan reads JSON through the same Reader whether the document arrives over http or off disk, and everything specific to it — JPaths, stepping, the response shapes — is easier to follow alongside the request that fetched it.