Skip to content

Step 8 – Auditing

Every time IMan runs an integration it compiles an audit report. For a job that ran overnight, the report is often all that anyone sees. By default it shows only a couple of warnings and whether the job finished. It does not show which customers were imported, or what order numbers Sage 200 gave them.

In this step you add that information to the report.

An integration that fails should say which record failed

A good integration works, and when it fails, its report points you to the data that caused the failure.

See Audit Report, Auditing, and Log Keys in the IMan User Guide for the full description.

Where the audit output appears

You can read the audit output in two places.
Preview > Audit, on each transform What this transform would report, without running the job. Use it while you are building.
Audit Log, in the side menu The report for the whole job, covering every transform. IMan keeps it after the run and emails it to the report group.

The preview Audit tab is new in IMan 6.1. Before it, you had to save, switch to the scheduler, run the job and open the report every time you wanted to check an audit summary.

Design > Audit

The integration's own Audit tab sits alongside Options and Transform Setup. It holds the settings for the job as a whole, not for any one transform.

  1. Press the Audit tab.

    1. Report Groups
      • Named groups that you assign transforms to. Each group produces its own report for its own recipients, so each department receives only the part that concerns it.
      • Every integration starts with one, RptGrp.
      • For training, leave it as it is.
    2. Email Groups
      • Who receives this report group's report. Email groups are defined in Setup.
    3. Produce Audit Report
      • For training, leave as: Always
    4. Default Action on Transform Error
      • The setting every transform in the integration starts with. Because of it, the connectors you are about to open are already set to Abort.
      • For training, leave as: Abort

    The integration's Audit tab showing the Report Groups panel with RptGrp, the Email Groups and Email Subject fields, Integration and Audit Settings with Produce Audit Report set to Always, and the Job Defaults section

Action on Transform Error

At the top of every transform's own Audit tab is a drop-down that controls what happens when that transform hits an error. It starts at the integration default set above.

On most transforms, leave it at Abort. On connectors, choose with care, because a connector is where data reaches the accounting system. There are three settings:

  1. Abort
    • The whole process stops.
  2. Reject Record
    • IMan processes the valid transactions and rejects the erroneous ones.
    • IMan treats an order, despatch, customer or item as an atomic transaction and never leaves a half-written record. If an order has an invalid item code, IMan rejects the entire order, not just the invalid line. You will see this happen at the end of this step.
  3. Continue
    • Deprecated. Do not use.

Add a Customer Import summary

Here you add a line to the report that gives the number of customers imported and the number of errors.

  1. Open the Sage 200 connector for the customer import and press its Audit tab.

    1. Summary Header
      • The heading this transform's block appears under in the report.
      • For training, enter: Customer Import
    2. Audit Summary
      • Type the line into the empty box at the bottom of the list and press the green + button to its right. Enter it as a single line:
      • %Customer.PROCESSED Customers Processed. %Customer.ERRORS Errors. %Customer.INSERTED Inserted. %Customer.UPDATED Updated.

    The customer connector's Audit tab with Summary Header set to Customer Import and the audit summary line entered beneath it

The prompt in the box is not the syntax

The prompt in the empty box reads e.g. %[Record.Processed] records processed. That is not the syntax. Write a counter as %<transaction id>.<counter>, with no brackets, and use the transaction id the connector uses: Customer, not Orders.

This is the renaming described in Step 7. A connector replaces the incoming transaction ids and field names with the Sage 200 ones. By the time IMan evaluates the audit summary, the customer transaction is Customer and the order transaction is SOPOrder.

The available counters are PROCESSED, INSERTED, UPDATED, DELETED, ERRORS and WARNINGS. They count what the connector did. On a run where every customer already exists and is unchanged, INSERTED and UPDATED can both read zero. That is the case here, because Step 5 already imported the customers.

Checking it without running the job

  1. With the connector still open, press Refresh, then press the Audit tab in the preview pane on the right. This is the tab beside Trace, not the connector's own Audit tab on the left.

    The connector's preview pane with the Audit tab selected, showing the Customer Import heading and its summary line

The preview shows the report for this transform exactly as the finished job will. If the summary line is wrong, you find out here instead of after a scheduler run. A mistyped transaction id, for example, shows as literal text instead of a number.

Add an Order Import summary

For the order import there are two things to add: a count of orders processed and created, and a list matching each incoming order to the order number Sage 200 allocated it.

Connector > Audit

  1. Open the Sage 200 connector for the order import and press its Audit tab.

    1. Action on Transform Error
      • For training, set to: Reject Record
    2. Summary Header
      • For training, enter: Order Import
    3. Audit Summary — first line
      • %SOPOrder.PROCESSED Orders Processed. %SOPOrder.INSERTED Orders Created. %SOPOrder.ERRORS Errors.
    4. Audit Summary — second line
      • Press + again and add:
      • Web Order %SOPOrder.CustomerDocumentNo - Sage 200 Order %SOPOrder.DocumentNo

    The order connector's Audit tab with Action on Transform Error set to Reject Record, Summary Header set to Order Import, and the two audit summary lines listed beneath it

The second line names fields instead of counters, so IMan repeats it once per record. The result is a reconciliation list of each web order against its Sage 200 order. The field names are the connector's, not the workbook's: CustomerDocumentNo and DocumentNo.

Lines print in the order they are listed

Lines print in the order they are listed. To reorder them, drag a line by the handle on its left; to remove one, hover over it and press the bin.

Why the two connectors are set differently

The order import is set to Reject Record and the customer import stays on Abort:

  • Order Import — Reject Record. On Abort, the process would stop at the first bad order and leave the orders already written in Sage 200. You would probably need to delete those before re-running, or you would get duplicates. On Reject Record, IMan rejects the bad orders and carries on to the end, and only the rejected orders need fixing.
  • Customer Import — Abort. If the customer import fails, the whole integration should stop. You can run the customer import repeatedly without duplicating data. Its Update Operation is Insert. Insert creates only customers that do not exist yet and skips the ones that do.

There are more advanced ways to prevent duplicates

Filters and lookups can prevent duplicates in more advanced ways than this training covers. The Integration Cookbook's Preventing Duplicate Transactions builds one, and gives the lookup for this integration.

Connector > Field Mapping — Log Keys

A Log Key marks a field as identifying the record. When a row causes a warning or an error, IMan writes the fields you have given log keys into the Source column of the report. You can then tell which order went wrong, not only that one did.

The Log Key column has moved to the end of the grid

In IMan 6 the Log Key column is the last column of the field mapping grid, to the right of the Sage 200 field. Earlier versions placed it between Type and the Sage 200 field.

  1. Press the Field Mapping tab, make sure Orders is the current transaction, and press Edit.

  2. Set the Log Key on OrderId to 1.

    The order header field grid with the Log Key column at the right-hand end and OrderId set to 1

  3. Save the grid with the green tick.

  4. Change the current transaction to OrderDetails and press Edit again.

    1. OrderId
      • Set Log Key to: 1
    2. SkuCode
      • Set Log Key to: 2

    The order detail field grid with OrderId set to Log Key 1 and SkuCode set to Log Key 2

  5. Save the grid, close the connector and save the integration.

Numbering matters. IMan joins the log keys in numeric order to build the Source value. Giving OrderId 1 and SkuCode 2 produces the order reference followed by the item code.

Running the job and reading the report

  1. Go to Scheduling, select the job by its description and press RUN NOW (see Scheduling and Running Integrations).

  2. When it finishes, go to Audit Log in the side menu, choose the integration and press SEARCH.

    The Audit Log results grid listing the summary and detail rows for the run, with the order reference and item code in the Source column

  3. Click any row of the run to open the full report.

    The full audit report showing the Customer Import and Order Import summary sections together, the three web orders listed against the Sage 200 numbers they were given, and the Detail Processing Results table

Both connectors now report under their own headings. The report lists the three orders against the Sage 200 numbers they were given. The stock warnings in Detail Processing Results name the order and item they came from in the Source column. Before you set the log keys, that column was empty and the warnings could have belonged to any of the three.

20180 is a warning, not an error: the orders are created

20180 - The specified quantity exceeds the stock available at the location is a warning, not an error. Sage 200 still creates the orders. The demo company does not hold enough of those items.

Watching a record be rejected

So far every run has been clean. To see Reject Record work, run the same integration against a workbook with one bad item code in it.

Download the invalid workbook and save it beside the original in C:\IMan\InputData\Training. It is the training workbook with a single change: the first line of order FBRN-309243 has INVALID as its item code.

  1. Open the Excel Read transform and change File Name to Sage200OrdersFile-invalid.xlsx. Close and save.

  2. Run the job from Scheduling again and open the new report.

    The audit report for the run against the invalid workbook, reading 3 Orders Processed, 2 Orders Created, 1 Errors, with no Sage 200 order number beside FBRN-309243 and two rows in Detail Processing Results discarding first the order line and then the whole order

Look for three things in it:

  • The summary counts 3 Orders Processed. 2 Orders Created. 1 Errors. The bad order did not stop the other two.
  • The reconciliation line for FBRN-309243 has no order number after it, because no order was created for it.

  • Detail Processing Results has two rows for that order: IMan discards the line, then the whole order. This is the atomic transaction described above. One bad line rejects the whole order, so Sage 200 never holds the order without that line.

  • Set File Name back to Sage200OrdersFile.xlsx before going on to Step 9.

Every run creates real orders

The two good orders from this run are in Sage 200 alongside the three from the run before, so your order numbers will not match the ones in these screenshots.

Using the Log Key for Unmapped Fields

A non-zero log key makes an unmapped field flow out

Fields you do not map in a connector do not flow out the other side (see Step 7).

To make an unmapped field flow out the other side, set its log key to a non-zero value.

If you number a field's log key in sequence from 1, the field becomes part of the value IMan writes to the Source column when an error or warning occurs.

To make a field flow out without joining the Source column, give it a log key outside that sequence: set it to 20, and add one for each further field.

The order header field grid with the unmapped SYS.INPUTFILE field given a Log Key of 20 so that it flows through the connector without joining the Source column

This training needs it for SYS.INPUTFILE. Step 12 archives the workbook the orders came from, and it can do that only if the file name survives the connector.

Step 9: Order Status >