Relationship between encoding conversion, the recovery database, SMAT, and resend

Conceptually, inbound encoding conversion happens after a message is pulled from the inbound pre-TPS queue and before it is passed to the inbound TPS. Outbound encoding conversion happens after a message is returned from the prewrite procedure.

Inbound SMAT is written immediately when a message is received from a protocol driver. Outbound SMAT is written from a copy of the post-TPS queue message after a successful driver write and before the SendOK procedure is called. Inbound SMAT files contain raw data received from the outside. The inbound encoding conversion has not been performed.

Outbound SMAT files contain UTF-8 Unicode data. The outbound encoding conversion has not been performed.

Recovery database state 1 On IB pre-TPS queue contains raw data from the outside. All other recovery database states contain UTF-8 Unicode data.

Error database state 416 (Inbound Encoding Conversion error) contains raw encoded data. All other error database states contain Unicode.

Messages can be resent from SMAT files or from any arbitrary file using NetMonitor resend. The resend places the message in IB pre-TPS, IB post-TPS, OB pre-TPS or OB post-TPS under user control. Messages resent to IB pre-TPS are considered raw data. The normal inbound encoding conversion happens when the message is pulled from the IB pre-TPS queue. Messages resent to the other three queues are considered UTF-8 Unicode.

This is a consistent treatment of the data. Inbound SMAT is usually resent to IB pre-TPS and outbound SMAT is usually resent to OB post-TPS. The other resend points are only used in special cases.

It may seem inconsistent for inbound SMAT and recovery state 1 to be raw data but there is a good reason for this. Inbound encoding conversion errors can make it impossible to convert the inbound data to Unicode. This is required for inbound SMAT and recovery state 1 to be raw data. Messages with encoding errors can then be examined, corrected and resent in a consistent fashion.

Outbound encoding conversion is implemented as an automatic pre-write.

Note: Outbound encoding conversion is implemented as an automatic pre-write.