Backward compatibility of SMAT files
Unicode stores raw inbound data in SMAT files, but stores UTF-8 Unicode in outbound SMAT files. This causes some backward compatibility issues with existing SMAT files because they contain raw binary data for both inbound and outbound. A single SMAT file could contain raw and UTF-8 data if a site is upgraded with outbound SMAT files in place.
When the Unicode engine is started, it continues writing to the existing
SMAT files using UTF-8. This results in mixed raw/UTF-8 files. To handle this properly,
the Unicode engine inserts an additional setting into the outbound .idx file. This setting is {ENCODING
UTF-8}.
These settings are stored on a per-message basis, so there is no problem
even with a mixed file. When a SMAT file is viewed or resent, the ENCODING
setting must be checked. If it does not exist, then the message
data is assumed to be raw. An automatic binary to UTF-8 conversion is then required when
the message is resent to any point other than Inbound pre-TPS.
An automatic UTF-8 to binary conversion is required for any UTF-8
message resent to Inbound pre-TPS. The only valid value for
ENCODING
is UTF-8 and it is only saved for an outbound
SMAT file. This feature ensures that both old and new SMAT files can be used
without any requirement for explicitly converting the files.