Queues
A queue is an engine internal data structure that stores messages in a given sequence. As messages flow through the engine, they flow through the queues.
A FIFO (First In First Out) sequence is used within a given message priority. Messages that are placed into a queue with the same priority are removed from the queue in the same order: oldest first.
- Inbound pre-TPS queue:
Pre-TPS queues are used only if there are Tcl procedures to run. When the protocol driver has successfully read a message from a connection, it places the message into the Inbound pre-TPS queue. When an inbound pre-TPS runs, it pulls the message from this queue.
If database logging is turned on for this inbound connection, then the message is placed into the recovery database at this point. If the message is not put into the database at this point, then it is not recoverable unless custom Tcl code defines it as recoverable. After the message is placed in the recovery database, it moves forward through each subsequent processing phase within the database.
- Inbound post-TPS queue:
The engine places any inbound messages produced by a TPS into the inbound post-TPS queue. From there, they are sent by ICL to the translation thread within the same process as the inbound protocol thread.
- Pre-translation queue:
The engine places messages that arrive from any protocol thread into the pre-translation queue. When the translation thread runs, it takes the first message from the queue, oldest message first, and begins the translation and routing process.
- Route detail queue:
The route detail queue ensures that all messages from a given route detail are collected together until the current route detail finishes running. This is only used with SEND or CONTINUE. After the route detail completes running, the engine moves messages in this queue to the partial queue.
Primarily, the route detail queue holds messages that result from a translation that produces multiple messages. The goal is to give the user control, by translation UPoCs, over what should happen during a translation failure. The
xpmerror
command defines different error severities that control the flow of messages from this queue during abnormal translation termination, such as a power outage. - Partial queue:
The partial queue ensures that the engine performs atomic translations. The engine does this with a two-phase commit of messages. After all route details are run for a message, and any resulting messages are collected in the partial queue. The inbound source message is removed from the recovery database. The engine then begins moving messages from the partial queue to the post-translation queues.
If the engine is interrupted, then it can determine whether it was in message movement to the post-translation queues or in the process of translation. It does this by searching the recovery database for the inbound message.
- If it finds the inbound message, then the engine starts the routing and translation over again.
- If it does not find the message, then it continues moving messages from the partial queue to the post-translation queue.
This queue continues the engine semantic of all messages or no messages resulting from routing and translation, even across unexpected events such as power loss.
- Post-translation queue:
When all of the source message translations are complete, the engine removes the source message from the database. The engine moves the messages waiting in the partial queue to the correct post-translation queue. There is one post-translation queue per destination thread.
After the engine places messages in the post-translation queue, the translation thread attempts to deliver them to the associated protocol thread. Usually, delivery is successful unless the destination protocol thread is stopped, or the process that should contain the thread is not running. If delivery is not successful, then the messages remain indefinitely in the post-translation queue in sequence.
- Outbound pre-TPS queue:
The outbound pre-TPS queue is skipped if there are no outbound TPS procedures to run. Messages sent from the translation thread are placed here for the outbound protocol thread by ICL. When the outbound TPS is ready to process a message, it pulls it from this queue.
- Outbound post-TPS queue:
When the outbound TPS produces one or more messages, they are placed in this queue. The messages remain in this queue until the engine can deliver them to the destination connection, as defined from the protocol thread.