Add route table guide and formatting tweaks

[ric-plt/lib/rmr.git] / docs / rt_tables.rst

.. This work is licensed under a Creative Commons Attribution 4.0 International License. 
.. SPDX-License-Identifier: CC-BY-4.0 
.. CAUTION: this document is generated from source in doc/src/rtd. 
.. To make changes edit the source and recompile the document. 
.. Do NOT make changes directly to .rst or .md files. 
 
============================================================================================ 
Route Table Guide 
============================================================================================ 
-------------------------------------------------------------------------------------------- 
RIC Message Router -- RMR 
-------------------------------------------------------------------------------------------- 


Overview
========

Messages sent via the RIC Message Router (RMR) are routed to 
an endpoint (another application) based on a combination of 
the *message type* (MT) and *subscription ID* (SID), supplied 
in the message. RMR determines the endpoint by matching the 
MT and SID combination to an entry in a route table which has 
been supplied dynamically by a *Route Manager* service, or as 
a static table loaded during RMR initialisation. It is also 
possible to route messages directly to an endpoint which is 
the *managed entity* "owner," using the *managed entity ID* 
(MEID). 
 
For most xAPP developers the format of the RMR route table is 
not important beyond understanding how to create a static 
table for local testing. For developers of a *Route Manager* 
service, the need is certainly a requirement. This document 
describes the overall syntax of a route table and the 
interface between the *Route Manager* service and RMR. 


Contents of a Route Table
=========================

The table consists of a start record, one or more entry 
records, and an end record. Each entry record defines one 
message type, with an optional sender application, and the 
endpoint(s) which accept the indicated message type. All 
table records contain fields separated with vertical bars 
(|), and allow for trailing comments with the standard shell 
comment symbol (hash, #) provided that the start of the 
comment is separated from the last token on the record by one 
or more spaces. Leading and trailing white space in each 
field is ignored. Figure 1 illustrates a very basic route 
table with two message types, 1000 and 2000, and two 
subscription IDs for message type 1000. 
 
 
:: 
 
    newrt | start | rt-0928
    rte   | 2000  | logger:30311
    mse   | 1000  | 10 | forwarder:43086
    mse   | 1000  | 21 | app0:43086,app1:43086
    newrt | end   | 3
 
Figure 1: A basic route table. 


Entry record syntax
-------------------

Two types of table entries are supported for compatibility 
with the original RMR implementation, but only the *mse* 
entry type is needed and that should be the entry used when 
creating new tables. The following shows the syntax for both 
entry types: 
 
 
:: 
 
   rte | <msg-type>[,<sender-endpoint>] | <endpoint-group>[;<endpoint-group>;...]
   mse | <msg-type>[,<sender-endpoint>] | <sub-id> | <endpoint-group>[;<endpoint-group>;...]
 
 
Where: 
 
 
    .. list-table:: 
      :widths: 25,70 
      :header-rows: 0 
      :class: borderless 
       
      * - **mse, rte** 
        - 
          is the table entry type 
       
      * - **<msg-type>** 
        - 
          is the integer message type 
       
      * - **<sender-endpoint>** 
        - 
          is the endpoint description of the message sender; only that 
          sender will read the entry from the table, so a single table 
          may be used for all senders when a common message type is 
          delivered to varying endpoints based on senders. If the 
          sender endpoint is omitted from the entry, then the entry 
          will be used by all applications. 
       
      * - **<sub-id>** 
        - 
          is the subscription id (integer) for subscription-based 
          messages, or -1 if the message type is not 
          subscription-based. An *mse* entry with a sub-id of -1 is the 
          **same** as an *rte* entry with the same message type. 
       
      * - **<endpoint-group>** 
        - 
          is one or more, comma separated, endpoint descriptions. 
           
 
space When an application sends a message with the indicated 
type, the message will be sent to one endpoint in the group 
in a round-robin ordering. If multiple endpoint groups are 
given, then the message is sent to a member selected from 
each group; 3 groups, then three messages will be sent. The 
first group is required. 


Line separation
---------------

Table entries **must** end with a record termination sequence 
which may be one of the following three sequences: 
 
 
* a single newline (\\n) 
* a DOS style CRLF pair (\\r\\n) 
* a single carriage return (\\r) 
 
 
Care must be taken when manually editing a static table; some 
editors do **not** add a final record termination sequence to 
the last line of a file. RMR expects the final record to have 
a termination sequence to ensure that the record was not 
truncated (especially important when receiving dynamic 
tables). 


Table framing
-------------

The route table parser within RMR assumes that route table 
entries are sent via RMR messages as a stream. To ensure 
synchronisation and prevent malformed tables because of 
broken sessions or lost packets, each table must begin and 
end with an *newrt* record. Each *newrt* record has one of 
two possible syntax layouts as described below. 
 
 
:: 
 
    newrt | begin [| table-id-string]
    newrt | end  [| record-count]
 
Figure 2: Illustration of the newrt records in the table. 
 
The *table-id-string* is an optional string which is used by 
RMR when sending an acknowledgement back to the *Route 
Manager* service (see the *Route Manager Interface* section 
for more details). If a *record-count* is given as the final 
field on the *end* record, RMR will verify that the number of 
*mse* and *rte* entries in the table matches the count; if 
there is a mismatch in values the table is not used. 


Comments, spaces, and blank lines
---------------------------------

Comments may be placed to the right of any table entry line 
using the standard shell comment symbol (#). The start of a 
comment must be separated from any previous record content by 
at least one space or tab. Complete lines are treated as 
comments when the first non-whitespace character of a line is 
a comment symbol. Blank lines are also ignored. 
 
Fields on table records are separated using the vertical bar 
(|) character. Any white space (tabs or spaces) which appear 
immediately before or after a separator are ignored. 


Endpoint Description
--------------------

The endpoint description is either the hostname or IP address 
followed by a port number; the two are separated by a single 
colon. The illustration below assumes that host names (e.g. 
forwarder and app1) are defined; they also make the tables 
easier to read. The port number given is the port number that 
the user application provides to RMR when the RMR 
initialisation function is invoked (and thus is the port that 
RMR is listening on). 


Table Mechanics
===============

Creating a table from the two entry types is fairly simple, 
however there are some subtleties which should be pointed out 
to avoid unexpected behaviour. For this discussion the 
following complete table will be used. 
 
.. list-table:: 
  :widths: 75,10 
  :header-rows: 0 
  :class: borderless 
 
 
  * -  
        
       :: 
        
           newrt | start | rt-0928
           rte | 2000 | logger:30311
           mse | 1000 | 10 | forwarder:43086
           mse | 1000,forwarder:43086 | 10 | app2:43086
           mse | 1000 | -1 | app0:43086,app1:43086; logger:20311
           newrt | end | 4
        
    -  
        
       :: 
        
         (1)
         (2)
         (3)
         (4)
         (5)
         (6)
        
        
Figure 3: A complete RMR routing table (line numbers to the 
right for reference). 


Table Entry Ordering
--------------------

Whether a table is read from a file on disk, or is received 
from a *Route Manager* service, RMR parses the records to 
build an internal route table keeping only the relevant 
information. Entries are read in the order they appear (from 
the file or in messages received), and RMR will use only one 
entry for each MT/SID pair. 
 
For most tables, the ordering of entries is not important, 
but when there are entries which duplicate the MT/SID pair 
ordering becomes significant. RMR will use the **last** valid 
entry for a MT/SID pair that it encounters. An entry is 
considered valid if there is no sender identified with the 
message type (line 3), and when the sender (host and port) 
match the the applications' location and the port provided to 
RMR for listening. 
 
Using the table in figure 3 as an example, there are two 
entries which match the MT/SID pair of 1000/10. When this 
table is parsed on any host, RMR will recognise and add the 
first entry (line 3) to the internal representation; this 
entry is valid for all applications. The second 1000/10 entry 
(line 4) is valid when the table is parsed on the *forwarder* 
host, and only by the application which is listening on port 
43086. For this application the entry will override the more 
generic entry for the MT/SID combination. 
 
As a rule, the ordering of entries for a given MT/SID pair 
should be from most generic to most specific. 


Route Manager Communications
============================

During initialisation RMR will use the value of the 
``RMR_RTG_SVC`` environment variable to connect to the *Route 
Manager* service in order to request a route table. The 
connection between RMR and the *Route Manager* is also an RMR 
session and thus RMR messages will be used to exchange 
requests and responses. 


Table Request
-------------

During initialisation, RMR will establish a wormhole 
connection to the *Route Manager* and sends a message type of 
21 to request a new table. RMR will continue to send table 
requests until a table is received and accepted; in other 
words it is fine for the *Route Manager* to ignore the 
requests if it is not ready to respond. 


Sending Tables To RMR
---------------------

Table entry data is expected to arrive via RMR message with a 
message type of 20. The message may contain one or more 
entries provided that the entries are newline separated. 
Current versions of RMR support very large messages, however 
to ensure compatibility with an xAPP built using an older 
version of RMR (pre 3.8), messages should be limited to 4 
KiB. 


Table Acceptance and Acknowledgement
------------------------------------

When RMR receives the table end entry (newrt|end), it will 
send a state message back to the *Route Manager* to indicate 
the state of the received table. The message type is 22 and 
the payload will contain UTF-8 tokens which indicate the 
state. The second token will be the *table ID* supplied on 
the start record, or the string "<id-missing>." When the 
state is an error state, RMR might add a final set of tokens 
which contain the reason for the failure. 
 
Upon receipt of a status message which indicates an "OK" 
response, the *Route Manager* can assume that the table has 
been installed and is in use. Any other response indicates 
that RMR did not use the table and has dropped it; the 
previous table is still in use. 


Providing A Static Table
========================

For testing, or possibly bootstrapping purposes, a static 
route table can be supplied. During initialisation, RMR will 
check the ``RMR_SEED_RT`` environment variable. If it exists, 
and references a file, RMR will open and read the file 
expecting to find a static route table. This route table is 
used until an update is received from a *Route Manager*. 
Normally, when the RMR initialisation function is invoked, a 
listener is started to receive route table information from a 
route manager process. During testing it is often useful to 
supply a static table which is available should no route 
management process exist, or to provide a seed table to use 
before the first table is delivered. The environment variable 
``RMR_SEED_RT`` can be set to provide the RMR initialisation 
function with the name of the static table to use. If a 
static table is provided, it will be loaded only once, and 
will be overlaid if a dynamically delivered table is 
received. 


Routing Using MEID
==================

Starting with version 1.13.0, RMR provides the ability to 
select the endpoint for a message based on the MEID (managed 
entity ID) in the message, rather than selecting the endpoint 
from the round-robin list for the matching route table entry. 
When the MEID is used, the message is sent to the endpoint 
which *owns,* or is responsible for the managed entity. 
Should the *owner* change messages will be routed to the new 
owner when the route table is updated. To make use of MEID 
routing, there must be one or more route table entries which 
list the special endpoint name ``%meid`` instead of providing 
a round robin list. As an example, consider the following 
route table entry: 
 
 
:: 
 
   mse| 1000,forwarder:43086 | 10 | %meid
 
Figure 4: Sample route entry with the meid flag. 
 
The final field of the entry doesn't specify a round-robin 
group which means that when an application attempts to send a 
message with type 1000, and the subscription ID of 10, the 
MEID in the message will be used to select the endpoint. 


MEID endpoint selection
-----------------------

To select an endpoint for the message based on the MEID in a 
message, RMR must know which endpoint owns the MEID. This 
information, known as an MEID map, is provided by the *Route 
Manager* over the same communication path as the route table 
is supplied. The following is the syntax for an MEID map. 
 
 
:: 
 
   meid_map | start | <table-id>
   mme_ar | <owner-endpoint> | <meid> [<meid>...]
   mme_del | <meid> [<meid>...]
   meid_map | end | <count> [| <md5sum> ]
 
Figure 5: Meid map table. 
 
The mme_ar records are add/update records and allow for the 
list of MEIDs to be associated with (owned by) the indicated 
endpoint. The <owner-endpoint> is the hostname:port, or IP 
address and port, of the application which owns the MEID and 
thus should receive any messages which are routed based on a 
route table entry with %meid as the round-robin group. The 
mme_del records allow for MEIDs to be deleted from RMR's 
view. Finally, the <count> is the number of add/replace and 
delete records which were sent; if RMR does not match the 
<count> value to the number of records, then it will not add 
the data to the table. Updates only need to list the 
ownership changes that are necessary; in other words, the 
*Route Manager* does not need to supply all of the MEID 
relationships with each update. 
 
The optional <md5sum> field on the end record should be the 
MD5 hash of all of the records between the start and end 
records. This allows for a precise verification that the 
transmitted data was correctly received. 
 
If a static seed file is being used for the route table, a 
second section can be given which supplies the MEID map. The 
following is a small example of a seed file: 
 
 
:: 
 
  newrt|start | id-64306
  mse|0|-1| %meid
  mse|1|-1|172.19.0.2:4560
  mse|2|-1|172.19.0.2:4560
  mse|3|-1|172.19.0.2:4560
  mse|4|-1|172.19.0.2:4560
  mse|5|-1|172.19.0.2:4560
  newrt|end
  
  meid_map | start | id-028919
  mme_ar| 172.19.0.2:4560 | meid000 meid001 meid002 meid003 meid004 meid005
  mme_ar| 172.19.0.42:4560 | meid100 meid101 meid102 meid103
  mme_del | meid1000
  meid_map | end | 1
 
Figure 6: Illustration of both a route table and meid map in 
the same file. 
 
The tables above will route all messages with a message type 
of 0 based on the MEID. There are 10 meids which are owned by 
two different endpoints. The table also deletes the MEID 
meid1000 from RMR's view. 


Reserved Message Types
======================

RMR is currently reserving message types in the range of 0 
through 99 (inclusive) for its own use. Please do not use 
these types in any production or test environment as the 
results may be undesired. 
 


Appendix A -- Glossary
======================

Many terms in networking can be interpreted with multiple 
meanings, and several terms used in various RMR documentation 
are RMR specific. The following definitions are the meanings 
of terms used within RMR documentation and should help the 
reader to understand the intent of meaning. 
 
   .. list-table:: 
     :widths: 25,70 
     :header-rows: 0 
     :class: borderless 
      
     * - **application** 
       - 
         A programme which uses RMR to send and/or receive messages 
         to/from another RMR based application. 
      
     * - **Critical error** 
       - 
         An error that RMR has encountered which will prevent further 
         successful processing by RMR. Critical errors usually 
         indicate that the application should abort. 
      
     * - **Endpoint** 
       - 
         An RMR based application that is defined as being capable of 
         receiving one or more types of messages (as defined by a 
         *message key.*) 
      
     * - **Environment variable** 
       - 
         A key/value pair which is set externally to the application, 
         but which is available to the application (and referenced 
         libraries) through the ``getenv`` system call. Environment 
         variables are the main method of communicating information 
         such as port numbers to RMR. 
      
     * - **Error** 
       - 
         An abnormal condition that RMR has encountered, but will not 
         affect the overall processing by RMR, but may impact certain 
         aspects such as the ability to communicate with a specific 
         endpoint. Errors generally indicate that something, usually 
         external to RMR, must be addressed. 
      
     * - **Host name** 
       - 
         The name of the host as returned by the ``gethostbyname`` 
         system call. In a containerised environment this might be the 
         container or service name depending on how the container is 
         started. From RMR's point of view, a host name can be used to 
         resolve an *endpoint* definition in a *route* table.) 
      
     * - **IP** 
       - 
         Internet protocol. A low level transmission protocol which 
         governs the transmission of datagrams across network 
         boundaries. 
      
     * - **Listen socket** 
       - 
         A *TCP* socket used to await incoming connection requests. 
         Listen sockets are defined by an interface and port number 
         combination where the port number is unique for the 
         interface. 
      
     * - **Message** 
       - 
         A series of bytes transmitted from the application to another 
         RMR based application. A message is comprised of RMR specific 
         data (a header), and application data (a payload). 
      
     * - **Message buffer** 
       - 
         A data structure used to describe a message which is to be 
         sent or has been received. The message buffer includes the 
         payload length, message type, message source, and other 
         information. 
      
     * - **Messgae type** 
       - 
         A signed integer (0-32000) which identifies the type of 
         message being transmitted, and is one of the two components 
         of a *routing key.* See *Subscription ID.* 
      
     * - **Payload** 
       - 
         The portion of a message which holds the user data to be 
         transmitted to the remote *endpoint.* The payload contents 
         are completely application defined. 
      
     * - **RMR context** 
       - 
         A set of information which defines the current state of the 
         underlying transport connections that RMR is managing. The 
         application will be give a context reference (pointer) that 
         is supplied to most RMR functions as the first parameter. 
      
     * - **Round robin** 
       - 
         The method of selecting an *endpoint* from a list such that 
         all *endpoints* are selected before starting at the head of 
         the list. 
      
     * - **Route table** 
       - 
         A series of "rules" which define the possible *endpoints* for 
         each *message key.* 
      
     * - **Route table manager** 
       - 
         An application responsible for building a *route table* and 
         then distributing it to all applicable RMR based 
         applications. 
      
     * - **Routing** 
       - 
         The process of selecting an *endpoint* which will be the 
         recipient of a message. 
      
     * - **Routing key** 
       - 
         A combination of *message type* and *subscription ID* which 
         RMR uses to select the destination *endpoint* when sending a 
         message. 
      
     * - **Source** 
       - 
         The sender of a message. 
      
     * - **Subscription ID** 
       - 
         A signed integer value (0-32000) which identifies the 
         subscription characteristic of a message. It is used in 
         conjunction with the *message type* to determine the *routing 
         key.* 
      
     * - **Target** 
       - 
         The *endpoint* selected to receive a message. 
      
     * - **TCP** 
       - 
         Transmission Control Protocol. A connection based internet 
         protocol which provides for lossless packet transportation, 
         usually over IP. 
      
     * - **Thread** 
       - 
         Also called a *process thread, or pthread.* This is a 
         lightweight process which executes in concurrently with the 
         application and shares the same address space. RMR uses 
         threads to manage asynchronous functions such as route table 
         updates. &Term An optional portion of the message buffer that 
         the application may populate with data that allows for 
         tracing the progress of the transaction or application 
         activity across components. RMR makes no use of this data. 
      
     * - **Transaction ID** 
       - 
         A fixed number of bytes in the *message* buffer) which the 
         application may populate with information related to the 
         transaction. RMR makes use of the transaction ID for matching 
         response messages with the &c function is used to send a 
         message. 
      
     * - **Transient failure** 
       - 
         An error state that is believed to be short lived and that 
         the operation, if retried by the application, might be 
         successful. C programmers will recognise this as 
         ``EAGAIN.`` 
      
     * - **Warning** 
       - 
         A warning occurs when RMR has encountered something that it 
         believes isn't correct, but has a defined work round. 
      
     * - **Wormhole** 
       - 
         A direct connection managed by RMR between the user 
         application and a remote, RMR based, application.