.if false ================================================================================== Copyright (c) 2020 Nokia Copyright (c) 2020 AT&T Intellectual Property. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. ================================================================================== .fi .if false Mnemonic: rt_tables.xfm Abstract: An overview of route tables in RMR. Date: 25 February 2020 Author: E. Scott Daniels .fi .** constants .dv MT_REQ_TABLE 21 .dv MT_TABLE_STATE 22 .dv MT_TABLE_DATA 20 .dv RTMGR ^&ital(Route Manager) .** ------------------------------------------------------------------------------- .** force setup to generate a title and to reverse title/subtitle for rst docs .dv GEN_TITLE 1 .dv doc_title RIC Message Router -- RMR .dv doc_subtitle Route Table Guide .dv reverse_titles 1 .dv textfont Helvetica .dv textsize 10p .dv index_snare_file index_snatch.im .dv orig_date 30 April 2020 .gv e XFM_PASS pass .** setup will do the right thing with the index configuration; a common snatch .** file is used for all documents in this directory. .** .dv index_snare_file index_snatch.im .im setup.im &line_len( &line_size ) .** ------------------------------------------------------------------------------- &h1(Overview) Messages sent via the RIC Message Router (RMR) are routed to an endpoint (another application) based on a combination of the &ital(message type) (MT) and &ital(subscription ID) (SID) supplied in the message. RMR determines the endpoint by matching the MT and SID combination to an entry in a route table which has been supplied dynamically by a &RTMGR service, or as a static table loaded during RMR initialisation. It is also possible to route messages directly to an endpoint which is the &ital(managed entity) "owner," using the &ital(managed entity ID) (MEID). &space For most xAPP developers the format of the RMR route table is not important beyond understanding how to create a static table for local testing. For developers of a &RTMGR service, the need is certainly a requirement. This document describes the overall syntax of a route table and the interface between the &RTMGR service and RMR. &h1(Contents of a Route Table) The table consists of a start record, one or more entry records, and an end record. Each entry record defines one message type, with an optional sender application, and the endpoint(s) which accept the indicated message type. All table records contain fields separated with vertical bars (|), and allow for trailing comments with the standard shell comment symbol (hash, #) provided that the start of the comment is separated from the last token on the record by one or more spaces. Leading and trailing white space in each field is ignored. .gv fig Figure &_fig illustrates a very basic route table with two message types, 1000 and 2000, and two subscription IDs for message type 1000. &half_space &ex_start newrt | start | rt-0928 rte | 2000 | logger:30311 mse | 1000 | 10 | forwarder:43086 mse | 1000 | 21 | app0:43086,app1:43086 newrt | end | 3 &ex_end &fig(A basic route table.) &h2(Entry record syntax) Two types of table entries are supported for compatibility with the original RMR implementation, but only the &ital(mse) entry type is needed and that should be the entry used when creating new tables. The following shows the syntax for both entry types: &space &ex_start rte | [,] | [;;...] mse | [,] | | [;;...] &ex_end &space Where: &half_space &indent &beg_dlist( 1i : Helvetica-bold : 25,70 ) &ditem(mse, rte) is the table entry type &ditem() is the integer message type &ditem() is the endpoint description of the message sender; only that sender will read the entry from the table, so a single table may be used for all senders when a common message type is delivered to varying endpoints based on senders. If the sender endpoint is omitted from the entry, then the entry will be used by all applications. &ditem() is the subscription id (integer) for subscription-based messages, or -1 if the message type is not subscription-based. An &ital(mse) entry with a sub-id of -1 is the &bold(same) as an &ital(rte) entry with the same message type. &ditem() is one or more, comma separated, endpoint descriptions. &end_dlist &uindent &space When an application sends a message with the indicated type, the message will be sent to one endpoint in the group in a round-robin ordering. If multiple endpoint groups are given, then the message is sent to a member selected from each group; 3 groups, then three messages will be sent. The first group is required. &h3(Line separation) Table entries &bold(must) end with a record termination sequence which may be one of the following three sequences: &half_space &indent &beg_list(&lic1) &li a single newline (&{backslant}n) &li a DOS style CRLF pair (&{backslant}r&{backslant}n) &li a single carriage return (&{backslant}r) &end_list &uindent &space Care must be taken when manually editing a static table; some editors do &bold(not) add a final record termination sequence to the last line of a file. RMR expects the final record to have a termination sequence to ensure that the record was not truncated (especially important when receiving dynamic tables). &h2(Table framing) The route table parser within RMR assumes that route table entries are sent via RMR messages as a stream. To ensure synchronisation and prevent malformed tables because of broken sessions or lost packets, each table must begin and end with an &ital(newrt) record. Each &ital(newrt) record has one of two possible syntax layouts as described below. &half_space &ex_start newrt | begin [| table-id-string] newrt | end [| record-count] &ex_end &fig(Illustration of the newrt records in the table.) &space The &ital(table-id-string) is an optional string which is used by RMR when sending an acknowledgement back to the &RTMGR service (see the &ital(Route Manager Interface) section for more details). If a &ital(record-count) is given as the final field on the &ital(end) record, RMR will verify that the number of &ital(mse) and &ital(rte) entries in the table matches the count; if there is a mismatch in values the table is not used. &h2(Comments, spaces, and blank lines) Comments may be placed to the right of any table entry line using the standard shell comment symbol (#). The start of a comment must be separated from any previous record content by at least one space or tab. Complete lines are treated as comments when the first non-whitespace character of a line is a comment symbol. Blank lines are also ignored. &space Fields on table records are separated using the vertical bar (|) character. Any white space (tabs or spaces) which appear immediately before or after a separator are ignored. &h2(Endpoint Description) The endpoint description is either the hostname or IP address followed by a port number; the two are separated by a single colon. The illustration below assumes that host names (e.g. forwarder and app1) are defined; they also make the tables easier to read. The port number given is the port number that the user application provides to RMR when the RMR initialisation function is invoked (and thus is the port that RMR is listening on). &h1(Table Mechanics) Creating a table from the two entry types is fairly simple, however there are some subtleties which should be pointed out to avoid unexpected behaviour. For this discussion the following complete table will be used. .cc 20 &space &beg_table_nb( 75,10 : 4.5i .3i) &row &ex_start newrt | start | rt-0928 rte | 2000 | logger:30311 mse | 1000 | 10 | forwarder:43086 mse | 1000,forwarder:43086 | 10 | app2:43086 mse | 1000 | -1 | app0:43086,app1:43086; logger:20311 newrt | end | 4 &ex_end &col &ex_start (1) (2) (3) (4) (5) (6) &ex_end &end_table .gv fig .dv table_fig &_fig &fig( A complete RMR routing table (line numbers to the right for reference). ) &h2(Table Entry Ordering) Whether a table is read from a file on disk, or is received from a &RTMGR service, RMR parses the records to build an internal route table keeping only the relevant information. Entries are read in the order they appear (from the file or in messages received), and RMR will use only one entry for each MT/SID pair. &space For most tables, the ordering of entries is not important, but when there are entries which duplicate the MT/SID pair ordering becomes significant. RMR will use the &bold(last) valid entry for a MT/SID pair that it encounters. An entry is considered valid if there is no sender identified with the message type (line 3), and when the sender (host and port) match the the applications' location and the port provided to RMR for listening. &space Using the table in figure &table_fig as an example, there are two entries which match the MT/SID pair of 1000/10. When this table is parsed on any host, RMR will recognise and add the first entry (line 3) to the internal representation; this entry is valid for all applications. The second 1000/10 entry (line 4) is valid when the table is parsed on the &ital(forwarder) host, and only by the application which is listening on port 43086. For this application the entry will override the more generic entry for the MT/SID combination. &space As a rule, the ordering of entries for a given MT/SID pair should be from most generic to most specific. &h1(Route Manager Communications) During initialisation RMR will use the value of the &cw(RMR_RTG_SVC) environment variable to connect to the &RTMGR service in order to request a route table. The connection between RMR and the &RTMGR is also an RMR session and thus RMR messages will be used to exchange requests and responses. &h2(Table Request) During initialisation, RMR establishes a wormhole connection to the &RTMGR and sends a message type of &MT_REQ_TABLE to request a new table. RMR will continue to send table requests until a table is received and accepted; in other words it is fine for the &RTMGR to ignore the requests if it is not ready to respond. &h2(Sending Tables To RMR) Table entry data is expected to arrive via RMR message with a message type of &MT_TABLE_DATA. The message may contain one or more entries provided that the entries are newline separated. Current versions of RMR support very large messages, however to ensure compatibility with an xAPP built using an older version of RMR (pre 3.8), messages should be limited to 4 KiB. &h2(Table Acceptance and Acknowledgement) When RMR receives the table end entry (newrt|end), it will send a state message back to the &RTMGR to indicate the state of the received table. The message type is &MT_TABLE_STATE and the payload will contain UTF-8 tokens which indicate the state. The second token will be the &ital(table ID) supplied on the start record, or the string "." When the state is an error state, RMR might add a final set of tokens which contain the reason for the failure. &space Upon receipt of a status message which indicates an "OK" response, the &RTMGR can assume that the table has been installed and is in use. Any other response indicates that RMR did not use the table and has dropped it; the previous table is still in use. &h2(Using A Static Route Table) A static route table can be provided to assist with testing, or to provide a bootstrap set of route information until a dynamic table is received from a routing manager. The environment variable &cw(RMR_SEED_RT) is checked during RMR initialisation and if set is expected to reference a file containing a route table. This table will be loaded and used until overlaid by a table sent by the &RTMGR. &space To simulate dynamic reloads during testing, and for some specialised use cases, the static table will be reloaded periodically if the &cw(RMR_RTG_SVC) environment variable is set to -1. When set to -1 RMR will not listen for &RTMGR connections, nor will it attempt to request a dynamic table. &space If the file given by the &cw(RMR_SEED_RT) variable does not exist, and the &cw(RMR_RTG_SVC) variable is set to -1, RMR will block until the table is created. This simulates a delayed dynamic load during testing, and can be used when the xAPP is reading the route table saved by another local process rather than one sent directly by the &RTMGR. &h2(Table Stashing) To assist with debugging, and to allow an application to share the route table received from &RTMGR, RMR will stash the route table updates it received. Updates are stashed in a file named by the &cw(RMR_STASH_RT) environment variable, and if that variable is not present, the &cw(RR_SEED_RT) variable will be used with an added &cw(.stash) extension. &space The primary use of route table stashing is to assist with debugging of applications, and because there are risks for an application to share its table, table sharing is &bold(NOT) recommended. Table sharing can be enabled by setting the &cw(RMR_STASH_RT) variable for the application that will be the source of the route table updates, and using the file named for tha application when defining the &cw(RMR_SEED_RT) variable for applications which are to read the table information. Obviously, all applications must be running in the same container, on the same host, or have a common disk volum between their environments. Known risks to using table sharing include &half_space &indent &beg_list(&lic1) &li An update to the table (not a complete table) may be received prior to one or more readers accessing the file, and thus the reader may not receive a valid or complete table. &half_space &li Any entry which has a sender:port associated with the message type will likely be ignored by all readers. &end_list &uindent &h1(Routing Using MEID) Starting with version 1.13.0, RMR provides the ability to select the endpoint for a message based on the MEID (managed entity ID) in the message, rather than selecting the endpoint from the round-robin list for the matching route table entry. When the MEID is used, the message is sent to the endpoint which &ital(owns,) or is responsible for the managed entity. Should the &ital(owner) change messages will be routed to the new owner when the route table is updated. To make use of MEID routing, there must be one or more route table entries which list the special endpoint name &cw(%meid) instead of providing a round robin list. As an example, consider the following route table entry: &space &ex_start mse| 1000,forwarder:43086 | 10 | %meid &ex_end &fig( Sample route entry with the meid flag.) &space The final field of the entry doesn't specify a round-robin group which means that when an application attempts to send a message with type 1000, and the subscription ID of 10, the MEID in the message will be used to select the endpoint. &h2(MEID endpoint selection) To select an endpoint for the message based on the MEID in a message, RMR must know which endpoint owns the MEID. This information, known as an MEID map, is provided by the &RTMGR over the same communication path as the route table is supplied. The following is the syntax for an MEID map. &half_space &ex_start meid_map | start | mme_ar | | [...] mme_del | [...] meid_map | end | [| ] &ex_end &fig( Meid map table.) &space The mme_ar records are add/update records and allow for the list of MEIDs to be associated with (owned by) the indicated endpoint. The is the hostname:port, or IP address and port, of the application which owns the MEID and thus should receive any messages which are routed based on a route table entry with %meid as the round-robin group. The mme_del records allow for MEIDs to be deleted from RMR's view. Finally, the is the number of add/replace and delete records which were sent; if RMR does not match the value to the number of records, then it will not add the data to the table. Updates only need to list the ownership changes that are necessary; in other words, the &RTMGR does not need to supply all of the MEID relationships with each update. &space The optional field on the end record should be the MD5 hash of all of the records between the start and end records. This allows for a precise verification that the transmitted data was correctly received. &space If a static seed file is being used for the route table, a second section can be given which supplies the MEID map. The following is a small example of a seed file: &half_space &ex_start newrt|start | id-64306 mse|0|-1| %meid mse|1|-1|172.19.0.2:4560 mse|2|-1|172.19.0.2:4560 mse|3|-1|172.19.0.2:4560 mse|4|-1|172.19.0.2:4560 mse|5|-1|172.19.0.2:4560 newrt|end meid_map | start | id-028919 mme_ar| 172.19.0.2:4560 | meid000 meid001 meid002 meid003 meid004 meid005 mme_ar| 172.19.0.42:4560 | meid100 meid101 meid102 meid103 mme_del | meid1000 meid_map | end | 1 &ex_end &fig( Illustration of both a route table and meid map in the same file. ) &space The tables above will route all messages with a message type of 0 based on the MEID. There are 10 meids which are owned by two different endpoints. The table also deletes the MEID meid1000 from RMR's view. &h1(Reserved Message Types) RMR is currently reserving message types in the range of 0 through 99 (inclusive) for its own use. Please do not use these types in any production or test environment as the results may be undesired. .** -- appendices ------------------------------ .dv gloss_appendix A .pa .im glossary.im .** ----------- end housekeeping --------------- .** if pfm and index was setup, include it now .if index_here .st 8p &index_here .st &textsize .fi .pa .** capture all interesting variables to be used as forward references during pass 2 .ca expand start p1var_setup.ca .** pass 1 variable settings -- do NOT commit to repo .** set vars which need to be referenced prior to use, for example: .** .dv qr_appendix &qr_appendix .ca end