2 ==================================================================================
3 Copyright (c) 2019 Nokia
4 Copyright (c) 2018-2019 AT&T Intellectual Property.
6 Licensed under the Apache License, Version 2.0 (the "License");
7 you may not use this file except in compliance with the License.
8 You may obtain a copy of the License at
10 http://www.apache.org/licenses/LICENSE-2.0
12 Unless required by applicable law or agreed to in writing, software
13 distributed under the License is distributed on an "AS IS" BASIS,
14 WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
15 See the License for the specific language governing permissions and
16 limitations under the License.
17 ==================================================================================
21 Mnemonic: general_use.im
22 Abstract: Major section -- Describes the general use of RMR.
24 Author: E. Scott Daniels
28 To use, the RMR based application simply needs to initialise the RMR
29 environment, wait for RMR to have received a routing table (become
30 ready), and then invoke either the send or receive functions. These
31 steps, and some behind the scenes details, are described in the
35 The RMR function &func(rmr_init:) is used to set up the RMR
36 environment and must be called before messages can be sent or
37 received. One of the few parameters that the application must
38 communicate to RMR is the port number that will be used as the listen
39 port for new connections. The port number is passed on the
40 initialisation function call and a TCP listen socket will be opened
41 with this port. If the port is already in use RMR will report a
42 failure; the application will need to reinitialise with a different
43 port number, abort, or take some other action appropriate for the
46 In addition to creating a TCP listen port, RMR will start a process
47 thread which will be responsible for receiving dynamic updates to the
48 route table. This thread also causes a TCP listen port to be opened
49 as it is expected that the process which generates route table updates
50 will connect and send new information when needed. The route table
51 update port is &bold(not) supplied by the application, but is supplied
52 via an environment variable as this value is likely determined by the
53 mechanism which is starting and configuring the application.
56 On successful initialisation, a void pointer, often called a
57 &ital(handle) by some programming languages, is returned to the
58 application. This is a reference to the RMR control information and
59 must be passed as the first parameter on most RMR function calls.
60 RMR refers to this as the context, or ctx.
63 An application which is only receiving messages does not need to wait
64 for RMR to &ital(become ready) after the call to the initialization
65 function. However, before the application can successfully send a
66 message, RMR must have loaded a route table, and the application must
67 wait for RMR to report that it has done so. The RMR function
68 &func(rmr_ready:) will return the value &ital(true) (1) when a
69 complete route table has been loaded and can be used to determine the
70 endpoint for a send request.
72 &h2(Receiving Messages)
73 The process of receiving is fairly straight forward. The application
74 invokes the RMR &func(rmr_rcv_msg:) function which will block until a
75 message is received. The function returns a pointer to a message
76 block which provides all of the details about the message.
77 Specifically, the application has access to the following information
78 either directly or indirectly:
83 &li The payload (actual data)
85 &li The total payload length in bytes
87 &li The number of bytes of the payload which contain valid data
89 &li The message type and subscription ID values
91 &li The hostname and IP address of the source of the message (the sender)
93 &li The transaction ID
95 &li Tracing data (if provided)
100 &h3(The Message Payload)
101 The message payload contains the &ital(raw) data that was sent by the
102 peer application. The format will likely depend on the message type,
103 and is expected to be known by the application. A direct pointer to
104 the payload is available from the message buffer (see appendix
105 &mbuf_appendix for specific message buffer details).
108 Two payload-related length values are also directly available: the
109 total payload length, and the number of bytes actually filled with
110 data. The used length is set by the caller, and may or not be an
111 accurate value. The total payload length is determined when the
112 buffer is created for sending, and is the maximum number of bytes that
113 the application may modify should the buffer be used to return a
116 &h3(Message Type and Subscription ID)
117 The message type and subscription ID are both directly available from
118 the message buffer, and are the values which were used to by RMR in
119 the sending application to select the endpoint. If the application
120 resends the message, as opposed to returning the message buffer as a
121 response, the message number and/or the subscription ID might need to
122 be changed to avoid potential issues ¬e .sm .
123 .cn l=&cn_line_len i=&cn_ident start &atbot Times-roman 8p 1i
124 It is entirely possible to design a routing table, and
125 application group, such that the same message type is
126 is left unchanged and the message is forwarded by an
127 application after updating the payload. This type of behaviour
128 is often referred to as service chaining, and can be done
129 without any "knowledge" by an application with respect to
130 where the message goes next. Service chaining is supported
131 by RMR in as much as it allows the message to be resent, but
132 the actual complexities of designing and implementing service
133 chaining lie with the route table generator process.
136 &h3(Sender Information)
137 The source, or sender information, is indirectly available to the
138 application via the &func(rmr_get_src:) and &func(rmr_get_ip:)
139 functions. The former returns a string containing
140 &cw(hostname^:port,) while the string &cw(ip^:port) is returned by the
144 The message buffer contains a fixed length set of bytes which
145 applications can set to track related messages across the application
146 concept of a transaction. RMR will use the transaction ID for
147 matching a response message when the &func(rmr_call:) function is used
150 &h3(Trace Information)
151 RMR supports the addition of an optional trace information to any
152 message. The presence and size is controlled by the application, and
153 can vary from message to message if desired. The actual contents of
154 the trace information is determined by the application; RMR provides
155 only the means to set, extract, and obtain a direct reference to the
156 trace bytes. The trace data field in a message buffer is discussed in
157 greater detail in the &ital(Trace Data) section.
159 &h2(Sending Messages)
160 Sending requires only slightly more work on the part of the
161 application than receiving a message. The application must allocate
162 an RMR message buffer, populate the message payload with data, set the
163 message type and length, and optionally set the subscription ID.
164 Information such as the source IP address, hostname, and port are
165 automatically added to the message buffer by RMR, so there is no need
166 for the application to worry about these.
168 &h3(Message Buffer Allocation)
169 The function &func(rmr_msg_alloc:) allocates a &ital(zero copy) buffer
170 and returns a pointer to the RMR &cw(rmr_mbuf_t) structure. The
171 message buffer provides direct access to the payload, length, message
172 type and subscription ID fields. The buffer must be preallocated in
173 order to allow the underlying transport mechanism to allocate the
174 payload space from its internal memory pool; this eliminates multiple
175 copies as the message is sent, and thus is more efficient.
178 If a message buffer has been received, and the application wishes to
179 use the buffer to send a response, or to forward the buffer to another
180 application, a new buffer does &bold(not) need to be allocated. The
181 application may set the necessary information (message type, etc.),
182 and adjust the payload, as is necessary and then pass the message
183 buffer to &func(rmr_send_msg:) or &func(rmr_rts_msg:) to be sent or
184 returned to the sender.
186 &h3(Populating the Message Buffer)
187 The application has direct access to several of the message buffer
188 fields, and should set them appropriately.
191 &beg_dlist( 1i &ditext )
192 &di(len) This is the number of bytes that the application
193 placed into the payload. Setting length to 0
194 is allowed, and length may be less than the
195 allocated payload size.
198 &di(mtype) The message type that RMR will use to determine the
199 endpoint used as the target of the send.
202 &di(sub_id) The subscription ID if the message is to be routed
203 based on the combination of message type and
204 subscription ID. If no subscription ID is
205 valid for the message, the application should
206 set the field with the RMR constant
210 &di(payload) The application should obtain the reference
211 (pointer) to the payload from the message
212 buffer and place any data into the payload.
213 The application is responsible for ensuring
214 that the maximum payload size is not exceeded.
215 The application may obtain the maximum size
216 via the &func(rmr_payload_size:) function.
219 &di(trace data) Optionally, the application may add trace
220 information to the message buffer.
226 &h3(Sending a Message Buffer)
227 Once the application has populated the necessary bits of a message, it
228 may be sent by passing the buffer to the &func(rmr_send_msg:)
229 function. This function will select an endpoint to receive the
230 message, based on message type and subscription ID, and will pass the
231 message to the underlying transport mechanism for actual transmission
232 on the connection. (Depending on the underlying transport mechanism,
233 the actual connection to the endpoint may happen at the time of the
234 first message sent to the endpoint, and thus the latency of the first
235 send might be longer than expected.)
238 On success, the send function will return a reference to a message
239 buffer; the status within that message buffer will indicate what the
240 message buffer contains. When the status is &cw(RMR_OK) the reference
241 is to a &bold(new) message buffer for the application to use for the
242 next send; the payload size is the same as the payload size allocated
243 for the message that was just sent. This is a convenience as it
244 eliminates the need for the application to call the message allocation
245 function at some point in the future, and assumes the application will
246 send many messages which will require the same payload dimensions.
249 If the message contains any status other than &cw(RMR_OK,) then the
250 message could &bold(not) be sent, and the reference is to the unsent
251 message buffer. The value of the status will indicate whether the
252 nature of the failure was transient ( .sm &cw(RMR_ERR_RETRY) .sm ) or
253 not. Transient failures are likely to be successful if the
254 application attempts to send the message at a later time.
255 Unfortunately, it is impossible for RMR to know the exact transient
256 failure (e.g. connection being established, or TCP buffer shortage),
257 and thus it is not possible to communicate how long the application
258 should wait before attempting to resend, if the application wishes to
259 resend the message. (More discussion with respect to message retries
260 can be found in the &ital(Handling Failures) section.)