RMR User's Guide
============================================================================================
-The RIC Message Router (RMR) is a library which applications
-use to send and receive messages where the message routing,
-endpoint selection, is based on the message type rather than
-on traditional DNS names or IP addresses. Because the user
-documentation for RMR is a collection of UNIX manpages
-(included in the development package, and available via the
-man command when installed), there is no separate "User's
-Guide." To provide something for the document scrapers to
-find, this is a collection of the RMR manual pages formatted
-directly from their source which might be a bit ragged when
-combined into a single markup document. Read the manual pages
-:)
+The RIC Message Router (RMR) is a library for peer-to-peer
+communication. Applications use the library to send and
+receive messages where the message routing and endpoint
+selection is based on the message type rather than DNS host
+name-IP port combinations.
+
+This document contains information that developers need to
+know to use the RMR library. Because the primary
+documentation for the RMR library is a collection of UNIX
+manpages (included in the development package, and available
+via the man command when installed), there is no separate
+"User's Guide." To provide something for the document
+scrapers to find, this is a collection of the RMR manual
+pages formatted directly from their source, which might be a
+bit ragged when combined into a single markup document. Read
+the manual pages :)
and wait for the reply; the underlying RMR processing
expects that the matching reply message will also contain
the same data in the *xaction* field.
-
-
+
sub_id
For C applications making use of RMR, the state of a
transport based failure will often be available via errno.
- However, some wrapper environments may not have direct access
- to the C-lib errno value. RMR send and receive operations
- will place the current value of errno into this field which
- should make it available to wrapper functions. User
- applications are strongly cautioned against relying on the
- value of errno as some transport mechanisms may not set this
- value on all calls. This value should also be ignored any
- time the message status is RMR_OK.
+ However, some wrapper environments may not have direct
+ access to the C-lib errno value. RMR send and receive
+ operations will place the current value of errno into this
+ field which should make it available to wrapper functions.
+ User applications are strongly cautioned against relying
+ on the value of errno as some transport mechanisms may not
+ set this value on all calls. This value should also be
+ ignored any time the message status is RMR_OK.
RETURN VALUE
--------------------------------------------------------------------------------------------
-The function returns a pointer to a rmr_mbuf structure, or NULL
-on error.
+The function returns a pointer to a rmr_mbuf structure, or
+NULL on error.
ERRORS
--------------------------------------------------------------------------------------------
SEE ALSO
--------------------------------------------------------------------------------------------
-rmr_tralloc_msg(3), rmr_call(3), rmr_free_msg(3), rmr_init(3),
-rmr_init_trace(3), rmr_get_trace(3), rmr_get_trlen(3),
-rmr_payload_size(3), rmr_send_msg(3), rmr_rcv_msg(3),
-rmr_rcv_specific(3), rmr_rts_msg(3), rmr_ready(3), rmr_fib(3),
-rmr_has_str(3), rmr_tokenise(3), rmr_mk_ring(3),
-rmr_ring_free(3), rmr_set_trace(3)
+rmr_tralloc_msg(3), rmr_call(3), rmr_free_msg(3),
+rmr_init(3), rmr_init_trace(3), rmr_get_trace(3),
+rmr_get_trlen(3), rmr_payload_size(3), rmr_send_msg(3),
+rmr_rcv_msg(3), rmr_rcv_specific(3), rmr_rts_msg(3),
+rmr_ready(3), rmr_fib(3), rmr_has_str(3), rmr_tokenise(3),
+rmr_mk_ring(3), rmr_ring_free(3), rmr_set_trace(3)
NAME
Enable multi-threaded call support.
- &ditem Some underlying transport providers (e.g. SI95)
- enable locking to be turned off if the user application is
- single threaded, or otherwise can guarantee that RMR
- functions will not be invoked concurrently from different
- threads. Turning off locking can help make message receipt
- more efficient. If this flag is set when the underlying
+
+RMRFL_NOLOCK
+
+ Some underlying transport providers (e.g. SI95) enable
+ locking to be turned off if the user application is single
+ threaded, or otherwise can guarantee that RMR functions
+ will not be invoked concurrently from different threads.
+ Turning off locking can help make message receipt more
+ efficient. If this flag is set when the underlying
transport does not support disabling locks, it will be
ignored.
-RMR_SEED_RT
+RMR_ASYNC_CONN
+
+ Allows the async connection mode to be turned off (by
+ setting the value to 0. When set to 1, or missing from the
+ environment, RMR will invoke the connection interface in
+ the transport mechanism using the non-blocking (async)
+ mode. This will likely result in many "soft failures"
+ (retry) until the connection is established, but allows
+ the application to continue unimpeeded should the
+ connection be slow to set up.
+
+
+RMR_BIND_IF
+
+ This provides the interface that RMR will bind listen
+ ports to allowing for a single interface to be used rather
+ than listening across all interfaces. This should be the
+ IP address assigned to the interface that RMR should
+ listen on, and if not defined RMR will listen on all
+ interfaces.
+
+
+RMR_CTL_PORT
+
+ This variable defines the port that RMR should open for
+ communications with Route Manager, and other RMR control
+ applications. If not defined, the port 4561 is assumed.
- Assumes this is the filename of the seed route table file
- to use. In normal situations, the library will wait for an
- update from the route table generator (expected within a
- few seconds of initialisation) before being able to send
- messages. However, in some situations where a bootstrap
- table is necessary, this is the means to supply it to the
- library.
+ Previously, the RMR_RTG_SVC (route table generator service
+ port) was used to define this port. However, a future
+ version of Route Manager will require RMR to connect and
+ request tables, thus that variable is now used to supply
+ the Route Manager well known address and port.
+
+ To maintain backwards compatablibility with the older
+ Route Manager versions, the presence of this variable in
+ the environment will shift RMR's behaviour with respect to
+ the default value used when RMR_RTG_SVC is **not**
+ defined.
+
+ When RMR_CTL_PORT is **defined:** RMR assumes that Route
+ Manager requires RMR to connect and request table updates
+ is made, and the default well known address for Route
+ manager is used (routemgr:4561).
+
+ When RMR_CTL_PORT is **undefined:** RMR assumes that Route
+ Manager will connect and push table updates, thus the
+ default listen port (4561) is used.
+
+ To avoid any possible misinterpretation and/or incorrect
+ assumptions on the part of RMR, it is recommended that
+ both the RMR_CTL_PORT and RMR_RTG_SVC be defined. In the
+ case where both variables are defined, RMR will behave
+ exactly as is communicated with the variable's values.
RMR_RTG_SVC
- The route table generator assumes that RMr is listening on
- a well known port (4561) by default, but this environment
- variable can be used to change the listening port if
- needed. The value of the variable is expected to be just
- the port.
+ The value of this variable depends on the Route Manager in
+ use.
+
+ When the Route Manager is expecting to connect to an xAPP
+ and push route tables, this variable must indicate the
+ port which RMR should use to listen for these connections.
+
+ When the Route Manager is expecting RMR to connect and
+ request a table update during initialisation, the variable
+ should be the host of the Route Manager process.
+
+ The RMR_CTL_PORT variable (added with the support of
+ sending table update requests to Route manager), controls
+ the behaviour if this variable is not set. See the
+ description of that variable for details.
+
+
+RMR_HR_LOG
+
+ By default RMR writes messages to standard error
+ (incorrectly referred to as log messages) in human
+ readable format. If this environment variable is set to 0,
+ the format of standard error messages might be written in
+ some format not easily read by humans. If missing, a value
+ of 1 is assumed.
+
+
+RMR_LOG_VLEVEL
+
+ This is a numeric value which corresponds to the verbosity
+ level used to limit messages written to standard error.
+ The lower the number the less chatty RMR functions are
+ during execution. The following is the current
+ relationship between the value set on this variable and
+ the messages written:
+
+
+0
+
+ Off; no messages of any sort are written.
+
+
+1
+
+ Only critical messages are written (default if this
+ variable does not exist)
+
+
+2
+
+ Errors and all messages written with a lower value.
+
+
+3
+
+ Warnings and all messages written with a lower value.
+
+
+4
+
+ Informational and all messages written with a lower
+ value.
+
+
+5
+
+ Debugging mode -- all messages written, however this
+ requires RMR to have been compiled with debugging
+ support enabled.
+
+
+
+RMR_RTG_ISRAW
+
+ **Deprecated.** Should be set to 1 if the route table
+ generator is sending "plain" messages (not using RMR to
+ send messages, 0 if the rtg is using RMR to send. The
+ default is 1 as we don't expect the rtg to use RMR.
+
+ This variable is only recognised when using the NNG
+ transport library as it is not possible to support NNG
+ "raw" communications with other transport libraries. It is
+ also necessary to match the value of this variable with
+ the capabilities of the Route Manager; at some point in
+ the future RMR will assume that all Route Manager messages
+ will arrive via an RMR connection and will ignore this
+ variable.
+
+RMR_SEED_RT
+
+ This is used to supply a static route table which can be
+ used for debugging, testing, or if no route table
+ generator process is being used to supply the route table.
+ If not defined, no static table is used and RMR will not
+ report *ready* until a table is received. The static route
+ table may contain both the route table (between newrt
+ start and end records), and the MEID map (between meid_map
+ start and end records)
+
+RMR_SRC_ID
+
+ This is either the name or IP address which is placed into
+ outbound messages as the message source. This will used
+ when an RMR based application uses the rmr_rts_msg()
+ function to return a response to the sender. If not
+ supplied RMR will use the hostname which in some container
+ environments might not be routable.
+
+ The value of this variable is also used for Route Manager
+ messages which are sent via an RMR connection.
+
+RMR_VCTL_FILE
+
+ This supplies the name of a verbosity control file. The
+ core RMR functions do not produce messages unless there is
+ a critical failure. However, the route table collection
+ thread, not a part of the main message processing
+ component, can write additional messages to standard
+ error. If this variable is set, RMR will extract the
+ verbosity level for these messages (0 is silent) from the
+ first line of the file. Changes to the file are detected
+ and thus the level can be changed dynamically, however RMR
+ will only suss out this variable during initialisation, so
+ it is impossible to enable verbosity after startup.
+
+RMR_WARNINGS
+
+ If set to 1, RMR will write some warnings which are
+ non-performance impacting. If the variable is not defined,
+ or set to 0, RMR will not write these additional warnings.
RETURN VALUE
--------------------------------------------------------------------------------------------
The *state* field in the message buffer will indicate either
-RMR_OK or RMR_ERR_EMPTY if an empty message was received. If
-a nil pointer is returned, or any other state value was set
-in the message buffer, errno will be set to one of the
-following:
+RMR_OK when the message receive process was successful and
+the message can be used by the caller. Depending on the
+underlying transport mechanism, one of the following RMR
+error stats may be returned:
+
+
+
+RMR_ERR_EMPTY
+
+ The message received had no payload, or was completely
+ empty.
+
+
+RMR_ERR_TIMEOUT
+
+ For some transport mechanisms, or if reading the receive
+ queue from multiple threads, it is possible for one thread
+ to find no data waiting when it queries the queue. When
+ this state is reported, the message buffer does not
+ contain message data and the user application should
+ reinvoke the receive function.
+
+
+When an RMR error state is reported, the underlying errno
+value might provide more information. The following is a list
+of possible values that might accompany the states listed
+above:
+
+RMR_ERR_EMPTY if an empty message was received. If a nil
+pointer is returned, or any other state value was set in the
+message buffer, errno will be set to one of the following:
rmr_ring_free(3), rmr_torcv_rcv(3), rmr_wh_send_msg(3)
+NAME
+--------------------------------------------------------------------------------------------
+
+rmr_set_fack
+
+SYNOPSIS
+--------------------------------------------------------------------------------------------
+
+
+::
+
+ #include <rmr/rmr.h>
+ void rmr_set_fack( void* vctx );
+
+
+
+DESCRIPTION
+--------------------------------------------------------------------------------------------
+
+The rmr_set_fack function enables *fast TCP acknowledgements*
+if the underlying transport library supports it. This might
+be useful for applications which must send messages as a
+maximum rate.
+
+RETURN VALUE
+--------------------------------------------------------------------------------------------
+
+There is no return value.
+
+ERRORS
+--------------------------------------------------------------------------------------------
+
+This function does not generate any errors.
+
+SEE ALSO
+--------------------------------------------------------------------------------------------
+
+rmr_init(3),
+
+
NAME
--------------------------------------------------------------------------------------------
::
#include <rmr/rmr.h>
- int rmr_bytes2payload( rmr_mbuf_t* mbuf, unsigned char* data, int len )
+ int rmr_set_trace( rmr_mbuf_t* mbuf, unsigned char* data, int len )
rmr_wh_open(3), rmr_wh_send_msg(3)
+NAME
+--------------------------------------------------------------------------------------------
+
+rmr_set_trace
+
+SYNOPSIS
+--------------------------------------------------------------------------------------------
+
+
+::
+
+ #include <rmr/rmr.h>
+ #include <rmr/rmr_logging.h>
+ void rmr_set_vlevel( int new_level )
+
+
+
+DESCRIPTION
+--------------------------------------------------------------------------------------------
+
+The rmr_set_vlevel allows the user programme to set the
+verbosity level which is used to determine the messages RMR
+writes to standard error. The new_vlevel value must be one of
+the following constants which have the indicated meanings:
+
+
+RMR_VL_OFF
+
+ Turns off all message writing. This includes the stats and
+ debugging messages generated by the route collector thread
+ which are normally affected only by the externally managed
+ verbose level file (and related environment variable).
+
+
+RMR_VL_CRIT
+
+ Write only messages of critical importance. From the point
+ of view of RMR, when a critical proper behaviour of the
+ library cannot be expected or guaranteed.
+
+RMR_VL_ERR
+
+ Include error messages in the output. An error is an event
+ from which RMR has no means to recover. Continued proper
+ execution is likely except where the affected connection
+ and/or component mentioned in the error is concerned.
+
+RMR_VL_WARN
+
+ Include warning messages in the output. A warning
+ indicates an event which is not considered to be normal,
+ but is expected and continued acceptable behaviour of the
+ system is assured.
+
+RMR_VL_INFO
+
+ Include informational messagees in the output.
+ Informational messages include some diagnostic information
+ which explain the activities of RMR.
+
+RMR_VL_DEBUG
+
+ Include all debugging messages in the output. Debugging
+ must have also been enabled during the build as a
+ precaution to accidentally enabling this level of output
+ as it can grossly affect performance.
+
+
+generally RMR does not write messages to the standard error
+device from *critical path* functions, therefore it is
+usually not harmful to enable a verbosity level of either
+RMR_VL_CRIT, or RMR_VL_ERR.
+
+Messages written from the route table collection thread are
+still governed by the value placed into the verbose level
+control file (see the man page for rmr_init()); those
+messages are affected only when logging is completely
+disabled by passing RMR_VL_OFF to this function.
+
+The verbosity level can also be set via an environment
+variable prior to the start of the RMR based application. The
+environment variable is read only during initialisation; if
+the programme must change the value during execution, this
+function must be used. The default value, if this function is
+never called, and the environment variable is not present, is
+RMR_VL_ERR.
+
+SEE ALSO
+--------------------------------------------------------------------------------------------
+
+rmr_init(3)
+
+
NAME
--------------------------------------------------------------------------------------------
rmr_mk_ring(3), rmr_ring_free(3), rmr_set_trace(3)
+NAME
+--------------------------------------------------------------------------------------------
+
+rmr_wh_call
+
+SYNOPSIS
+--------------------------------------------------------------------------------------------
+
+
+::
+
+ #include <rmr/rmr.h>
+ rmr_mbuf_t* rmr_wh_call( void* vctx, rmr_whid_t whid, rmr_mbuf_t* msg, int call_id, int max_wait )
+
+
+
+DESCRIPTION
+--------------------------------------------------------------------------------------------
+
+The rmr_wh_call function accepts a message buffer (msg) from
+the user application and attempts to send it using the
+wormhole ID provided (whid). If the send is successful, the
+call will block until either a response message is received,
+or the max_wait number of milliseconds has passed. In order
+for the response to be recognised as a response, the remote
+process **must** use rmr_rts_msg() to send their response.
+
+Like *rmr_wh_send_msg,* this function attempts to send the
+message directly to a process at the other end of a wormhole
+which was created with *rmr_wh-open().* When sending message
+via wormholes, the normal RMr routing based on message type
+is ignored, and the caller may leave the message type
+unspecified in the message buffer (unless it is needed by the
+receiving process). The call_id parameter is a number in the
+range of 2 through 255 and is used to identify the calling
+thread in order to properly match a response message when it
+arrives. Providing this value, and ensuring the proper
+uniqueness, is the responsibility of the user application and
+as such the ability to use the rmr_wh_call() function from
+potentially non-threaded concurrent applications (such as
+Go's goroutines) is possible.
+
+Retries
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The send operations in RMr will retry *soft* send failures
+until one of three conditions occurs:
+
+
+
+1.
+
+ The message is sent without error
+
+
+2.
+
+ The underlying transport reports a * hard * failure
+
+
+3.
+
+ The maximum number of retry loops has been attempted
+
+
+A retry loop consists of approximately 1000 send attemps **
+without** any intervening calls to * sleep() * or * usleep().
+* The number of retry loops defaults to 1, thus a maximum of
+1000 send attempts is performed before returning to the user
+application. This value can be set at any point after RMr
+initialisation using the * rmr_set_stimeout() * function
+allowing the user application to completely disable retires
+(set to 0), or to increase the number of retry loops.
+
+Transport Level Blocking
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The underlying transport mechanism used to send messages is
+configured in *non-blocking* mode. This means that if a
+message cannot be sent immediately the transport mechanism
+will **not** pause with the assumption that the inability to
+send will clear quickly (within a few milliseconds). This
+means that when the retry loop is completely disabled (set to
+0), that the failure to accept a message for sending by the
+underlying mechanisms (software or hardware) will be reported
+immediately to the user application.
+
+It should be noted that depending on the underlying transport
+mechanism being used, it is extremly possible that during
+normal operations that retry conditions are very likely to
+happen. These are completely out of RMr's control, and there
+is nothing that RMr can do to avoid or midigate these other
+than by allowing RMr to retry the send operation, and even
+then it is possible (e.g. during connection reattempts), that
+a single retry loop is not enough to guarentee a successful
+send.
+
+RETURN VALUE
+--------------------------------------------------------------------------------------------
+
+On success, new message buffer, with the payload containing
+the response from the remote endpoint is returned. The state
+in this buffer will reflect the overall send operation state
+and should be RMR_OK.
+
+If a message is returned with a state which is anything other
+than RMR_OK, the indication is that the send was not
+successful. The user application must check the state and
+determine the course of action. If the return value is NULL,
+no message, the indication is that there was no response
+received within the timeout (max_wait) period of time.
+
+ERRORS
+--------------------------------------------------------------------------------------------
+
+The following values may be passed back in the *state* field
+of the returned message buffer.
+
+
+
+RMR_ERR_WHID
+
+ The wormhole ID passed in was not associated with an open
+ wormhole, or was out of range for a valid ID.
+
+RMR_ERR_NOWHOPEN
+
+ No wormholes exist, further attempt to validate the ID are
+ skipped.
+
+RMR_ERR_BADARG
+
+ The message buffer pointer did not refer to a valid
+ message.
+
+RMR_ERR_NOHDR
+
+ The header in the message buffer was not valid or
+ corrupted.
+
+
+EXAMPLE
+--------------------------------------------------------------------------------------------
+
+The following is a simple example of how the a wormhole is
+created (rmr_wh_open) and then how rmr_wh_send_msg function
+is used to send messages. Some error checking is omitted for
+clarity.
+
+
+::
+
+ #include <rmr/rmr.h> .// system headers omitted for clarity
+ int main() {
+ rmr_whid_t whid = -1; // wormhole id for sending
+ void* mrc; //msg router context
+ int i;
+ rmr_mbuf_t* sbuf; // send buffer
+ int count = 0;
+ mrc = rmr_init( "43086", RMR_MAX_RCV_BYTES, RMRFL_NONE );
+ if( mrc == NULL ) {
+ fprintf( stderr, "[FAIL] unable to initialise RMr environment\\n" );
+ exit( 1 );
+ }
+ while( ! rmr_ready( mrc ) ) { e i// wait for routing table info
+ sleep( 1 );
+ }
+ sbuf = rmr_alloc_msg( mrc, 2048 );
+ while( 1 ) {
+ if( whid < 0 ) {
+ whid = rmr_wh_open( mrc, "localhost:6123" ); // open fails if endpoint refuses conn
+ w if( RMR_WH_CONNECTED( wh ) ) {
+ snprintf( sbuf->payload, 1024, "periodic update from sender: %d", count++ );
+ sbuf->len = strlen( sbuf->payload );
+ sbuf = rmr_wh_call( mrc, whid, sbuf, 1000 ); f s// expect a response in 1s or less
+ if( sbuf != NULL && sbuf->state = RMR_OK ) {
+ sprintf( stderr, "response: %s\\n", sbuf->payload ); x// assume they sent a string
+ } else {
+ sprintf( stderr, "response not received, or send error\\n" );
+ }
+ }
+ }
+ sleep( 5 );
+ }
+ }
+
+
+
+SEE ALSO
+--------------------------------------------------------------------------------------------
+
+rmr_alloc_msg(3), rmr_call(3), rmr_free_msg(3), rmr_init(3),
+rmr_payload_size(3), rmr_rcv_msg(3), rmr_rcv_specific(3),
+rmr_rts_msg(3), rmr_ready(3), rmr_fib(3), rmr_has_str(3),
+rmr_tokenise(3), rmr_mk_ring(3), rmr_ring_free(3),
+rmr_set_stimeout(3), rmr_wh_open(3), rmr_wh_close(3),
+rmr_wh_state(3)
+
+
NAME
--------------------------------------------------------------------------------------------
rmr_get_rcvfd(3), rmr_payload_size(3), rmr_send_msg(3),
rmr_rcv_msg(3), rmr_rcv_specific(3), rmr_rts_msg(3),
rmr_ready(3), rmr_fib(3), rmr_has_str(3), rmr_tokenise(3),
-rmr_mk_ring(3), rmr_ring_free(3), rmr_wh_send_msg(3),
-rmr_wh_close(3)
+rmr_mk_ring(3), rmr_ring_free(3), rmr_wh_close(3),
+rmr_wh_send_msg(3), rmr_wh_state(3)
NAME
rmr_payload_size(3), rmr_rcv_msg(3), rmr_rcv_specific(3),
rmr_rts_msg(3), rmr_ready(3), rmr_fib(3), rmr_has_str(3),
rmr_tokenise(3), rmr_mk_ring(3), rmr_ring_free(3),
-rmr_set_stimeout(3), rmr_wh_open(3), rmr_wh_close(3)
+rmr_set_stimeout(3), rmr_wh_open(3), rmr_wh_close(3),
+rmr_wh_state(3)
+
+
+NAME
+--------------------------------------------------------------------------------------------
+
+rmr_wh_state
+
+SYNOPSIS
+--------------------------------------------------------------------------------------------
+
+
+::
+
+ #include <rmr/rmr.h>
+ int rmr_wh_state( void* vctx, rmr_whid_t whid )
+
+
+
+DESCRIPTION
+--------------------------------------------------------------------------------------------
+
+The rmr_wh_state function will return the current state of
+the connection associated with the given wormhole (whid). The
+return value indicates whether the connection is open
+(RMR_OK), or closed (any other return value).
+
+When using some transport mechanisms (e.g. NNG), it may not
+be possible for RMR to know the actual state and the
+connection may always be reported as "open."
+
+RETURN
+--------------------------------------------------------------------------------------------
+
+The following values are potential return values.
+
+
+
+RMR_OK
+
+ The wormhole ID is valid and the connection is "open."
+
+
+RMR_ERR_WHID
+
+ THe wormhole ID passed into the function was not valid.
+
+
+RMR_ERR_NOENDPT
+
+ The wormhole is not open (not connected).
+
+
+RMR_ERR_BADARG
+
+ The context passed to the function was nil or invalid.
+
+
+RMR_ERR_NOWHOPEN
+
+ Wormholes have not been initialised (no wormhole open call
+ has been made).
+
+
+
+SEE ALSO
+--------------------------------------------------------------------------------------------
+rmr_wh_open(3), rmr_wh_send_msg(3), rmr_wh_close(3)
Code Review / ric-plt / lib / rmr.git / blobdiff