.. Do NOT make changes directly to .rst or .md files.
-
-RMR User's Guide
============================================================================================
-
-The RIC Message Router (RMR) is a library for peer-to-peer
-communication. Applications use the library to send and
-receive messages where the message routing and endpoint
-selection is based on the message type rather than DNS host
-name-IP port combinations.
-
-This document contains information that developers need to
-know to use the RMR library. Because the primary
-documentation for the RMR library is a collection of UNIX
-manpages (included in the development package, and available
-via the man command when installed), there is no separate
-"User's Guide." To provide something for the document
-scrapers to find, this is a collection of the RMR manual
-pages formatted directly from their source, which might be a
-bit ragged when combined into a single markup document. Read
-the manual pages :)
-
-
-
-NAME
---------------------------------------------------------------------------------------------
-
-rmr_alloc_msg
-
-SYNOPSIS
---------------------------------------------------------------------------------------------
-
-
-::
-
- #include <rmr/rmr.h>
- rmr_mbuf_t* rmr_alloc_msg( void* ctx, int size );
-
-
-
-DESCRIPTION
---------------------------------------------------------------------------------------------
-
-The rmr_alloc_msg function is used to allocate a buffer which
-the user programme can write into and then send through the
-RMR library. The buffer is allocated such that sending it
-requires no additional copying out of the buffer. If the
-value passed in size is less than or equal to 0, then the
-*normal maximum size* supplied on the *rmr_init* call will be
-used. When *size* is greater than zero, the message allocated
-will have at least the indicated number of bytes in the
-payload. There is no maximum size imposed by RMR, however the
-underlying system memory managerment (e.g. malloc) functions
-may impose a limit.
-
-The *ctx* parameter is the void context pointer that was
-returned by the *rmr_init* function.
-
-The pointer to the message buffer returned is a structure
-which has some user application visible fields; the structure
-is described in rmr.h, and is illustrated below.
-
-
-::
-
- typedef struct {
- int state;
- int mtype;
- int len;
- unsigned char* payload;
- unsigned char* xaction;
- int sub_id;
- int tp_state;
- } rmr_mbuf_t;
-
-
-
-
-
-state
-
- Is the current buffer state. Following a call to
- rmr_send_msg the state indicates whether the buffer was
- successfully sent which determines exactly what the
- payload points to. If the send failed, the payload
- referenced by the buffer is the message that failed to
- send (allowing the application to attempt a
- retransmission). When the state is RMR_OK the buffer
- represents an empty buffer that the application may fill
- in in preparation to send.
-
-
-mtype
-
- When sending a message, the application is expected to set
- this field to the appropriate message type value (as
- determined by the user programme). Upon send this value
- determines how the RMR library will route the message. For
- a buffer which has been received, this field will contain
- the message type that was set by the sending application.
-
-
-len
-
- The application using a buffer to send a message is
- expected to set the length value to the actual number of
- bytes that it placed into the message. This is likely less
- than the total number of bytes that the message can carry.
- For a message buffer that is passed to the application as
- the result of a receive call, this will be the value that
- the sending application supplied and should indicate the
- number of bytes in the payload which are valid.
-
-
-payload
-
- The payload is a pointer to the actual received data. The
- user programme may read and write from/to the memory
- referenced by the payload up until the point in time that
- the buffer is used on a rmr_send, rmr_call or rmr_reply
- function call. Once the buffer has been passed back to a
- RMR library function the user programme should **NOT**
- make use of the payload pointer.
-
-
-xaction
-
- The *xaction* field is a pointer to a fixed sized area in
- the message into which the user may write a transaction
- ID. The ID is optional with the exception of when the user
- application uses the rmr_call function to send a message
- and wait for the reply; the underlying RMR processing
- expects that the matching reply message will also contain
- the same data in the *xaction* field.
-
-
-sub_id
-
- This value is the subscription ID. It, in combination with
- the message type is used by rmr to determine the target
- endpoint when sending a message. If the application to
- application protocol does not warrant the use of a
- subscription ID, the RMR constant RMR_VOID_SUBID should be
- placed in this field. When an application is forwarding or
- returning a buffer to the sender, it is the application's
- responsibility to set/reset this value.
-
-
-tp_state
-
- For C applications making use of RMR, the state of a
- transport based failure will often be available via errno.
- However, some wrapper environments may not have direct
- access to the C-lib errno value. RMR send and receive
- operations will place the current value of errno into this
- field which should make it available to wrapper functions.
- User applications are strongly cautioned against relying
- on the value of errno as some transport mechanisms may not
- set this value on all calls. This value should also be
- ignored any time the message status is RMR_OK.
-
-
-RETURN VALUE
---------------------------------------------------------------------------------------------
-
-The function returns a pointer to a rmr_mbuf structure, or
-NULL on error.
-
-ERRORS
---------------------------------------------------------------------------------------------
-
-
-
-ENOMEM
-
- Unable to allocate memory.
-
-
-SEE ALSO
---------------------------------------------------------------------------------------------
-
-rmr_tralloc_msg(3), rmr_call(3), rmr_free_msg(3),
-rmr_init(3), rmr_init_trace(3), rmr_get_trace(3),
-rmr_get_trlen(3), rmr_payload_size(3), rmr_send_msg(3),
-rmr_rcv_msg(3), rmr_rcv_specific(3), rmr_rts_msg(3),
-rmr_ready(3), rmr_fib(3), rmr_has_str(3), rmr_tokenise(3),
-rmr_mk_ring(3), rmr_ring_free(3), rmr_set_trace(3)
-
-
-NAME
---------------------------------------------------------------------------------------------
-
-rmr_bytes2meid
-
-SYNOPSIS
---------------------------------------------------------------------------------------------
-
-
-::
-
- #include <rmr/rmr.h>
- int rmr_bytes2meid( rmr_mbuf_t* mbuf, unsigned char* src, int len )
-
-
-
-DESCRIPTION
---------------------------------------------------------------------------------------------
-
-The rmr_bytes2meid function will copy up to *len* butes from
-*src* to the managed entity ID (meid) field in the message.
-The field is a fixed length, gated by the constant
-RMR_MAX_MEID and if len is larger than this value, only
-RMR_MAX_MEID bytes will actually be copied.
-
-RETURN VALUE
---------------------------------------------------------------------------------------------
-
-On success, the actual number of bytes copied is returned, or
--1 to indicate a hard error. If the length is less than 0, or
-not the same as length passed in, errno is set to one of the
-errors described in the *Errors* section.
-
-ERRORS
---------------------------------------------------------------------------------------------
-
-If the returned length does not match the length passed in,
-errno will be set to one of the following constants with the
-meaning listed below.
-
-
-
-EINVAL
-
- The message, or an internal portion of the message, was
- corrupted or the pointer was invalid.
-
-
-EOVERFLOW
-
- The length passed in was larger than the maximum length of
- the field; only a portion of the source bytes were copied.
-
-
-EXAMPLE
---------------------------------------------------------------------------------------------
-
-
-SEE ALSO
---------------------------------------------------------------------------------------------
-
-rmr_alloc_msg(3), rmr_bytes2xact(3), rmr_call(3),
-rmr_free_msg(3), rmr_get_rcvfd(3), rmr_get_meid(3),
-rmr_payload_size(3), rmr_send_msg(3), rmr_rcv_msg(3),
-rmr_rcv_specific(3), rmr_rts_msg(3), rmr_ready(3),
-rmr_fib(3), rmr_has_str(3), rmr_tokenise(3), rmr_mk_ring(3),
-rmr_ring_free(3), rmr_str2meid(3), rmr_str2xact(3),
-rmr_wh_open(3), rmr_wh_send_msg(3)
-
-
-NAME
---------------------------------------------------------------------------------------------
-
-rmr_bytes2payload
-
-SYNOPSIS
---------------------------------------------------------------------------------------------
-
-
-::
-
- #include <rmr/rmr.h>
- void rmr_bytes2payload( rmr_mbuf_t* mbuf, unsigned char* src, int len )
-
-
-
-DESCRIPTION
---------------------------------------------------------------------------------------------
-
-This is a convenience function as some wrapper languages
-might not have the ability to directly copy into the payload
-buffer. The bytes from *src* for the length given are copied
-to the payload. It is the caller's responsibility to ensure
-that the payload is large enough. Upon successfully copy, the
-len field in the message buffer is updated to reflect the
-number of bytes copied.
-
-There is little error checking, and no error reporting.
-
-RETURN VALUE
---------------------------------------------------------------------------------------------
-
-None.
-
-EXAMPLE
---------------------------------------------------------------------------------------------
-
-
-SEE ALSO
---------------------------------------------------------------------------------------------
-
-rmr_alloc_msg(3), rmr_bytes2xact(3), rmr_bytes2payload(3),
-rmr_call(3), rmr_free_msg(3), rmr_get_rcvfd(3),
-rmr_get_meid(3), rmr_payload_size(3), rmr_send_msg(3),
-rmr_rcv_msg(3), rmr_rcv_specific(3), rmr_rts_msg(3),
-rmr_ready(3), rmr_fib(3), rmr_has_str(3), rmr_tokenise(3),
-rmr_mk_ring(3), rmr_ring_free(3), rmr_str2meid(3),
-rmr_str2xact(3), rmr_wh_open(3), rmr_wh_send_msg(3)
-
-
-NAME
---------------------------------------------------------------------------------------------
-
-rmr_bytes2xact
-
-SYNOPSIS
---------------------------------------------------------------------------------------------
-
-
-::
-
- #include <rmr/rmr.h>
- int rmr_bytes2xact( rmr_mbuf_t* mbuf, unsigned char* src, int len )
-
-
-
-DESCRIPTION
---------------------------------------------------------------------------------------------
-
-The rmr_bytes2xact function will copy up to *len* butes from
-*src* to the transaction ID (xaction) field in the message.
-The field is a fixed length, gated by the constant
-RMR_MAX_XID and if len is larger than this value, only
-RMR_MAX_XID bytes will actually be copied.
-
-
-RETURN VALUE
+RIC Message Router -- RMR
+============================================================================================
--------------------------------------------------------------------------------------------
-
-On success, the actual number of bytes copied is returned,
-or -1 to indicate a hard error. If the length is less than
-0, or not the same as length passed in, errno is set to
-one of the errors described in the *Errors* section.
-
-ERRORS
+User's Manual
--------------------------------------------------------------------------------------------
-If the returned length does not match the length passed
-in, errno will be set to one of the following constants
-with the meaning listed below.
-
-
-EINVAL
-
- The message, or an internal portion of the message, was
- corrupted or the pointer was invalid.
-
-
-EOVERFLOW
-
- The length passed in was larger than the maximum length of
- the field; only a portion of the source bytes were copied.
+Overview
+============================================================================================
+The RIC Message Router (RMR) is a library for peer-to-peer communication.
+Applications use the library to send and receive messages where the message
+routing and endpoint selection is based on the message type rather than DNS host
+name-IP port combinations. The library provides the following major features:
-EXAMPLE
---------------------------------------------------------------------------------------------
++ Routing and endpoint selection is based on *message type.*
-SEE ALSO
---------------------------------------------------------------------------------------------
++ Application is insulated from the underlying transport mechanism and/or protocols.
-rmr_alloc_msg(3), rmr_bytes2meid(3), rmr_call(3),
-rmr_free_msg(3), rmr_get_meid(3), rmr_get_rcvfd(3),
-rmr_get_xact(3), rmr_payload_size(3), rmr_send_msg(3),
-rmr_rcv_msg(3), rmr_rcv_specific(3), rmr_rts_msg(3),
-rmr_ready(3), rmr_fib(3), rmr_has_str(3), rmr_tokenise(3),
-rmr_mk_ring(3), rmr_ring_free(3), rmr_str2meid(3),
-rmr_wh_open(3), rmr_wh_send_msg(3)
++ Message distribution (round robin or fanout) is selectable by message type.
++ Route management updates are received and processed asynchronously and without overt application involvement.
-NAME
---------------------------------------------------------------------------------------------
-rmr_call
-SYNOPSIS
+Purpose
--------------------------------------------------------------------------------------------
+RMR's main purpose is to provide an application with the
+ability to send and receive messages to/from other peer
+applications with minimal effort on the application's part.
+To achieve this, RMR manages all endpoint information,
+connections, and routing information necessary to establish
+and maintain communication. From the application's point of
+view, all that is required to send a message is to allocate
+(via RMR) a message buffer, add the payload data, and set the
+message type. To receive a message, the application needs
+only to invoke the receive function; when a message arrives a
+message buffer will be returned as the function result.
-::
-
- #include <rmr/rmr.h>
- extern rmr_mbuf_t* rmr_call( void* vctx, rmr_mbuf_t* msg );
-
-
-
-DESCRIPTION
+Message Routing
--------------------------------------------------------------------------------------------
-The rmr_call function sends the user application message to a
-remote endpoint, and waits for a corresponding response
-message before returning control to the user application. The
-user application supplies a completed message buffer, as it
-would for a rmr_send call, but unlike with the send, the
-buffer returned will have the response from the application
-that received the message.
-
-Messages which are received while waiting for the response
-are queued internally by RMR, and are returned to the user
-application when rmr_rcv_msg is invoked. These messages are
-returned in the order received, one per call to rmr_rcv_msg.
-
-Call Timeout
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+Applications are required to place a message type into a
+message before sending, and may optionally add a subscription
+ID when appropriate. The combination of message type, and
+subscription ID are refered to as the *message key,* and is
+used to match an entry in a routing table which provides the
+possible endpoints expecting to receive messages with the
+matching key.
-The rmr_call function implements a timeout failsafe to
-prevent, in most cases, the function from blocking forever.
-The timeout period is **not** based on time (calls to clock
-are deemed too expensive for a low latency system level
-library), but instead the period is based on the number of
-received messages which are not the response. Using a
-non-time mechanism for *timeout* prevents the async queue
-from filling (which would lead to message drops) in an
-environment where there is heavy message traffic.
-
-When the threshold number of messages have been queued
-without receiving a response message, control is returned to
-the user application and a nil pointer is returned to
-indicate that no message was received to process. Currently
-the threshold is fixed at 20 messages, though in future
-versions of the library this might be extended to be a
-parameter which the user application may set.
-
-Retries
+Round Robin Delivery
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-The send operations in RMR will retry *soft* send failures
-until one of three conditions occurs:
-
-
-
-1.
-
- The message is sent without error
-
-
-2.
-
- The underlying transport reports a *hard* failure
-
-
-3.
-
- The maximum number of retry loops has been attempted
-
-
-A retry loop consists of approximately 1000 send attempts
-**without** any intervening calls to *sleep()* or *usleep().*
-The number of retry loops defaults to 1, thus a maximum of
-1000 send attempts is performed before returning to the user
-application. This value can be set at any point after RMR
-initialisation using the *rmr_set_stimeout()* function
-allowing the user application to completely disable retires
-(set to 0), or to increase the number of retry loops.
-
-Transport Level Blocking
+An endpoint from RMR's perspective is an application to which
+RMR may establish a connection, and expect to send messages
+with one or more defined message keys. Each entry in the
+route table consists of one or more endpoint groups, called
+round robin groups. When a message matches a specific entry,
+the entry's groups are used to select the destination of the
+message. A message is sent once to each group, with messages
+being *balanced* across the endpoints of a group via round
+robin selection. Care should be taken when defining multiple
+groups for a message type as there is extra overhead required
+and thus the overall message latency is somewhat increased.
+
+Routing Table Updates
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-The underlying transport mechanism used to send messages is
-configured in *non-blocking* mode. This means that if a
-message cannot be sent immediately the transport mechanism
-will **not** pause with the assumption that the inability to
-send will clear quickly (within a few milliseconds). This
-means that when the retry loop is completely disabled (set to
-0), that the failure to accept a message for sending by the
-underlying mechanisms (software or hardware) will be reported
-immediately to the user application.
-
-It should be noted that depending on the underlying transport
-mechanism being used, it is extremely likely that retry
-conditions will happen during normal operations. These are
-completely out of RMR's control, and there is nothing that
-RMR can do to avoid or mitigate these other than by allowing
-RMR to retry the send operation, and even then it is possible
-(e.g., during connection reattempts), that a single retry
-loop is not enough to guarantee a successful send.
-
-RETURN VALUE
---------------------------------------------------------------------------------------------
-
-The rmr_call function returns a pointer to a message buffer
-with the state set to reflect the overall state of call
-processing (see Errors below). In some cases a nil pointer
-will be returned; when this is the case only *errno* will be
-available to describe the reason for failure.
-
-ERRORS
---------------------------------------------------------------------------------------------
-
-These values are reflected in the state field of the returned
-message.
-
-
-
-RMR_OK
-
- The call was successful and the message buffer references
- the response message.
-
-
-RMR_ERR_CALLFAILED
-
- The call failed and the value of *errno,* as described
- below, should be checked for the specific reason.
-
-
-The global "variable" *errno* will be set to one of the
-following values if the overall call processing was not
-successful.
-
-
-
-ETIMEDOUT
-
- Too many messages were queued before receiving the
- expected response
-
-
-ENOBUFS
-
- The queued message ring is full, messages were dropped
-
-
-EINVAL
-
- A parameter was not valid
-
-
-EAGAIN
-
- The underlying message system was interrupted or the
- device was busy; the message was **not** sent, and the
- user application should call this function with the
- message again.
-
-
-EXAMPLE
---------------------------------------------------------------------------------------------
-
-The following code snippet shows one way of using the
-rmr_call function, and illustrates how the transaction ID
-must be set.
-
-
-::
-
- int retries_left = 5; // max retries on dev not available
- int retry_delay = 50000; // retry delay (usec)
- static rmr_mbuf_t* mbuf = NULL; // response msg
- msg_t* pm; // application struct for payload
- // get a send buffer and reference the payload
- mbuf = rmr_alloc_msg( mr, sizeof( pm->req ) );
- pm = (msg_t*) mbuf->payload;
- // generate an xaction ID and fill in payload with data and msg type
- snprintf( mbuf->xaction, RMR_MAX_XID, "%s", gen_xaction() );
- snprintf( pm->req, sizeof( pm->req ), "{ \\"req\\": \\"num users\\"}" );
- mbuf->mtype = MT_REQ;
- msg = rmr_call( mr, msg );
- if( ! msg ) { // probably a timeout and no msg received
- return NULL; // let errno trickle up
- }
- if( mbuf->state != RMR_OK ) {
- while( retries_left-- > 0 && // loop as long as eagain
- errno == EAGAIN &&
- (msg = rmr_call( mr, msg )) != NULL &&
- mbuf->state != RMR_OK ) {
- usleep( retry_delay );
- }
- if( mbuf == NULL || mbuf->state != RMR_OK ) {
- rmr_free_msg( mbuf ); // safe if nil
- return NULL;
- }
- }
- // do something with mbuf
-
-
-
-SEE ALSO
---------------------------------------------------------------------------------------------
-
-rmr_alloc_msg(3), rmr_free_msg(3), rmr_init(3),
-rmr_payload_size(3), rmr_send_msg(3), rmr_rcv_msg(3),
-rmr_rcv_specific(3), rmr_rts_msg(3), rmr_ready(3),
-rmr_fib(3), rmr_has_str(3), rmr_set_stimeout(3),
-rmr_tokenise(3), rmr_mk_ring(3), rmr_ring_free(3)
-
-
-NAME
---------------------------------------------------------------------------------------------
-
-rmr_close
-
-SYNOPSIS
---------------------------------------------------------------------------------------------
-
-
-::
-
- #include <rmr/rmr.h>
- void rmr_close( void* vctx )
-
-
-
-DESCRIPTION
---------------------------------------------------------------------------------------------
-
-The rmr_close function closes the listen socket effectively
-cutting the application off. The route table listener is also
-stopped. Calls to rmr_rcv_msg() will fail with unpredictable
-error codes, and calls to rmr_send_msg(), rmr_call(), and
-rmr_rts_msg() will have unknown results.
-
-
-SEE ALSO
---------------------------------------------------------------------------------------------
-
-rmr_alloc_msg(3), rmr_call(3), rmr_free_msg(3),
-rmr_get_rcvfd(3), rmr_payload_size(3), rmr_send_msg(3),
-rmr_rcv_msg(3), rmr_rcv_specific(3), rmr_rts_msg(3),
-rmr_ready(3), rmr_fib(3), rmr_has_str(3), rmr_tokenise(3),
-rmr_mk_ring(3), rmr_ring_free(3), rmr_wh_open(3),
-rmr_wh_send_msg(3)
-
-
-NAME
---------------------------------------------------------------------------------------------
-
-rmr_free_msg
-
-SYNOPSIS
---------------------------------------------------------------------------------------------
-
-
-::
-
- #include <rmr/rmr.h>
- void rmr_free_msg( rmr_mbuf_t* mbuf );
-
-
-
-DESCRIPTION
---------------------------------------------------------------------------------------------
-
-The message buffer is returned to the pool, or the associated
-memory is released depending on the needs of the underlying
-messaging system. This allows the user application to release
-a buffer that is not going to be used. It is safe to pass a
-nil pointer to this function, and doing so does not result in
-a change to the value of errrno.
-
-After calling, the user application should **not** use any of
-the pointers (transaction ID, or payload) which were
-available.
-
-SEE ALSO
---------------------------------------------------------------------------------------------
-
-rmr_alloc_msg(3), rmr_call(3), rmr_init(3),
-rmr_payload_size(3), rmr_send_msg(3), rmr_rcv_msg(3),
-rmr_rcv_specific(3), rmr_rts_msg(3), rmr_ready(3),
-rmr_fib(3), rmr_has_str(3), rmr_tokenise(3), rmr_mk_ring(3),
-rmr_ring_free(3)
-
-
-NAME
---------------------------------------------------------------------------------------------
-
-rmr_get_const
-
-SYNOPSIS
---------------------------------------------------------------------------------------------
-
-
-::
-
- #include <rmr/rmr.h>
- unsigned char* rmr_get_const();
-
-
-
-DESCRIPTION
---------------------------------------------------------------------------------------------
-
-The rmr_get_const function is a convenience function for
-wrappers which do not have the ability to "compile in" RMR
-constants. The function will build a nil terminated string
-containing JSON which defines the RMR constants that C and Go
-applications have at compile time via the rmr.h header file.
-
-All values are represented as strings and the JSON format is
-illustrated in the following (partial) example:
-
-
-::
-
- {
- "RMR_MAX_XID": "32",
- "RMR_OK": "0",
- "RMR_ERR_BADARG", "1",
- "RMR_ERR_NOENDPT" "2"
- }
-
-
-
-RETURN VALUE
---------------------------------------------------------------------------------------------
-
-On success, a pointer to a string containing the JSON
-defining constant and value pairs. On failure a nil pointer
-is returned.
-
-SEE ALSO
---------------------------------------------------------------------------------------------
-
-rmr(7)
-
-
-NAME
---------------------------------------------------------------------------------------------
-
-rmr_get_meid
-
-SYNOPSIS
---------------------------------------------------------------------------------------------
-
-
-::
-
- #include <rmr/rmr.h>
- char* rmr_get_meid( rmr_mbuf_t* mbuf, unsigned char* dest )
-
-
-
-DESCRIPTION
---------------------------------------------------------------------------------------------
-
-The rmr_get_meid function will copy the managed entity ID
-(meid) field from the message into the *dest* buffer provided
-by the user. The buffer referenced by *dest* is assumed to be
-at least RMR_MAX_MEID bytes in length. If *dest* is NULL,
-then a buffer is allocated (the calling application is
-expected to free when the buffer is no longer needed).
-
-RETURN VALUE
---------------------------------------------------------------------------------------------
-
-On success, a pointer to the extracted string is returned. If
-*dest* was supplied, then this is just a pointer to the
-caller's buffer. If *dest* was NULL, this is a pointer to the
-allocated buffer. If an error occurs, a nil pointer is
-returned and errno is set as described below.
-
-ERRORS
---------------------------------------------------------------------------------------------
-
-If an error occurs, the value of the global variable errno
-will be set to one of the following with the indicated
-meaning.
-
-
-
-EINVAL
-
- The message, or an internal portion of the message, was
- corrupted or the pointer was invalid.
-
-
-ENOMEM
-
- A nil pointer was passed for *dest,* however it was not
- possible to allocate a buffer using malloc().
-
-
-SEE ALSO
---------------------------------------------------------------------------------------------
-
-rmr_alloc_msg(3), rmr_bytes2xact(3), rmr_bytes2meid(3),
-rmr_call(3), rmr_free_msg(3), rmr_get_rcvfd(3),
-rmr_get_xact(3), rmr_payload_size(3), rmr_send_msg(3),
-rmr_rcv_msg(3), rmr_rcv_specific(3), rmr_rts_msg(3),
-rmr_ready(3), rmr_fib(3), rmr_has_str(3), rmr_tokenise(3),
-rmr_mk_ring(3), rmr_ring_free(3), rmr_str2meid(3),
-rmr_str2xact(3), rmr_wh_open(3), rmr_wh_send_msg(3)
-
-
-NAME
---------------------------------------------------------------------------------------------
-
-rmr_get_rcvfd
-
-SYNOPSIS
---------------------------------------------------------------------------------------------
-
-
-::
-
- #include <rmr/rmr.h>
- void* rmr_get_rcvfd( void* ctx )
-
-
-
-DESCRIPTION
---------------------------------------------------------------------------------------------
-
-The rmr_get_rcvfd function returns a file descriptor which
-may be given to epoll_wait() by an application that wishes to
-use event poll in a single thread rather than block on the
-arrival of a message via calls to rmr_rcv_msg(). When
-epoll_wait() indicates that this file descriptor is ready, a
-call to rmr_rcv_msg() will not block as at least one message
-has been received.
-
-The context (ctx) pointer passed in is the pointer returned
-by the call to rmr_init().
-
-RETURN VALUE
---------------------------------------------------------------------------------------------
-
-The rmr_get_rcvfd function returns a file descriptor greater
-or equal to 0 on success and -1 on error.
-
-ERRORS
---------------------------------------------------------------------------------------------
-
-The following error values are specifically set by this RMR
-function. In some cases the error message of a system call is
-propagated up, and thus this list might be incomplete.
-
-
-EINVAL
-
- The use of this function is invalid in this environment.
-
-
-EXAMPLE
---------------------------------------------------------------------------------------------
-
-The following short code bit illustrates the use of this
-function. Error checking has been omitted for clarity.
-
-
-::
-
- #include <stdio.h>
- #include <stdlib.h>
- #include <sys/epoll.h>
- #include <rmr/rmr.h>
- int main() {
- int rcv_fd; // pollable fd
- void* mrc; //msg router context
- struct epoll_event events[10]; // support 10 events to poll
- struct epoll_event epe; // event definition for event to listen to
- int ep_fd = -1;
- rmr_mbuf_t* msg = NULL;
- int nready;
- int i;
- int norm_msg_size = 1500; // 95% messages are less than this
- mrc = rmr_init( "43086", norm_msg_size, RMRFL_NONE );
- rcv_fd = rmr_get_rcvfd( mrc );
- ep_fd = epoll_create1( 0 ); // initialise epoll environment
- epe.events = EPOLLIN;
- epe.data.fd = rcv_fd;
- epoll_ctl( ep_fd, EPOLL_CTL_ADD, rcv_fd, &epe ); // add our info to the mix
- while( 1 ) {
- nready = epoll_wait( ep_fd, events, 10, -1 ); // -1 == block forever (no timeout)
- for( i = 0; i < nready && i < 10; i++ ) { // loop through to find what is ready
- if( events[i].data.fd == rcv_fd ) { // RMR has something
- msg = rmr_rcv_msg( mrc, msg );
- if( msg ) {
- // do something with msg
- }
- }
- // check for other ready fds....
- }
- }
- }
-
-
-
-SEE ALSO
---------------------------------------------------------------------------------------------
-
-rmr_alloc_msg(3), rmr_call(3), rmr_free_msg(3),
-rmr_payload_size(3), rmr_send_msg(3), rmr_rcv_msg(3),
-rmr_rcv_specific(3), rmr_rts_msg(3), rmr_ready(3),
-rmr_fib(3), rmr_has_str(3), rmr_tokenise(3), rmr_mk_ring(3),
-rmr_ring_free(3)
-
-
-NAME
---------------------------------------------------------------------------------------------
-
-rmr_get_src
-
-SYNOPSIS
---------------------------------------------------------------------------------------------
-
-
-::
-
- #include <rmr/rmr.h>
- unsigned char* rmr_get_src( rmr_mbuf_t* mbuf, unsigned char* dest )
-
-
-
-DESCRIPTION
---------------------------------------------------------------------------------------------
-
-The rmr_get_src function will copy the *source* information
-from the message to a buffer (dest) supplied by the user. In
-an RMR message, the source is the sender's information that
-is used for return to sender function calls, and is generally
-the hostname and port in the form *name*. The source might be
-an IP address port combination; the data is populated by the
-sending process and the only requirement is that it be
-capable of being used to start a TCP session with the sender.
-
-The maximum size allowed by RMR is 64 bytes (including the
-nil string terminator), so the user must ensure that the
-destination buffer given is at least 64 bytes.
-
-RETURN VALUE
---------------------------------------------------------------------------------------------
-
-On success, a pointer to the destination buffer is given as a
-convenience to the user programme. On failure, a nil pointer
-is returned and the value of errno is set.
-
-ERRORS
---------------------------------------------------------------------------------------------
-
-If an error occurs, the value of the global variable errno
-will be set to one of the following with the indicated
-meaning.
-
-
-
-EINVAL
-
- The message, or an internal portion of the message, was
- corrupted or the pointer was invalid.
-
-
-SEE ALSO
---------------------------------------------------------------------------------------------
-
-rmr_alloc_msg(3), rmr_bytes2xact(3), rmr_bytes2meid(3),
-rmr_call(3), rmr_free_msg(3), rmr_get_rcvfd(3),
-rmr_get_srcip(3), rmr_payload_size(3), rmr_send_msg(3),
-rmr_rcv_msg(3), rmr_rcv_specific(3), rmr_rts_msg(3),
-rmr_ready(3), rmr_fib(3), rmr_has_str(3), rmr_tokenise(3),
-rmr_mk_ring(3), rmr_ring_free(3), rmr_str2meid(3),
-rmr_str2xact(3), rmr_wh_open(3), rmr_wh_send_msg(3)
-
-
-NAME
---------------------------------------------------------------------------------------------
-
-rmr_get_srcip
-
-SYNOPSIS
---------------------------------------------------------------------------------------------
-
-
-::
-
- #include <rmr/rmr.h>
- unsigned char* rmr_get_srcip( rmr_mbuf_t* mbuf, unsigned char* dest )
-
-
-
-DESCRIPTION
---------------------------------------------------------------------------------------------
-
-The rmr_get_srcip function will copy the *source IP address*
-from the message to a buffer (dest) supplied by the user. In
-an RMR message, the source IP address is the sender's
-information that is used for return to sender function calls;
-this function makes it available to the user application. The
-address is maintained as IP:port where *IP* could be either
-an IPv6 or IPv4 address depending on what was provided by the
-sending application.
-
-The maximum size allowed by RMR is 64 bytes (including the
-nil string terminator), so the user must ensure that the
-destination buffer given is at least 64 bytes. The user
-application should use the RMR constant RMR_MAX_SRC to ensure
-that the buffer supplied is large enough, and to protect
-against future RMR enhancements which might increase the
-address buffer size requirement.
-
-RETURN VALUE
---------------------------------------------------------------------------------------------
-
-On success, a pointer to the destination buffer is given as a
-convenience to the user programme. On failure, a nil pointer
-is returned and the value of errno is set.
-
-ERRORS
---------------------------------------------------------------------------------------------
-
-If an error occurs, the value of the global variable errno
-will be set to one of the following with the indicated
-meaning.
-
-
-
-EINVAL
-
- The message, or an internal portion of the message, was
- corrupted or the pointer was invalid.
-
-
-SEE ALSO
---------------------------------------------------------------------------------------------
-
-rmr_alloc_msg(3), rmr_bytes2xact(3), rmr_bytes2meid(3),
-rmr_call(3), rmr_free_msg(3), rmr_get_rcvfd(3),
-rmr_get_src(3), rmr_payload_size(3), rmr_send_msg(3),
-rmr_rcv_msg(3), rmr_rcv_specific(3), rmr_rts_msg(3),
-rmr_ready(3), rmr_fib(3), rmr_has_str(3), rmr_tokenise(3),
-rmr_mk_ring(3), rmr_ring_free(3), rmr_str2meid(3),
-rmr_str2xact(3), rmr_wh_open(3), rmr_wh_send_msg(3)
-
-
-NAME
---------------------------------------------------------------------------------------------
-
-rmr_get_trace
-
-SYNOPSIS
---------------------------------------------------------------------------------------------
-
-
-::
-
- #include <rmr/rmr.h>
- int rmr_get_trace( rmr_mbuf_t* mbuf, unsigned char* dest, int size )
-
-
-
-DESCRIPTION
---------------------------------------------------------------------------------------------
-
-The rmr_get_trace function will copy the trace information
-from the message into the user's allocated memory referenced
-by dest. The size parameter is assumed to be the maximum
-number of bytes which can be copied (size of the destination
-buffer).
-
-RETURN VALUE
---------------------------------------------------------------------------------------------
-
-On success, the number of bytes actually copied is returned.
-If the return value is 0, no bytes copied, then the reason
-could be that the message pointer was nil, or the size
-parameter was <= 0.
-
-SEE ALSO
---------------------------------------------------------------------------------------------
-
-rmr_alloc_msg(3), rmr_tralloc_msg(3), rmr_bytes2xact(3),
-rmr_bytes2meid(3), rmr_call(3), rmr_free_msg(3),
-rmr_get_rcvfd(3), rmr_get_trlen(3), rmr_init(3),
-rmr_init_trace(3), rmr_payload_size(3), rmr_send_msg(3),
-rmr_rcv_msg(3), rmr_rcv_specific(3), rmr_rts_msg(3),
-rmr_ready(3), rmr_fib(3), rmr_has_str(3), rmr_tokenise(3),
-rmr_mk_ring(3), rmr_ring_free(3), rmr_str2meid(3),
-rmr_str2xact(3), rmr_wh_open(3), rmr_wh_send_msg(3),
-rmr_set_trace(3), rmr_trace_ref(3)
-
-
-NAME
---------------------------------------------------------------------------------------------
-
-rmr_get_trlen
-
-SYNOPSIS
---------------------------------------------------------------------------------------------
-
-
-::
-
- #include <rmr/rmr.h>
- int rmr_get_trlen( rmr_mbuf_t* msg );
-
-
-
-DESCRIPTION
---------------------------------------------------------------------------------------------
-
-Given a message buffer, this function returns the amount of
-space (bytes) that have been allocated for trace data. If no
-trace data has been allocated, then 0 is returned.
-
-RETURN VALUE
---------------------------------------------------------------------------------------------
-
-The number of bytes allocated for trace information in the
-given message.
-
-ERRORS
---------------------------------------------------------------------------------------------
-
-
-
-INVAL
-
- Parameter(s) passed to the function were not valid.
-
-
-SEE ALSO
---------------------------------------------------------------------------------------------
-
-rmr_alloc_msg(3), rmr_call(3), rmr_free_msg(3),
-rmr_get_trace(3), rmr_init(3), rmr_init_trace(3),
-rmr_send_msg(3), rmr_rcv_msg(3), rmr_rcv_specific(3),
-rmr_rts_msg(3), rmr_ready(3), rmr_fib(3), rmr_has_str(3),
-rmr_tokenise(3), rmr_mk_ring(3), rmr_ring_free(3),
-rmr_set_trace(3), rmr_tralloc_msg(3)
-
-
-NAME
---------------------------------------------------------------------------------------------
-
-rmr_get_xact
-
-SYNOPSIS
---------------------------------------------------------------------------------------------
-
-
-::
-
- #include <rmr/rmr.h>
- char* rmr_get_xact( rmr_mbuf_t* mbuf, unsigned char* dest )
-
-
-
-DESCRIPTION
---------------------------------------------------------------------------------------------
-
-The rmr_get_xact function will copy the transaction field
-from the message into the *dest* buffer provided by the user.
-The buffer referenced by *dest* is assumed to be at least
-RMR_MAX_XID bytes in length. If *dest* is NULL, then a buffer
-is allocated (the calling application is expected to free
-when the buffer is no longer needed).
-
-RETURN VALUE
---------------------------------------------------------------------------------------------
-
-On success, a pointer to the extracted string is returned. If
-*dest* was supplied, then this is just a pointer to the
-caller's buffer. If *dest* was NULL, this is a pointer to the
-allocated buffer. If an error occurs, a nil pointer is
-returned and errno is set as described below.
-
-ERRORS
---------------------------------------------------------------------------------------------
-
-If an error occurs, the value of the global variable errno
-will be set to one of the following with the indicated
-meaning.
-
-
-
-EINVAL
-
- The message, or an internal portion of the message, was
- corrupted or the pointer was invalid.
-
-
-ENOMEM
-
- A nil pointer was passed for *dest,* however it was not
- possible to allocate a buffer using malloc().
-
-
-SEE ALSO
---------------------------------------------------------------------------------------------
-
-rmr_alloc_msg(3), rmr_bytes2xact(3), rmr_bytes2meid(3),
-rmr_call(3), rmr_free_msg(3), rmr_get_rcvfd(3),
-rmr_get_meid(3), rmr_payload_size(3), rmr_send_msg(3),
-rmr_rcv_msg(3), rmr_rcv_specific(3), rmr_rts_msg(3),
-rmr_ready(3), rmr_fib(3), rmr_has_str(3), rmr_tokenise(3),
-rmr_mk_ring(3), rmr_ring_free(3), rmr_str2meid(3),
-rmr_str2xact(3), rmr_wh_open(3), rmr_wh_send_msg(3)
-
-
-NAME
---------------------------------------------------------------------------------------------
-
-rmr_init
-
-SYNOPSIS
---------------------------------------------------------------------------------------------
-
-
-::
-
- #include <rmr/rmr.h>
- void* rmr_init( char* proto_port, int norm_msg_size, int flags );
-
-
-
-DESCRIPTION
---------------------------------------------------------------------------------------------
-
-The rmr_init function prepares the environment for sending
-and receiving messages. It does so by establishing a worker
-thread (pthread) which subscribes to a route table generator
-which provides the necessary routing information for the RMR
-library to send messages.
-
-*Port* is used to listen for connection requests from other
-RMR based applications. The *norm_msg_size* parameter is used
-to allocate receive buffers and should be set to what the
-user application expects to be a size which will hold the
-vast majority of expected messages. When computing the size,
-the application should consider the usual payload size
-**and** the maximum trace data size that will be used. This
-value is also used as the default message size when
-allocating message buffers (when a zero size is given to
-rmr_alloc_msg(); see the rmr_alloc_msg() manual page).
-Messages arriving which are longer than the given normal size
-will cause RMR to allocate a new buffer which is large enough
-for the arriving message.
-
-Starting with version 3.8.0 RMR no longer places a maximum
-buffer size for received messages. The underlying system
-memory manager might impose such a limit and the attempt to
-allocate a buffer larger than that limit will likely result
-in an application abort. Other than the potential performance
-impact from extra memory allocation and release, there is no
-penality to the user programme for specifyning a normal
-buffer size which is usually smaller than received buffers.
-Similarly, the only penality to the application for over
-specifying the normal buffer size might be a larger memory
-footprint.
-
-*Flags* allows for selection of some RMR options at the time
-of initialisation. These are set by ORing RMRFL constants
-from the RMR header file. Currently the following flags are
-supported:
-
-
-
-RMRFL_NONE
-
- No flags are set.
-
-
-RMRFL_NOTHREAD
-
- The route table collector thread is not to be started.
- This should only be used by the route table generator
- application if it is based on RMR.
-
-
-RMRFL_MTCALL
-
- Enable multi-threaded call support.
-
-
-RMRFL_NOLOCK
-
- Some underlying transport providers (e.g. SI95) enable
- locking to be turned off if the user application is single
- threaded, or otherwise can guarantee that RMR functions
- will not be invoked concurrently from different threads.
- Turning off locking can help make message receipt more
- efficient. If this flag is set when the underlying
- transport does not support disabling locks, it will be
- ignored.
-
-
-Multi-threaded Calling
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-The support for an application to issue a *blocking call* by
-the rmr_call() function was limited such that only user
-applications which were operating in a single thread could
-safely use the function. Further, timeouts were message count
-based and not time unit based. Multi-threaded call support
-adds the ability for a user application with multiple threads
-to invoke a blocking call function with the guarantee that
-the correct response message is delivered to the thread. The
-additional support is implemented with the *rmr_mt_call()*
-and *rmr_mt_rcv()* function calls.
-
-Multi-threaded call support requires the user application to
-specifically enable it when RMR is initialised. This is
-necessary because a second, dedicated, receiver thread must
-be started, and requires all messages to be examined and
-queued by this thread. The additional overhead is minimal,
-queuing information is all in the RMR message header, but as
-an additional process is necessary the user application must
-"opt in" to this approach.
-
-
-ENVIRONMENT
---------------------------------------------------------------------------------------------
-
-As a part of the initialisation process rmr_init reads
-environment variables to configure itself. The following
-variables are used if found.
-
-
-
-RMR_ASYNC_CONN
-
- Allows the async connection mode to be turned off (by
- setting the value to 0). When set to 1, or missing from
- the environment, RMR will invoke the connection interface
- in the transport mechanism using the non-blocking (async)
- mode. This will likely result in many "soft failures"
- (retry) until the connection is established, but allows
- the application to continue unimpeded should the
- connection be slow to set up.
-
-
-RMR_BIND_IF
-
- This provides the interface that RMR will bind listen
- ports to, allowing for a single interface to be used
- rather than listening across all interfaces. This should
- be the IP address assigned to the interface that RMR
- should listen on, and if not defined RMR will listen on
- all interfaces.
-
-
-RMR_CTL_PORT
-
- This variable defines the port that RMR should open for
- communications with Route Manager, and other RMR control
- applications. If not defined, the port 4561 is assumed.
-
- Previously, the RMR_RTG_SVC (route table generator service
- port) was used to define this port. However, a future
- version of Route Manager will require RMR to connect and
- request tables, thus that variable is now used to supply
- the Route Manager's well-known address and port.
-
- To maintain backwards compatibility with the older Route
- Manager versions, the presence of this variable in the
- environment will shift RMR's behaviour with respect to the
- default value used when RMR_RTG_SVC is **not** defined.
-
- When RMR_CTL_PORT is **defined:** RMR assumes that Route
- Manager requires RMR to connect and request table updates
- is made, and the default well-known address for Route
- manager is used (routemgr:4561).
-
- When RMR_CTL_PORT is **undefined:** RMR assumes that Route
- Manager will connect and push table updates, thus the
- default listen port (4561) is used.
-
- To avoid any possible misinterpretation and/or incorrect
- assumptions on the part of RMR, it is recommended that
- both the RMR_CTL_PORT and RMR_RTG_SVC be defined. In the
- case where both variables are defined, RMR will behave
- exactly as is communicated with the variable's values.
-
-
-RMR_RTG_SVC
-
- The value of this variable depends on the Route Manager in
- use.
-
- When the Route Manager is expecting to connect to an xAPP
- and push route tables, this variable must indicate the
- port which RMR should use to listen for these connections.
-
- When the Route Manager is expecting RMR to connect and
- request a table update during initialisation, the variable
- should be the host of the Route Manager process.
-
- The RMR_CTL_PORT variable (added with the support of
- sending table update requests to Route manager), controls
- the behaviour if this variable is not set. See the
- description of that variable for details.
-
-
-RMR_HR_LOG
-
- By default RMR writes messages to standard error
- (incorrectly referred to as log messages) in human
- readable format. If this environment variable is set to 0,
- the format of standard error messages might be written in
- some format not easily read by humans. If missing, a value
- of 1 is assumed.
-
-
-RMR_LOG_VLEVEL
-
- This is a numeric value which corresponds to the verbosity
- level used to limit messages written to standard error.
- The lower the number the less chatty RMR functions are
- during execution. The following is the current
- relationship between the value set on this variable and
- the messages written:
-
-
-0
-
- Off; no messages of any sort are written.
-
-
-1
-
- Only critical messages are written (default if this
- variable does not exist)
-
-
-2
-
- Errors and all messages written with a lower value.
-
-
-3
-
- Warnings and all messages written with a lower value.
-
-
-4
-
- Informational and all messages written with a lower
- value.
-
-
-5
-
- Debugging mode -- all messages written, however this
- requires RMR to have been compiled with debugging
- support enabled.
-
-
-
-RMR_RTG_ISRAW
-
- **Deprecated.** Should be set to 1 if the route table
- generator is sending "plain" messages (not using RMR to
- send messages), 0 if the RTG is using RMR to send. The
- default is 1 as we don't expect the RTG to use RMR.
-
- This variable is only recognised when using the NNG
- transport library as it is not possible to support NNG
- "raw" communications with other transport libraries. It is
- also necessary to match the value of this variable with
- the capabilities of the Route Manager; at some point in
- the future RMR will assume that all Route Manager messages
- will arrive via an RMR connection and will ignore this
- variable.
-
-RMR_SEED_RT
-
- This is used to supply a static route table which can be
- used for debugging, testing, or if no route table
- generator process is being used to supply the route table.
- If not defined, no static table is used and RMR will not
- report *ready* until a table is received. The static route
- table may contain both the route table (between newrt
- start and end records), and the MEID map (between meid_map
- start and end records).
-
-RMR_SRC_ID
-
- This is either the name or IP address which is placed into
- outbound messages as the message source. This will used
- when an RMR based application uses the rmr_rts_msg()
- function to return a response to the sender. If not
- supplied RMR will use the hostname which in some container
- environments might not be routable.
-
- The value of this variable is also used for Route Manager
- messages which are sent via an RMR connection.
-
-RMR_VCTL_FILE
-
- This supplies the name of a verbosity control file. The
- core RMR functions do not produce messages unless there is
- a critical failure. However, the route table collection
- thread, not a part of the main message processing
- component, can write additional messages to standard
- error. If this variable is set, RMR will extract the
- verbosity level for these messages (0 is silent) from the
- first line of the file. Changes to the file are detected
- and thus the level can be changed dynamically, however RMR
- will only suss out this variable during initialisation, so
- it is impossible to enable verbosity after startup.
-
-RMR_WARNINGS
-
- If set to 1, RMR will write some warnings which are
- non-performance impacting. If the variable is not defined,
- or set to 0, RMR will not write these additional warnings.
-
-
-RETURN VALUE
---------------------------------------------------------------------------------------------
-
-The rmr_init function returns a void pointer (a contex if you
-will) that is passed as the first parameter to nearly all
-other RMR functions. If rmr_init is unable to properly
-initialise the environment, NULL is returned and errno is set
-to an appropriate value.
-
-ERRORS
---------------------------------------------------------------------------------------------
-
-The following error values are specifically set by this RMR
-function. In some cases the error message of a system call is
-propagated up, and thus this list might be incomplete.
-
-
-ENOMEM
-
- Unable to allocate memory.
-
-
-EXAMPLE
---------------------------------------------------------------------------------------------
-
-
-::
-
- void* uh;
- rmr_mbuf* buf = NULL;
- uh = rmr_init( "43086", 4096, 0 );
- buf = rmr_rcv_msg( uh, buf );
-
-
-
-SEE ALSO
---------------------------------------------------------------------------------------------
-
-rmr_alloc_msg(3), rmr_call(3), rmr_free_msg(3),
-rmr_get_rcvfd(3), rmr_mt_call(3), rmr_mt_rcv(3),
-rmr_payload_size(3), rmr_send_msg(3), rmr_rcv_msg(3),
-rmr_rcv_specific(3), rmr_rts_msg(3), rmr_ready(3),
-rmr_fib(3), rmr_has_str(3), rmr_tokenise(3), rmr_mk_ring(3),
-rmr_ring_free(3)
-
-
-NAME
---------------------------------------------------------------------------------------------
-
-rmr_init_trace
-
-SYNOPSIS
---------------------------------------------------------------------------------------------
-
-
-::
-
- #include <rmr/rmr.h>
- void* rmr_init_trace( void* ctx )
-
-
-
-DESCRIPTION
---------------------------------------------------------------------------------------------
-
-The rmr_init_trace function establishes the default trace
-space placed in each message buffer allocated with
-rmr_alloc_msg(). If this function is never called, then no
-trace space is allocated by default into any message buffer.
-
-Trace space allows the user application to pass some trace
-token, or other data with the message, but outside of the
-payload. Trace data may be added to any message with
-rmr_set_trace(), and may be extracted from a message with
-rmr_get_trace(). The number of bytes that a message contains
-for/with trace data can be determined by invoking
-rmr_get_trlen().
-
-This function may be safely called at any time during the
-life of the user programme to (re)set the default trace space
-reserved. If the user programme needs to allocate a message
-with trace space of a different size than is allocated by
-default, without fear of extra overhead of reallocating a
-message later, the rmr_tralloc_msg() function can be used.
-
-RETURN VALUE
---------------------------------------------------------------------------------------------
-
-A value of 1 is returned on success, and 0 on failure. A
-failure indicates that the RMR context (a void pointer passed
-to this function was not valid.
-
-SEE ALSO
---------------------------------------------------------------------------------------------
-
-rmr_alloc_msg(3), rmr_tr_alloc_msg(3), rmr_call(3),
-rmr_free_msg(3), rmr_get_rcvfd(3), rmr_get_trace(3),
-rmr_get_trlen(3), rmr_payload_size(3), rmr_send_msg(3),
-rmr_rcv_msg(3), rmr_rcv_specific(3), rmr_rts_msg(3),
-rmr_ready(3), rmr_fib(3), rmr_has_str(3), rmr_tokenise(3),
-rmr_mk_ring(3), rmr_ring_free(3), rmr_set_trace(3)
-
-
-NAME
---------------------------------------------------------------------------------------------
-
-rmr_mt_call
-
-SYNOPSIS
---------------------------------------------------------------------------------------------
-
-
-::
-
- #include <rmr/rmr.h>
- extern rmr_mbuf_t* rmr_mt_call( void* vctx, rmr_mbuf_t* msg, int id, int timeout );
-
-
-
-DESCRIPTION
---------------------------------------------------------------------------------------------
-
-The rmr_mt_call function sends the user application message
-to a remote endpoint, and waits for a corresponding response
-message before returning control to the user application. The
-user application supplies a completed message buffer, as it
-would for a rmr_send_msg call, but unlike with a send, the
-buffer returned will have the response from the application
-that received the message. The thread invoking the
-*rmr_mt_call()* will block until a message arrives or until
-*timeout* milliseconds has passed; which ever comes first.
-Using a timeout value of zero (0) will cause the thread to
-block without a timeout.
-
-The *id* supplied as the third parameter is an integer in the
-range of 2 through 255 inclusive. This is a caller defined
-"thread number" and is used to match the response message
-with the correct user application thread. If the ID value is
-not in the proper range, the attempt to make the call will
-fail.
-
-Messages which are received while waiting for the response
-are queued on a *normal* receive queue and will be delivered
-to the user application with the next invocation of
-*rmr_mt_rcv()* or *rmr_rvv_msg().* by RMR, and are returned
-to the user application when rmr_rcv_msg is invoked. These
-messages are returned in the order received, one per call to
-rmr_rcv_msg.
-
-The Transaction ID
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-The user application is responsible for setting the value of
-the transaction ID field before invoking *rmr_mt_call.* The
-transaction ID is a RMR_MAX_XID byte field that is used to
-match the response message when it arrives. RMR will compare
-**all** of the bytes in the field, so the caller must ensure
-that they are set correctly to avoid missing the response
-message. The application which returns the response message
-is also expected to ensure that the return buffer has the
-matching transaction ID. This can be done transparently if
-the application uses the *rmr_rts_msg()* function and does
-not adjust the transaction ID.
-
-Retries
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-The send operations in RMR will retry *soft* send failures
-until one of three conditions occurs:
-
-
-
-1.
-
- The message is sent without error
-
-
-2.
-
- The underlying transport reports a *hard* failure
-
-
-3.
-
- The maximum number of retry loops has been attempted
-
-
-A retry loop consists of approximately 1000 send attempts
-**without** any intervening calls to *sleep()* or *usleep().*
-The number of retry loops defaults to 1, thus a maximum of
-1000 send attempts is performed before returning to the user
-application. This value can be set at any point after RMR
-initialisation using the *rmr_set_stimeout()* function
-allowing the user application to completely disable retires
-(set to 0), or to increase the number of retry loops.
-
-Transport Level Blocking
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-The underlying transport mechanism used to send messages is
-configured in *non-blocking* mode. This means that if a
-message cannot be sent immediately the transport mechanism
-will **not** pause with the assumption that the inability to
-send will clear quickly (within a few milliseconds). This
-means that when the retry loop is completely disabled (set to
-0), that the failure to accept a message for sending by the
-underlying mechanisms (software or hardware) will be reported
-immediately to the user application.
-
-It should be noted that depending on the underlying transport
-mechanism being used, it is extremely likely that retry
-conditions will happen during normal operations. These are
-completely out of RMR's control, and there is nothing that
-RMR can do to avoid or mitigate these other than by allowing
-RMR to retry the send operation, and even then it is possible
-(e.g., during connection reattempts), that a single retry
-loop is not enough to guarantee a successful send.
-
-RETURN VALUE
---------------------------------------------------------------------------------------------
-
-The rmr_mt_call function returns a pointer to a message
-buffer with the state set to reflect the overall state of
-call processing. If the state is RMR_OK then the buffer
-contains the response message; otherwise the state indicates
-the error encountered while attempting to send the message.
-
-If no response message is received when the timeout period
-has expired, a nil pointer will be returned (NULL).
-
-ERRORS
---------------------------------------------------------------------------------------------
-
-These values are reflected in the state field of the returned
-message.
-
-
-
-RMR_OK
-
- The call was successful and the message buffer references
- the response message.
-
-
-RMR_ERR_BADARG
-
- An argument passed to the function was invalid.
-
-
-RMR_ERR_CALLFAILED
-
- The call failed and the value of *errno,* as described
- below, should be checked for the specific reason.
-
-
-RMR_ERR_NOENDPT
-
- An endpoint associated with the message type could not be
- found in the route table.
-
-
-RMR_ERR_RETRY
-
- The underlying transport mechanism was unable to accept
- the message for sending. The user application can retry
- the call operation if appropriate to do so.
-
-
-The global "variable" *errno* will be set to one of the
-following values if the overall call processing was not
-successful.
-
-
-
-ETIMEDOUT
-
- Too many messages were queued before receiving the
- expected response
-
-
-ENOBUFS
-
- The queued message ring is full, messages were dropped
-
-
-EINVAL
-
- A parameter was not valid
-
-
-EAGAIN
-
- The underlying message system wsa interrupted or the
- device was busy; the message was **not** sent, and user
- application should call this function with the message
- again.
-
-
-EXAMPLE
---------------------------------------------------------------------------------------------
-
-The following code bit shows one way of using the rmr_mt_call
-function, and illustrates how the transaction ID must be set.
-
-
-::
-
- int retries_left = 5; // max retries on dev not available
- static rmr_mbuf_t* mbuf = NULL; // response msg
- msg_t* pm; // appl message struct (payload)
- // get a send buffer and reference the payload
- mbuf = rmr_alloc_msg( mr, sizeof( pm->req ) );
- pm = (msg_t*) mbuf->payload;
- // generate an xaction ID and fill in payload with data and msg type
- rmr_bytes2xact( mbuf, xid, RMR_MAX_XID );
- snprintf( pm->req, sizeof( pm->req ), "{ \\"req\\": \\"num users\\"}" );
- mbuf->mtype = MT_USR_RESP;
- msg = rmr_mt_call( mr, msg, my_id, 100 ); // wait up to 100ms
- if( ! msg ) { // probably a timeout and no msg received
- return NULL; // let errno trickle up
- }
- if( mbuf->state != RMR_OK ) {
- while( retries_left-- > 0 && // loop as long as eagain
- mbuf->state == RMR_ERR_RETRY &&
- (msg = rmr_mt_call( mr, msg )) != NULL &&
- mbuf->state != RMR_OK ) {
- usleep( retry_delay );
- }
- if( mbuf == NULL || mbuf->state != RMR_OK ) {
- rmr_free_msg( mbuf ); // safe if nil
- return NULL;
- }
- }
- // do something with mbuf
-
-
-
-SEE ALSO
---------------------------------------------------------------------------------------------
-
-rmr_alloc_msg(3), rmr_free_msg(3), rmr_init(3),
-rmr_mt_rcv(3), rmr_payload_size(3), rmr_send_msg(3),
-rmr_rcv_msg(3), rmr_rcv_specific(3), rmr_rts_msg(3),
-rmr_ready(3), rmr_fib(3), rmr_has_str(3),
-rmr_set_stimeout(3), rmr_tokenise(3), rmr_mk_ring(3),
-rmr_ring_free(3)
-
-
-NAME
---------------------------------------------------------------------------------------------
-
-rmr_mt_rcv
-
-SYNOPSIS
---------------------------------------------------------------------------------------------
-
-
-::
-
- #include <rmr/rmr.h>
- rmr_mbuf_t* rmr_mt_rcv( void* vctx, rmr_mbuf_t* old_msg, int timeout );
-
-
-
-DESCRIPTION
---------------------------------------------------------------------------------------------
-
-The rmr_mt_rcv function blocks until a message is received,
-or the timeout period (milliseconds) has passed. The result
-is an RMR message buffer which references a received message.
-In the case of a timeout the state will be reflected in an
-"empty buffer" (if old_msg was not nil, or simply with the
-return of a nil pointer. If a timeout value of zero (0) is
-given, then the function will block until the next message
-received.
-
-The *vctx* pointer is the pointer returned by the rmr_init
-function. *Old_msg* is a pointer to a previously used message
-buffer or NULL. The ability to reuse message buffers helps to
-avoid alloc/free cycles in the user application. When no
-buffer is available to supply, the receive function will
-allocate one.
-
-The *old_msg* parameter allows the user to pass a previously
-generated RMR message back to RMR for reuse. Optionally, the
-user application may pass a nil pointer if no reusable
-message is available. When a timeout occurs, and old_msg was
-not nil, the state will be returned by returning a pointer to
-the old message with the state set.
-
-It is possible to use the *rmr_rcv_msg()* function instead of
-this function. Doing so might be advantageous if the user
-programme does not always start the multi-threaded mode and
-the use of *rmr_rcv_msg()* would make the flow of the code
-more simple. The advantages of using this function are the
-ability to set a timeout without using epoll, and a small
-performance gain (if multi-threaded mode is enabled, and the
-*rmr_rcv_msg()* function is used, it simply invokes this
-function without a timeout value, thus there is the small
-cost of a second call that results). Similarly, the
-*rmr_torcv_msg()* call can be used when in multi-threaded
-mode with the same "pass through" overhead to using this
-function directly.
-
-RETURN VALUE
---------------------------------------------------------------------------------------------
-
-When a message is received before the timeout period expires,
-a pointer to the RMR message buffer which describes the
-message is returned. This will, with a high probability, be a
-different message buffer than *old_msg;* the user application
-should not continue to use *old_msg* after it is passed to
-this function.
-
-In the event of a timeout the return value will be the old
-msg with the state set, or a nil pointer if no old message
-was provided.
-
-ERRORS
---------------------------------------------------------------------------------------------
-
-The *state* field in the message buffer will be set to one of
-the following values:
-
-
-
-RMR_OK
-
- The message was received without error.
-
-
-RMR_ERR_BADARG
-
- A parameter passed to the function was not valid (e.g. a
- nil pointer). indicate either RMR_OK or RMR_ERR_EMPTY if
- an empty message was received.
-
-
-RMR_ERR_EMPTY
-
- The message received had no associated data. The length of
- the message will be 0.
-
-
-RMR_ERR_NOTSUPP
-
- The multi-threaded option was not enabled when RMR was
- initialised. See the man page for *rmr_init()* for
- details.
-
-
-RMR_ERR_RCVFAILED
-
- A hard error occurred preventing the receive from
- completing.
-
-When a nil pointer is returned, or any other state value was
-set in the message buffer, errno will be set to one of the
-following:
-
-
-
-INVAL
-
- Parameter(s) passed to the function were not valid.
-
-
-EBADF
-
- The underlying message transport is unable to process the
- request.
-
-
-ENOTSUP
-
- The underlying message transport is unable to process the
- request.
-
-
-EFSM
-
- The underlying message transport is unable to process the
- request.
-
-
-EAGAIN
-
- The underlying message transport is unable to process the
- request.
-
-
-EINTR
-
- The underlying message transport is unable to process the
- request.
-
-
-ETIMEDOUT
-
- The underlying message transport is unable to process the
- request.
-
-
-ETERM
-
- The underlying message transport is unable to process the
- request.
-
-
-EXAMPLE
---------------------------------------------------------------------------------------------
-
-
-
-::
-
- rmr_mbuf_t* mbuf = NULL; // received msg
- msg = rmr_mt_recv( mr, mbuf, 100 ); // wait up to 100ms
- if( msg != NULL ) {
- switch( msg->state ) {
- case RMR_OK:
- printf( "got a good message\\n" );
- break;
- case RMR_ERR_EMPTY:
- printf( "received timed out\\n" );
- break;
- default:
- printf( "receive error: %d\\n", mbuf->state );
- break;
- }
- } else {
- printf( "receive timeout (nil)\\n" );
- }
-
-
-
-SEE ALSO
---------------------------------------------------------------------------------------------
-
-rmr_alloc_msg(3), rmr_call(3), rmr_free_msg(3),
-rmr_get_rcvfd(3), rmr_init(3), rmr_mk_ring(3),
-rmr_mt_call(3), rmr_payload_size(3), rmr_send_msg(3),
-rmr_torcv_msg(3), rmr_rcv_specific(3), rmr_rts_msg(3),
-rmr_ready(3), rmr_ring_free(3), rmr_torcv_msg(3)
-
-
-NAME
---------------------------------------------------------------------------------------------
-
-rmr_payload_size
-
-SYNOPSIS
---------------------------------------------------------------------------------------------
-
-
-::
-
- #include <rmr/rmr.h>
- int rmr_payload_size( rmr_mbuf_t* msg );
-
-
-
-DESCRIPTION
---------------------------------------------------------------------------------------------
-
-Given a message buffer, this function returns the amount of
-space (bytes) available for the user application to consume
-in the message payload. This is different than the message
-length available as a field in the message buffer.
-
-RETURN VALUE
---------------------------------------------------------------------------------------------
-
-The number of bytes available in the payload.
-
-ERRORS
---------------------------------------------------------------------------------------------
-
-
-
-INVAL
-
- Parameter(s) passed to the function were not valid.
-
-
-SEE ALSO
---------------------------------------------------------------------------------------------
-
-rmr_alloc_msg(3), rmr_call(3), rmr_free_msg(3), rmr_init(3),
-rmr_send_msg(3), rmr_rcv_msg(3), rmr_rcv_specific(3),
-rmr_rts_msg(3), rmr_ready(3), rmr_fib(3), rmr_has_str(3),
-rmr_tokenise(3), rmr_mk_ring(3), rmr_ring_free(3)
-
-
-NAME
---------------------------------------------------------------------------------------------
-
-rmr_rcv_msg
-
-SYNOPSIS
---------------------------------------------------------------------------------------------
-
-
-::
-
- #include <rmr/rmr.h>
- rmr_mbuf_t* rmr_rcv_msg( void* vctx, rmr_mbuf_t* old_msg );
-
-
-
-DESCRIPTION
---------------------------------------------------------------------------------------------
-
-The rmr_rcv_msg function blocks until a message is received,
-returning the message to the caller via a pointer to a
-rmr_mbuf_t structure type. If messages were queued while
-waiting for the response to a previous invocation of
-rmr_call, the oldest message is removed from the queue and
-returned without delay.
-
-The *vctx* pointer is the pointer returned by the rmr_init
-function. *Old_msg* is a pointer to a previously used message
-buffer or NULL. The ability to reuse message buffers helps to
-avoid alloc/free cycles in the user application. When no
-buffer is available to supply, the receive function will
-allocate one.
-
-RETURN VALUE
---------------------------------------------------------------------------------------------
-
-The function returns a pointer to the rmr_mbuf_t structure
-which references the message information (state, length,
-payload), or a nil pointer in the case of an extreme error.
-
-ERRORS
---------------------------------------------------------------------------------------------
-
-The *state* field in the message buffer will indicate RMR_OK
-when the message receive process was successful and the
-message can be used by the caller. Depending on the
-underlying transport mechanism, one of the following RMR
-error stats may be returned:
-
-
-
-RMR_ERR_EMPTY
-
- The message received had no payload, or was completely
- empty.
-
-
-RMR_ERR_TIMEOUT
-
- For some transport mechanisms, or if reading the receive
- queue from multiple threads, it is possible for one thread
- to find no data waiting when it queries the queue. When
- this state is reported, the message buffer does not
- contain message data and the user application should
- reinvoke the receive function.
-
-
-When an RMR error state is reported, the underlying errno
-value might provide more information. The following is a list
-of possible values that might accompany the states listed
-above:
-
-RMR_ERR_EMPTY if an empty message was received. If a nil
-pointer is returned, or any other state value was set in the
-message buffer, errno will be set to one of the following:
-
-
-
-INVAL
-
- Parameter(s) passed to the function were not valid.
-
-
-EBADF
-
- The underlying message transport is unable to process the
- request.
-
-
-ENOTSUP
-
- The underlying message transport is unable to process the
- request.
-
-
-EFSM
-
- The underlying message transport is unable to process the
- request.
-
-
-EAGAIN
-
- The underlying message transport is unable to process the
- request.
-
-
-EINTR
-
- The underlying message transport is unable to process the
- request.
-
-
-ETIMEDOUT
-
- The underlying message transport is unable to process the
- request.
-
-
-ETERM
-
- The underlying message transport is unable to process the
- request.
-
-
-EXAMPLE
---------------------------------------------------------------------------------------------
-
-
-SEE ALSO
---------------------------------------------------------------------------------------------
-
-rmr_alloc_msg(3), rmr_call(3), rmr_free_msg(3),
-rmr_get_rcvfd(3), rmr_init(3), rmr_mk_ring(3),
-rmr_payload_size(3), rmr_send_msg(3), rmr_torcv_msg(3),
-rmr_rcv_specific(3), rmr_rts_msg(3), rmr_ready(3),
-rmr_ring_free(3), rmr_torcv_msg(3)
-
-
-NAME
---------------------------------------------------------------------------------------------
-
-rmr_ready
-
-SYNOPSIS
---------------------------------------------------------------------------------------------
-
-
-::
-
- #include <rmr/rmr.h>
- int rmr_ready( void* vctx );
-
-
-
-DESCRIPTION
---------------------------------------------------------------------------------------------
-
-The rmr_ready function checks to see if a routing table has
-been successfully received and installed. The return value
-indicates the state of readiness.
-
-RETURN VALUE
---------------------------------------------------------------------------------------------
-
-A return value of 1 (true) indicates that the routing table
-is in place and attempts to send messages can be made. When 0
-is returned (false) the routing table has not been received
-and thus attempts to send messages will fail with *no
-endpoint* errors.
-
-SEE ALSO
---------------------------------------------------------------------------------------------
-
-rmr_alloc_msg(3), rmr_call(3), rmr_free_msg(3), rmr_init(3),
-rmr_payload_size(3), rmr_send_msg(3), rmr_rcv_msg(3),
-rmr_rcv_specific(3), rmr_rts_msg(3), rmr_fib(3),
-rmr_has_str(3), rmr_tokenise(3), rmr_mk_ring(3),
-rmr_ring_free(3)
-
-
-NAME
---------------------------------------------------------------------------------------------
-
-rmr_realloc_payload
-
-SYNOPSIS
---------------------------------------------------------------------------------------------
-
-
-::
-
- #include <rmr/rmr.h>
- extern rmr_mbuf_t* rmr_realloc_payload( rmr_mbuf_t* msg, int new_len, int copy, int clone );
-
-
-
-DESCRIPTION
---------------------------------------------------------------------------------------------
-
-The rmr_realloc_payload function will return a pointer to an
-RMR message buffer struct (rmr_mbuf_t) which has a payload
-large enough to accomodate *new_len* bytes. If necessary, the
-underlying payload is reallocated, and the bytes from the
-original payload are copied if the *copy* parameter is true
-(1). If the message passed in has a payload large enough,
-there is no additional memory allocation and copying.
-
-Cloning The Message Buffer
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-This function can also be used to generate a separate copy of
-the original message, with the desired payload size, without
-destroying the original message buffer or the original
-payload. A standalone copy is made only when the *clone*
-parameter is true (1). When cloning, the payload is copied to
-the cloned message **only** if the *copy* parameter is true.
-
-Message Buffer Metadata
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-The metadata in the original message buffer (message type,
-subscription ID, and payload length) will be preserved if the
-*copy* parameter is true. When this parameter is not true
-(0), then these values are set to the uninitialised value
-(-1) for type and ID, and the length is set to 0.
-
-RETURN VALUE
---------------------------------------------------------------------------------------------
-
-The rmr_realloc_payload function returns a pointer to the
-message buffer with the payload which is large enough to hold
-*new_len* bytes. If the *clone* option is true, this will be
-a pointer to the newly cloned message buffer; the original
-message buffer pointer may still be used to reference that
-message. It is the calling application's responsibility to
-free the memory associateed with both messages using the
-rmr_free_msg() function.
-
-When the *clone* option is not used, it is still good
-practice by the calling application to capture and use this
-reference as it is possible that the message buffer, and not
-just the payload buffer, was reallocated. In the event of an
-error, a nil pointer will be returned and the value of
-*errno* will be set to reflect the problem.
-
-ERRORS
---------------------------------------------------------------------------------------------
-
-These value of *errno* will reflect the error condition if a
-nil pointer is returned:
-
-
-
-ENOMEM
-
- Memory allocation of the new payload failed.
-
-
-EINVAL
-
- The pointer passed in was nil, or refrenced an invalid
- message, or the required length was not valid.
-
-
-EXAMPLE
---------------------------------------------------------------------------------------------
-
-The following code bit illustrates how this function can be
-used to reallocate a buffer for a return to sender
-acknowledgement message which is larger than the message
-received.
-
-
-::
-
- if( rmr_payload_size( msg ) < ack_sz ) { // received message too small for ack
- msg = rmr_realloc_payload( msg, ack_sz, 0, 0 ); // reallocate the message with a payload big enough
- if( msg == NULL ) {
- fprintf( stderr, "[ERR] realloc returned a nil pointer: %s\\n", strerror( errno ) );
- } else {
- // populate and send ack message
- }
- }
-
-
-
-SEE ALSO
---------------------------------------------------------------------------------------------
-
-rmr_alloc_msg(3), rmr_free_msg(3), rmr_init(3),
-rmr_payload_size(3), rmr_send_msg(3), rmr_rcv_msg(3),
-rmr_rcv_specific(3), rmr_rts_msg(3), rmr_ready(3),
-rmr_fib(3), rmr_has_str(3), rmr_set_stimeout(3),
-rmr_tokenise(3), rmr_mk_ring(3), rmr_ring_free(3)
-
-
-NAME
---------------------------------------------------------------------------------------------
-
-rmr_rts_msg
-
-SYNOPSIS
---------------------------------------------------------------------------------------------
-
-
-::
-
- #include <rmr/rmr.h>
- rmr_mbuf_t* rmr_rts_msg( void* vctx, rmr_mbuf_t* msg );
-
-
-
-DESCRIPTION
---------------------------------------------------------------------------------------------
-
-The rmr_rts_msg function sends a message returning it to the
-endpoint which sent the message rather than selecting an
-endpoint based on the message type and routing table. Other
-than this small difference, the behaviour is exactly the same
-as rmr_send_msg.
-
-Retries
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-The send operations in RMR will retry *soft* send failures
-until one of three conditions occurs:
-
-
-
-1.
-
- The message is sent without error
-
-
-2.
-
- The underlying transport reports a *hard* failure
-
-
-3.
-
- The maximum number of retry loops has been attempted
-
-
-A retry loop consists of approximately 1000 send attempts
-**without** any intervening calls to *sleep()* or *usleep().*
-The number of retry loops defaults to 1, thus a maximum of
-1000 send attempts is performed before returning to the user
-application. This value can be set at any point after RMR
-initialisation using the *rmr_set_stimeout()* function
-allowing the user application to completely disable retires
-(set to 0), or to increase the number of retry loops.
-
-Transport Level Blocking
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-The underlying transport mechanism used to send messages is
-configured in *non-blocking* mode. This means that if a
-message cannot be sent immediately the transport mechanism
-will **not** pause with the assumption that the inability to
-send will clear quickly (within a few milliseconds). This
-means that when the retry loop is completely disabled (set to
-0), that the failure to accept a message for sending by the
-underlying mechanisms (software or hardware) will be reported
-immediately to the user application.
-
-It should be noted that depending on the underlying transport
-mechanism being used, it is extremely likely that retry
-conditions will happen during normal operations. These are
-completely out of RMR's control, and there is nothing that
-RMR can do to avoid or mitigate these other than by allowing
-RMR to retry the send operation, and even then it is possible
-(e.g., during connection reattempts), that a single retry
-loop is not enough to guarantee a successful send.
-
-PAYLOAD SIZE
---------------------------------------------------------------------------------------------
-
-When crafting a response based on a received message, the
-user application must take care not to write more bytes to
-the message payload than the allocated message has. In the
-case of a received message, it is possible that the response
-needs to be larger than the payload associated with the
-inbound message. In order to use the return to sender
-function, the source information in the original message must
-be present in the response; information which cannot be added
-to a message buffer allocated through the standard RMR
-allocation function. To allocate a buffer with a larger
-payload, and which retains the necessary sender data needed
-by this function, the *rmr_realloc_payload()* function must
-be used to extend the payload to a size suitable for the
-response.
-
-RETURN VALUE
---------------------------------------------------------------------------------------------
-
-On success, a new message buffer, with an empty payload, is
-returned for the application to use for the next send. The
-state in this buffer will reflect the overall send operation
-state and should be RMR_OK.
-
-If the state in the returned buffer is anything other than
-RMR_OK, the user application may need to attempt a
-retransmission of the message, or take other action depending
-on the setting of errno as described below.
-
-In the event of extreme failure, a nil pointer is returned.
-In this case the value of errno might be of some use, for
-documentation, but there will be little that the user
-application can do other than to move on.
-
-ERRORS
---------------------------------------------------------------------------------------------
-
-The following values may be passed back in the *state* field
-of the returned message buffer.
-
-
-
-RMR_ERR_BADARG
-
- The message buffer pointer did not refer to a valid
- message.
-
-RMR_ERR_NOHDR
-
- The header in the message buffer was not valid or
- corrupted.
-
-RMR_ERR_NOENDPT
-
- The message type in the message buffer did not map to a
- known endpoint.
-
-RMR_ERR_SENDFAILED
-
- The send failed; errno has the possible reason.
-
-
-The following values may be assigned to errno on failure.
-
-
-INVAL
-
- Parameter(s) passed to the function were not valid, or the
- underlying message processing environment was unable to
- interpret the message.
-
-
-ENOKEY
-
- The header information in the message buffer was invalid.
-
-
-ENXIO
-
- No known endpoint for the message could be found.
-
-
-EMSGSIZE
-
- The underlying transport refused to accept the message
- because of a size value issue (message was not attempted
- to be sent).
-
-
-EFAULT
-
- The message referenced by the message buffer is corrupt
- (nil pointer or bad internal length).
-
-
-EBADF
-
- Internal RMR error; information provided to the message
- transport environment was not valid.
-
-
-ENOTSUP
-
- Sending was not supported by the underlying message
- transport.
-
-
-EFSM
-
- The device is not in a state that can accept the message.
-
-
-EAGAIN
-
- The device is not able to accept a message for sending.
- The user application should attempt to resend.
-
-
-EINTR
-
- The operation was interrupted by delivery of a signal
- before the message was sent.
-
-
-ETIMEDOUT
-
- The underlying message environment timed out during the
- send process.
-
-
-ETERM
-
- The underlying message environment is in a shutdown state.
-
-
-EXAMPLE
---------------------------------------------------------------------------------------------
-
-
-SEE ALSO
---------------------------------------------------------------------------------------------
-
-rmr_alloc_msg(3), rmr_call(3), rmr_free_msg(3), rmr_init(3),
-rmr_payload_size(3), rmr_send_msg(3), rmr_rcv_msg(3),
-rmr_rcv_specific(3), rmr_ready(3), rmr_fib(3),
-rmr_has_str(3), rmr_set_stimeout(3), rmr_tokenise(3),
-rmr_mk_ring(3), rmr_ring_free(3)
-
-
-NAME
---------------------------------------------------------------------------------------------
-
-rmr_send_msg
-
-SYNOPSIS
---------------------------------------------------------------------------------------------
-
-
-::
-
- #include <rmr/rmr.h>
- rmr_mbuf_t* rmr_send_msg( void* vctx, rmr_mbuf_t* msg );
-
-
-
-DESCRIPTION
---------------------------------------------------------------------------------------------
-
-The rmr_send_msg function accepts a message buffer from the
-user application and attempts to send it. The destination of
-the message is selected based on the message type specified
-in the message buffer, and the matching information in the
-routing tables which are currently in use by the RMR library.
-This may actually result in the sending of the message to
-multiple destinations which could degrade expected overall
-performance of the user application. (Limiting excessive
-sending of messages is the responsibility of the
-application(s) responsible for building the routing table
-used by the RMR library, and not the responsibility of the
-library.)
-
-Retries
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-The send operations in RMR will retry *soft* send failures
-until one of three conditions occurs:
-
-
-
-1.
-
- The message is sent without error
-
-
-2.
-
- The underlying transport reports a *hard* failure
-
-
-3.
-
- The maximum number of retry loops has been attempted
-
-
-A retry loop consists of approximately 1000 send attempts
-**without** any intervening calls to *sleep()* or *usleep().*
-The number of retry loops defaults to 1, thus a maximum of
-1000 send attempts is performed before returning to the user
-application. This value can be set at any point after RMR
-initialisation using the *rmr_set_stimeout()* function
-allowing the user application to completely disable retires
-(set to 0), or to increase the number of retry loops.
-
-Transport Level Blocking
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-The underlying transport mechanism used to send messages is
-configured in *non-blocking* mode. This means that if a
-message cannot be sent immediately the transport mechanism
-will **not** pause with the assumption that the inability to
-send will clear quickly (within a few milliseconds). This
-means that when the retry loop is completely disabled (set to
-0), that the failure to accept a message for sending by the
-underlying mechanisms (software or hardware) will be reported
-immediately to the user application.
-
-It should be noted that depending on the underlying transport
-mechanism being used, it is extremely likely that retry
-conditions will happen during normal operations. These are
-completely out of RMR's control, and there is nothing that
-RMR can do to avoid or mitigate these other than by allowing
-RMR to retry the send operation, and even then it is possible
-(e.g., during connection reattempts), that a single retry
-loop is not enough to guarantee a successful send.
-
-RETURN VALUE
---------------------------------------------------------------------------------------------
-
-On success, a new message buffer, with an empty payload, is
-returned for the application to use for the next send. The
-state in this buffer will reflect the overall send operation
-state and will be RMR_OK when the send was successful.
-
-When the message cannot be successfully sent this function
-will return the unsent (original) message buffer with the
-state set to indicate the reason for failure. The value of
-*errno* may also be set to reflect a more detailed failure
-reason if it is known.
-
-In the event of extreme failure, a nil pointer is returned.
-In this case the value of errno might be of some use, for
-documentation, but there will be little that the user
-application can do other than to move on.
-
-**CAUTION:** In some cases it is extremely likely that the
-message returned by the send function does **not** reference
-the same memory structure. Thus is important for the user
-programme to capture the new pointer for future use or to be
-passed to rmr_free(). If you are experiencing either double
-free errors or segment faults in either rmr_free() or
-rmr_send_msg(), ensure that the return value from this
-function is being captured and used.
-
-ERRORS
---------------------------------------------------------------------------------------------
-
-The following values may be passed back in the *state* field
-of the returned message buffer.
-
-
-
-RMR_RETRY
-
- The message could not be sent, but the underlying
- transport mechanism indicates that the failure is
- temporary. If the send operation is tried again it might
- be successful.
-
-RMR_SEND_FAILED
-
- The send operation was not successful and the underlying
- transport mechanism indicates a permanent (hard) failure;
- retrying the send is not possible.
-
-RMR_ERR_BADARG
-
- The message buffer pointer did not refer to a valid
- message.
-
-RMR_ERR_NOHDR
-
- The header in the message buffer was not valid or
- corrupted.
-
-RMR_ERR_NOENDPT
-
- The message type in the message buffer did not map to a
- known endpoint.
-
-
-The following values may be assigned to errno on failure.
-
-
-INVAL
-
- Parameter(s) passed to the function were not valid, or the
- underlying message processing environment was unable to
- interpret the message.
-
-
-ENOKEY
-
- The header information in the message buffer was invalid.
-
-
-ENXIO
-
- No known endpoint for the message could be found.
-
-
-EMSGSIZE
-
- The underlying transport refused to accept the message
- because of a size value issue (message was not attempted
- to be sent).
-
-
-EFAULT
-
- The message referenced by the message buffer is corrupt
- (nil pointer or bad internal length).
-
-
-EBADF
-
- Internal RMR error; information provided to the message
- transport environment was not valid.
-
-
-ENOTSUP
-
- Sending was not supported by the underlying message
- transport.
-
-
-EFSM
-
- The device is not in a state that can accept the message.
-
-
-EAGAIN
-
- The device is not able to accept a message for sending.
- The user application should attempt to resend.
-
-
-EINTR
-
- The operation was interrupted by delivery of a signal
- before the message was sent.
-
-
-ETIMEDOUT
-
- The underlying message environment timed out during the
- send process.
-
-
-ETERM
-
- The underlying message environment is in a shutdown state.
-
-
-EXAMPLE
---------------------------------------------------------------------------------------------
-
-The following is a simple example of how the rmr_send_msg
-function is called. In this example, the send message buffer
-is saved between calls and reused eliminating alloc/free
-cycles.
-
-
-::
-
- static rmr_mbuf_t* send_msg = NULL; // message to send; reused on each call
- msg_t* send_pm; // payload for send
- msg_t* pm; // our message format in the received payload
- if( send_msg == NULL ) {
- send_msg = rmr_alloc_msg( mr, MAX_SIZE ); // new buffer to send
- }
- // reference payload and fill in message type
- pm = (msg_t*) send_msg->payload;
- send_msg->mtype = MT_ANSWER;
- msg->len = generate_data( pm ); // something that fills the payload in
- msg = rmr_send_msg( mr, send_msg ); // ensure new pointer used after send
- if( ! msg ) {
- return ERROR;
- } else {
- if( msg->state != RMR_OK ) {
- // check for RMR_ERR_RETRY, and resend if needed
- // else return error
- }
- }
- return OK;
-
-
-
-SEE ALSO
---------------------------------------------------------------------------------------------
-
-rmr_alloc_msg(3), rmr_call(3), rmr_free_msg(3), rmr_init(3),
-rmr_payload_size(3), rmr_rcv_msg(3), rmr_rcv_specific(3),
-rmr_rts_msg(3), rmr_ready(3), rmr_mk_ring(3),
-rmr_ring_free(3), rmr_torcv_rcv(3), rmr_wh_send_msg(3)
-
-
-NAME
---------------------------------------------------------------------------------------------
-
-rmr_set_fack
-
-SYNOPSIS
---------------------------------------------------------------------------------------------
-
-
-::
-
- #include <rmr/rmr.h>
- void rmr_set_fack( void* vctx );
-
-
-
-DESCRIPTION
---------------------------------------------------------------------------------------------
-
-The rmr_set_fack function enables *fast TCP acknowledgements*
-if the underlying transport library supports it. This might
-be useful for applications which must send messages at a
-maximum rate.
-
-RETURN VALUE
---------------------------------------------------------------------------------------------
-
-There is no return value.
-
-ERRORS
---------------------------------------------------------------------------------------------
-
-This function does not generate any errors.
-
-SEE ALSO
---------------------------------------------------------------------------------------------
-
-rmr_init(3),
-
-
-NAME
---------------------------------------------------------------------------------------------
-
-rmr_set_stimeout
-
-SYNOPSIS
---------------------------------------------------------------------------------------------
-
-
-::
-
- #include <rmr/rmr.h>
- rmr_mbuf_t* rmr_set_stimeout( void* vctx, int rloops );
-
-
-
-DESCRIPTION
---------------------------------------------------------------------------------------------
-
-The rmr_set_stimeout function sets the configuration for how
-RMR will retry message send operations which complete with
-either a *timeout* or *again* completion value. (Send
-operations include all of the possible message send
-functions: *rmr_send_msg(), rmr_call(), rmr_rts_msg()* and
-*rmr_wh_send_msg().* The *rloops* parameter sets the maximum
-number of retry loops that will be attempted before giving up
-and returning the unsuccessful state to the user application.
-Each retry loop is approximately 1000 attempts, and RMR does
-**not** invoke any sleep function between retries in the
-loop; a small, 1 mu-sec, sleep is executed between loop sets
-if the *rloops* value is greater than 1.
-
-
-Disabling Retries
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-By default, the send operations will execute with an *rloop*
-setting of 1; each send operation will attempt to resend the
-message approximately 1000 times before giving up. If the
-user application does not want to have send operations retry
-when the underlying transport mechanism indicates *timeout*
-or *again,* the application should invoke this function and
-pass a value of 0 (zero) for *rloops.* With this setting, all
-RMR send operations will attempt a send operation only
-**once,** returning immediately to the caller with the state
-of that single attempt.
-
-RETURN VALUE
---------------------------------------------------------------------------------------------
-
-This function returns a -1 to indicate that the *rloops*
-value could not be set, and the value *RMR_OK* to indicate
-success.
-
-ERRORS
---------------------------------------------------------------------------------------------
-
-Currently errno is **not** set by this function; the only
-cause of a failure is an invalid context (*vctx*) pointer.
-
-EXAMPLE
---------------------------------------------------------------------------------------------
-
-The following is a simple example of how the rmr_set_stimeout
-function is called.
-
-
-::
-
- #define NO_FLAGS 0
- char* port = "43086"; // port for message router listen
- int max_size = 4096; // max message size for default allocations
- void* mr_context; // message router context
- mr_context = rmr_init( port, max_size, NO_FLAGS );
- if( mr_context != NULL ) {
- rmr_set_stimeout( mr_context, 0 ); // turn off retries
- }
-
-
-
-SEE ALSO
---------------------------------------------------------------------------------------------
-
-rmr_alloc_msg(3), rmr_call(3), rmr_free_msg(3), rmr_init(3),
-rmr_payload_size(3), rmr_rcv_msg(3), rmr_rcv_specific(3),
-rmr_rts_msg(3), rmr_ready(3), rmr_mk_ring(3),
-rmr_ring_free(3), rmr_send_msg(3), rmr_torcv_rcv(3),
-rmr_wh_send_msg(3)
-
-
-NAME
---------------------------------------------------------------------------------------------
-
-rmr_set_trace
-
-SYNOPSIS
---------------------------------------------------------------------------------------------
-
-
-::
-
- #include <rmr/rmr.h>
- int rmr_set_trace( rmr_mbuf_t* mbuf, unsigned char* data, int len )
-
-
-
-DESCRIPTION
---------------------------------------------------------------------------------------------
-
-The rmr_set_trace function will copy len bytes from data into
-the trace portion of mbuf. If the trace area of mbuf is not
-the correct size, the message buffer will be reallocated to
-ensure that enough space is available for the trace data.
-
-RETURN VALUE
---------------------------------------------------------------------------------------------
-
-The rmr_set_trace function returns the number of bytes
-successfully copied to the message. If 0 is returned either
-the message pointer was nil, or the size in the parameters
-was <= 0.
-
-SEE ALSO
---------------------------------------------------------------------------------------------
-
-rmr_alloc_msg(3), rmr_tralloc_msg(3), rmr_bytes2xact(3),
-rmr_bytes2payload(3), rmr_call(3), rmr_free_msg(3),
-rmr_get_rcvfd(3), rmr_get_meid(3), rmr_get_trace(3),
-rmr_get_trlen(3), rmr_init(3), rmr_init_trace(3),
-rmr_payload_size(3), rmr_send_msg(3), rmr_rcv_msg(3),
-rmr_rcv_specific(3), rmr_rts_msg(3), rmr_ready(3),
-rmr_fib(3), rmr_has_str(3), rmr_tokenise(3), rmr_mk_ring(3),
-rmr_ring_free(3), rmr_str2meid(3), rmr_str2xact(3),
-rmr_wh_open(3), rmr_wh_send_msg(3)
-
-
-NAME
---------------------------------------------------------------------------------------------
-
-rmr_set_vlevel
-
-SYNOPSIS
---------------------------------------------------------------------------------------------
-
-
-::
-
- #include <rmr/rmr.h>
- #include <rmr/rmr_logging.h>
- void rmr_set_vlevel( int new_level )
-
-
-
-DESCRIPTION
---------------------------------------------------------------------------------------------
-
-The rmr_set_vlevel allows the user programme to set the
-verbosity level which is used to determine the messages RMR
-writes to standard error. The new_vlevel value must be one of
-the following constants which have the indicated meanings:
-
-
-RMR_VL_OFF
-
- Turns off all message writing. This includes the stats and
- debugging messages generated by the route collector thread
- which are normally affected only by the externally managed
- verbose level file (and related environment variable).
-
-
-RMR_VL_CRIT
-
- Write only messages of critical importance. From the point
- of view of RMR, when a critical proper behaviour of the
- library cannot be expected or guaranteed.
-
-RMR_VL_ERR
-
- Include error messages in the output. An error is an event
- from which RMR has no means to recover. Continued proper
- execution is likely except where the affected connection
- and/or component mentioned in the error is concerned.
-
-RMR_VL_WARN
-
- Include warning messages in the output. A warning
- indicates an event which is not considered to be normal,
- but is expected and continued acceptable behaviour of the
- system is assured.
-
-RMR_VL_INFO
-
- Include informational messagees in the output.
- Informational messages include some diagnostic information
- which explain the activities of RMR.
-
-RMR_VL_DEBUG
-
- Include all debugging messages in the output. Debugging
- must have also been enabled during the build as a
- precaution to accidentally enabling this level of output
- as it can grossly affect performance.
-
-
-Generally RMR does not write messages to the standard error
-device from *critical path* functions, therefore it is
-usually not harmful to enable a verbosity level of either
-RMR_VL_CRIT or RMR_VL_ERR.
-
-Messages written from the route table collection thread are
-still governed by the value placed into the verbose level
-control file (see the man page for rmr_init()); those
-messages are affected only when logging is completely
-disabled by passing RMR_VL_OFF to this function.
-
-The verbosity level can also be set via an environment
-variable prior to the start of the RMR based application. The
-environment variable is read only during initialisation; if
-the programme must change the value during execution, this
-function must be used. The default value, if this function is
-never called, and the environment variable is not present, is
-RMR_VL_ERR.
-
-SEE ALSO
---------------------------------------------------------------------------------------------
-
-rmr_init(3)
-
-
-NAME
---------------------------------------------------------------------------------------------
-
-rmr_str2meid
-
-SYNOPSIS
---------------------------------------------------------------------------------------------
-
-
-::
-
- #include <rmr/rmr.h>
- int rmr_str2meid( rmr_mbuf_t* mbuf, unsigned char* src, int len )
-
-
-
-DESCRIPTION
---------------------------------------------------------------------------------------------
-
-The rmr_str2meid function will copy the string pointed to by
-src to the managed entity ID (meid) field in the given
-message. The field is a fixed length, gated by the constant
-RMR_MAX_MEID and if string length is larger than this value,
-then **nothing** will be copied. (Note, this differs slightly
-from the behaviour of the lrmr_bytes2meid() function.)
-
-RETURN VALUE
---------------------------------------------------------------------------------------------
-
-On success, the value RMR_OK is returned. If the string
-cannot be copied to the message, the return value will be one
-of the errors listed below.
-
-ERRORS
---------------------------------------------------------------------------------------------
-
-If the return value is not RMR_OK, then it will be set to one
-of the values below.
-
-
-
-RMR_ERR_BADARG
-
- The message, or an internal portion of the message, was
- corrupted or the pointer was invalid.
-
-
-RMR_ERR_OVERFLOW
-
- The length passed in was larger than the maximum length of
- the field; only a portion of the source bytes were copied.
-
-
-EXAMPLE
---------------------------------------------------------------------------------------------
-
-
-SEE ALSO
---------------------------------------------------------------------------------------------
-
-rmr_alloc_msg(3), rmr_call(3), rmr_free_msg(3),
-rmr_get_meid(3), rmr_get_rcvfd(3), rmr_payload_size(3),
-rmr_send_msg(3), rmr_rcv_msg(3), rmr_rcv_specific(3),
-rmr_rts_msg(3), rmr_ready(3), rmr_fib(3), rmr_has_str(3),
-rmr_tokenise(3), rmr_mk_ring(3), rmr_ring_free(3),
-rmr_bytes2meid(3), rmr_wh_open(3), rmr_wh_send_msg(3)
-
-
-NAME
---------------------------------------------------------------------------------------------
-
-rmr_str2xact
-
-SYNOPSIS
---------------------------------------------------------------------------------------------
-
-
-::
-
- #include <rmr/rmr.h>
- int rmr_str2xact( rmr_mbuf_t* mbuf, unsigned char* src, int len )
-
-
-
-DESCRIPTION
---------------------------------------------------------------------------------------------
-
-The rmr_str2xact function will copy the string pointed to by
-src to the transaction ID (xaction) field in the given
-message. The field is a fixed length, gated by the constant
-RMR_MAX_XID and if string length is larger than this value,
-then **nothing** will be copied. (Note, this differs slightly
-from the behaviour of the lrmr_bytes2xact() function.)
-
-
-RETURN VALUE
---------------------------------------------------------------------------------------------
-
-On success, the value RMR_OK is returned. If the string
-cannot be copied to the message, the return value will be
-one of the errors listed below.
-
-ERRORS
---------------------------------------------------------------------------------------------
-
-If the return value is not RMR_OK, then it will be set to
-one of the values below.
-
-
-RMR_ERR_BADARG
-
- The message, or an internal portion of the message, was
- corrupted or the pointer was invalid.
-
-
-RMR_ERR_OVERFLOW
-
- The length passed in was larger than the maximum length of
- the field; only a portion of the source bytes were copied.
-
-
-EXAMPLE
---------------------------------------------------------------------------------------------
-
-
-SEE ALSO
---------------------------------------------------------------------------------------------
-
-rmr_alloc_msg(3), rmr_bytes2meid(3), rmr_bytes2xact(3),
-rmr_call(3), rmr_free_msg(3), rmr_get_meid(3),
-rmr_get_rcvfd(3), rmr_get_xact(3), rmr_payload_size(3),
-rmr_send_msg(3), rmr_rcv_msg(3), rmr_rcv_specific(3),
-rmr_rts_msg(3), rmr_ready(3), rmr_fib(3), rmr_has_str(3),
-rmr_tokenise(3), rmr_mk_ring(3), rmr_ring_free(3),
-rmr_str2meid(3), rmr_wh_open(3), rmr_wh_send_msg(3)
-
-
-NAME
---------------------------------------------------------------------------------------------
-
-RMR support functions
-
-SYNOPSIS
---------------------------------------------------------------------------------------------
-
-
-::
-
- #include <rmr/rmr.h>
- #include <rmr/ring_inline.h>
- char* rmr_fib( char* fname );
- int rmr_has_str( char const* buf, char const* str, char sep, int max );
- int rmr_tokenise( char* buf, char** tokens, int max, char sep );
- void* rmr_mk_ring( int size );
- void rmr_ring_free( void* vr );
- static inline void* rmr_ring_extract( void* vr )
- static inline int rmr_ring_insert( void* vr, void* new_data )
-
-
-
-DESCRIPTION
---------------------------------------------------------------------------------------------
-
-These functions support the RMR library, and are made
-available to user applications as some (e.g. route table
-generators) might need and/or want to make use of them. The
-rmr_fib function accepts a file name and reads the entire
-file into a single buffer. The intent is to provide an easy
-way to load a static route table without a lot of buffered
-I/O hoops.
-
-The rmr_has_str function accepts a *buffer* containing a set
-of delimited tokens (e.g. foo,bar,goo) and returns true if
-the target string, *str,* matches one of the tokens. The
-*sep* parameter provides the separation character in the
-buffer (e.g a comma) and *max* indicates the maximum number
-of tokens to split the buffer into before checking.
-
-The rmr_tokenise function is a simple tokeniser which splits
-*buf* into tokens at each occurrence of *sep*. Multiple
-occurrences of the separator character (e.g. a,,b) result in
-a nil token. Pointers to the tokens are placed into the
-*tokens* array provided by the caller which is assumed to
-have at least enough space for *max* entries.
-
-The rmr_mk_ring function creates a buffer ring with *size*
-entries.
-
-The rmr_ring_free function accepts a pointer to a ring
-context and frees the associated memory.
-
-The rmr_ring_insert and rmr_ring_extract functions are
-provided as static inline functions via the
-*rmr/ring_inline.h* header file. These functions both accept
-the ring *context* returned by mk_ring, and either insert a
-pointer at the next available slot (tail) or extract the data
-at the head.
-
-RETURN VALUES
---------------------------------------------------------------------------------------------
-
-The following are the return values for each of these
-functions.
-
-The rmr_fib function returns a pointer to the buffer
-containing the contents of the file. The buffer is terminated
-with a single nil character (0) making it a legitimate C
-string. If the file was empty or nonexistent, a buffer with
-an immediate nil character. If it is important to the calling
-programme to know if the file was empty or did not exist, the
-caller should use the system stat function call to make that
-determination.
-
-The rmr_has_str function returns 1 if *buf* contains the
-token referenced by &ita and false (0) if it does not. On
-error, a -1 value is returned and errno is set accordingly.
-
-The rmr_tokenise function returns the actual number of token
-pointers placed into *tokens*
-
-The rmr_mk_ring function returns a void pointer which is the
-*context* for the ring.
-
-The rmr_ring_insert function returns 1 if the data was
-successfully inserted into the ring, and 0 if the ring is
-full and the pointer could not be deposited.
-
-The rmr_ring_extract will return the data which is at the
-head of the ring, or NULL if the ring is empty.
-
-ERRORS
---------------------------------------------------------------------------------------------
-
-Not many of these functions set the value in errno, however
-the value may be one of the following:
-
-
-INVAL
-
- Parameter(s) passed to the function were not valid.
-
-
-EXAMPLE
---------------------------------------------------------------------------------------------
-
-
-SEE ALSO
---------------------------------------------------------------------------------------------
-
-rmr_alloc_msg(3), rmr_call(3), rmr_free_msg(3), rmr_init(3),
-rmr_payload_size(3), rmr_send_msg(3), rmr_rcv_msg(3),
-rmr_rcv_specific(3), rmr_rts_msg(3), rmr_ready(3),
-
-
-NAME
---------------------------------------------------------------------------------------------
-
-rmr_torcv_msg
-
-SYNOPSIS
---------------------------------------------------------------------------------------------
-
-
-::
-
- #include <rmr/rmr.h>
- rmr_mbuf_t* rmr_torcv_msg( void* vctx, rmr_mbuf_t* old_msg, int ms_to );
-
-
-
-DESCRIPTION
---------------------------------------------------------------------------------------------
-
-The rmr_torcv_msg function will pause for *ms_to*
-milliseconds waiting for a message to arrive. If a message
-arrives before the timeout expires the message buffer
-returned will have a status of RMR_OK and the payload will
-contain the data received. If the timeout expires before the
-message is received, the status will have the value
-RMR_ERR_TIMEOUT. When a received message is returned the
-message buffer will also contain the message type and length
-set by the sender. If messages were queued while waiting for
-the response to a previous invocation of rmr_call, the oldest
-message is removed from the queue and returned without delay.
-
-The *vctx* pointer is the pointer returned by the rmr_init
-function. *Old_msg* is a pointer to a previously used message
-buffer or NULL. The ability to reuse message buffers helps to
-avoid alloc/free cycles in the user application. When no
-buffer is available to supply, the receive function will
-allocate one.
-
-RETURN VALUE
---------------------------------------------------------------------------------------------
-
-The function returns a pointer to the rmr_mbuf_t structure
-which references the message information (state, length,
-payload), or a nil pointer in the case of an extreme error.
-
-ERRORS
---------------------------------------------------------------------------------------------
-
-The *state* field in the message buffer will be one of the
-following:
-
-
-
-RMR_OK
-
- The message buffer (payload) references the received data.
-
-
-RMR_ERR_INITFAILED
-
- The first call to this function must initialise an
- underlying system notification mechanism. On failure, this
- error is returned and errno will have the system error
- status set. If this function fails to intialise, the poll
- mechansim, it is likely that message receives will never
- be successful.
-
-
-RMR_ERR_TIMEOUT
-
- The timeout expired before a complete message was
- received. All other fields in the message buffer are not
- valid.
-
-
-RMR_ERR_EMPTY
-
- A message was received, but it had no payload. All other
- fields in the message buffer are not valid.
-
-
-
-
-INVAL
-
- Parameter(s) passed to the function were not valid.
-
-
-EBADF
-
- The underlying message transport is unable to process the
- request.
-
-
-ENOTSUP
-
- The underlying message transport is unable to process the
- request.
-
-
-EFSM
-
- The underlying message transport is unable to process the
- request.
-
-
-EAGAIN
-
- The underlying message transport is unable to process the
- request.
-
-
-EINTR
-
- The underlying message transport is unable to process the
- request.
-
-
-ETIMEDOUT
-
- The underlying message transport is unable to process the
- request.
-
-
-ETERM
-
- The underlying message transport is unable to process the
- request.
-
-
-EXAMPLE
---------------------------------------------------------------------------------------------
-
-
-SEE ALSO
---------------------------------------------------------------------------------------------
+Route table information is made available to RMR a static
+file (loaded once), or by updates sent from a separate route
+manager application. If a static table is provided, it is
+loaded during RMR initialization and will remain in use until
+an external process connects and delivers a route table
+update (often referred to as a dynamic update). Dynamic
+updates are listened for in a separate process thread and
+applied automatically; the application does not need to allow
+for, or trigger, updates.
+
+Latency And Throughput
+--------------------------------------------------------------------------------------------
+
+While providing insulation from the underlying message
+transport mechanics, RMR must also do so in such a manner
+that message latency and throughput are not impacted. In
+general, the RMR induced overhead, incurred due to the
+process of selecting an endpoint for each message, is minimal
+and should not impact the overall latency or throughput of
+the application. This impact has been measured with test
+applications running on the same physical host and the
+average latency through RMR for a message was on the order of
+0.02 milliseconds.
+
+As an application's throughput increases, it becomes easy for
+the application to overrun the underlying transport mechanism
+(e.g. NNG), consume all available TCP transmit buffers, or
+otherwise find itself in a situation where a send might not
+immediately complete. RMR offers different *modes* which
+allow the application to manage these states based on the
+overall needs of the application. These modes are discussed
+in the *Configuration* section of this document.
+
+General Use
+============================================================================================
-rmr_alloc_msg(3), rmr_call(3), rmr_free_msg(3),
-rmr_get_rcvfd(3), rmr_init(3), rmr_payload_size(3),
-rmr_rcv_msg(3), rmr_send_msg(3), rmr_rcv_specific(3),
-rmr_rts_msg(3), rmr_ready(3), rmr_fib(3), rmr_has_str(3),
-rmr_tokenise(3), rmr_mk_ring(3), rmr_ring_free(3)
+To use, the RMR based application simply needs to initialise
+the RMR environment, wait for RMR to have received a routing
+table (become ready), and then invoke either the send or
+receive functions. These steps, and some behind the scenes
+details, are described in the following paragraphs.
+
+Initialisation
+--------------------------------------------------------------------------------------------
+
+The RMR function is used to set up the RMR environment and
+must be called before messages can be sent or received. One
+of the few parameters that the application must communicate
+to RMR is the port number that will be used as the listen
+port for new connections. The port number is passed on the
+initialisation function call and a TCP listen socket will be
+opened with this port. If the port is already in use RMR will
+report a failure; the application will need to reinitialise
+with a different port number, abort, or take some other
+action appropriate for the application.
+
+In addition to creating a TCP listen port, RMR will start a
+process thread which will be responsible for receiving
+dynamic updates to the route table. This thread also causes a
+TCP listen port to be opened as it is expected that the
+process which generates route table updates will connect and
+send new information when needed. The route table update port
+is **not** supplied by the application, but is supplied via
+an environment variable as this value is likely determined by
+the mechanism which is starting and configuring the
+application.
+
+The RMR Context
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+On successful initialisation, a void pointer, often called a
+*handle* by some programming languages, is returned to the
+application. This is a reference to the RMR control
+information and must be passed as the first parameter on most
+RMR function calls. RMR refers to this as the context, or
+ctx.
-NAME
+Wait For Ready
--------------------------------------------------------------------------------------------
-rmr_trace_ref
+An application which is only receiving messages does not need
+to wait for RMR to *become ready* after the call to the
+initialization function. However, before the application can
+successfully send a message, RMR must have loaded a route
+table, and the application must wait for RMR to report that
+it has done so. The RMR function will return the value *true*
+(1) when a complete route table has been loaded and can be
+used to determine the endpoint for a send request.
-SYNOPSIS
+Receiving Messages
--------------------------------------------------------------------------------------------
+The process of receiving is fairly straight forward. The
+application invokes the RMR function which will block until a
+message is received. The function returns a pointer to a
+message block which provides all of the details about the
+message. Specifically, the application has access to the
+following information either directly or indirectly:
-::
-
- #include <rmr/rmr.h>
- int rmr_trace_ref( rmr_mbuf_t* mbuf, int* sizeptr )
++ The payload (actual data)
++ The total payload length in bytes
-DESCRIPTION
---------------------------------------------------------------------------------------------
++ The number of bytes of the payload which contain valid data
-The rmr_trace_ref function returns a pointer to the trace
-area in the message, and optionally populates the user
-programme supplied size integer with the trace area size, if
-*sizeptr* is not nil.
++ The message type and subscription ID values
-RETURN VALUE
---------------------------------------------------------------------------------------------
++ The hostname and IP address of the source of the message (the sender)
-On success, a void pointer to the trace area of the message
-is returned. A nil pointer is returned if the message has no
-trace data area allocated, or if the message itself is
-invalid.
++ The transaction ID
-SEE ALSO
---------------------------------------------------------------------------------------------
++ Tracing data (if provided)
-rmr_alloc_msg(3), rmr_tralloc_msg(3), rmr_bytes2xact(3),
-rmr_bytes2meid(3), rmr_call(3), rmr_free_msg(3),
-rmr_get_rcvfd(3), rmr_get_trlen(3), rmr_init(3),
-rmr_init_trace(3), rmr_payload_size(3), rmr_send_msg(3),
-rmr_rcv_msg(3), rmr_rcv_specific(3), rmr_rts_msg(3),
-rmr_ready(3), rmr_fib(3), rmr_has_str(3), rmr_tokenise(3),
-rmr_mk_ring(3), rmr_ring_free(3), rmr_str2meid(3),
-rmr_str2xact(3), rmr_wh_open(3), rmr_wh_send_msg(3),
-rmr_set_trace(3)
-NAME
---------------------------------------------------------------------------------------------
+The Message Payload
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-rmr_tralloc_msg
+The message payload contains the *raw* data that was sent by
+the peer application. The format will likely depend on the
+message type, and is expected to be known by the application.
+A direct pointer to the payload is available from the message
+buffer (see appendix B for specific message buffer details).
+
+Two payload-related length values are also directly
+available: the total payload length, and the number of bytes
+actually filled with data. The used length is set by the
+caller, and may or not be an accurate value. The total
+payload length is determined when the buffer is created for
+sending, and is the maximum number of bytes that the
+application may modify should the buffer be used to return a
+response.
-SYNOPSIS
---------------------------------------------------------------------------------------------
+Message Type and Subscription ID
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+The message type and subscription ID are both directly
+available from the message buffer, and are the values which
+were used to by RMR in the sending application to select the
+endpoint. If the application resends the message, as opposed
+to returning the message buffer as a response, the message
+number and/or the subscription ID might need to be changed to
+avoid potential issues[1].
-::
-
- #include <rmr/rmr.h>
- rmr_mbuf_t* rmr_tralloc_msg( void* vctx, int size,
- int trace_size, unsigned const char *tr_data );
+Sender Information
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+The source, or sender information, is indirectly available to
+the application via the and functions. The former returns a
+string containing hostname:port, while the string ip:port is
+returned by the latter.
+Transaction ID
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-DESCRIPTION
---------------------------------------------------------------------------------------------
+The message buffer contains a fixed length set of bytes which
+applications can set to track related messages across the
+application concept of a transaction. RMR will use the
+transaction ID for matching a response message when the
+function is used to send a message.
-The rmr_tralloc_msg function is used to allocate a buffer
-which the user programme can write into and then send through
-the library. The buffer is allocated such that sending it
-requires no additional copying from the buffer as it passes
-through the underlying transport mechanism.
+Trace Information
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-The *size* parameter is used to set the payload length in the
-message. If it is 0, then the default size supplied on the
-*rmr_init* call will be used. In addition to allocating the
-payload, a space in the buffer is reserved for *trace* data
-(tr_size bytes), and the bytes pointed to by *tr_data* are
-copied into that portion of the message. The *vctx* parameter
-is the void context pointer that was returned by the
-*rmr_init* function.
+RMR supports the addition of an optional trace information to
+any message. The presence and size is controlled by the
+application, and can vary from message to message if desired.
+The actual contents of the trace information is determined by
+the application; RMR provides only the means to set, extract,
+and obtain a direct reference to the trace bytes. The trace
+data field in a message buffer is discussed in greater detail
+in the *Trace Data* section.
-The pointer to the message buffer returned is a structure
-which has some user application visible fields; the structure
-is described in rmr.h, and is illustrated below.
+Sending Messages
+--------------------------------------------------------------------------------------------
+Sending requires only slightly more work on the part of the
+application than receiving a message. The application must
+allocate an RMR message buffer, populate the message payload
+with data, set the message type and length, and optionally
+set the subscription ID. Information such as the source IP
+address, hostname, and port are automatically added to the
+message buffer by RMR, so there is no need for the
+application to worry about these.
-::
-
- typedef struct {
- int state;
- int mtype;
- int len;
- unsigned char* payload;
- unsigned char* xaction;
- } rmr_mbuf_t;
+Message Buffer Allocation
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+The function allocates a *zero copy* buffer and returns a
+pointer to the RMR rmr_mbuf_t structure. The message buffer
+provides direct access to the payload, length, message type
+and subscription ID fields. The buffer must be preallocated
+in order to allow the underlying transport mechanism to
+allocate the payload space from its internal memory pool;
+this eliminates multiple copies as the message is sent, and
+thus is more efficient.
+
+If a message buffer has been received, and the application
+wishes to use the buffer to send a response, or to forward
+the buffer to another application, a new buffer does **not**
+need to be allocated. The application may set the necessary
+information (message type, etc.), and adjust the payload, as
+is necessary and then pass the message buffer to or to be
+sent or returned to the sender.
+
+Populating the Message Buffer
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+The application has direct access to several of the message
+buffer fields, and should set them appropriately.
-state
+len
- Is the current buffer state. Following a call to
- rmr_send_msg the state indicates whether the buffer was
- successfully sent which determines exactly what the
- payload points to. If the send failed, the payload
- referenced by the buffer is the message that failed to
- send (allowing the application to attempt a
- retransmission). When the state is a_OK the buffer
- represents an empty buffer that the application may fill
- in in preparation to send.
+ This is the number of bytes that the application placed
+ into the payload. Setting length to 0 is allowed, and
+ length may be less than the allocated payload size.
mtype
- When sending a message, the application is expected to set
- this field to the appropriate message type value (as
- determined by the user programme). Upon send this value
- determines how the a library will route the message. For a
- buffer which has been received, this field will contain
- the message type that was set by the sending application.
+ The message type that RMR will use to determine the
+ endpoint used as the target of the send.
-len
+sub_id
- The application using a buffer to send a message is
- expected to set the length value to the actual number of
- bytes that it placed into the message. This is likely less
- than the total number of bytes that the message can carry.
- For a message buffer that is passed to the application as
- the result of a receive call, this will be the value that
- the sending application supplied and should indicate the
- number of bytes in the payload which are valid.
+ The subscription ID if the message is to be routed based
+ on the combination of message type and subscription ID. If
+ no subscription ID is valid for the message, the
+ application should set the field with the RMR constant
+ RMR_VOID_SUBID.
payload
- The payload is a pointer to the actual received data. The
- user programme may read and write from/to the memory
- referenced by the payload up until the point in time that
- the buffer is used on a rmr_send, rmr_call or rmr_reply
- function call. Once the buffer has been passed back to a a
- library function the user programme should **NOT** make
- use of the payload pointer.
+ The application should obtain the reference (pointer) to
+ the payload from the message buffer and place any data
+ into the payload. The application is responsible for
+ ensuring that the maximum payload size is not exceeded.
+ The application may obtain the maximum size via the
+ function.
-xaction
+trace data
- The *xaction* field is a pointer to a fixed sized area in
- the message into which the user may write a transaction
- ID. The ID is optional with the exception of when the user
- application uses the rmr_call function to send a message
- and wait for the reply; the underlying processing expects
- that the matching reply message will also contain the same
- data in the *xaction* field.
+ Optionally, the application may add trace information to
+ the message buffer.
-RETURN VALUE
---------------------------------------------------------------------------------------------
-The function returns a pointer to a rmr_mbuf structure, or
-NULL on error.
+Sending a Message Buffer
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Once the application has populated the necessary bits of a
+message, it may be sent by passing the buffer to the
+function. This function will select an endpoint to receive
+the message, based on message type and subscription ID, and
+will pass the message to the underlying transport mechanism
+for actual transmission on the connection. (Depending on the
+underlying transport mechanism, the actual connection to the
+endpoint may happen at the time of the first message sent to
+the endpoint, and thus the latency of the first send might be
+longer than expected.)
+
+On success, the send function will return a reference to a
+message buffer; the status within that message buffer will
+indicate what the message buffer contains. When the status is
+RMR_OK the reference is to a **new** message buffer for the
+application to use for the next send; the payload size is the
+same as the payload size allocated for the message that was
+just sent. This is a convenience as it eliminates the need
+for the application to call the message allocation function
+at some point in the future, and assumes the application will
+send many messages which will require the same payload
+dimensions.
+
+If the message contains any status other than RMR_OK, then
+the message could **not** be sent, and the reference is to
+the unsent message buffer. The value of the status will
+indicate whether the nature of the failure was transient (
+RMR_ERR_RETRY) or not. Transient failures are likely to be
+successful if the application attempts to send the message at
+a later time. Unfortunately, it is impossible for RMR to know
+the exact transient failure (e.g. connection being
+established, or TCP buffer shortage), and thus it is not
+possible to communicate how long the application should wait
+before attempting to resend, if the application wishes to
+resend the message. (More discussion with respect to message
+retries can be found in the *Handling Failures* section.)
+
+Advanced Usage
+============================================================================================
+
+Several forms of usage fall into a more advanced category and
+are described in the following sections. These include
+blocking call, return to sender and wormhole functions.
-ERRORS
+The Call Function
--------------------------------------------------------------------------------------------
+The RMR function sends a message in the exact same manner as
+the rmr_send_msg() function, with the endpoint selection
+based on the message key. But unlike the send function, will
+block and wait for a response from the application that is
+selected to receive the message. The matching message is
+determined by the transaction ID which the application must
+place into the message buffer prior to invoking. Similarly,
+the responding application must ensure that the same
+transaction ID is placed into the message buffer before
+returning its response.
+The return from the call is a message buffer with the
+response message; there is no difference between a message
+buffer returned by the receive function and one returned by
+the function. If a response is not received in a reasonable
+amount of time, a nil message buffer is returned to the
+calling application.
-ENOMEM
-
- Unable to allocate memory.
+Returning a Response
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+Because of the nature of RMR's routing policies, it is
+generally not possible for an application to control exactly
+which endpoint is sent a message. There are cases, such as
+responding to a message delivered via that the application
+must send a message and guarantee that RMR routes it to an
+exact destination. To enable this, RMR provides the return to
+sender, function. Upon receipt of any message, an application
+may alter the payload, and if necessary the message type and
+subscription ID, and pass the altered message buffer to the
+function to return the altered message to the application
+which sent it. When this function is used, RMR will examine
+the message buffer for the source information and use that to
+select the connection on which to write the response.
+
+Multi-threaded Calls
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-SEE ALSO
---------------------------------------------------------------------------------------------
+The basic call mechanism described above is **not** thread
+safe, as it is not possible to guarantee that a response
+message is delivered to the correct thread. The RMR function
+accepts an additional parameter which identifies the calling
+thread in order to ensure that the response is delivered
+properly. In addition, the application must specifically
+initialise the multi-threaded call environment by passing the
+RMRFL_MTCALL flag as an option to the function[2].
-rmr_alloc_msg(3), rmr_mbuf(3) rmr_call(3), rmr_free_msg(3),
-rmr_init(3), rmr_init_trace(3), rmr_get_trace(3),
-rmr_get_trlen(3), rmr_payload_size(3), rmr_send_msg(3),
-rmr_rcv_msg(3), rmr_rcv_specific(3), rmr_rts_msg(3),
-rmr_ready(3), rmr_fib(3), rmr_has_str(3), rmr_tokenise(3),
-rmr_mk_ring(3), rmr_ring_free(3), rmr_set_trace(3)
+One advantage of the multi-threaded call capability in RMR is
+the fact that only the calling thread is blocked. Messages
+received which are not responses to the call are continued to
+be delivered via normal calls.
+While the process is blocked waiting for the response, it is
+entirely possible that asynchronous, non-matching, messages
+will arrive. When this happens, RMR will queues the messages
+and return them to the application over the next calls to
-NAME
+Wormholes
--------------------------------------------------------------------------------------------
-rmr_wh_call
+As was mentioned earlier, the design of RMR is to eliminate
+the need for an application to know a specific endpoint, even
+when a response message is being sent. In some rare cases it
+may be necessary for an application to establish a direct
+connection to an RMR-based application rather than relying on
+message type and subscription ID based routing. The
+*wormhole* functions provide an application with the ability
+to create a direct connection and then to send and receive
+messages across the connection. The following are the RMR
+functions which provide wormhole communications:
-SYNOPSIS
---------------------------------------------------------------------------------------------
-::
-
- #include <rmr/rmr.h>
- rmr_mbuf_t* rmr_wh_call( void* vctx, rmr_whid_t whid, rmr_mbuf_t* msg, int call_id, int max_wait )
+rmr_wh_open
+
+ Open a connection to an endpoint. Name or IP address and
+ port of the endpoint is supplied. Returns a wormhole ID
+ that the application must use when sending a direct
+ message.
+
+
+rmr_wh_send_msg
+
+ Sends an RMR message buffer to the connected application.
+ The message type and subscription ID may be set in the
+ message, but RMR will ignore both.
+
+rmr_wh_close
+
+ Closes the direct connection.
-DESCRIPTION
---------------------------------------------------------------------------------------------
-The rmr_wh_call function accepts a message buffer (msg) from
-the user application and attempts to send it using the
-wormhole ID provided (whid). If the send is successful, the
-call will block until either a response message is received,
-or the max_wait number of milliseconds has passed. In order
-for the response to be recognised as a response, the remote
-process **must** use rmr_rts_msg() to send their response.
-
-Like *rmr_wh_send_msg,* this function attempts to send the
-message directly to a process at the other end of a wormhole
-which was created with *rmr_wh_open().* When sending message
-via wormholes, the normal RMR routing based on message type
-is ignored, and the caller may leave the message type
-unspecified in the message buffer (unless it is needed by the
-receiving process). The call_id parameter is a number in the
-range of 2 through 255 and is used to identify the calling
-thread in order to properly match a response message when it
-arrives. Providing this value, and ensuring the proper
-uniqueness, is the responsibility of the user application and
-as such the ability to use the rmr_wh_call() function from
-potentially non-threaded concurrent applications (such as
-Go's goroutines) is possible.
-
-Retries
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+Handling Failures
+============================================================================================
-The send operations in RMR will retry *soft* send failures
-until one of three conditions occurs:
+The vast majority of states reported by RMR are fatal; if
+encountered during setup or initialization, then it is
+unlikely that any message oriented processing should
+continue, and when encountered on a message operation
+continued operation on that message should be abandoned.
+Specifically with regard to message sending, it is very
+likely that the underlying transport mechanism will report a
+*soft,* or transient, failure which might be successful if
+the operation is retried at a later point in time. The
+paragraphs below discuss the methods that an application
+might deal with these soft failures.
+Failure Notification
+--------------------------------------------------------------------------------------------
+When a soft failure is reported, the returned message buffer
+returned by the RMR function will be RMR_ERR_RETRY. These
+types of failures can occur for various reasons; one of two
+reasons is typically the underlying cause:
-1.
-
- The message is sent without error
-
-2.
-
- The underlying transport reports a *hard* failure
-
++ The session to the targeted recipient (endpoint) is not connected.
-3.
-
- The maximum number of retry loops has been attempted
++ The transport mechanism buffer pool is full and cannot accept another buffer.
-A retry loop consists of approximately 1000 send attempts
-**without** any intervening calls to *sleep()* or *usleep().*
-The number of retry loops defaults to 1, thus a maximum of
-1000 send attempts is performed before returning to the user
-application. This value can be set at any point after RMR
-initialisation using the *rmr_set_stimeout()* function
-allowing the user application to completely disable retires
-(set to 0), or to increase the number of retry loops.
-Transport Level Blocking
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+Unfortunately, it is not possible for RMR to determine which
+of these two cases is occurring, and equally as unfortunate
+the time to resolve each is different. The first, no
+connection, may require up to a second before a message can
+be accepted, while a rejection because of buffer shortage is
+likely to resolve in less than a millisecond.
-The underlying transport mechanism used to send messages is
-configured in *non-blocking* mode. This means that if a
-message cannot be sent immediately the transport mechanism
-will **not** pause with the assumption that the inability to
-send will clear quickly (within a few milliseconds). This
-means that when the retry loop is completely disabled (set to
-0), that the failure to accept a message for sending by the
-underlying mechanisms (software or hardware) will be reported
-immediately to the user application.
-
-It should be noted that depending on the underlying transport
-mechanism being used, it is extremely likely that retry
-conditions will happen during normal operations. These are
-completely out of RMR's control, and there is nothing that
-RMR can do to avoid or mitigate these other than by allowing
-RMR to retry the send operation, and even then it is possible
-(e.g., during connection reattempts), that a single retry
-loop is not enough to guarantee a successful send.
-
-RETURN VALUE
+Application Response
--------------------------------------------------------------------------------------------
-On success, new message buffer, with the payload containing
-the response from the remote endpoint is returned. The state
-in this buffer will reflect the overall send operation state
-and should be RMR_OK.
+The action which an application takes when a soft failure is
+reported ultimately depends on the nature of the application
+with respect to factors such as tolerance to extended message
+latency, dropped messages, and over all message rate.
+
+RMR Retry Modes
+--------------------------------------------------------------------------------------------
+
+In an effort to reduce the workload of an application
+developer, RMR has a default retry policy such that RMR will
+attempt to retransmit a message up to 1000 times when a soft
+failure is reported. These retries generally take less than 1
+millisecond (if all 1000 are attempted) and in most cases
+eliminates nearly all reported soft failures to the
+application. When using this mode, it might allow the
+application to simply treat all bad return values from a send
+attempt as permanent failures.
+
+If an application is so sensitive to any delay in RMR, or the
+underlying transport mechanism, it is possible to set RMR to
+return a failure immediately on any kind of error (permanent
+failures are always reported without retry). In this mode,
+RMR will still set the state in the message buffer to
+RMR_ERR_RETRY, but will **not** make any attempts to resend
+the message. This zero-retry policy is enabled by invoking
+the with a value of 0; this can be done once immediately
+after is invoked.
-If a message is returned with a state which is anything other
-than RMR_OK, the indication is that the send was not
-successful. The user application must check the state and
-determine the course of action. If the return value is NULL,
-no message, the indication is that there was no response
-received within the timeout (max_wait) period of time.
+Regardless of the retry mode which the application sets, it
+will ultimately be up to the application to handle failures
+by queuing the message internally for resend, retrying
+immediately, or dropping the send attempt all together. As
+stated before, only the application can determine how to best
+handle send failures.
-ERRORS
+Other Failures
--------------------------------------------------------------------------------------------
-The following values may be passed back in the *state* field
-of the returned message buffer.
+RMR will return the state of processing for message based
+operations (send/receive) as the status in the message
+buffer. For non-message operations, state is returned to the
+caller as the integer return value for all functions which
+are not expected to return a pointer (e.g. and a brief
+description of their meaning.
-RMR_ERR_WHID
+RMR_OK
- The wormhole ID passed in was not associated with an open
- wormhole, or was out of range for a valid ID.
-
-RMR_ERR_NOWHOPEN
+ state is good; operation finished successfully
- No wormholes exist, further attempt to validate the ID are
- skipped.
RMR_ERR_BADARG
- The message buffer pointer did not refer to a valid
- message.
+ argument passed to function was unusable
+
+
+RMR_ERR_NOENDPT
+
+ send/call could not find an endpoint based on msg type
+
+
+RMR_ERR_EMPTY
+
+ msg received had no payload; attempt to send an empty
+ message
+
RMR_ERR_NOHDR
- The header in the message buffer was not valid or
- corrupted.
+ message didn't contain a valid header
+
+RMR_ERR_SENDFAILED
+
+ send failed; errno may contain the transport provider
+ reason
+
-EXAMPLE
---------------------------------------------------------------------------------------------
+RMR_ERR_CALLFAILED
+
+ unable to send the message for a call function; errno may
+ contain the transport provider reason
+
-The following is a simple example of how the a wormhole is
-created (rmr_wh_open) and then how rmr_wh_send_msg function
-is used to send messages. Some error checking is omitted for
-clarity.
+RMR_ERR_NOWHOPEN
+
+ no wormholes are open
+
+RMR_ERR_WHID
+
+ the wormhole id provided was invalid
+
-::
-
- #include <rmr/rmr.h> // system headers omitted for clarity
- int main() {
- rmr_whid_t whid = -1; // wormhole id for sending
- void* mrc; //msg router context
- int i;
- rmr_mbuf_t* sbuf; // send buffer
- int count = 0;
- int norm_msg_size = 1500; // most messages fit in this size
- mrc = rmr_init( "43086", norm_msg_size, RMRFL_NONE );
- if( mrc == NULL ) {
- fprintf( stderr, "[FAIL] unable to initialise RMR environment\\n" );
- exit( 1 );
- }
- while( ! rmr_ready( mrc ) ) { // wait for routing table info
- sleep( 1 );
- }
- sbuf = rmr_alloc_msg( mrc, 2048 );
- while( 1 ) {
- if( whid < 0 ) {
- whid = rmr_wh_open( mrc, "localhost:6123" ); // open fails if endpoint refuses conn
- if( RMR_WH_CONNECTED( wh ) ) {
- snprintf( sbuf->payload, 1024, "periodic update from sender: %d", count++ );
- sbuf->len = strlen( sbuf->payload );
- sbuf = rmr_wh_call( mrc, whid, sbuf, 1000 ); // expect a response in 1s or less
- if( sbuf != NULL && sbuf->state = RMR_OK ) {
- sprintf( stderr, "response: %s\\n", sbuf->payload ); // assume they sent a string
- } else {
- sprintf( stderr, "response not received, or send error\\n" );
- }
- }
- }
- sleep( 5 );
- }
- }
+RMR_ERR_OVERFLOW
+
+ operation would have busted through a buffer/field size
+
+RMR_ERR_RETRY
+
+ request (send/call/rts) failed, but caller should retry
+ (EAGAIN for wrappers)
+
+RMR_ERR_RCVFAILED
+
+ receive failed (hard error)
+
-SEE ALSO
---------------------------------------------------------------------------------------------
+RMR_ERR_TIMEOUT
+
+ response message not received in a reasonable amount of
+ time
+
-rmr_alloc_msg(3), rmr_call(3), rmr_free_msg(3), rmr_init(3),
-rmr_payload_size(3), rmr_rcv_msg(3), rmr_rcv_specific(3),
-rmr_rts_msg(3), rmr_ready(3), rmr_fib(3), rmr_has_str(3),
-rmr_tokenise(3), rmr_mk_ring(3), rmr_ring_free(3),
-rmr_set_stimeout(3), rmr_wh_open(3), rmr_wh_close(3),
-rmr_wh_state(3)
+RMR_ERR_UNSET
+
+ the message hasn't been populated with a transport buffer
+
+RMR_ERR_TRUNC
+
+ length in the received buffer is longer than the size of
+ the allocated payload, received message likely truncated
+ (length set by sender could be wrong, but we can't know
+ that)
+
-NAME
---------------------------------------------------------------------------------------------
+RMR_ERR_INITFAILED
+
+ initialisation of something (probably message) failed
+
-rmr_wh_close
+RMR_ERR_NOTSUPP
+
+ the request is not supported, or RMR was not initialised
+ for the request
-SYNOPSIS
---------------------------------------------------------------------------------------------
+Depending on the underlying transport mechanism, and the
+nature of the call that RMR attempted, the system errno value
+might reflect additional detail about the failure.
+Applications should **not** rely on errno as some transport
+mechanisms do not set it with any consistency.
-::
-
- #include <rmr/rmr.h>
- void rmr_close( void* vctx, rmr_whid_t whid )
+Configuration and Control
+============================================================================================
+With the assumption that most RMR based applications will be
+executed in a containerised environment, there are some
+underlying mechanics which the developer may need to know in
+order to properly provide a configuration specification to
+the container management system. The following paragraphs
+briefly discuss these.
-DESCRIPTION
+TCP Ports
--------------------------------------------------------------------------------------------
-The rmr_wh_close function closes the wormhole associated with
-the wormhole id passed in. Future calls to rmr_wh_send_msg
-with this ID will fail.
+RMR requires two (2) TCP listen ports: one for general
+application-to-application communications and one for
+route-table updates. The general communication port is
+specified by the application at the time RMR is initialised.
+The port used to listen for route table updates is likely to
+be a constant port shared by all applications provided they
+are running in separate containers. To that end, the port
+number defaults to 4561, but can be configured with an
+environment variable (see later paragraph in this section).
-The underlying TCP connection to the remote endpoint is
-**not** closed as this session may be required for regularly
-routed messages (messages routed based on message type).
-There is no way to force a TCP session to be closed at this
-point in time.
-
-SEE ALSO
+Host Names
--------------------------------------------------------------------------------------------
-rmr_alloc_msg(3), rmr_call(3), rmr_free_msg(3),
-rmr_get_rcvfd(3), rmr_payload_size(3), rmr_send_msg(3),
-rmr_rcv_msg(3), rmr_rcv_specific(3), rmr_rts_msg(3),
-rmr_ready(3), rmr_fib(3), rmr_has_str(3), rmr_tokenise(3),
-rmr_mk_ring(3), rmr_ring_free(3), rmr_wh_open(3),
-rmr_wh_send_msg(3)
-
+RMR is typically host name agnostic. Route table entries may
+contain endpoints defined either by host name or IP address.
+In the container world the concept of a *service name* might
+exist, and likely is different than a host name. RMR's only
+requirement with respect to host names is that a name used on
+a route table entry must be resolvable via the gethostbyname
+system call.
-NAME
+Environment Variables
--------------------------------------------------------------------------------------------
-rmr_wh_open
+Several environment variables are recognised by RMR which, in
+general, are used to define interfaces and listen ports (e.g.
+the route table update listen port), or debugging
+information. Generally this information is system controlled
+and thus RMR expects this information to be defined in the
+environment rather than provided by the application. The
+following is a list of the environment variables which RMR
+recognises:
-SYNOPSIS
---------------------------------------------------------------------------------------------
-::
-
- #include <rmr/rmr.h>
- void* rmr_wh_open( void* vctx, char* target )
+RMR_BIND_IF
+
+ The interface to bind to listen ports to. If not defined
+ 0.0.0.0 (all interfaces) is assumed.
+
+RMR_RTG_SVC
+
+ The port RMR will listen on for route manager connections.
+ If not defined 4561 is used.
+
+RMR_SEED_RT
+
+ Where RMR expects to find the name of the seed (static)
+ route table. If not defined no static table is read.
+
-DESCRIPTION
---------------------------------------------------------------------------------------------
+RMR_RTG_ISRAW
+
+ If the value set to 0, RMR expects the route table manager
+ messages to be messages with and RMR header. If this is
+ not defined messages are assumed to be "raw" (without an
+ RMR header.
+
+
+RMR_VCTL_FILE
+
+ Provides a file which is used to set the verbose level of
+ the route table collection thread. The first line of the
+ file is read and expected to contain an integer value to
+ set the verbose level. The value may be changed at any
+ time and the route table thread will adjust accordingly.
+
+
+RMR_SRC_NAMEONLY
+
+ If the value of this variable is greater than 0, RMR will
+ not permit the IP address to be sent as the message
+ source. Only the host name will be sent as the source in
+ the message header.
-The rmr_wh_open function creates a direct link for sending, a
-wormhole, to another RMR based process. Sending messages
-through a wormhole requires that the connection be
-established overtly by the user application (via this
-function), and that the ID returned by rmr_wh_open be passed
-to the rmr_wh_send_msg function.
-
-*Target* is the *name* or *IP-address* combination of the
-processess that the wormhole should be connected to. *Vctx*
-is the RMR void context pointer that was returned by the
-rmr_init function.
-
-When invoked, this function immediatly attempts to connect to
-the target process. If the connection cannot be established,
-an error is returned to the caller, and no direct messages
-can be sent to the target. Once a wormhole is connected, the
-underlying transport mechanism (e.g. NNG) will provide
-reconnects should the connection be lost, however the
-handling of messages sent when a connection is broken is
-undetermined as each underlying transport mechanism may
-handle buffering and retries differently.
-
-RETURN VALUE
---------------------------------------------------------------------------------------------
-The rmr_wh_open function returns a type rmr_whid_t which must
-be passed to the rmr_wh_send_msg function when sending a
-message. The id may also be tested to determine success or
-failure of the connection by using the RMR_WH_CONNECTED macro
-and passing the ID as the parameter; a result of 1 indicates
-that the connection was esablished and that the ID is valid.
-ERRORS
+Logging
--------------------------------------------------------------------------------------------
-The following error values are specifically set by this RMR
-function. In some cases the error message of a system call is
-propagated up, and thus this list might be incomplete.
+RMR does **not** use any logging libraries; any error or
+warning messages are written to standard error. RMR messages
+are written with one of three prefix strings:
-EINVAL
+
+[CRI]
+
+ The event is of a critical nature and it is unlikely that
+ RMR will continue to operate correctly if at all. It is
+ almost certain that immediate action will be needed to
+ resolve the issue.
- A parameter passed was not valid.
-EACCESS
+[ERR]
+
+ The event is not expected and RMR is not able to handle
+ it. There is a small chance that continued operation will
+ be negatively impacted. Eventual action to diagnose and
+ correct the issue will be necessary.
- The user application does not have the ability to
- establish a wormhole to the indicated target (or maybe any
- target).
-ECONNREFUSED
+[WRN]
- The connection was refused.
+ The event was not expected by RMR, but can be worked
+ round. Normal operation will continue, but it is
+ recommended that the cause of the problem be investigated.
-EXAMPLE
---------------------------------------------------------------------------------------------
+_____________________________________________________________
-::
-
- void* rmc;
- rmr_whid_t wh;
- rmc = rmr_init( "43086", 4096, 0 ); // init context
- wh = rmr_wh_open( rmc, "localhost:6123" );
- if( !RMR_WH_CONNECTED( wh ) ) {
- fprintf( stderr, "unable to connect wormhole: %s\\n",
- strerror( errno ) );
- }
+[1] It is entirely possible to design a routing table, and
+application group, such that the same message type is is
+left unchanged and the message is forwarded by an
+application after updating the payload. This type of
+behaviour is often referred to as service chaining, and can
+be done without any "knowledge" by an application with
+respect to where the message goes next. Service chaining is
+supported by RMR in as much as it allows the message to be
+resent, but the actual complexities of designing and
+implementing service chaining lie with the route table
+generator process.
-SEE ALSO
---------------------------------------------------------------------------------------------
+[2] There is additional overhead to support multi-threaded
+call as a special listener thread must be used in order to
+deliver responses to the proper application thread.
-rmr_alloc_msg(3), rmr_call(3), rmr_free_msg(3),
-rmr_get_rcvfd(3), rmr_payload_size(3), rmr_send_msg(3),
-rmr_rcv_msg(3), rmr_rcv_specific(3), rmr_rts_msg(3),
-rmr_ready(3), rmr_fib(3), rmr_has_str(3), rmr_tokenise(3),
-rmr_mk_ring(3), rmr_ring_free(3), rmr_wh_close(3),
-rmr_wh_send_msg(3), rmr_wh_state(3)
-NAME
---------------------------------------------------------------------------------------------
-rmr_wh_send_msg
-SYNOPSIS
---------------------------------------------------------------------------------------------
+Appendix A -- Quick Reference
+============================================================================================
-::
-
- #include <rmr/rmr.h>
- rmr_mbuf_t* rmr_wh_send_msg( void* vctx, rmr_whid_t id, rmr_mbuf_t* msg );
+Please refer to the RMR manual pages on the Read the Docs
+site
+
+https://docs.o-ran-sc.org/projects/o-ran-sc-ric-plt-lib-rmr/en/latest/index.html
+Appendix B -- Message Buffer Details
+============================================================================================
+
+The RMR message buffer is a C structure which is exposed in
+the rmr.h header file. It is used to manage a message
+received from a peer endpoint, or a message that is being
+sent to a peer. Fields include payload length, amount of
+payload actually used, status, and a reference to the
+payload. There are also fields which the application should
+ignore, and could be hidden in the header file, but we chose
+not to. These fields include a reference to the RMR header
+information, and to the underlying transport mechanism
+message struct which may or may not be the same as the RMR
+header reference.
-DESCRIPTION
+The Structure
--------------------------------------------------------------------------------------------
-The rmr_wh_send_msg function accepts a message buffer from
-the user application and attempts to send it using the
-wormhole ID provided (id). Unlike *rmr_send_msg,* this
-function attempts to send the message directly to a process
-at the other end of a wormhole which was created with
-*rmr_wh_open().* When sending message via wormholes, the
-normal RMR routing based on message type is ignored, and the
-caller may leave the message type unspecified in the message
-buffer (unless it is needed by the receiving process).
-
-The message buffer (msg) used to send is the same format as
-used for regular RMR send and reply to sender operations,
-thus any buffer allocated by these means, or calls to
-*rmr_rcv_msg()* can be passed to this function.
-
-Retries
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+The following is the C structure. Readers are cautioned to
+examine the rmr.h header file directly; the information here
+may be out of date (old document in some cache), and thus it
+may be incorrect.
-The send operations in RMR will retry *soft* send failures
-until one of three conditions occurs:
+::
+
+ typedef struct {
+ int state; // state of processing
+ int mtype; // message type
+ int len; // length of data in the payload (send or received)
+ unsigned char* payload; // transported data
+ unsigned char* xaction; // pointer to fixed length transaction id bytes
+ int sub_id; // subscription id
+ int tp_state; // transport state (errno)
+ // these things are off limits to the user application
+ void* tp_buf; // underlying transport allocated pointer (e.g. nng message)
+ void* header; // internal message header (whole buffer: header+payload)
+ unsigned char* id; // if we need an ID in the message separate from the xaction id
+ int flags; // various MFL_ (private) flags as needed
+ int alloc_len; // the length of the allocated space (hdr+payload)
+ void* ring; // ring this buffer should be queued back to
+ int rts_fd; // SI fd for return to sender
+ int cookie; // cookie to detect user misuse of free'd msg
+ } rmr_mbuf_t;
-1.
-
- The message is sent without error
-
-2.
-
- The underlying transport reports a *hard* failure
-
-3.
-
- The maximum number of retry loops has been attempted
+State vs Transport State
+--------------------------------------------------------------------------------------------
+The state field reflects the state at the time the message
+buffer is returned to the calling application. For a send
+operation, if the state is not RMR_OK then the message buffer
+references the payload that could not be sent, and when the
+state is RMR_OK the buffer references a *fresh* payload that
+the application may fill in.
-A retry loop consists of approximately 1000 send attempts
-**without** any intervening calls to *sleep()* or *usleep().*
-The number of retry loops defaults to 1, thus a maximum of
-1000 send attempts is performed before returning to the user
-application. This value can be set at any point after RMR
-initialisation using the *rmr_set_stimeout()* function
-allowing the user application to completely disable retires
-(set to 0), or to increase the number of retry loops.
+When the state is not RMR_OK, C programmes may examine the
+global errno value which RMR will have left set, if it was
+set, by the underlying transport mechanism. In some cases,
+wrapper modules are not able to directly access the C-library
+errno value, and to assist with possible transport error
+details, the send and receive operations populate tp_state
+with the value of errno.
-Transport Level Blocking
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+Regardless of whether the application makes use of the
+tp_state, or the errno value, it should be noted that the
+underlying transport mechanism may not actually update the
+errno value; in other words: it might not be accurate. In
+addition, RMR populates the tp_state value in the message
+buffer **only** when the state is not RMR_OK.
-The underlying transport mechanism used to send messages is
-configured in *non-blocking* mode. This means that if a
-message cannot be sent immediately the transport mechanism
-will **not** pause with the assumption that the inability to
-send will clear quickly (within a few milliseconds). This
-means that when the retry loop is completely disabled (set to
-0), that the failure to accept a message for sending by the
-underlying mechanisms (software or hardware) will be reported
-immediately to the user application.
-
-It should be noted that depending on the underlying transport
-mechanism being used, it is extremely likely that retry
-conditions will happen during normal operations. These are
-completely out of RMR's control, and there is nothing that
-RMR can do to avoid or mitigate these other than by allowing
-RMR to retry the send operation, and even then it is possible
-(e.g., during connection reattempts), that a single retry
-loop is not enough to guarantee a successful send.
-
-RETURN VALUE
+Field References
--------------------------------------------------------------------------------------------
-On success, a new message buffer, with an empty payload, is
-returned for the application to use for the next send. The
-state in this buffer will reflect the overall send operation
-state and should be RMR_OK.
+The transaction field was exposed in the first version of
+RMR, and in hindsight this shouldn't have been done. Rather
+than break any existing code the reference was left, but
+additional fields such as trace data, were not directly
+exposed to the application. The application developer is
+strongly encouraged to use the functions which get and set
+the transaction ID rather than using the pointer directly;
+any data overruns will not be detected if the reference is
+used directly.
-If the state in the returned buffer is anything other than
-RMR_OK, the user application may need to attempt a
-retransmission of the message, or take other action depending
-on the setting of errno as described below.
+In contrast, the payload reference should be used directly by
+the application in the interest of speed and ease of
+programming. The same care to prevent writing more bytes to
+the payload buffer than it can hold must be taken by the
+application. By the nature of the allocation of the payload
+in transport space, RMR is unable to add guard bytes and/or
+test for data overrun.
-In the event of extreme failure, a nil pointer is returned.
-In this case the value of errno might be of some use, for
-documentation, but there will be little that the user
-application can do other than to move on.
-
-ERRORS
+Actual Transmission
--------------------------------------------------------------------------------------------
-The following values may be passed back in the *state* field
-of the returned message buffer.
+When RMR sends the application's message, the message buffer
+is **not** transmitted. The transport buffer (tp_buf) which
+contains the RMR header and application payload is the only
+set of bytes which are transmitted. While it may seem to the
+caller like the function is returning a new message buffer,
+the same struct is reused and only a new transport buffer is
+allocated. The intent is to keep the alloc/free cycles to a
+minimum.
+
+Appendix C -- Glossary
+============================================================================================
+Many terms in networking can be interpreted with multiple
+meanings, and several terms used in this document are RMR
+specific. The following definitions are the meanings of terms
+used within this document and should help the reader to
+understand the intent of meaning.
-RMR_ERR_WHID
+
+
+application
+
+ A programme which uses RMR to send and/or receive messages
+ to/from another RMR based application.
- The wormhole ID passed in was not associated with an open
- wormhole, or was out of range for a valid ID.
-RMR_ERR_NOWHOPEN
+Critical error
+
+ An error that RMR has encountered which will prevent
+ further successful processing by RMR. Critical errors
+ usually indicate that the application should abort.
- No wormholes exist, further attempt to validate the ID are
- skipped.
-RMR_ERR_BADARG
+Endpoint
+
+ An RMR based application that is defined as being capable
+ of receiving one or more types of messages (as defined by
+ a *message key.*)
- The message buffer pointer did not refer to a valid
- message.
-RMR_ERR_NOHDR
+Environment variable
+
+ A key/value pair which is set externally to the
+ application, but which is available to the application
+ (and referenced libraries) through the getenv system call.
+ Environment variables are the main method of communicating
+ information such as port numbers to RMR.
- The header in the message buffer was not valid or
- corrupted.
+Error
+
+ An abnormal condition that RMR has encountered, but will
+ not affect the overall processing by RMR, but may impact
+ certain aspects such as the ability to communicate with a
+ specific endpoint. Errors generally indicate that
+ something, usually external to RMR, must be addressed.
+
-The following values may be assigned to errno on failure.
+Host name
+
+ The name of the host as returned by the gethostbyname
+ system call. In a containerised environment this might be
+ the container or service name depending on how the
+ container is started. From RMR's point of view, a host
+ name can be used to resolve an *endpoint* definition in a
+ *route* table.)
+
+IP
+
+ Internet protocol. A low level transmission protocol which
+ governs the transmission of datagrams across network
+ boundaries.
+
-INVAL
+Listen socket
- Parameter(s) passed to the function were not valid, or the
- underlying message processing environment was unable to
- interpret the message.
+ A *TCP* socket used to await incoming connection requests.
+ Listen sockets are defined by an interface and port number
+ combination where the port number is unique for the
+ interface.
-ENOKEY
+Message
- The header information in the message buffer was invalid.
+ A series of bytes transmitted from the application to
+ another RMR based application. A message is comprised of
+ RMR specific data (a header), and application data (a
+ payload).
-ENXIO
+Message buffer
- No known endpoint for the message could be found.
+ A data structure used to describe a message which is to be
+ sent or has been received. The message buffer includes the
+ payload length, message type, message source, and other
+ information.
-EMSGSIZE
+Messgae type
- The underlying transport refused to accept the message
- because of a size value issue (message was not attempted
- to be sent).
+ A signed integer (0-32000) which identifies the type of
+ message being transmitted, and is one of the two
+ components of a *routing key.* See *Subscription ID.*
-EFAULT
+Payload
- The message referenced by the message buffer is corrupt
- (nil pointer or bad internal length).
+ The portion of a message which holds the user data to be
+ transmitted to the remote *endpoint.* The payload contents
+ are completely application defined.
-EBADF
+RMR context
- Internal RMR error; information provided to the message
- transport environment was not valid.
+ A set of information which defines the current state of
+ the underlying transport connections that RMR is managing.
+ The application will be give a context reference (pointer)
+ that is supplied to most RMR functions as the first
+ parameter.
-ENOTSUP
+Round robin
- Sending was not supported by the underlying message
- transport.
+ The method of selecting an *endpoint* from a list such
+ that all *endpoints* are selected before starting at the
+ head of the list.
-EFSM
+Route table
- The device is not in a state that can accept the message.
+ A series of "rules" which define the possible *endpoints*
+ for each *message key.*
-EAGAIN
+Route table manager
- The device is not able to accept a message for sending.
- The user application should attempt to resend.
+ An application responsible for building a *route table*
+ and then distributing it to all applicable RMR based
+ applications.
-EINTR
+Routing
- The operation was interrupted by delivery of a signal
- before the message was sent.
+ The process of selecting an *endpoint* which will be the
+ recipient of a message.
-ETIMEDOUT
+Routing key
- The underlying message environment timed out during the
- send process.
+ A combination of *message type* and *subscription ID*
+ which RMR uses to select the destination *endpoint* when
+ sending a message.
-ETERM
+Source
+
+ The sender of a message.
- The underlying message environment is in a shutdown state.
+Subscription ID
+
+ A signed integer value (0-32000) which identifies the
+ subscription characteristic of a message. It is used in
+ conjunction with the *message type* to determine the
+ *routing key.*
+
-EXAMPLE
---------------------------------------------------------------------------------------------
+Target
+
+ The *endpoint* selected to receive a message.
+
-The following is a simple example of how the a wormhole is
-created (rmr_wh_open) and then how rmr_wh_send_msg function
-is used to send messages. Some error checking is omitted for
-clarity.
+TCP
+
+ Transmission Control Protocol. A connection based internet
+ protocol which provides for lossless packet
+ transportation, usually over IP.
+
+Thread
+
+ Also called a *process thread, or pthread.* This is a
+ lightweight process which executes in concurrently with
+ the application and shares the same address space. RMR
+ uses threads to manage asynchronous functions such as
+ route table updates. &Term An optional portion of the
+ message buffer that the application may populate with data
+ that allows for tracing the progress of the transaction or
+ application activity across components. RMR makes no use
+ of this data.
+
-::
-
- #include <rmr/rmr.h> // system headers omitted for clarity
- int main() {
- rmr_whid_t whid = -1; // wormhole id for sending
- void* mrc; //msg router context
- int i;
- rmr_mbuf_t* sbuf; // send buffer
- int count = 0;
- int norm_msg_size = 1500; // most msg fit in this size
- mrc = rmr_init( "43086", norm_msg_size, RMRFL_NONE );
- if( mrc == NULL ) {
- fprintf( stderr, "[FAIL] unable to initialise RMR environment\\n" );
- exit( 1 );
- }
- while( ! rmr_ready( mrc ) ) { // wait for routing table info
- sleep( 1 );
- }
- sbuf = rmr_alloc_msg( mrc, 2048 );
- while( 1 ) {
- if( whid < 0 ) {
- whid = rmr_wh_open( mrc, "localhost:6123" ); // open fails if endpoint refuses conn
- if( RMR_WH_CONNECTED( wh ) ) {
- snprintf( sbuf->payload, 1024, "periodic update from sender: %d", count++ );
- sbuf->len = strlen( sbuf->payload );
- sbuf = rmr_wh_send_msg( mrc, whid, sbuf );
- }
- }
- sleep( 5 );
- }
- }
+Transaction ID
+
+ A fixed number of bytes in the *message* buffer) which the
+ application may populate with information related to the
+ transaction. RMR makes use of the transaction ID for
+ matching response messages with the &c function is used to
+ send a message.
+
+Transient failure
+
+ An error state that is believed to be short lived and that
+ the operation, if retried by the application, might be
+ successful. C programmers will recognise this as EAGAIN.
+
+Warning
+
+ A warning occurs when RMR has encountered something that
+ it believes isn't correct, but has a defined work round.
+
-SEE ALSO
---------------------------------------------------------------------------------------------
+Wormhole
+
+ A direct connection managed by RMR between the user
+ application and a remote, RMR based, application.
-rmr_alloc_msg(3), rmr_call(3), rmr_free_msg(3), rmr_init(3),
-rmr_payload_size(3), rmr_rcv_msg(3), rmr_rcv_specific(3),
-rmr_rts_msg(3), rmr_ready(3), rmr_fib(3), rmr_has_str(3),
-rmr_tokenise(3), rmr_mk_ring(3), rmr_ring_free(3),
-rmr_set_stimeout(3), rmr_wh_open(3), rmr_wh_close(3),
-rmr_wh_state(3)
-NAME
---------------------------------------------------------------------------------------------
+Appendix D -- Code Examples
+============================================================================================
-rmr_wh_state
+The following snippet of code illustrate some of the basic
+operation of the RMR library. Please refer to the examples
+and test directories in the RMR repository for complete RMR
+based programmes.
-SYNOPSIS
+Sender Sample
--------------------------------------------------------------------------------------------
+The following code segment shows how a message buffer can be
+allocated, populated, and sent. The snippet also illustrates
+how the result from the function is used to send the next
+message. It does not illustrate error and/or retry handling.
+
::
- #include <rmr/rmr.h>
- int rmr_wh_state( void* vctx, rmr_whid_t whid )
+ mrc = rmr_init( listen_port, MAX_BUF_SZ, RMRFL_NOFLAGS );
+ rmr_set_stimeout( mrc, rmr_retries );
+ while( ! rmr_ready( mrc ) ) {
+ sleep( 1 );
+ }
+ sbuf = rmr_alloc_msg( mrc, 256 ); // 1st send buffer
+ while( TRUE ) {
+ sbuf->len = gen_status( (status_msg *) sbuf->payload );
+ sbuf->mtype = STATUS_MSG;
+ sbuf->sub_id = RMR_VOID_SUBID; // subscription not used
+ sbuf = rmr_send_msg( mrc, sbuf );
+ sleep( delay_sec );
+ }
+ rmr_close( mrc );
-DESCRIPTION
+Receiver Sample
--------------------------------------------------------------------------------------------
-The rmr_wh_state function will return the current state of
-the connection associated with the given wormhole (whid). The
-return value indicates whether the connection is open
-(RMR_OK), or closed (any other return value).
-
-When using some transport mechanisms (e.g. NNG), it may not
-be possible for RMR to know the actual state and the
-connection may always be reported as "open."
+The receiver code is even simpler than the sender code as it
+does not need to wait for a route table to arrive (only
+senders need to do that), nor does it need to allocate an
+initial buffer. The example assumes that the sender is
+transmitting a zero terminated string as the payload.
-RETURN
---------------------------------------------------------------------------------------------
-The following values are potential return values.
+::
+
+ rmr_mbuf_t* rbuf = NULL;
+ void* mrc = rmr_init( listen_port, MAX_BUF_SZ, RMRFL_NOFLAGS );
+ while( TRUE ) {
+ rbuf = rmr_rcv_msg( mrc, rbuf ); // reuse buffer on all but first loop
+ if( rbuf == NULL || rbuf->state != RMR_OK ) {
+ break;
+ }
+ fprintf( stdout, "mtype=%d sid=%d pay=%s\\n",
+ rbuf->mtype, rbuf->sub_id, rbuf->payload );
+ sleep( delay_sec );
+ }
+ fprintf( stderr, "receive error\\n" );
+ rmr_close( mrc );
-RMR_OK
-
- The wormhole ID is valid and the connection is "open."
-
+Receive and Send Sample
+--------------------------------------------------------------------------------------------
-RMR_ERR_WHID
-
- THe wormhole ID passed into the function was not valid.
-
+The following code snippet receives messages and responds to
+the sender if the message type is odd. The code illustrates
+how the received message may be used to return a message to
+the source. Variable type definitions are omitted for clarity
+and should be obvious.
-RMR_ERR_NOENDPT
-
- The wormhole is not open (not connected).
-
+It should also be noted that things like the message type
+which id returned to the sender (99) is a random value that
+these applications would have agreed on in advance and is
+**not** an RMR definition.
-RMR_ERR_BADARG
-
- The context passed to the function was nil or invalid.
-
-RMR_ERR_NOWHOPEN
-
- Wormholes have not been initialised (no wormhole open call
- has been made).
-
+::
+
+ mrc = rmr_init( listen_port, MAX_BUF_SZ, RMRFL_NOFLAGS );
+ rmr_set_stimeout( mrc, 1 ); // allow RMR to retry failed sends for ~1ms
+ while( ! rmr_ready( mrc ) ) { // we send, therefore we need a route table
+ sleep( 1 );
+ }
+ mbuf = NULL; // ensure our buffer pointer is nil for 1st call
+ while( TRUE ) {
+ mbuf = rmr_rcv_msg( mrc, mbuf ); // wait for message
+ if( mbuf == NULL || mbuf->state != RMR_OK ) {
+ break;
+ }
+ if( mbuf->mtype % 2 ) { // respond to odd message types
+ plen = rmr_payload_size( mbuf ); // max size
+ // reset necessary fields in msg
+ mbuf->mtype = 99; // response type
+ mbuf->sub_id = RMR_VOID_SUBID; // we turn subid off
+ mbuf->len = snprintf( mbuf->payload, plen, "pong: %s", get_info() );
+ mbuf = rmr_rts_msg( mrc, mbuf ); // return to sender
+ if( mbuf == NULL || mbuf->state != RMR_OK ) {
+ fprintf( stderr, "return to sender failed\\n" );
+ }
+ }
+ }
+ fprintf( stderr, "abort: receive failure\\n" );
+ rmr_close( mrc );
-SEE ALSO
---------------------------------------------------------------------------------------------
-rmr_wh_open(3), rmr_wh_send_msg(3), rmr_wh_close(3)
Code Review / ric-plt / lib / rmr.git / commitdiff