#==================================================================================
#
-
Building RMR
-The RIC Message Router (RMR) is built with CMake, and requires
-a modern gcc compiler and make to be installed on the build
-system. Typically, installing the following list of packages
-in a container (Ubuntu) is all that is needed to craft a
-development environment (containerised builds are also the
-recommended approach):
+The RIC Message Router (RMR) is built with CMake, and requires cmake
+version 3, make and a modern gcc compiler to be installed on the build
+system. Typically, installing the following list of packages in a
+container (Ubuntu) is all that is needed to craft a development
+environment (containerised builds are also the recommended approach):
gcc git make vim cmake g++ ksh bash
make package
-This will create a .deb (provided the system supports this) in
-the build directory. It's that simple.
+On a Debian/Ubuntu system, this will create a .deb (provided the
+system supports this) in the build directory. It's that simple.
-The following flags may be given on the 'cmake' command line
-(options) which are outside of "normal" CMake flags and affect
-the configuration:
+The following flags may be given on the 'cmake' command line (options)
+which are outside of "normal" CMake flags and affect the
+configuration:
-DBUILD_DOC=1 Man pages generated
-DDEV_PKG=1 Development package configuration
Packages
-The build can be configured to generate either a run-time or
-development package. The run-time generation is the default and
-the -DDEV_PKG=1 option must be given to generate the development
-package. The run-time package contains only the shared library
-files (*.so), and the development package contains the headers,
-man pages (if the man option is set) and archive (.a) files.
-Resulting package names are illustrated in the CI section below.
+The build can be configured to generate either a run-time or
+development package. The run-time generation is the default and the
+-DDEV_PKG=1 option must be given to generate the development package.
+The run-time package contains only the shared library files (*.so),
+and the development package contains the headers, man pages (if the
+man option is set) and archive (.a) files. Resulting package names
+are illustrated in the CI section below.
Continuous Integration Build
-Use the Dockerfile in the ci/ subdirectory. This installs all
-the required tools, then builds RMR and executes the unit and
-program tests. If tests pass, then an image is created in the
-local registry with both run-time and development packages.
-To support the distribution of package(s) created during the
-build by the CI process, a YAML file is left in the /tmp
-directory (build_packages.yml) which contains a list of the
-packages available from the image. Currently, both .deb and
-.rpm packages are generated.
+Use the Dockerfile in the ci/ subdirectory. This installs all the
+required tools, then builds RMR and executes the unit and program
+tests. If tests pass, then an image is created in the local registry
+with both run-time and development packages.
+
+To support the distribution of package(s) created during the build by
+the CI process, a YAML file is left in the /tmp directory
+(build_packages.yml) which contains a list of the packages available
+from the image. Currently, both .deb and .rpm packages are generated.
The following is a sample YAML file generated during this process:
Alternatives
-To build in a non-Linux environment, or to build with an
-alternate install path (or both) read on.
-Instead of using 'make package' as listed above, using
-'make install' will build and install on the local system.
-By default, the target install is into /usr/local which may
-not be desired. To install into an alternate path add
-these two options when the 'cmake ..' command is given:
+To build in a non-Linux environment, or to build with an alternate
+install path (or both) read on.
+
+Instead of using 'make package' as listed above, using 'make install'
+will build and install on the local system. By default, the target
+install is into /usr/local which may not be desired. To install into
+an alternate path add these two options when the 'cmake ..' command is
+given:
-DCMAKE_INSTALL_PREFIX=/path/to/dir
-DMAN_PREFIX=/path/to/dir
The first will cause the make process to install into the named
-directory, which can be in your home directory. The second
-defines where manual pages are placed (if not defined
-/usr/share/man is the target). Manual pages are generally
-NOT built as the required tool has yet to be incorporated into
-the build process and generally is not available on most systems.
+directory, which can be in your home directory. The second defines
+where manual pages are placed (if not defined /usr/share/man is the
+target). Manual pages are generally NOT built as the required tool
+has yet to be incorporated into the build process and generally is not
+available on most systems.
Compiling and Linking User Applications
+
Should the Rmr and NNG/Nano libraries be installed in a directory
-outside of the normal system spots (e.g. not in /usr/local)
-it might be necessary to define the specific directory for
-libraries (.e.g -L) on the command line, or via environment
-variables (e.g.. C_INCLUDE_PATH, LD_LIBRARY_PATH, LIBRARY_PATH).
-It may also be necessary to have the library directory defined
-in the environment at run time. It is difficult to know what
-each system needs, but the following linker options work when
-libraries are installed in the system spots:
+outside of the normal system spots (e.g. not in /usr/local) it might
+be necessary to define the specific directory for libraries (.e.g -L)
+on the command line, or via environment variables
+(e.g.. C_INCLUDE_PATH, LD_LIBRARY_PATH, LIBRARY_PATH). It may also be
+necessary to have the library directory defined in the environment at
+run time. It is difficult to know what each system needs, but the
+following linker options work when libraries are installed in the
+system spots:
-lrmr_nng -lnng -lpthread
-Adding -L is one way to compensate when libraries are installed
-a different spot (e.g. in $HOME/usr):
+Adding -L is one way to compensate when libraries are installed in a
+different spot (e.g. in $HOME/usr):
-L $HOME/usr -lrmr_nng -lnng -lpthread
Libraries
-RMR supports only NNG as the underlying transport. Nanomsg
-support was dropped starting with version 1.0.45 as Nanomsg
-has reached end of life. The package generation and install
-will produce a single RMR library: librmr_nng. RMR is designed
-to support different underlying transport mechanisms, which
-might require separate libraries, and thus the library name is
-given a suffix of _nng to reflect the transport mechanism
-in use.
+
+RMR is designed to support different underlying transport mechanisms,
+which might require separate libraries, and thus the library name is
+given a suffix that reflects the transport mechanism in use. RMR
+supports NNG and SI95 as the underlying transports. Nanomsg support
+was dropped starting with version 1.0.45 as Nanomsg has reached end of
+life. The package generation and install will produce two RMR
+libraries: librmr_nng and librmr_si.
NNG version with a commit ID of 906d5ea1b3d67bece941d8a4e0a049e5f6c65051
is required to build RMR. That version (as yet untagged) adds a
If you do not set this flag, the proper NNG source will be used
automatically.
-Regardless of transport mechanism supported by an RMR library,
-the RMR API will be identical, thus it is possible for an application
-to shift mechanisms simply by referencing a different library (should
-multiple RMR libraries become available).
-
+Regardless of transport mechanism supported by an RMR library, the RMR
+API will be identical, thus it is possible for an application to shift
+mechanisms simply by referencing a different library (should multiple
+RMR libraries become available).
Manual Pages
+
By default the deb created does not include the manual pages. To
-enable their creation, and subsequent inclusion in the deb, add
-the following option to the cmake command:
+enable their creation, and subsequent inclusion in the deb, add the
+following option to the cmake command:
-DBUILD_DOC=1
This will cause the {X}fm text formatting package to be fetched
-(github) and built at cmake time (must exist before building)
-and will trigger the generation of the man pages in both postscript
-and troff format. The troff pages are placed into the deb and
-the postscript pages are left in the build directory for the
-developer to convert to PDF, or otherwise use.
+(github) and built at cmake time (must exist before building) and will
+trigger the generation of the man pages in both postscript and troff
+format. The troff pages are placed into the deb and the postscript
+pages are left in the build directory for the developer to convert to
+PDF, or otherwise use.
Debug Mode
+
Because RMR is designed to keep its overhead to an absolute minimum,
-messages written to standard error are by default very limited.
-The route table collection thread provides the means to enable
-debug messages on the fly, but only because that thread does not
-impact the sending and receiving of user messages.
+messages written to standard error are by default very limited. The
+route table collection thread provides the means to enable debug
+messages on the fly, but only because that thread does not impact the
+sending and receiving of user messages.
If it becomes necessary, for development or problem soving, to have
the RMR functions generate debugging messages the following
The value for 'n' should be 1 or 2 to enable debugging. The default
when not given is the same as setting n to zero.
-When running in debug mode, RMR will log messages received, sent,
-and other useful information. Because debugging uses fprintf() there
-is a significant amount of overhead with logging this information and
-thus in debugging mode the user should not expect that usual message
-rates can be achieved, and in some cases may cause messages to drop
-if TCP queues become full.
+When running in debug mode, RMR will log messages received, sent, and
+other useful information. Because debugging uses fprintf() there is a
+significant amount of overhead with logging this information and thus
+in debugging mode the user should not expect that usual message rates
+can be achieved, and in some cases may cause messages to drop if TCP
+queues become full.
-This directory contains the main RMR library documentation
+This directory contains the main RMR library documentation
such as the user guide and any internal doc; all of which will
-likely share a common set of figures, glossary, and index
+likely share a common set of figures, glossary, and index
lists. The "main" for each doc is the single .xfm file and
all modules are contained in .im files to allow sharing between
documents as makes sense (e.g. glossary).
Illustrations are all located in the ./figures directory and
-are created as .fig files [1] which describe drawings using an
-open ASCII format (rather than as a binary and/or propriatory
-descritption). ASCII descriptions are easily maintained as
+are created as .fig files [1] which describe drawings using an
+open ASCII format (rather than as a binary and/or proprietary
+description). ASCII descriptions are easily maintained as
source code to track changes and revert when necessary.
-A Dockerfile in the tools directory can be used to build a
+A Dockerfile in the tools directory can be used to build a
"development image" with all of the tools needed to render
the PNG and encapsulated postscript derivations from the ASCII
descriptions. Use of this image is not required, but might be
-needed if a development environment doesn't provide Xfig and
+needed if a development environment doesn't provide Xfig and
the related tools.
-[1]
- Defacto standard tools for creating and managing .fig files:
+[1]
+ De facto standard tools for creating and managing .fig files:
Install Xfig on a Linux system with:
- apt-get install xfig (ubuntu-ish)
- zypper install xfig (OpenSuse)
+ apt-get install xfig (ubuntu-ish)
+ zypper install xfig (OpenSuse)
- The fig2dev tool is used to "batch" process .fig files in
+ The fig2dev tool is used to "batch" process .fig files in
order to generate desired output (.eps, .png, etc.). These
can be installed with commands such as:
apt-get install fig2dev
.fi
&h1(Advanced Usage)
-Several forms of usage fall into a more advanced category and are described in the following sections.
-These include blocking call, return to sender and wormhole functions.
+Several forms of usage fall into a more advanced category and are
+described in the following sections. These include blocking call,
+return to sender and wormhole functions.
&h2(The Call Function)
-The RMR function &func(rmr_call) will send a message in the exact same manner as the &cw(rmr_send_msg()) function,
-with the endpoint selection based on the message key.
-Unlike the send function, &func(rmr_call) will block and wait for a response from the application that is selected
-to receive the message.
-The matching message is determined by the transaction ID which the application must place into the message buffer
-prior to invoking &func(rmr_call) .sm .
-Similarly, the responding application must ensure that the same transaction ID is placed into the message buffer
-before returning its response.
+The RMR function &func(rmr_call) sends a message in the exact same
+manner as the &cw(rmr_send_msg()) function, with the endpoint
+selection based on the message key. But unlike the send function,
+&func(rmr_call) will block and wait for a response from the
+application that is selected to receive the message. The matching
+message is determined by the transaction ID which the application must
+place into the message buffer prior to invoking &func(rmr_call) .sm .
+Similarly, the responding application must ensure that the same
+transaction ID is placed into the message buffer before returning its
+response.
&space
-The return from the call is a message buffer with the response message; there is no difference between a message
-buffer returned by the receive function and one returned by the &fucnt(rmr_call) function.
-If a response is not received in a reasonable amount of time, a nil message buffer is returned to the calling
-application.
+The return from the call is a message buffer with the response
+message; there is no difference between a message buffer returned by
+the receive function and one returned by the &func(rmr_call)
+function. If a response is not received in a reasonable amount of
+time, a nil message buffer is returned to the calling application.
&h3(Returning a Response)
-Because of the nature of RMR's routing policies, it is generally not possible for an application to control
-exactly which endpoint is sent a message.
-There are cases, such as responding to a message delivered via &func(rmr_call:) that the application must
-send a message and guarantee that RMR routes it to an exact destination.
-To enable this, RMR provides the &func(rmr_rts_msg:,) return to sender, function.
-Upon receipt of any message, an application may alter the payload, and if necessary the message type and subscription
-ID, an pass the altered message buffer to the &func(rmr_rts_msg:) function to return the altered message
-to the application which sent it.
-When this function is used, RMR will examine the message buffer for the source information and use that to
-select the connection on which to write the response.
+Because of the nature of RMR's routing policies, it is generally not
+possible for an application to control exactly which endpoint is sent
+a message. There are cases, such as responding to a message delivered
+via &func(rmr_call:) that the application must send a message and
+guarantee that RMR routes it to an exact destination. To enable this,
+RMR provides the &func(rmr_rts_msg:,) return to sender, function.
+Upon receipt of any message, an application may alter the payload, and
+if necessary the message type and subscription ID, and pass the altered
+message buffer to the &func(rmr_rts_msg:) function to return the
+altered message to the application which sent it. When this function
+is used, RMR will examine the message buffer for the source
+information and use that to select the connection on which to write
+the response.
&h3(Multi-threaded Calls)
-The basic call mechanism described above is &bold(not) thread safe, as it is not possible to guarantee that a response
-message is delivered to the correct thread.
-The RMR function &func(rmr_mt_call) accepts an additional parameter which identifies the calling thread in order
-to ensure that the response is delivered properly.
-In addition, the application must specifically initialise the multi-threaded call environment by passing the
-&cw(RMRFL_MTCALL) flag as an option to the &func(rmr_init) function ¬e .sm .
+
+The basic call mechanism described above is &bold(not) thread safe, as
+it is not possible to guarantee that a response message is delivered
+to the correct thread. The RMR function &func(rmr_mt_call) accepts an
+additional parameter which identifies the calling thread in order to
+ensure that the response is delivered properly. In addition, the
+application must specifically initialize the multi-threaded call
+environment by passing the &cw(RMRFL_MTCALL) flag as an option to the
+&func(rmr_init) function ¬e .sm .
.if pfm
-.dv cnopts l=&cn_line_len i=&cn_indent
+.dv cnopts l=&cn_line_len i=&cn_indent
.dv cncmd .cn start &cnopts &atbot Times-roman 8p .7i
.ei
.dv cncmd .cn start &atbot Times-roman 8p .7i
.fi
&cncmd
- There is additional overhead to support multi-threaded call as a special listener thread must be used
- in order to deliver responses to the proper application thread.
+ There is additional overhead to support multi-threaded call as
+ a special listener thread must be used in order to deliver
+ responses to the proper application thread.
.cn end
&space
-One advantage of the multi-threaded call capability in RMR is the fact that only the calling thread is
-blocked. Messages received which are not responses to the call are continued to be delivered via normal
+One advantage of the multi-threaded call capability in RMR is the fact
+that only the calling thread is blocked. Messages received which are
+not responses to the call are continued to be delivered via normal
&func(rmr_rcv_msg) calls.
&space
-While the process is blocked waiting for the response, it is entirely possible that asynchronous, nonmatching,
-messages will arrive.
-When this happens, RMR will queues the messages and return them to the application over the next calls to
-&func(rmr_rcv_msg:.)
+While the process is blocked waiting for the response, it is entirely
+possible that asynchronous, nonmatching, messages will arrive. When
+this happens, RMR will queues the messages and return them to the
+application over the next calls to &func(rmr_rcv_msg:.)
&h2(Wormholes)
-As was mentioned earlier, the design of RMR is to eliminate the need for an application to know a specific
-endpoint, even when a response message is being sent.
-In some rare cases it may be necessary for an application to establish a direct connection to an RMR based
-application rather than relying on message type and subscription ID based routing.
-The &ital(wormhole) functions provide an application with the ability to create a direct connection and then
-to send and receive messages across the connection.
-The following are the RMR functions which provide wormhole communications:
+As was mentioned earlier, the design of RMR is to eliminate the need
+for an application to know a specific endpoint, even when a response
+message is being sent. In some rare cases it may be necessary for an
+application to establish a direct connection to an RMR-based
+application rather than relying on message type and subscription ID
+based routing. The &ital(wormhole) functions provide an application
+with the ability to create a direct connection and then to send and
+receive messages across the connection. The following are the RMR
+functions which provide wormhole communications:
&space
&indent
&beg_dlist( 1i Helvetica )
- &di(rmr_wh_open) Open a connection to an endpoint. Name or IP address and port of the endpoint is supplied.
- Returns a wormhole ID that the application must use when sending a direct message.
+ &di(rmr_wh_open) Open a connection to an endpoint. Name or IP
+ address and port of the endpoint is supplied. Returns a
+ wormhole ID that the application must use when sending a
+ direct message.
&half_space
- &di(rmr_wh_send_msg) Sends an RMR message buffer to the connected application. The message type and subscription
- ID may be set in the message, but RMR will ignore both.
+ &di(rmr_wh_send_msg) Sends an RMR message buffer to the
+ connected application. The message type and subscription ID
+ may be set in the message, but RMR will ignore both.
&half_space
&di(rmr_wh_close) Closes the direct connection.
Abstract: A quick reference to all external function calls
Date: 2 August, 2019
Author: E. Scott Daniels
-.fi
+.fi
&h1(Appendix &qr_appendix -- Quick Reference)
-The prototype for each of the externally available functions which comprise the RMR API is listed
-in alphabetical order below.
-For each prototype a brief description of the function is given.
-The developer is encouraged to install the RMR development package which contains the manual pages.
-The manual pages completely describe each function in a level of detail consistent with UNIX man pages.
+
+The prototypes for all externally available functions which comprise
+the RMR API are listed in alphabetical order below. For each
+prototype a brief description of the function is given. The developer
+is encouraged to install the RMR development package which contains
+the manual pages. The manual pages completely describe each function
+in a level of detail consistent with UNIX man pages.
&h2(Context Specific Functions)
-These functions require that the RMR context (provided as the result of an &func(rmr_init:) call, be
-passed as the first argument (vctx).
+These functions require that the RMR context (provided as the result
+of an &func(rmr_init:) call, be passed as the first argument (vctx).
&proto_start
rmr_mbuf_t* rmr_alloc_msg( void* vctx, int size );
&proto_end
-This function allocates a &ital(zero copy) message buffer.
-The payload portion is allocated from the underlying transport space such that on send the buffer
-does not require a second copy.
-The size parameter is the size of the payload portion of the buffer; if the recipient of the
-message is expected to send a response, this should be large enough to accomodate the response
-which will enable the remote application to reuse the message for the response and avoid a costly
-reallocation.
+This function allocates a &ital(zero copy) message buffer. The
+payload portion is allocated from the underlying transport space such
+that on send the buffer does not require a second copy. The size
+parameter is the size of the payload portion of the buffer. If the
+recipient of the message is expected to send a response, this should
+be large enough to accomodate the response which will enable the
+remote application to reuse the message for the response and avoid a
+costly reallocation.
&proto_start
rmr_mbuf_t* rmr_call( void* vctx, rmr_mbuf_t* msg );
&proto_end
-The call function accepts a message, selects an endpoint to receive the message and sends the message.
-RMR will block the return to the application until a matching response is received, or a timeout period
-is reached.
-The transaction ID in the outbound message is used to match a response, so it is imperative that
-the application making the response reuse the received message, or copy the transaction ID to the new
-message before sending.
-Messages arriving which do not match the expected response are queued and will be delivered to the
-application on subsequent calls to &func(rmr_rcv_msg:.)
+The call function accepts a message, selects an endpoint to receive
+the message and sends the message. RMR will block the return to the
+application until a matching response is received, or a timeout period
+is reached. The transaction ID in the outbound message is used to
+match a response, so it is imperative that the application making the
+response reuse the received message, or copy the transaction ID to the
+new message before sending. Messages arriving which do not match the
+expected response are queued and will be delivered to the application
+on subsequent calls to &func(rmr_rcv_msg:.)
&proto_start
void rmr_close( void* vctx );
&proto_end
-This function terminates all sessions with the underlying transport mechanism.
-Buffers pending may or may not be flushed (depending on the underlying mechanism), thus it is
-recommended that before using this function the application pause for a second or two to
-ensure that the pending transmissions have completed.
+This function terminates all sessions with the underlying transport
+mechanism. Buffers pending may or may not be flushed (depending on
+the underlying mechanism), thus it is recommended that before using
+this function the application pause for a second or two to ensure that
+the pending transmissions have completed.
&proto_start
int rmr_get_rcvfd( void* vctx );
&proto_end
-When the underlying transport mechanism supports this, a file descriptor suitable for use with
-the &cw(select, poll) and &cw(epoll) system calls is returned.
-The file descriptor may not be used for read or write operations; doing so will have unpredictable
-results.
+When the underlying transport mechanism supports this, a file
+descriptor suitable for use with the &cw(select, poll) and &cw(epoll)
+system calls is returned. The file descriptor may not be used for
+read or write operations; doing so will have unpredictable results.
&proto_start
void* rmr_init( char* proto_port, int max_msg_size, int flags );
&proto_end
-This function must be used before all other RMR functions to initialise the environment.
-The &cw(max_msg_size) parameter defines the maximum receive buffer size which will be
-used if required by the underlying transport mechanism.
-The value is also used as the default payload size if a zero length is given to &func(rmr_alloc_msg:.)
+This function must be used before all other RMR functions to
+initialize the environment. The &cw(max_msg_size) parameter defines
+the maximum receive buffer size which will be used if required by the
+underlying transport mechanism. The value is also used as the default
+payload size if a zero length is given to &func(rmr_alloc_msg:.)
&proto_start
rmr_mbuf_t* rmr_mt_call( void* vctx, rmr_mbuf_t* mbuf, int call_id, int max_wait );
&proto_end
-Similar to the &func(rmr_call:) function, the message is sent to an endpoint and a response message
-is waited for.
-This call is thread safe, and while the thread that invokes this function blocks on the response,
-it is possible for other application threads to continue to receive messages via the &func(rmr_rcv_msg:)
-function.
+Similar to the &func(rmr_call:) function, the message is sent to an
+endpoint and a response message is waited for. This call is thread
+safe, and while the thread that invokes this function blocks on the
+response, it is possible for other application threads to continue to
+receive messages via the &func(rmr_rcv_msg:) function.
&space
-In order to use the multi-threaded call functions, the option to enable threaded receive support
-must be set on the call to &func(rmr_init:.)
+In order to use the multi-threaded call functions, the option to
+enable threaded receive support must be set on the call to
+&func(rmr_init:.)
&proto_start
rmr_mbuf_t* rmr_mtosend_msg( void* vctx, rmr_mbuf_t* msg, int max_to );
&proto_end
-This function sends the provided message allowing RMR to retry soft failures for approximately
-&ccw(max_to) milliseconds.
-When a value of zero (0) is given for the maximum timeout, RMR will not attempt any retires
-and will return the state after the first attempt.
-It is unlikely that a user application will use this function as it is possible (and recommended)
-to set the max timeout via the specific, one time, function call, and then to allow that value
-to be used as the default when &func(rmr_send_msg:) is invoked.
+This function sends the provided message allowing RMR to retry soft
+failures for approximately &ccw(max_to) milliseconds. When a value of
+zero (0) is given for the maximum timeout, RMR will not attempt any
+retries and will return the state after the first attempt. It is
+unlikely that a user application will use this function as it is
+possible (and recommended) to set the max timeout via the specific,
+one time, function call, and then to allow that value to be used as
+the default when &func(rmr_send_msg:) is invoked.
&proto_start
rmr_mbuf_t* rmr_mt_rcv( void* vctx, rmr_mbuf_t* mbuf, int max_wait );
&proto_end
-This function waits for a message to arrive and returns a message buffer with the received
-message.
-The function will timeout after &cw(max_wait) milliseconds (approximately) if no message is
-received.
+This function waits for a message to arrive and returns a message
+buffer with the received message. The function will timeout after
+&cw(max_wait) milliseconds (approximately) if no message is received.
&proto_start
rmr_mbuf_t* rmr_send_msg( void* vctx, rmr_mbuf_t* msg );
&proto_end
-This function accepts a message, selects a destination endpoint using the message type and subscription ID,
-and then attempts to send the message to the destination.
-The function returns a message buffer to the caller with the state set to indicate the state of the
-send operation.
-On success, the message buffer is a new buffer that the application can "fill in," and send without the
-need to overtly allocate a new buffer.
-On failure, the message buffer is the buffer that the application attempted to send.
+This function accepts a message, selects a destination endpoint using
+the message type and subscription ID, and then attempts to send the
+message to the destination. The function returns a message buffer to
+the caller with the state set to indicate the state of the send
+operation. On success, the message buffer is a new buffer that the
+application can "fill in," and send without the need to overtly
+allocate a new buffer. On failure, the message buffer is the buffer
+that the application attempted to send.
&proto_start
int rmr_init_trace( void* vctx, int size );
&proto_end
-This function is used to set the default trace data size.
-The size defaults to zero until this function is called; after the application sets a non-zero value
-messages created with the &func(rmr_alloc_msg:) function will be allocated with trace data set to
-the size provided.
+This function is used to set the default trace data size. The size
+defaults to zero until this function is called; after the application
+sets a non-zero value, messages created with the &func(rmr_alloc_msg:)
+function will be allocated with trace data set to the size provided.
&proto_start
rmr_mbuf_t* rmr_rcv_msg( void* vctx, rmr_mbuf_t* old_msg );
&proto_end
-The &func(rmr_rcv_msg:) function is used to block and wait for a message to arrive.
-When a message is received a pointer to an RMR message buffer structure is returned to the caller.
-The &cw(old_msg) parameter allows the application to to pass a message buffer for reuse.
-If the application does not have an old buffer, a nil pointer is given and the function will
-allocate a new buffer.
+The &func(rmr_rcv_msg:) function is used to block and wait for a
+message to arrive. When a message is received, a pointer to an RMR
+message buffer structure is returned to the caller. The &cw(old_msg)
+parameter allows the application to to pass a message buffer for
+reuse. If the application does not have an old buffer, a nil pointer
+is given and the function will allocate a new buffer.
&proto_start
-rmr_mbuf_t* rmr_rcv_specific( void* uctx, rmr_mbuf_t* msg, char* expect,
+rmr_mbuf_t* rmr_rcv_specific( void* uctx, rmr_mbuf_t* msg, char* expect,
int allow2queue );
&proto_end
-This function blocks until a message with a specific transaction ID is received.
-If the &cw(allowd2queue) parameter is set to 1, messagess which do not match the ID are queued
-and returned to the application on subsequent calls to &func(rmr_rcv_msg:.)
-
+This function blocks until a message with a specific transaction ID is
+received. If the &cw(allowd2queue) parameter is set to 1, messages
+which do not match the ID are queued and returned to the application
+on subsequent calls to &func(rmr_rcv_msg:.)
&proto_start
int rmr_ready( void* vctx );
&proto_end
-This function is used to test whether RMR is capable of sending messages.
-In other words once this function returns true (!0) RMR has received a route table (either from a static
-file or from a route manager process, and can map message types to endpoints.
-If the application attempts to send a message before this function returns true, the sends
-will fail.
-Applications which are only message receivers do not need to use this function.
+This function is used to test whether RMR is capable of sending
+messages. In other words once this function returns true (!0) RMR has
+received a route table (either from a static file or from a route
+manager process) and can map message types to endpoints. If the
+application attempts to send a message before this function returns
+true, the sends will fail. Applications which are only message
+receivers do not need to use this function.
&proto_start
rmr_mbuf_t* rmr_rts_msg( void* vctx, rmr_mbuf_t* msg );
&proto_end
-The &func(rmr_rts_msg:) function allows the application to send a response message to the endpoint
-from which the message originated.
-This requires that the application use the same message buffer that was received for the
-response as it contains the sender information that is needed for this function to be successful.
-If the message cannot be sent, a pointer to the message buffer is returned and the status in the
-message buffer is set to indicate the reason.
-On success, a nil pointer will be returned.
-
+The &func(rmr_rts_msg:) function allows the application to send a
+response message to the endpoint from which the message originated.
+This requires that the application use the same message buffer that
+was received for the response as it contains the sender information
+that is needed for this function to be successful. If the message
+cannot be sent, a pointer to the message buffer is returned and the
+status in the message buffer is set to indicate the reason. On
+success, a nil pointer is returned.
&proto_start
int rmr_set_rtimeout( void* vctx, int time );
&proto_end
-This function provides the ability to return from a receive operation after a timeout threshold
-is reached before a message is received, and is intended only to support the underlying Nanomsg
-transport mechanism (support for Nanomsg is deprecated).
-The &func(rmr_torcv_msg) function should be used if timed receives are required.
+This function provides the ability to return from a receive operation
+after a timeout threshold is reached before a message is received, and
+is intended only to support the underlying Nanomsg transport mechanism
+(support for Nanomsg is deprecated). The &func(rmr_torcv_msg)
+function should be used if timed receives are required.
&space
-
-For transport mechanisms which support a receive timeout, this function allows the application to
-set a default timeout for receive operations.
-If a call to &func(rmr_rcv_msg) does not complete before &ital(time) milliseconds has elapsed,
-the receive function will return a nil message buffer.
-This may not be supported by the underlying transport mechanism, and if it is not the return
-from this function will be -1.
+For transport mechanisms which support a receive timeout, this
+function allows the application to set a default timeout for receive
+operations. If a call to &func(rmr_rcv_msg) does not complete before
+&ital(time) milliseconds has elapsed, the receive function will return
+a nil message buffer. This may not be supported by the underlying
+transport mechanism, and if it is not the return from this function
+will be -1.
&proto_start
int rmr_set_stimeout( void* vctx, int rounds );
&proto_end
-This function allows the application to set a maximum number of retry &ital(rounds) that RMR will
-attempt when send operations report a transient (retry) failure.
-Each &ital(round) of retries requires approximately 1 millisecond, and setting the number of
-rounds to zero (0) causes RMR to report the transient failure to the application without
-any retry attempts.
-If the user application does not invoke this function, the default is one (1) round of retries.
+This function allows the application to set a maximum number of retry
+&ital(rounds) that RMR will attempt when send operations report a
+transient (retry) failure. Each &ital(round) of retries requires
+approximately 1 millisecond, and setting the number of rounds to zero
+(0) causes RMR to report the transient failure to the application
+without any retry attempts. If the user application does not invoke
+this function, the default is one (1) round of retries.
&proto_start
rmr_mbuf_t* rmr_torcv_msg( void* vctx, rmr_mbuf_t* old_msg, int ms_to );
&proto_end
-This function because identically to the &func(rmr_rcv_msg) function, and allows the application to
-set a specific timeout value for the receive operation.
-If a message is not received before the timeout period expires (ms_to milliseconds), a message buffer
-is returned with the state set to &cw(RMR_ERR_TIMEOUT.)
-
+This function behaves identically to the &func(rmr_rcv_msg) function,
+but allows the application to set a specific timeout value for the
+receive operation. If a message is not received before the timeout
+period expires (ms_to milliseconds), a message buffer is returned with
+the state set to &cw(RMR_ERR_TIMEOUT.)
&proto_start
-rmr_mbuf_t* rmr_tralloc_msg( void* context, int msize, int trsize,
+rmr_mbuf_t* rmr_tralloc_msg( void* context, int msize, int trsize,
unsigned const char* data );
&proto_end
-Similar to the &func(rmr_alloc_msg) this function allocates a message buffer, and adds the
-referenced trace data to the buffer.
-The new message buffer is returned.
+Similar to the &func(rmr_alloc_msg) this function allocates a message
+buffer, and adds the referenced trace data to the buffer. The new
+message buffer is returned.
&proto_start
void rmr_wh_close( void* vctx, int whid );
&proto_end
This function closes an existing wormhole connection.
-
&proto_start
rmr_whid_t rmr_wh_open( void* vctx, char const* target );
&proto_end
-This function allows the application to create a &ital(wormhole,) direct, connection to another application.
-The peer application must also be using RMR (messages sent on a wormhole connection are RMR messages).
-The target may be a hostname:port or IP-address:port combination.
-Upon successful completion, the &ital(wormhole ID) is returned; this ID must be passed on all
-subsequent calls to wormhole functions for this connection.
+This function allows the application to create a &ital(wormhole,) a
+direct connection to another application. The peer application must
+also be using RMR (messages sent on a wormhole connection are RMR
+messages). The target may be a hostname:port or IP-address:port
+combination. Upon successful completion, the &ital(wormhole ID) is
+returned; this ID must be passed on all subsequent calls to wormhole
+functions for this connection.
&proto_start
rmr_mbuf_t* rmr_wh_send_msg( void* vctx, rmr_whid_t whid, rmr_mbuf_t* msg );
&proto_end
-Once a wormhole has been established to a peer application, this function allows the application
-to send a message to the peer.
-All semantics of normal RMR sends (retries, etc.) are observed.
-The application may opt not to supply the message type or subscription ID in the message as
-neither are used by RMR; they may be required by the peer application depending on the application
-level protocol(s) in use.
-
+Once a wormhole has been established to a peer application, this
+function allows the application to send a message to the peer. All
+semantics of normal RMR sends (retries, etc.) are observed. The
+application may opt not to supply the message type or subscription ID
+in the message as neither are used by RMR; they may be required by the
+peer application depending on the application level protocol(s) in
+use.
.** ------------------------------------
&space
&h2(Message Buffer Functions)
-The message buffer functions operate directly on a message buffer, and as such do not
-require that RMR context as a parameter.
+The message buffer functions operate directly on a message buffer, and
+as such do not require that RMR context as a parameter.
&space
&proto_start
int rmr_bytes2meid( rmr_mbuf_t* mbuf, unsigned char const* src, int len );
&proto_end
-Copy the bytes referenced by &ital(src) to the &ital(meid) field in the RMR message header.
-Up to &ital(len) bytes are copied, though the maximum length of the field as
-governed by &cw(RMR_MAX_MEID) is enforced.
+Copy the bytes referenced by &ital(src) to the &ital(meid) field in
+the RMR message header. Up to &ital(len) bytes are copied, though the
+maximum length of the field as governed by &cw(RMR_MAX_MEID) is
+enforced.
&proto_start
void rmr_bytes2payload( rmr_mbuf_t* mbuf, unsigned char const* src, int len );
&proto_end
-This function copies &ital(len) bytes from &ital(src) into the message buffer payload.
-This function is primarily provided to support wrappers which don't directly support
-C pointers.
-
+This function copies &ital(len) bytes from &ital(src) into the message
+buffer payload. This function is primarily provided to support
+wrappers which don't directly support C pointers.
&proto_start
int rmr_bytes2xact( rmr_mbuf_t* mbuf, unsigned char const* src, int len );
&proto_end
-This function copies &ital(len) bytes of data from &ital(src) to the transaction ID
-field in the message buffer.
-The number of bytes provided will be limited to a maximum of &cw(RMR_MAX_XID.)
+This function copies &ital(len) bytes of data from &ital(src) to the
+transaction ID field in the message buffer. The number of bytes
+provided will be limited to a maximum of &cw(RMR_MAX_XID.)
&proto_start
void rmr_free_msg( rmr_mbuf_t* mbuf );
&proto_end
-This function should be used by the application to release the storage used by a message buffer.
+This function should be used by the application to release the storage
+used by a message buffer.
&proto_start
unsigned char* rmr_get_meid( rmr_mbuf_t* mbuf, unsigned char* dest );
&proto_end
-The bytes from the &cw(meid) field of the message buffer are copied to the &ital(dest) buffer
-provided by the application.
-The full field of &cw(RMR_MAX_MEID) bytes are copied; the caller must ensure that &ital(dest)
-is large enough.
-If the destination buffer pointer passed in is a nil pointer, the function will allocate a buffer
-and return a pointer, which the caller is expected to free, to the buffer.
+The bytes from the &cw(meid) field of the message buffer are copied to
+the &ital(dest) buffer provided by the application. The full field of
+&cw(RMR_MAX_MEID) bytes are copied; the caller must ensure that
+&ital(dest) is large enough. If the destination buffer pointer passed
+in is a nil pointer, the function will allocate a buffer and return a
+pointer to the buffer, which the caller is expected to free.
&proto_start
unsigned char* rmr_get_src( rmr_mbuf_t* mbuf, unsigned char* dest );
&proto_end
-The source of a message is copied to the &ital(dest) buffer provided by the caller.
-This is generally the hostname and port, separated by a colon, of the application
-which sent the message, and is a zero terminated string.
-Up to &cw(RMR_MAX_SRC) bytes will be copied, so the caller must ensure that &ital(dest)
-is at least this large.
+The source of a message is copied to the &ital(dest) buffer provided
+by the caller. This is generally the hostname and port, separated by
+a colon, of the application which sent the message, and is a zero
+terminated string. Up to &cw(RMR_MAX_SRC) bytes will be copied, so
+the caller must ensure that &ital(dest) is at least this large.
&proto_start
unsigned char* rmr_get_srcip( rmr_mbuf_t* msg, unsigned char* dest );
&proto_end
-This function copies the source IP address and port, separated by a colon, to the
-&ital(dest) buffer provided by the caller.
-This is the address of the application which sent the message.
-Up to &cw(RMR_MAX_SRC) bytes will be copied, so the caller must ensure that &ital(dest)
-is at least this large.
+This function copies the source IP address and port, separated by a
+colon, to the &ital(dest) buffer provided by the caller. This is the
+address of the application which sent the message. Up to
+&cw(RMR_MAX_SRC) bytes will be copied, so the caller must ensure that
+&ital(dest) is at least this large.
&proto_start
int rmr_get_trlen( rmr_mbuf_t* msg );
&proto_end
-This function can be used to determine the size of the trace information in the message buffer.
-If no trace data is present, then 0 is returned.
+This function can be used to determine the size of the trace
+information in the message buffer. If no trace data is present, then 0
+is returned.
&proto_start
int rmr_get_trace( rmr_mbuf_t* msg, unsigned char* dest, int size );
&proto_end
-The bytes from the trace data, up to &tial(size) bytes, is copied from the message buffer
-to the &ital(dest) buffer provided by the caller.
+The bytes from the trace data, up to &tial(size) bytes, is copied from
+the message buffer to the &ital(dest) buffer provided by the caller.
The return value is the number of bytes actually copied.
&proto_start
unsigned char* rmr_get_xact( rmr_mbuf_t* mbuf, unsigned char* dest );
&proto_end
-The bytes from the &ital(transaction) field in the message buffer are copied to the &ital(dest) buffer
-provided by the application.
-The full field of &cw(RMR_MAX_MEID) bytes are copied; the caller must ensure that &ital(dest)
-is large enough.
-If the destination buffer pointer passed in is a nil pointer, the function will allocate a buffer
-and return a pointer, which the caller is expected to free, to the buffer.
+The bytes from the &ital(transaction) field in the message buffer are
+copied to the &ital(dest) buffer provided by the application. The
+full field of &cw(RMR_MAX_MEID) bytes are copied; the caller must
+ensure that &ital(dest) is large enough. If the destination buffer
+pointer passed in is a nil pointer, the function will allocate a
+buffer and return a pointer to the buffer, which the caller is
+expected to free.
&proto_start
int rmr_payload_size( rmr_mbuf_t* msg );
&proto_end
-This function returns the number of bytes in the message buffer's payload that are available
-for the application to use.
-
+This function returns the number of bytes in the message buffer's
+payload that are available for the application to use.
&proto_start
rmr_mbuf_t* rmr_realloc_msg( rmr_mbuf_t* mbuf, int new_tr_size );
&proto_end
-This function allows the application to reallocate a message buffer with a different trace
-data size.
-The contents of the message buffer supplied are copied to the new buffer, and a reference to
-the new buffer is returned.
+This function allows the application to reallocate a message buffer
+with a different trace data size. The contents of the message buffer
+supplied are copied to the new buffer, and a reference to the new
+buffer is returned.
&proto_start
int rmr_set_trace( rmr_mbuf_t* msg, unsigned const char* data, int size );
&proto_end
-The &ital(size) bytes, up to the size of the trace data in the message buffer,
-at &ital(data) are copied to the trace portion of the message buffer.
-The return value is the actual number of bytes copied which could be less than the number
-requested.
-
+The &ital(size) bytes, up to the size of the trace data in the message
+buffer, at &ital(data) are copied to the trace portion of the message
+buffer. The return value is the actual number of bytes copied which
+could be less than the number requested.
&proto_start
int rmr_str2meid( rmr_mbuf_t* mbuf, unsigned char const* str );
&proto_end
-Accepts a pointer to a zero terminated string an copies it to the &cw(meid) field in the
-message header.
-Up to &cw(RMR_MAX_MEID) bytes are copied (including the final 0), and the number
-copied is returned.
+Accepts a pointer to a zero terminated string an copies it to the
+&cw(meid) field in the message header. Up to &cw(RMR_MAX_MEID) bytes
+are copied (including the final 0), and the number copied is returned.
&proto_start
void rmr_str2payload( rmr_mbuf_t* mbuf, unsigned char const* str );
&proto_end
-Accepts a pointer to a zero terminated string, and copies the string to the payload
-portion of the message buffer.
-If the string is longer than the allocated payload, the string will be truncated
-and will &bold(not) be terminated with a zero byte.
-
+Accepts a pointer to a zero terminated string, and copies the string
+to the payload portion of the message buffer. If the string is longer
+than the allocated payload, the string will be truncated and will
+&bold(not) be terminated with a zero byte.
&proto_start
int rmr_str2xact( rmr_mbuf_t* mbuf, unsigned char const* str );
&proto_end
-Accepts a pointer to a zero terminated string and copies the string to the transaction ID
-portion of the message buffer.
-If the string is longer than &cw(RMR_MAX_XID,) the string will be truncated and will &bold(not)
-be zero terminated.
+Accepts a pointer to a zero terminated string and copies the string to
+the transaction ID portion of the message buffer. If the string is
+longer than &cw(RMR_MAX_XID,) the string will be truncated and will
+&bold(not) be zero terminated.
&proto_start
void* rmr_trace_ref( rmr_mbuf_t* msg, int* sizeptr );
&proto_end
-This function returns a pointer to the trace information in the message buffer.
-The intent is that the application will treat this as a read/only field and will not
-write trace data into the message buffer.
-The length of data available should be determined by calling &func(rmr_get_trlen:.)
-
-
-
+This function returns a pointer to the trace information in the
+message buffer. The intent is that the application will treat this as
+a read/only field and will not write trace data into the message
+buffer. The length of data available should be determined by calling
+&func(rmr_get_trlen:.)
.fi
&h1(Appendix &code_appendix -- Code Examples)
-The following snippet of code illustrate some of the basic operation of the RMR
-library.
-Please refer to the examples and test directories in the RMR repository for
-complete RMR based programmes.
+The following snippet of code illustrate some of the basic operation
+of the RMR library. Please refer to the examples and test directories
+in the RMR repository for complete RMR based programmes.
.im code_send.im
.im code_rcv.im
.** example sender code
&h2(Receiver Sample)
-The receiver code is even more simple than the sender code as it does not
-need to wait for a route table to arrive (only senders need to do that), nor
-does it need to allocate an initial buffer.
-The example assumes that the sender is transmitting a zero terminated string
-as the payload.
+The receiver code is even simpler than the sender code as it does not
+need to wait for a route table to arrive (only senders need to do
+that), nor does it need to allocate an initial buffer. The example
+assumes that the sender is transmitting a zero terminated string as
+the payload.
&space
.** example sender code
&h2(Sender Sample)
-The following code segment shows how a message buffer can be allocated,
-populated, and sent.
-The snippet also illustrates how the result from the &func(rmr_send_msg)
-function is used to send the next message.
-It does not illustrate error and/or retry handling.
+The following code segment shows how a message buffer can be
+allocated, populated, and sent. The snippet also illustrates how the
+result from the &func(rmr_send_msg) function is used to send the next
+message. It does not illustrate error and/or retry handling.
&space
&indent
.** example sender code
&h2(Receive and Send Sample)
-The following code snippet receives messages and responds to the sender
-if the message type is odd.
-The code illustrates how the received message may be used to return
-a message to the source.
-Variable type definitions are omitted for clarity and should be obvious.
+The following code snippet receives messages and responds to the
+sender if the message type is odd. The code illustrates how the
+received message may be used to return a message to the source.
+Variable type definitions are omitted for clarity and should be
+obvious.
&space
-It should also be noted that
-things like the message type which id returned to the sender (99) is
-a random value that these applications would have agreed on in advance
-and is &bold(not) an RMR definition.
+It should also be noted that things like the message type which id
+returned to the sender (99) is a random value that these applications
+would have agreed on in advance and is &bold(not) an RMR definition.
&space
.fi
&h1(Configuration and Control)
-With the assumption that most RMR based applications will be executed in a containerised
-environment, there are some underlying mechanics which the developer may need to know
-in order to properly provide a configuration specification to the container management
-system.
-The following paragraphs briefly discuss these.
+With the assumption that most RMR based applications will be executed
+in a containerised environment, there are some underlying mechanics
+which the developer may need to know in order to properly provide a
+configuration specification to the container management system. The
+following paragraphs briefly discuss these.
.sp .1
&h2(TCP Ports)
-RMR requires two (2) TCP listen ports: one for general application to application communications
-and one for route table updates.
-The general communication port is specified by the application at the time RMR is initialised.
-The port used to listen for route table updates is likely to be a constant port shared by all
-applications provided they are running in separate containers.
-To that end, the port number defaults to 4561, but can be configured with an environment variable
-(see later paragraph in this section).
+RMR requires two (2) TCP listen ports: one for general
+application-to-application communications and one for route-table
+updates. The general communication port is specified by the
+application at the time RMR is initialised. The port used to listen
+for route table updates is likely to be a constant port shared by all
+applications provided they are running in separate containers. To
+that end, the port number defaults to 4561, but can be configured with
+an environment variable (see later paragraph in this section).
&h2(Host Names)
-RMR is typically host name agnostic.
-Route table entries may contain endpoints defined either by host name or IP address.
-In the container world the concept of a &ital(service name) might exist, and likely is different
-than a host name.
-RMR's only requirement with respect to host names is that a name used on a route table entry must
-be resolvable via the &cw(gethostbyname) system call.
+RMR is typically host name agnostic. Route table entries may contain
+endpoints defined either by host name or IP address. In the container
+world the concept of a &ital(service name) might exist, and likely is
+different than a host name. RMR's only requirement with respect to
+host names is that a name used on a route table entry must be
+resolvable via the &cw(gethostbyname) system call.
&h2(Environment Variables)
-Several environment variables are recognised by RMR which,
-in general, are used to define interfaces and listen ports (e.g. the route table update
-listen port), or debugging information.
-Generally this information is system controlled and thus RMR expects this information to
-be defined in the environment rather than provided by the application.
-The following is a list of the environment variables which RMR recognises:
+Several environment variables are recognised by RMR which, in general,
+are used to define interfaces and listen ports (e.g. the route table
+update listen port), or debugging information. Generally this
+information is system controlled and thus RMR expects this information
+to be defined in the environment rather than provided by the
+application. The following is a list of the environment variables
+which RMR recognises:
&half_space
.st 8p
&di(RMR_SEED_RT) Where RMR expects to find the name of the seed (static) route table. If not defined no static table is read.
&half_space
- &di(RMR_RTG_ISRAW) If the value set to 0, RMR expects the route table manager messages to be messages with and RMR header.
+ &di(RMR_RTG_ISRAW) If the value set to 0, RMR expects the route table manager messages to be messages with and RMR header.
If this is not defined messages are assumed to be "raw" (without an RMR header.
&half_space
- &di(RMR_VCTL_FILE) Provides a file which is used to set the verbose level of the route table collection thread.
- The first line of the file is read and expected to contain an integer value to set the verbose level.
+ &di(RMR_VCTL_FILE) Provides a file which is used to set the verbose level of the route table collection thread.
+ The first line of the file is read and expected to contain an integer value to set the verbose level.
The value may be changed at any time and the route table thread will adjust accordingly.
&half_space
&space
&h2(Logging)
-RMR does &bold(not) use any logging libraries; any error or warning messages are written to standard error.
-.if false
+RMR does &bold(not) use any logging libraries; any error or warning messages are written to standard error.
+.if false
¬e .sm .
.cn l=&cn_line_len i=0 start &atbot Times-roman 8p .7i
This is standard practice for container based applications as it makes error output easily available to operations.
&half_space
&di(^[ERR]) The event is not expected and RMR is not able to handle it. There is a small chance that continued operation
- will be negatively impacted.
+ will be negatively impacted.
Eventual action to diagnose and correct the issue will be necessary.
&half_space
.fi
&h1(Handling Failures)
-The vast majority of states reported by RMR are fatal; if encountered during setup or initialisation,
-then it is unlikely that any message oriented processing should continue, and when encountered on
-a message operation continued operation on that message should be abandoned.
-Specifically with regard to message sending, it is very likely that the underlying transport mechanism
-will report a &ital(soft,) or transient, failure which might be successful if the operation is retried at a
-later point in time.
-The paragraphs below discuss the methods that an application might deal with these soft failures.
+The vast majority of states reported by RMR are fatal; if encountered
+during setup or initialization, then it is unlikely that any message
+oriented processing should continue, and when encountered on a message
+operation continued operation on that message should be abandoned.
+Specifically with regard to message sending, it is very likely that
+the underlying transport mechanism will report a &ital(soft,) or
+transient, failure which might be successful if the operation is
+retried at a later point in time. The paragraphs below discuss the
+methods that an application might deal with these soft failures.
&h2(Failure Notification)
-When a soft failure is reported, the returned message buffer returned by the RMR function will be &cw(RMR_ERR_RETRY.)
-These types of failures can occur for various reasons; one of two reasons is typically the underlying cause:
+When a soft failure is reported, the returned message buffer returned
+by the RMR function will be &cw(RMR_ERR_RETRY.) These types of
+failures can occur for various reasons; one of two reasons is
+typically the underlying cause:
&half_space
&indent
&uindent
&space
-Unfortunately, it is not possible for RMR to determine which of these two cases is occurring, and equally
-as unfortunate the time to resolve each is different.
-The first, no connection, may require up to a second before a message can be accepted, while a rejection
-because of buffer shortage is likely to resolve in less than a millisecond.
+Unfortunately, it is not possible for RMR to determine which of these
+two cases is occurring, and equally as unfortunate the time to resolve
+each is different. The first, no connection, may require up to a
+second before a message can be accepted, while a rejection because of
+buffer shortage is likely to resolve in less than a millisecond.
&h2(Application Response)
-The action which an application takes when a soft failure is reported ultimately depends on the nature
-of the application with respect to factors such as tolerance to extended message latency, dropped messages,
-and over all message rate.
+The action which an application takes when a soft failure is reported
+ultimately depends on the nature of the application with respect to
+factors such as tolerance to extended message latency, dropped
+messages, and over all message rate.
&h2(RMR Retry Modes)
-In an effort to reduce the workload of an application developer, RMR has a default retry policy such that
-RMR will attempt to retransmit a message up to 1000 times when a soft failure is reported.
-These retries generally take less than 1 millisecond (if all 1000 are attempted) and in most cases eliminates
-nearly all reported soft failures to the application.
-When using this mode, it might allow the application to simply treat all bad return values from a send attempt
-as permanent failures.
-&space
-
-If an application is so sensitive to any delay in RMR, or the underlying transport mechanism, it is possible to
-set RMR to return a failure immediately on any kind of error (permanent failures are always reported without retry).
-In this mode, RMR will still set the state in the message buffer to &cw(RMR_ERR_RETRY,) but will &bold(not)
-make any attempts to resend the message.
-This zero-retry policy is enabled by invoking the &func(rmr_set_stimeout) with a value of 0; this can be done once
-immediately after &func(rmr_init:) is invoked.
+In an effort to reduce the workload of an application developer, RMR
+has a default retry policy such that RMR will attempt to retransmit a
+message up to 1000 times when a soft failure is reported. These
+retries generally take less than 1 millisecond (if all 1000 are
+attempted) and in most cases eliminates nearly all reported soft
+failures to the application. When using this mode, it might allow the
+application to simply treat all bad return values from a send attempt
+as permanent failures. &space
+
+If an application is so sensitive to any delay in RMR, or the
+underlying transport mechanism, it is possible to set RMR to return a
+failure immediately on any kind of error (permanent failures are
+always reported without retry). In this mode, RMR will still set the
+state in the message buffer to &cw(RMR_ERR_RETRY,) but will &bold(not)
+make any attempts to resend the message. This zero-retry policy is
+enabled by invoking the &func(rmr_set_stimeout) with a value of 0;
+this can be done once immediately after &func(rmr_init:) is invoked.
&space
-Regardless of the retry mode which the application sets, it will ultimately be up to the application to
-handle failures by queuing the message internally for resend, retrying immediately, or dropping the
-send attempt all together.
-As stated before, only the application can determine how to best handle send failures.
+Regardless of the retry mode which the application sets, it will
+ultimately be up to the application to handle failures by queuing the
+message internally for resend, retrying immediately, or dropping the
+send attempt all together. As stated before, only the application can
+determine how to best handle send failures.
&h2(Other Failures)
-RMR will return the state of processing for message based operations (send/receive) as the status in
-the message buffer.
-For non-message operations, state is returned to the caller as the integer return value for all functions
-which are not expected to return a pointer (e.g. &func(rmr_init:).)
-The following are the RMR state constants and a brief description of their meaning.
+RMR will return the state of processing for message based operations
+(send/receive) as the status in the message buffer. For non-message
+operations, state is returned to the caller as the integer return
+value for all functions which are not expected to return a pointer
+(e.g. &func(rmr_init:).) The following are the RMR state constants
+and a brief description of their meaning.
&space
.st 8p
&di(RMR_ERR_UNSET) the message hasn't been populated with a transport buffer
&half_space
- &di(RMR_ERR_TRUNC) length in the received buffer is longer than the size of the allocated payload,
+ &di(RMR_ERR_TRUNC) length in the received buffer is longer than the size of the allocated payload,
received message likely truncated (length set by sender could be wrong, but we can't know that)
&half_space
.st &textsize
&space
-Depending on the underlying transport mechanism, and the nature of the call that RMR attempted, the
-system &cw(errno) value might reflect additional detail about the failure.
-Applications should &bold(not) rely on errno as some transport mechanisms do not set it with
-any consistency.
+Depending on the underlying transport mechanism, and the nature of the
+call that RMR attempted, the system &cw(errno) value might reflect
+additional detail about the failure. Applications should &bold(not)
+rely on errno as some transport mechanisms do not set it with any
+consistency.
.fi
&h1(General Use)
-To use, the RMR based application simply needs to initialise the RMR environment, wait for RMR to
-have received a routing table (become ready), and then invoke either the send or receive functions.
-These steps, and some behind the scenes details, are described in the following paragraphs.
+To use, the RMR based application simply needs to initialise the RMR
+environment, wait for RMR to have received a routing table (become
+ready), and then invoke either the send or receive functions. These
+steps, and some behind the scenes details, are described in the
+following paragraphs.
&h2(Initialisation)
-The RMR function &func(rmr_init:) is used to set up the RMR environment and must be called before messages
-can be sent or received.
-One of the few parameters that the application must communicate to RMR is the port number that will be
-used as the listen port for new connections.
-The port number is passed on the initialisation function call and a TCP listen socket will be opened
-with this port.
-If the port is already in use RMR will report a failure; the application will need to reinitialise with
-a different port number, abort, or take some other action appropriate for the application.
+The RMR function &func(rmr_init:) is used to set up the RMR
+environment and must be called before messages can be sent or
+received. One of the few parameters that the application must
+communicate to RMR is the port number that will be used as the listen
+port for new connections. The port number is passed on the
+initialisation function call and a TCP listen socket will be opened
+with this port. If the port is already in use RMR will report a
+failure; the application will need to reinitialise with a different
+port number, abort, or take some other action appropriate for the
+application.
.sp
-In addition to creating a TCP listen port, RMR will start a process thread which will be responsible for
-receiving dynamic updates to the route table.
-This thread also causes a TCP listen port to be opened as it is expected that the process which generates
-route table updates will connect and send new information when needed.
-The route table update port is &bold(not) supplied by the application, but is supplied via an environment
-variable as this value is likely determined by the mechanism which is starting and configuring the application.
+In addition to creating a TCP listen port, RMR will start a process
+thread which will be responsible for receiving dynamic updates to the
+route table. This thread also causes a TCP listen port to be opened
+as it is expected that the process which generates route table updates
+will connect and send new information when needed. The route table
+update port is &bold(not) supplied by the application, but is supplied
+via an environment variable as this value is likely determined by the
+mechanism which is starting and configuring the application.
&h3(The RMR Context)
-On successful initialisation, a void pointer, often called a &ital(handle) by some programming languages,
-is returned to the application. This is a reference to the RMR control information and must be passed as the
-first parameter on most RMR functions calls.
+On successful initialisation, a void pointer, often called a
+&ital(handle) by some programming languages, is returned to the
+application. This is a reference to the RMR control information and
+must be passed as the first parameter on most RMR function calls.
RMR refers to this as the context, or ctx.
&h2(Wait For Ready)
-An application which is only receiving messages does not need to wait for RMR to &ital(become ready) after
-the call to the initialisation function.
-However, before the application can successfully send a message, RMR must have loaded a route table, and
-the application must wait for RMR to report that it has done so.
-The RMR function &func(rmr_ready:) will return the value &ital(true) (1) when a complete route table has
-been loaded and can be used to determine the endpoint for a send request.
+An application which is only receiving messages does not need to wait
+for RMR to &ital(become ready) after the call to the initialization
+function. However, before the application can successfully send a
+message, RMR must have loaded a route table, and the application must
+wait for RMR to report that it has done so. The RMR function
+&func(rmr_ready:) will return the value &ital(true) (1) when a
+complete route table has been loaded and can be used to determine the
+endpoint for a send request.
&h2(Receiving Messages)
-The process of receiving is fairly straight forward.
-The application invokes the RMR &func(rmr_rcv_msg:) function which will block until a message is received.
-The function returns a pointer to a message block which provides all of the details about the message.
-Specifically, the application has access to the following information either directly or indirectly:
+The process of receiving is fairly straight forward. The application
+invokes the RMR &func(rmr_rcv_msg:) function which will block until a
+message is received. The function returns a pointer to a message
+block which provides all of the details about the message.
+Specifically, the application has access to the following information
+either directly or indirectly:
&half_space
&indent
&space
&h3(The Message Payload)
-The message payload contains the &ital(raw) data that was sent by the peer application.
-The format will likely depend on the message type, and is expected to be known by the application.
-A direct pointer to the payload is available from the message buffer (see appendix &mbuf_appendix
-for specific message buffer details).
+The message payload contains the &ital(raw) data that was sent by the
+peer application. The format will likely depend on the message type,
+and is expected to be known by the application. A direct pointer to
+the payload is available from the message buffer (see appendix
+&mbuf_appendix for specific message buffer details).
&space
-Two payload related length values are also directly available: the total payload length, and the number of bytes
-actually filled with data.
-The used length is set by the caller, and may or not be an accurate value.
-The total payload length is determined when the buffer is created for sending, and is the maximum
-number of bytes that the application may modify should the buffer be used to return a response.
+Two payload-related length values are also directly available: the
+total payload length, and the number of bytes actually filled with
+data. The used length is set by the caller, and may or not be an
+accurate value. The total payload length is determined when the
+buffer is created for sending, and is the maximum number of bytes that
+the application may modify should the buffer be used to return a
+response.
&h3(Message Type and Subscription ID)
-The message type and subscription ID are both directly available from the message buffer, and
-are the values which were used to by RMR in the sending application to select the endpoint.
-If the application resends the message, as opposed to returning the message buffer as a response,
-the message number and/or the subscription ID might need to be changed to avoid potential issues ¬e .sm .
+The message type and subscription ID are both directly available from
+the message buffer, and are the values which were used to by RMR in
+the sending application to select the endpoint. If the application
+resends the message, as opposed to returning the message buffer as a
+response, the message number and/or the subscription ID might need to
+be changed to avoid potential issues ¬e .sm .
.cn l=&cn_line_len i=&cn_ident start &atbot Times-roman 8p 1i
- It is entirely possible to design a routing table, and application group, such that the same message type is
- is left unchanged and the message is forwarded by an application after updating the payload. This type
- of behaviour is often referred to as service chaining, and can be done without any "knowledge" by
- an application with respect to where the message goes next. Service chaining is supported by RMR
- in as much as it allows the message to be resent, but the actual complexities of designing
- and implementing service chaining lie with the route table generator process.
+ It is entirely possible to design a routing table, and
+ application group, such that the same message type is
+ is left unchanged and the message is forwarded by an
+ application after updating the payload. This type of behaviour
+ is often referred to as service chaining, and can be done
+ without any "knowledge" by an application with respect to
+ where the message goes next. Service chaining is supported
+ by RMR in as much as it allows the message to be resent, but
+ the actual complexities of designing and implementing service
+ chaining lie with the route table generator process.
.cn end
&h3(Sender Information)
-The source, or sender information, is indirectly available to the application via the &func(rmr_get_src:) and
-&func(rmr_get_ip:) functions.
-The former returns a string containing &cw(hostname^:port,) while the string &cw(ip^:port) is returned
-by the latter.
+The source, or sender information, is indirectly available to the
+application via the &func(rmr_get_src:) and &func(rmr_get_ip:)
+functions. The former returns a string containing
+&cw(hostname^:port,) while the string &cw(ip^:port) is returned by the
+latter.
-&h3(Transaction ID)
-The message buffer contains a fixed length set of bytes which applications can set to track related messages
-across the application concept of a transaction.
-RMR will use the transaction ID for matching a response message when the &func(rmr_call:) function is used to
-send a message.
+&h3(Transaction ID)
+The message buffer contains a fixed length set of bytes which
+applications can set to track related messages across the application
+concept of a transaction. RMR will use the transaction ID for
+matching a response message when the &func(rmr_call:) function is used
+to send a message.
&h3(Trace Information)
-RMR supports the addition of an optional trace information to any message.
-The presence and size is controlled by the application, and can vary from message to message if desired.
-The actual contents of the trace information is determined by the application; RMR provides only the means to
-set, extract, and obtain a direct reference to the trace bytes.
-The trace data field in a message buffer is discussed in greater detail in the &ital(Trace Data) section.
+RMR supports the addition of an optional trace information to any
+message. The presence and size is controlled by the application, and
+can vary from message to message if desired. The actual contents of
+the trace information is determined by the application; RMR provides
+only the means to set, extract, and obtain a direct reference to the
+trace bytes. The trace data field in a message buffer is discussed in
+greater detail in the &ital(Trace Data) section.
&h2(Sending Messages)
-Sending requires only slightly more work on the part of the application than receiving a message.
-The application must allocate an RMR message buffer, populate the message payload with data, set the
+Sending requires only slightly more work on the part of the
+application than receiving a message. The application must allocate
+an RMR message buffer, populate the message payload with data, set the
message type and length, and optionally set the subscription ID.
-Information such as the source IP address, hostname, and port are automatically added to the message buffer
-by RMR, so there is no need for the application to worry about these.
-
+Information such as the source IP address, hostname, and port are
+automatically added to the message buffer by RMR, so there is no need
+for the application to worry about these.
&h3(Message Buffer Allocation)
-The function &func(rmr_msg_alloc:) allocates a &ital(zero copy) buffer and returns a pointer to the RMR
-&cw(rmr_mbuf_t) structure.
-The message buffer provides direct access to the payload, length, message type and subscription ID fields.
-The buffer must be preallocated in order to allow the underlying transport mechanism to allocate the payload
-space from it's internal memory pool; this eliminates multiple copies as the message is sent, and thus is
-more efficient.
+The function &func(rmr_msg_alloc:) allocates a &ital(zero copy) buffer
+and returns a pointer to the RMR &cw(rmr_mbuf_t) structure. The
+message buffer provides direct access to the payload, length, message
+type and subscription ID fields. The buffer must be preallocated in
+order to allow the underlying transport mechanism to allocate the
+payload space from its internal memory pool; this eliminates multiple
+copies as the message is sent, and thus is more efficient.
.sp
-If a message buffer has been received, and the application wishes to use the buffer to send a response, or
-to forward the buffer to another application, a new buffer does &bold(not) need to be allocated.
-The application may set the necessary information (message type, etc.), and adjust the payload, as is
-necessary and then pass the message buffer to &func(rmr_send_msg:) or &func(rmr_rts_msg:) to be sent or
+If a message buffer has been received, and the application wishes to
+use the buffer to send a response, or to forward the buffer to another
+application, a new buffer does &bold(not) need to be allocated. The
+application may set the necessary information (message type, etc.),
+and adjust the payload, as is necessary and then pass the message
+buffer to &func(rmr_send_msg:) or &func(rmr_rts_msg:) to be sent or
returned to the sender.
&h3(Populating the Message Buffer)
-The application has direct access to several of the message buffer fields, and should set them appropriately.
+The application has direct access to several of the message buffer
+fields, and should set them appropriately.
&half_space
&indent
&beg_dlist( 1i &ditext )
- &di(len) This is the number of bytes that the application placed into the payload. Setting length to 0 is
- allowed, and length may be less than the allocated payload size.
+ &di(len) This is the number of bytes that the application
+ placed into the payload. Setting length to 0
+ is allowed, and length may be less than the
+ allocated payload size.
&half_space
- &di(mtype) The message type that RMR will use to determine the endpoint used as the target of the send.
+ &di(mtype) The message type that RMR will use to determine the
+ endpoint used as the target of the send.
&half_space
- &di(sub_id) The subscription ID if the message is to be routed based on the combination of message type and subscription ID.
- If no subscription ID is valid for the message, the application should set the field with the
- RMR constant &cw(RMR_VOID_SUBID.)
+ &di(sub_id) The subscription ID if the message is to be routed
+ based on the combination of message type and
+ subscription ID. If no subscription ID is
+ valid for the message, the application should
+ set the field with the RMR constant
+ &cw(RMR_VOID_SUBID.)
&half_space
- &di(payload) The application should obtain the reference (pointer) to the payload from the message buffer
- and place any data into the payload. The application is responsible for ensuring that the
- maximum payload size is not exceeded. The application may obtain the maximum size via the &func(rmr_payload_size:)
- function.
+ &di(payload) The application should obtain the reference
+ (pointer) to the payload from the message
+ buffer and place any data into the payload.
+ The application is responsible for ensuring
+ that the maximum payload size is not exceeded.
+ The application may obtain the maximum size
+ via the &func(rmr_payload_size:) function.
&half_space
- &di(trace data) Optionally, the application may add trace information to the message buffer.
+ &di(trace data) Optionally, the application may add trace
+ information to the message buffer.
&end_dlist
&space
&uindent
&h3(Sending a Message Buffer)
-Once the application has populated the necessary bits of a message, it may be sent by passing the buffer to
-the &func(rmr_send_msg:) function.
-This function will select an endpoint to receive the message, based on message type and subscription ID, and
-will pass the message to the underlying transport mechanism for actual transmission on the connection.
-(Depending on the underlying transport mechanism, the actual connection to the endpoint may happen at the time of
-the first message sent to the endpoint, and thus the latency of the first send might be longer than expected.)
+Once the application has populated the necessary bits of a message, it
+may be sent by passing the buffer to the &func(rmr_send_msg:)
+function. This function will select an endpoint to receive the
+message, based on message type and subscription ID, and will pass the
+message to the underlying transport mechanism for actual transmission
+on the connection. (Depending on the underlying transport mechanism,
+the actual connection to the endpoint may happen at the time of the
+first message sent to the endpoint, and thus the latency of the first
+send might be longer than expected.)
&space
-On success, the send function will return a reference to a message buffer; the status within that message
-buffer will indicate what the message buffer contains.
-When the status is &cw(RMR_OK) the reference is to a &bold(new) message buffer for the application to use for the next
-send; the payload size is the same as the payload size allocated for the message that was just sent.
-This is a convenience as it eliminates the need for the application to call the message allocation function
-at some point in the future, and assumes the application will send many messages which will require the same
-payload dimensions.
+On success, the send function will return a reference to a message
+buffer; the status within that message buffer will indicate what the
+message buffer contains. When the status is &cw(RMR_OK) the reference
+is to a &bold(new) message buffer for the application to use for the
+next send; the payload size is the same as the payload size allocated
+for the message that was just sent. This is a convenience as it
+eliminates the need for the application to call the message allocation
+function at some point in the future, and assumes the application will
+send many messages which will require the same payload dimensions.
&space
-If the message contains any status other than &cw(RMR_OK,) then the message could &bold(not) be sent, and the
-reference is to the unsent message buffer.
-The value of the status will indicate whether the nature of the failure was transient ( .sm &cw(RMR_ERR_RETRY) .sm )
-or not.
-Transient failures are likely to be successful if the application attempts to send the message at a later time.
-Unfortunately, it is impossible for RMR to know the exact transient failure (e.g. connection being established, or
-TCP buffer shortage), and thus it is not possible to communicate how long the application should wait before
-attempting to resend, if the application wishes to resend the message.
-(More discussion with respect to message retries can be found in the &ital(Handling Failures) section.)
-
-
+If the message contains any status other than &cw(RMR_OK,) then the
+message could &bold(not) be sent, and the reference is to the unsent
+message buffer. The value of the status will indicate whether the
+nature of the failure was transient ( .sm &cw(RMR_ERR_RETRY) .sm ) or
+not. Transient failures are likely to be successful if the
+application attempts to send the message at a later time.
+Unfortunately, it is impossible for RMR to know the exact transient
+failure (e.g. connection being established, or TCP buffer shortage),
+and thus it is not possible to communicate how long the application
+should wait before attempting to resend, if the application wishes to
+resend the message. (More discussion with respect to message retries
+can be found in the &ital(Handling Failures) section.)
&h1(Appendix &gloss_appendix -- Glossary)
-Many terms in networking can be interpreted with multiple meanings, and several terms used
-in this document are RMR specific.
-The following definitions are the meanings of terms used within this document and should
-help the reader to understand the intent of meaning.
+Many terms in networking can be interpreted with multiple meanings,
+and several terms used in this document are RMR specific. The
+following definitions are the meanings of terms used within this
+document and should help the reader to understand the intent of
+meaning.
&beg_dlist( &1.0i Helvetica-bold )
-&term(application) A programme which uses RMR to send and/or receive messages to/from another RMR based application.
-
-&term(Critical error) An error that RMR has encountered which will prevent further successful processing by
- RMR. Critical errors usually indicate that the application should abort.
-
-&term(Endpoint) An RMR based application that is defined as being capable of receiving one or more types of messages
- (as defined by a &ital(message key.) .sm )
-
-&term(Environment variable) A key/value pair which is set externally to the application, but which is available
- to the application (and referenced libraries) through the &cw(getenv) system call. Environment variables
- are the main method of communicating information such as port numbers to RMR.
-
-&term(Error) An abnormal condition that RMR has encountered, but will not affect the overall processing by RMR,
- but may impact certain aspects such as the ability to communicate with a specific endpoint.
- Errors generally indicate that something, usually external to RMR, must be addressed.
-
-&term(Host name) The name of the host as returned by the &cw(gethostbyname) system call. In a containerised
- environment this might be the container or service name depending on how the container is started.
- From RMR's point of view, a host name can be used to resolve an &ital(endpoint) definition in
- a &ital(route table.)
-
-&term(IP) Internet protocol. A low level transmission protocol which governs the transmission of datagrams
+&term(application) A programme which uses RMR to send and/or receive
+ messages to/from another RMR based application.
+
+&term(Critical error) An error that RMR has encountered which will
+ prevent further successful processing by
+ RMR. Critical errors usually indicate that the
+ application should abort.
+
+&term(Endpoint) An RMR based application that is defined as being
+ capable of receiving one or more types of
+ messages (as defined by a &ital(message key.)
+ .sm )
+
+&term(Environment variable) A key/value pair which is set externally
+ to the application, but which is available to
+ the application (and referenced libraries)
+ through the &cw(getenv) system
+ call. Environment variables are the main
+ method of communicating information such as
+ port numbers to RMR.
+
+&term(Error) An abnormal condition that RMR has encountered, but will
+ not affect the overall processing by RMR, but
+ may impact certain aspects such as the ability
+ to communicate with a specific endpoint.
+ Errors generally indicate that something,
+ usually external to RMR, must be addressed.
+
+&term(Host name) The name of the host as returned by the
+ &cw(gethostbyname) system call. In a
+ containerised environment this might be the
+ container or service name depending on how the
+ container is started. From RMR's point of
+ view, a host name can be used to resolve an
+ &ital(endpoint) definition in a &ital(route
+ table.)
+
+&term(IP) Internet protocol. A low level transmission protocol
+ which governs the transmission of datagrams
across network boundaries.
-&term(Listen socket) A &ital(TCP) socket used to await incoming connection requests. Listen sockets are defined
- by an interface and port number combination where the port number is unique for the interface.
-
-&term(Message) A series of bytes transmitted from the application to another RMR based application. A message
- is comprised of RMR specific data (a header), and application data (a payload).
-
-&term(Message buffer) A data structure used to describe a message which is to be sent or has been received.
- The message buffer includes the payload length, message type, message source, and other information.
-
-&term(Messgae type) A signed integer (0-32000) which identifies the type of message being transmitted, and is
- one of the two components of a &ital(routing key.) See &ital(Subscription ID.)
-
-&term(Payload) The portion of a message which holds the user data to be transmitted to the remote &ital(endpoint.)
- The payload contents are completely application defined.
-
-&term(RMR context) A set of information which defines the current state of the underlying transport connections that
- RMR is managing. The application will be give a context reference (pointer) that is supplied to most
- RMR functions as the first parameter.
-
-&term(Round robin) The method of selecting an &ital(endpoint) from a list such that all &ital(endpoints) are selected
- before starting at the head of the list.
-
-&term(Route table) A series of "rules" which define the possible &ital(endpoints) for each &ital(message key.)
-
-&term(Route table manager) An application responsible for building a &ital(route table) and then distributing it to
+&term(Listen socket) A &ital(TCP) socket used to await incoming
+ connection requests. Listen sockets are
+ defined by an interface and port number
+ combination where the port number is unique
+ for the interface.
+
+&term(Message) A series of bytes transmitted from the application to
+ another RMR based application. A message is
+ comprised of RMR specific data (a header), and
+ application data (a payload).
+
+&term(Message buffer) A data structure used to describe a message
+ which is to be sent or has been received. The
+ message buffer includes the payload length,
+ message type, message source, and other
+ information.
+
+&term(Messgae type) A signed integer (0-32000) which identifies the
+ type of message being transmitted, and is one
+ of the two components of a &ital(routing key.)
+ See &ital(Subscription ID.)
+
+&term(Payload) The portion of a message which holds the user data to
+ be transmitted to the remote &ital(endpoint.)
+ The payload contents are completely
+ application defined.
+
+&term(RMR context) A set of information which defines the current
+ state of the underlying transport connections
+ that RMR is managing. The application will be
+ give a context reference (pointer) that is
+ supplied to most RMR functions as the first
+ parameter.
+
+&term(Round robin) The method of selecting an &ital(endpoint) from a
+ list such that all &ital(endpoints) are
+ selected before starting at the head of the list.
+
+&term(Route table) A series of "rules" which define the possible
+ &ital(endpoints) for each &ital(message key.)
+
+&term(Route table manager) An application responsible for building a
+ &ital(route table) and then distributing it to
all applicable RMR based applications.
-&term(Routing) The process of selecting an &ital(endpoint) which will be the recipient of a message.
+&term(Routing) The process of selecting an &ital(endpoint) which will
+be the recipient of a message.
-&term(Routing key) A combination of &ital(message type) and &ital(subscription ID) which RMR uses to select the
- destination &ital(endpoint) when sending a message.
+&term(Routing key) A combination of &ital(message type) and
+ &ital(subscription ID) which RMR uses to
+ select the destination &ital(endpoint) when
+ sending a message.
-&term(Source) The sender of a message.
+&term(Source) The sender of a message.
-&term(Subscription ID) A signed integer value (0-32000) which identifies the subscription characteristic of a message.
- It is used in conjunction with the &ital(message type) to determine the &ital(routing key.)
+&term(Subscription ID) A signed integer value (0-32000) which
+ identifies the subscription characteristic of
+ a message. It is used in conjunction with the
+ &ital(message type) to determine the
+ &ital(routing key.)
&term(Target) The &ital(endpoint) selected to receive a message.
-&term(TCP) Transmission Control Protocol. A connection based internet protocol which provides for
- lossless packet transportation, usually over IP.
-
-&term(Thread) Also called a &ital(process thread, or pthread.) This is a lightweight process which executes in concurrently with
- the application and shares the same address space.
- RMR uses threads to manage asynchronous functions such as route table updates.
-
-&Term(Trace information) An optional portion of the message buffer that the application may populate with data
- that allows for tracing the progress of the transaction or application activity across components.
- RMR makes no use of this data.
-
-&term(Transaction ID) A fixed number of bytes in the &ital(message buffer) which the application may populate with
- information related to the transaction. RMR makes use of the transaction ID for matching response messages
- with the &c(rmr_call) function is used to send a message.
-
-&term(Transient failure) An error state that is believed to be short lived and that the operation, if retried by the
- application, might be successful. C programmers will recognise this as &cw(EAGAIN.)
+&term(TCP) Transmission Control Protocol. A connection based
+ internet protocol which provides for lossless
+ packet transportation, usually over IP.
+
+&term(Thread) Also called a &ital(process thread, or pthread.) This
+ is a lightweight process which executes in
+ concurrently with the application and shares
+ the same address space. RMR uses threads to
+ manage asynchronous functions such as route
+ table updates.
+
+&Term(Trace information) An optional portion of the message buffer
+ that the application may populate with data
+ that allows for tracing the progress of the
+ transaction or application activity across
+ components. RMR makes no use of this data.
+
+&term(Transaction ID) A fixed number of bytes in the &ital(message
+ buffer) which the application may populate
+ with information related to the
+ transaction. RMR makes use of the transaction
+ ID for matching response messages with the
+ &c(rmr_call) function is used to send a
+ message.
+
+&term(Transient failure) An error state that is believed to be short
+ lived and that the operation, if retried by
+ the application, might be successful. C
+ programmers will recognise this as
+ &cw(EAGAIN.)
-&term(Warning) A warning occurs when RMR has encountered something that it believes isn't correct, but has a defined
- work round.
+&term(Warning) A warning occurs when RMR has encountered something
+ that it believes isn't correct, but has a
+ defined work round.
+
+&term(Wormhole) A direct connection managed by RMR between the user
+ application and a remote, RMR based, application.
-&term(Wormhole) A direct connection managed by RMR between the user application and a remote, RMR based, application.
&end_dlist
.ix resume
-
.** this file is imbedded after the cover page for Postscript generated
.** documents, and at the top for text based documents. It's purpose is
-.** to ensure the output doc has the license statement.
+.** to ensure the output doc has the license statement.
.if pfm
.sp 5
-.fi
+.fi
This document, and the source used to generate it, is governed by the following
-copyright statement(s).
+copyright statement(s).
.sp 2
.ln
.nf
Copyright (c) 2019 Nokia
Copyright (c) 2018-2019 AT&T Intellectual Property.
-
+
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
-
+
http://www.apache.org/licenses/LICENSE-2.0
-
+
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
.fo
-.ln
+.ln
.sp 2
&h1(Appendix &mbuf_appendix -- Message Buffer Details)
-The RMR message buffer is a C structure which is exposed in the &cw(rmr.h)
-header file.
-It is used to manage a message received from a peer endpoint, or a message
-that is being sent to a peer.
-Fields include payload length, amount of payload actually used, status,
-and a reference to the payload.
-There are also fields which the application should ignore, and could be
-hidden in the header file, but we chose not to. These fields include
-a reference to the RMR header information, and to the underlying transport
-mechanism message struct which may or may not be the same as the RMR
-header reference.
+The RMR message buffer is a C structure which is exposed in the
+&cw(rmr.h) header file. It is used to manage a message received from
+a peer endpoint, or a message that is being sent to a peer. Fields
+include payload length, amount of payload actually used, status, and a
+reference to the payload. There are also fields which the application
+should ignore, and could be hidden in the header file, but we chose
+not to. These fields include a reference to the RMR header
+information, and to the underlying transport mechanism message struct
+which may or may not be the same as the RMR header reference.
&h2(The Structure)
-The following is the C structure.
-Readers are cautioned to examine the header file directly; the information
-here may be out of date (old document in some cache), and thus it may
-be incorrect.
+The following is the C structure. Readers are cautioned to examine
+the &cw(rmr.h) header file directly; the information here may be out
+of date (old document in some cache), and thus it may be incorrect.
&space
&indent
unsigned char* id; // if we need an ID in the message separate from the xaction id
int flags; // various MFL_ (private) flags as needed
int alloc_len; // the length of the allocated space (hdr+payload)
+ void* ring; // ring this buffer should be queued back to
+ int rts_fd; // SI fd for return to sender
+ int cookie; // cookie to detect user misuse of free'd msg
} rmr_mbuf_t;
&ex_end
&uindent
&space
&h2(State vs Transport State)
-The state field reflects the state at the time the message buffer is returned to the
-calling applicaiton.
-For a send operation, if the state is not &cw(RMR_OK) then the message buffer references
-the payload that could not be sent, and when the state is &cw(RMR_OK) the buffer
-references a &ital(fresh) payload that the application may fill in.
+The state field reflects the state at the time the message buffer is
+returned to the calling application. For a send operation, if the
+state is not &cw(RMR_OK) then the message buffer references the
+payload that could not be sent, and when the state is &cw(RMR_OK) the
+buffer references a &ital(fresh) payload that the application may fill
+in.
&space
-When the state is not &cw(RMR_OK,) C programmes may examine the global &cw(errno) value
-which RMR will have left set, if it was set, by the underlying transport mechanism.
-In some cases, wrapper modules are not able to directly access the C-library &cw(errno)
-value, and to assist with possible transport error details, the send and receive
+When the state is not &cw(RMR_OK,) C programmes may examine the global
+&cw(errno) value which RMR will have left set, if it was set, by the
+underlying transport mechanism. In some cases, wrapper modules are
+not able to directly access the C-library &cw(errno) value, and to
+assist with possible transport error details, the send and receive
operations populate &cw(tp_state) with the value of &cw(errno.)
&space
-Regardless of whether the application makes use of the &cw(tp_state,) or the &cw(errno)
-value, it should be noted that the underlying transport mechanism may not actually update
-the errno value; in other words: it might not be accurate.
-In addition, RMR populates the &cw(tp_state) value in the message buffer &bold(only) when
-the state is not &cw(RMR_OK.)
+Regardless of whether the application makes use of the &cw(tp_state,)
+or the &cw(errno) value, it should be noted that the underlying
+transport mechanism may not actually update the errno value; in other
+words: it might not be accurate. In addition, RMR populates the
+&cw(tp_state) value in the message buffer &bold(only) when the state
+is not &cw(RMR_OK.)
&h2(Field References)
-The transaction field was exposed in the first version of RMR, and in hindsight this
-shouldn't have been done.
-Rather than break any existing code the reference was left, but additional fields
-such as trace data, were not directly exposed to the application.
-The application developer is strongly encouraged to use the functions which get and
-set the transaction ID rather than using the pointer directly; any data overruns will
-not be detected if the reference is used directly.
+The transaction field was exposed in the first version of RMR, and in
+hindsight this shouldn't have been done. Rather than break any
+existing code the reference was left, but additional fields such as
+trace data, were not directly exposed to the application. The
+application developer is strongly encouraged to use the functions
+which get and set the transaction ID rather than using the pointer
+directly; any data overruns will not be detected if the reference is
+used directly.
&space
-In contrast, the payload reference should be used directly by the application in the
-interest of speed and ease of programming.
-The same care to prevent writing more bytes to the payload buffer than it can hold
-must be taken by the application.
-By the nature of the allocation of the payload in transport space, RMR is unable to
-add guard bytes and/or test for data overrun.
+In contrast, the payload reference should be used directly by the
+application in the interest of speed and ease of programming. The
+same care to prevent writing more bytes to the payload buffer than it
+can hold must be taken by the application. By the nature of the
+allocation of the payload in transport space, RMR is unable to add
+guard bytes and/or test for data overrun.
&h2(Actual Transmission)
-When RMR sends the application's message, the message buffer is &bold(not) transmitted.
-The transport buffer (tp_buf) which contains the RMR header and application payload
-is the only set of bytes which are transmitted.
-While it may seem to the caller like the function &func(rmr_send_msg) is returning a
-new message buffer, the same struct is reused and only a new transport buffer is allocated.
-The intent is to keep the alloc/free cycles to a minimum.
+When RMR sends the application's message, the message buffer is
+&bold(not) transmitted. The transport buffer (tp_buf) which contains
+the RMR header and application payload is the only set of bytes which
+are transmitted. While it may seem to the caller like the function
+&func(rmr_send_msg) is returning a new message buffer, the same struct
+is reused and only a new transport buffer is allocated. The intent is
+to keep the alloc/free cycles to a minimum.
.dv doc_subtitle User's Manual
.if pfm
- .** add licence, a title page, and talbe of contents
- .im front_junk.im
+ .** add licence, a title page, and table of contents
+ .im front_junk.im
.ei
.** for text based things, nothing more than license
.im license.im
.st &textsize
&h1(Overview)
-The RIC Message Router (a.k.a. RMR) is a thin library which provides a latency sensitive application with the ability
-to send and receive messages with other RMR based applications.
+
+The RIC Message Router (RMR) is a library for peer-to-peer
+communication. Applications use the library to send and receive
+messages where the message routing and endpoint selection is based on
+the message type rather than DNS host name-IP port combinations.
The library provides the following major features:
&half_space
&li Message distribution (round robin or fanout) is selectable by message type.
&half_space
- &li Route management updates are received and processed asynchronously and without overt
- application involvement.
+ &li Route management updates are received and processed
+ asynchronously and without overt application involvement.
&end_list
&uindent
&space
&h2(Purpose)
-RMR's main purpose is to provide an application with the ability to send and receive messages to/from other
-peer applications with minimal effort on the application's part.
-To achieve this, RMR manages manages all endpoint information, connections, and routing information necessary
-to establish and maintain communication.
-From the application's point of view, all that is required to send a message is to allocate (via RMR) a message
-buffer, add the payload data, and set the message type.
-To receive a message, the application needs only to invoke the receive function; when a message
-arrives a message buffer will be returned as the function result.
-
+RMR's main purpose is to provide an application with the ability to
+send and receive messages to/from other peer applications with minimal
+effort on the application's part. To achieve this, RMR manages all
+endpoint information, connections, and routing information necessary
+to establish and maintain communication. From the application's point
+of view, all that is required to send a message is to allocate (via
+RMR) a message buffer, add the payload data, and set the message type.
+To receive a message, the application needs only to invoke the receive
+function; when a message arrives a message buffer will be returned as
+the function result.
&h2(Message Routing)
-Applications are required to place a message type into a message before sending, and may optionally add a
-subscription ID when appropriate.
-The combination of message type, and subscription ID are refered to as the &ital(message key,)
-and is used to match an entry in a routing table which provides the possible endpoints expecting
-to receive messages with the matching key.
+Applications are required to place a message type into a message
+before sending, and may optionally add a subscription ID when
+appropriate. The combination of message type, and subscription ID are
+refered to as the &ital(message key,) and is used to match an entry in
+a routing table which provides the possible endpoints expecting to
+receive messages with the matching key.
&h3(Round Robin Delivery)
-An endpoint from RMR's perspective is an application to which RMR may establish a connection, and expect to
-send messages with one or more defined message keys.
-Each entry in the route table consists of one or more endpoint groups, called round robin groups.
-When a message matches a specific entry, the entry's groups are used to select the destination of the message.
-A message is sent once to each group, with messages being &ital(balanced) across the endpoints of a group
-via round robin selection.
-Care should be taken when defining multiple groups for a message type as there is extra overhead required
-and thus the overall message latency is somewhat reduced.
+An endpoint from RMR's perspective is an application to which RMR may
+establish a connection, and expect to send messages with one or more
+defined message keys. Each entry in the route table consists of one
+or more endpoint groups, called round robin groups. When a message
+matches a specific entry, the entry's groups are used to select the
+destination of the message. A message is sent once to each group,
+with messages being &ital(balanced) across the endpoints of a group
+via round robin selection. Care should be taken when defining
+multiple groups for a message type as there is extra overhead required
+and thus the overall message latency is somewhat increased.
&h3(Routing Table Updates)
-Route table information is made available to RMR a static file (loaded once), or by updates sent from a
-separate route manager application.
-If a static table is provided, it is loaded during RMR initialisation and will remain in use until
-an external process connects and delivers a route table update (often referred to as a dynamic update).
-Dynamic updates are listened for in a separate process thread and applied automatically; the application does not
-need to allow for, or trigger, updates.
+Route table information is made available to RMR a static file (loaded
+once), or by updates sent from a separate route manager application.
+If a static table is provided, it is loaded during RMR initialization
+and will remain in use until an external process connects and delivers
+a route table update (often referred to as a dynamic update). Dynamic
+updates are listened for in a separate process thread and applied
+automatically; the application does not need to allow for, or trigger,
+updates.
&h2(Latency And Throughput)
-While providing insulation from the underlying message transport mechanics, RMR must also do so in such a
-manner that message latency and throughput are not impacted.
-In general, the RMR induced overhead, incurred due to the process of selecting an endpoint for each message,
-is minimal and should not impact the overall latency or throughput of the application.
-This impact has been measured with test applications running on the same physical host and the average
-latency through RMR for a message was on the order of 0.02 milliseconds.
+While providing insulation from the underlying message transport
+mechanics, RMR must also do so in such a manner that message latency
+and throughput are not impacted. In general, the RMR induced
+overhead, incurred due to the process of selecting an endpoint for
+each message, is minimal and should not impact the overall latency or
+throughput of the application. This impact has been measured with
+test applications running on the same physical host and the average
+latency through RMR for a message was on the order of 0.02
+milliseconds.
&space
-As an application's throughput increases, it becomes easy for the application to overrun the underlying
-transport mechanism (e.g. NNG), consume all available TCP transmit buffers, or otherwise find itself
-in a situation where a send might not immediately complete.
-RMR offers different &ital(modes) which allow the application to manage these states based on the
-overall needs of the application.
-These modes are discussed in the &ital(Configuration) section of this document.
+As an application's throughput increases, it becomes easy for the
+application to overrun the underlying transport mechanism (e.g. NNG),
+consume all available TCP transmit buffers, or otherwise find itself
+in a situation where a send might not immediately complete. RMR
+offers different &ital(modes) which allow the application to manage
+these states based on the overall needs of the application. These
+modes are discussed in the &ital(Configuration) section of this
+document.
.** snarf in the major sections (to avoid one huge source document and maybe to promote reuse)
# ditch anything that can be rebuilt
nuke: clean
- rm -f *.ps *.pdf *.rst *.md
+ rm -f *.ps *.pdf *.rst *.md *.txt
# make hack to force a rule to always be out of date
always:
-This directory contains source for the project mandated "read the docs"
-.rst documentation. The Makefile will both build the doc from the source
-here and "install" it into the docs directory at the repo root. While
-it is not good practice, the generated .rst files must be checked into
-the repo and committed with changes to the source.
+This directory contains source for the project mandated "read the
+docs" .rst documentation. If the "tfm" tool is available, the
+Makefile will both build the doc from the source here and "install" it
+into the docs directory at the repo root. Then the generated .rst
+files must be checked into the repo and committed with changes to the
+source (maybe not the best practice).
The command 'make all' should be all that is needed to build the
rtd documentation. Follow that with 'make publish' to actually move
files are moved.
Adding A New Man Page
-WHen a new manual page is added to the source (../man), it must be
+When a new manual page is added to the source (../man), it must be
added to the list in the user's guide source in this directory.
Rationale
Documentation is just code, and by maintaining the documentation as
source is is possible to generate various forms of output with a
single make. While it is possible to convert X to Y, a true document
-composition language is far better at geneating readable Postscript
+composition language is far better at generating readable Postscript
output with embedded figures as well as text and tables.
CAUTION:
.im setup.im
-&h1(RMR Configuration and Delpoyment)
+&h1(RMR Configuration and Deployment)
-The RIC Message Router (RMR) is a library which applications use to send and receive
-messages where the message routing, endpoint selection, is based on the message type
-rather than on traditional DNS names or IP addresses.
-This document contains information regarding the configuration of RMR when it is
-embedded by a &ital(user application).
-RMR itself is not a deployable entity.
+The RIC Message Router (RMR) is a library for peer-to-peer
+communication. Applications use the library to send and receive
+messages where the message routing and endpoint selection is based on
+the message type rather than DNS host name-IP port combinations.
+&space
+This document contains information regarding the configuration of RMR
+when it is embedded by a &ital(user application). RMR itself is a
+library, not a deployable entity.
&h2(Configuration)
-Several aspects of RMR behaviour is controlled via environment variables which are
-set at the time that a user application invokes the RMR initialisation function.
-This allows these variables to be set before the application is started as a
-function of the true environment, or set by the application as a means for the
-application to influence RMR's behaviour.
-The following is a list of RMR variables which RMR recognises (see the main
-RMR manual page in the development package for more details).
+
+Many aspects of RMR behavior are controlled via environment variables.
+These values are read when a user application invokes the RMR
+initialization function. This allows these variables to be set before
+the application is started as a function of the true environment, or
+set by the application as a means for the application to influence
+RMR's behaviour. The following is a list of environment variables
+which RMR recognizes. Also see the main RMR manual page in the
+development package for more details.
.** use the same list which is used in the manual pages
&h1(RMR Developer Guide)
-The RIC Message Router (RMR) is a library which applications use to send and receive
-messages where the message routing, endpoint selection, is based on the message type
-rather than on traditional DNS names or IP addresses.
-This document contains information that potential developers might need to know
-in order to contribute to the project
+The RIC Message Router (RMR) is a library for peer-to-peer
+communication. Applications use the library to send and receive
+messages where the message routing and endpoint selection is based on
+the message type rather than DNS host name-IP port combinations.
+&space
+This document contains information that developers need to know to
+contribute to the RMR project.
&h2(Language)
-RMR is written in C, and thus a contributing developer to the core library should
-have an excellent working knowledge of C.
-There currently is one set of cross languages bindings supporting Python, and a
-developer wishing to contribute to the bindings source should be familiar with
-Python (version 3.7+) and with the Python &ital(ctypes) library.
+RMR is written in C, and thus a contributing developer to the core
+library should have an excellent working knowledge of C. There
+currently is one set of cross-languages bindings supporting Python,
+and a developer wishing to contribute to the bindings source should be
+familiar with Python (version 3.7+) and with the Python &ital(ctypes)
+library.
&h2(Code Structure)
-RMR is designed to provide an insulation layer between user applications and
-the actual transport mechanism.
-Initially RMR was built on top of Nanosmg, and shortly after was ported to NNG
-(Nanomsg Next Generation).
-Because RMR presents the same API to the user application regardless of the underlying
-transport library, the resulting output when compiling RMR is a transport specific
-library.
-As an example, &cw(librmr_nng.a) is the library generated for use with the NNG
-transport.
-&space
+RMR is designed to provide an insulation layer between user
+applications and the actual transport mechanism. Initially RMR was
+built on top of the third-party library Nanosmg, shortly after was
+ported to the third-party library NNG (Nanomsg Next Generation), and
+then was ported to an internally developed socket library called SI95.
+RMR presents the same API to the user application regardless of the
+underlying transport library, but the resulting output when compiling
+RMR is always a transport-specific library. As an example,
+&cw(librmr_nng.a) is the library generated for use with the NNG
+transport. &space
As such the library source is organised into multiple components:
&beg_dlist(.75i : ^&bold_font )
-&di(common) Source in the common directory is agnostic to the underlying transport
- mechanism (Nanomsg or NNG), and thus can be used when generating either library.
+&di(common) Source in the common directory is agnostic to the
+ underlying transport mechanism (Nanomsg, NNG, SI95, ..), and
+ thus can be used when generating either library.
&di(nano) Source which is tightly coupled with the underlying Nanomsg library.
(Nanomsg has been deprecated, but the RMR source remains as an example.)
&di(nng) Source which is tightly coupled with the underlying NNG library.
+ (NNG has been deprecated, but the RMR source remains as an example.)
+
+&di(si) Source which is tightly coupled with the underlying SI95 library.
&end_dlist
&space
&h3(Internal Function Exposure)
-The decision to limit as much as practical the exposure of truely internal RMR functions
-was made, and as a result most of the RMR functions carry a &cw(static) label.
-In order to modularise the code as much as possible, this means that the primary
-module (e.g. rmr_nng.c) will directly include other RMR modules, rather than
-depending on referencing the internal functions during linking.
-While this is an infrequently used approach, it does mean that there are very
-few functions visible for the user application to reference, all of them having
-the prefix &cw(rmr&{esc}_,) while allowing internal functions to have shorter names
-while still being meaningful.
+The decision to limit as much as practical the exposure of truly
+internal RMR functions was made, and as a result most of the RMR
+functions carry a &cw(static) label. In order to modularise the code
+as much as possible, this means that the primary module
+(e.g. rmr_nng.c) directly includes other RMR modules, rather than
+depending on referencing the internal functions during linking. While
+this is an infrequently used approach, it does mean that there are
+very few functions visible for the user application to reference, all
+of them having the prefix &cw(rmr&{esc}_). This allows internal
+functions to have shorter names while still being meaningful.
&h2(Coding Style)
-There is a list of coding style guidelines in the top level directory, and as such
-they are not expanded upon here.
-The general practice is to follow the style when editing an existing module, respect
-the author's choice where style alternatives are not frowned upon.
-When creating new modules, select a style that fits the guidelines and is easy
-for you to work with.
-There are a few things that are insisted on by the maintainers of RMR, but for the
-most part style is up to the creator of a module.
+There is a list of coding style guidelines in the top level directory,
+and as such they are not expanded upon here. The general practice is
+to follow the style when editing an existing module, respect the
+author's choice where style alternatives are not frowned upon. When
+creating new modules, select a style that fits the guidelines and is
+easy for you to work with. There are a few things that the RMR
+maintainers insist on, but for the most part style is up to the
+creator of a module.
&h2(Building)
-RMR is constructed using CMake.
-While CMake's project description can be more cumbersome than most typical Makefiles,
-the tool provides convenience especially when it comes to creating packages.
+RMR is constructed using CMake. While CMake's project description can
+be more cumbersome than most typical Makefiles, the tool provides
+convenience especially when it comes to creating DEB/RPM packages.
&h1(RMR User's Guide)
-The RIC Message Router (RMR) is a library which applications use to send and receive
-messages where the message routing, endpoint selection, is based on the message type
-rather than on traditional DNS names or IP addresses.
-Because the user documentation for RMR is a collection of UNIX manpages (included
-in the development package, and available via the &cw(man) command when installed),
-there is no separate "User's Guide."
-To provide something for the document scrapers to find, this is a collection of
-the RMR manual pages formatted directly from their source which might be a bit
-ragged when combined into a single markup document.
-Read the manual pages :)
+The RIC Message Router (RMR) is a library for peer-to-peer
+communication. Applications use the library to send and receive
+messages where the message routing and endpoint selection is based on
+the message type rather than DNS host name-IP port combinations.
+&space
+This document contains information that developers need to know to use
+the RMR library. Because the primary documentation for the RMR
+library is a collection of UNIX manpages (included in the development
+package, and available via the &cw(man) command when installed), there
+is no separate "User's Guide." To provide something for the document
+scrapers to find, this is a collection of the RMR manual pages
+formatted directly from their source, which might be a bit ragged when
+combined into a single markup document. Read the manual pages :)
&space
-.** turn of header 1 so that the rmr library header isn't generated for each page
+.** turn off header 1 so that the rmr library header isn't generated for each page
.dv h1
.** CAUTION: man_list.im is generated by a script when make is used!
.im man_list.im
-
-
.. Do NOT make changes directly to .rst or .md files.
-RMR Configuration and Delpoyment
+RMR Configuration and Deployment
============================================================================================
-The RIC Message Router (RMR) is a library which applications
-use to send and receive messages where the message routing,
-endpoint selection, is based on the message type rather than
-on traditional DNS names or IP addresses. This document
-contains information regarding the configuration of RMR when
-it is embedded by a *user application* . RMR itself is not a
-deployable entity.
+The RIC Message Router (RMR) is a library for peer-to-peer
+communication. Applications use the library to send and
+receive messages where the message routing and endpoint
+selection is based on the message type rather than DNS host
+name-IP port combinations.
+
+This document contains information regarding the
+configuration of RMR when it is embedded by a *user
+application* . RMR itself is a library, not a deployable
+entity.
Configuration
--------------------------------------------------------------------------------------------
-Several aspects of RMR behaviour is controlled via
-environment variables which are set at the time that a user
-application invokes the RMR initialisation function. This
-allows these variables to be set before the application is
-started as a function of the true environment, or set by the
-application as a means for the application to influence RMR's
-behaviour. The following is a list of RMR variables which RMR
-recognises (see the main RMR manual page in the development
-package for more details).
+Many aspects of RMR behavior are controlled via environment
+variables. These values are read when a user application
+invokes the RMR initialization function. This allows these
+variables to be set before the application is started as a
+function of the true environment, or set by the application
+as a means for the application to influence RMR's behaviour.
+The following is a list of environment variables which RMR
+recognizes. Also see the main RMR manual page in the
+development package for more details.
RMR_ASYNC_CONN
RMR Developer Guide
============================================================================================
-The RIC Message Router (RMR) is a library which applications
-use to send and receive messages where the message routing,
-endpoint selection, is based on the message type rather than
-on traditional DNS names or IP addresses. This document
-contains information that potential developers might need to
-know in order to contribute to the project
+The RIC Message Router (RMR) is a library for peer-to-peer
+communication. Applications use the library to send and
+receive messages where the message routing and endpoint
+selection is based on the message type rather than DNS host
+name-IP port combinations.
+
+This document contains information that developers need to
+know to contribute to the RMR project.
Language
--------------------------------------------------------------------------------------------
RMR is written in C, and thus a contributing developer to the
core library should have an excellent working knowledge of C.
-There currently is one set of cross languages bindings
+There currently is one set of cross-languages bindings
supporting Python, and a developer wishing to contribute to
the bindings source should be familiar with Python (version
3.7+) and with the Python *ctypes* library.
RMR is designed to provide an insulation layer between user
applications and the actual transport mechanism. Initially
-RMR was built on top of Nanosmg, and shortly after was ported
-to NNG (Nanomsg Next Generation). Because RMR presents the
-same API to the user application regardless of the underlying
-transport library, the resulting output when compiling RMR is
-a transport specific library. As an example, librmr_nng.a is
-the library generated for use with the NNG transport.
+RMR was built on top of the third-party library Nanosmg,
+shortly after was ported to the third-party library NNG
+(Nanomsg Next Generation), and then was ported to an
+internally developed socket library called SI95. RMR presents
+the same API to the user application regardless of the
+underlying transport library, but the resulting output when
+compiling RMR is always a transport-specific library. As an
+example, librmr_nng.a is the library generated for use with
+the NNG transport.
As such the library source is organised into multiple
components:
common
Source in the common directory is agnostic to the
- underlying transport mechanism (Nanomsg or NNG), and thus
- can be used when generating either library.
+ underlying transport mechanism (Nanomsg, NNG, SI95, ..),
+ and thus can be used when generating either library.
nano
nng
Source which is tightly coupled with the underlying NNG
+ library. (NNG has been deprecated, but the RMR source
+ remains as an example.)
+
+si
+
+ Source which is tightly coupled with the underlying SI95
library.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The decision to limit as much as practical the exposure of
-truely internal RMR functions was made, and as a result most
+truly internal RMR functions was made, and as a result most
of the RMR functions carry a static label. In order to
modularise the code as much as possible, this means that the
-primary module (e.g. rmr_nng.c) will directly include other
-RMR modules, rather than depending on referencing the
-internal functions during linking. While this is an
-infrequently used approach, it does mean that there are very
-few functions visible for the user application to reference,
-all of them having the prefix rmr\_, while allowing internal
-functions to have shorter names while still being meaningful.
+primary module (e.g. rmr_nng.c) directly includes other RMR
+modules, rather than depending on referencing the internal
+functions during linking. While this is an infrequently used
+approach, it does mean that there are very few functions
+visible for the user application to reference, all of them
+having the prefix rmr\_. This allows internal functions to
+have shorter names while still being meaningful.
Coding Style
--------------------------------------------------------------------------------------------
existing module, respect the author's choice where style
alternatives are not frowned upon. When creating new modules,
select a style that fits the guidelines and is easy for you
-to work with. There are a few things that are insisted on by
-the maintainers of RMR, but for the most part style is up to
-the creator of a module.
+to work with. There are a few things that the RMR maintainers
+insist on, but for the most part style is up to the creator
+of a module.
Building
--------------------------------------------------------------------------------------------
RMR is constructed using CMake. While CMake's project
description can be more cumbersome than most typical
Makefiles, the tool provides convenience especially when it
-comes to creating packages.
+comes to creating DEB/RPM packages.
--------------------------------------------------------------------------------------------
+2020 April 2; version 3.6.5
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Correct potential nil pointer use when examining interfaces
+for use as a listen target (RIC-307)
+
+
+2020 April 1; version 3.6.4
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Correct potential nil pointer use in the NNG interface
+(RIC-303) Correct issue preventing CI build without a
+container
+
+
+2020 March 30; version 3.6.3
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Correct the max receive message size constant in rmr.h
+(RIC-301)
+
+
+2020 March 23; version 3.6.2
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Fix message initialisation bug when pulling a message from
+the pool (RIC-295)
+
+
+2020 March 19; version 3.6.1
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Fix problem with RPM package install
+
+
2020 March 18; version 3.6.0
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Add missing health check message types.
+
2020 March 9; version 3.5.0
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
RMR User's Guide
============================================================================================
-The RIC Message Router (RMR) is a library which applications
-use to send and receive messages where the message routing,
-endpoint selection, is based on the message type rather than
-on traditional DNS names or IP addresses. Because the user
-documentation for RMR is a collection of UNIX manpages
-(included in the development package, and available via the
-man command when installed), there is no separate "User's
-Guide." To provide something for the document scrapers to
-find, this is a collection of the RMR manual pages formatted
-directly from their source which might be a bit ragged when
-combined into a single markup document. Read the manual pages
-:)
+The RIC Message Router (RMR) is a library for peer-to-peer
+communication. Applications use the library to send and
+receive messages where the message routing and endpoint
+selection is based on the message type rather than DNS host
+name-IP port combinations.
+
+This document contains information that developers need to
+know to use the RMR library. Because the primary
+documentation for the RMR library is a collection of UNIX
+manpages (included in the development package, and available
+via the man command when installed), there is no separate
+"User's Guide." To provide something for the document
+scrapers to find, this is a collection of the RMR manual
+pages formatted directly from their source, which might be a
+bit ragged when combined into a single markup document. Read
+the manual pages :)
Code Review / ric-plt / lib / rmr.git / commitdiff