2 --------------------------------------------------------------------------------
4 Copyright (c) 2018-2019 AT&T Intellectual Property.
6 Licensed under the Apache License, Version 2.0 (the "License");
7 you may not use this file except in compliance with the License.
8 You may obtain a copy of the License at
10 http://www.apache.org/licenses/LICENSE-2.0
12 Unless required by applicable law or agreed to in writing, software
13 distributed under the License is distributed on an "AS IS" BASIS,
14 WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
15 See the License for the specific language governing permissions and
16 limitations under the License.
17 --------------------------------------------------------------------------------
23 This directory contains the source for the a simple message listener which
24 writes messages received via RMR into a fifo (named pipe) for an external
27 Fifos are named MT_xxxxxxxx where the Xs are replaced with the message
28 type with up to 7 leading zeros (e.g. MT_00000002). The data written
29 to a pipe has the form:
32 Where the first 8 bytes are the ASCII representation of the length of
33 the message (n) (the 8th byte is a zero allowing the bytes to be
34 treated as a C string if desired. The next n bytes are the unchanged
35 payload which was received. No RMR header information (e.g. source,
36 meid, etc.) is communicated.
38 If the listener is executed with the timestamp extension enabled, then
39 the leading 'header' is enhanced such that the time the message was
40 received by the listener is added. Additionally, a leading delimiter
41 is added to make synchronisation possible. The timestamp is also an
42 ASCII string of the form 1570110224356 (milliseconds past the epoch).
43 The enhanced header has the format:
45 <delimeter><length-bytes><time-bytes>
47 The <delimeter> is a series of 4 bytes which should always be: '@MCL'.
48 It is intended to be used to sequence "frames" in the pipe should there be
49 write errors which result in missing data. If the application reading from
50 a pipe does not see this delimeter, then it should read byte by byte from
51 the pipe until it does in order to synchronise with the stream.
53 The <length-bytes> are as descrbed previously: 8 byte ASCII string (nil
56 The <time-bytes> is an ASCII string (nil terminated) with a length of 16
59 The entire header will require 28 bytes.
62 There are multiple docker files; *.df.
63 mcl_runime.df -- builds an image with the runtime mc_listener binary
64 mcl_dev.df -- builds a development image that can be used to
65 interactively build and test the library and mc_listener
69 A very small set of unit tests are provided for the library functions in
70 mcl.c. Because of the nature of the fanout function, which blocks waiting
71 on RMR messages, it is not possible to unit test that bit of code.
75 In addition to writing received messages to a FIFO, each message is
76 captured in a raw data file. These files are created in a staging
77 directory (/tmp/rdc/stage by default) and moved to a final directory
78 (/tmp/rdc/final by default) every 5 minutes (300 seconds). The files
79 are named MCLT[src]_<timestamp><.suffix>; the default suffix is ".rdc".
80 The "src" string is optional and can be used to disabmiguate files
81 if merged with capture files from different environments.
83 The data capture is controlled by setting environment variables:
85 MCL_RDC_ENABLE: If set to 0 the raw data capture will be disabled.
86 (It is on by default if this variable is not defined.)
88 MCL_RDC_STAGE: overrides the directory where raw data capture
91 MCL_RDC_FINAL: overrides the directory where raw data capture files
92 are placed for export.
94 MCL_RDC_SUFFIX: the suffix written on each raw data capture file;
95 must include '.' if a dot is desired and is written only on
96 the final file (not the staged file).
98 MCL_RDC_SOURCE: a short string used as source identification in rdc file names.
99 This defaults to nothing and should begin, but not end,
100 with an underbar (e.g. _host1).
102 MCL_RDC_FREQ: The frequency with which files are closed and moved from the
103 staging to final directory. The default if 300 sec.
105 The captured data is saved in the following format:
106 <delim><mtype><len><fifo-buffer>
109 <delim> is a 4 character delimiter for synchronisation (@RDC)
111 <mtype> is an 8 character field containing a nil terminated ACII
112 value that is the message type of the received message.
114 <len> is an 8 character field containing a nil terminated ASCII
115 value that is the length of the <fifo-buffer>
117 <fifo-buffer> is the exact contents that were written to the FIFO.
119 Thus to "replay" the captured data, a decoder need read just the first
120 20 bytes, convert the message type and length, read the payload, and
121 then write the payload to a FIFO
123 Staging and Final Directories
124 If the staging and final directories (/tmp/rdc/stage and /tmp/rdc/final
125 by default) are on the same filsystem, then the rename() system call is
126 used to switch the inode reference to the file from one directory to the
127 other. This is the preferred setup. However, if it is not possible for
128 these directories to exist on the same filesystem, then the listener will
129 copy the file "manually." During the copy, the filename in the final
130 directory will have a leading dot (.) charcter, and the file will be
131 created with a write only by owner mode (0200) and not switched to readable
132 until after it is closed; just before renaming it to remove the leading
135 Any file capture utillity should either ignore files with leading dot
136 characters, or files which do not have a read bit set in their mode.
141 The pipe_reader programme is a simple application which uses the mcl.c
142 library functions to open and read from a single pipe. If the -e option
143 is given it will expect that data in the FIFO has extended headers. Use
144 the -? option (or -h) to generate a full usage statement.