2 .. Copyright (c) 2019 AT&T Intellectual Property.
3 .. Copyright (c) 2019 Nokia.
5 .. Licensed under the Creative Commons Attribution 4.0 International
6 .. Public License (the "License"); you may not use this file except
7 .. in compliance with the License. You may obtain a copy of the License at
9 .. https://creativecommons.org/licenses/by/4.0/
11 .. Unless required by applicable law or agreed to in writing, documentation
12 .. distributed under the License is distributed on an "AS IS" BASIS,
13 .. WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
15 .. See the License for the specific language governing permissions and
16 .. limitations under the License.
39 This is the user guide of O-RAN SC SDL C++ library.
40 Shared Data Layer (SDL) provides a lightweight, high-speed interface (API) for
41 accessing shared data storage. SDL can be used for storing and sharing any
42 data. Data can be shared at VNF level. One typical use case for SDL is sharing
43 the state data of stateful application processes. Thus enabling stateful
44 application processes to become stateless, conforming with, e.g., the
45 requirements of the fifth generation mobile networks.
47 Figure below illustrates some main points of SDL:
49 .. image:: ./_static/sdl_intro.png
51 :alt: SDL introduction
53 SDL has been implemented in many languages:
55 * C++ Linux shared library
59 This document focuses on C++ implementation of SDL but general principles are
60 the same in all implementations.
69 **Backend Data Storage**
71 Backend data storage refers to data storage technology behind SDL API which
72 handles the actual data storing. SDL API hides the backend data storage
73 implementation from SDL API clients, and therefore backend data storage
74 technology can be changed without affecting SDL API clients. Currently, Redis
75 database is the most commonly used backend data storage implementation.
77 Figure below illustrates how SDL API hides backend data storage technology
80 .. image:: ./_static/backend_data_storage.png
82 :alt: SDL API hides backend data storage technology from application
84 `SDL Deployment section <#sdl-deployment>`_ provides further information
85 about backend data storage deployment options.
89 Namespaces provide data isolation within SDL data storage. That is, data in
90 certain namespace is isolated from the data in other namespaces. Each SDL
91 client uses one or more namespaces.
93 Namespaces can be used, for example, to isolate data belonging to different
96 Figure below shows an example of the SDL namespace concept. There are two SDL
97 clients, both accessing SDL backend data storage using an SDL API instance
98 (C++ object). Client 1 uses both namespaces: A and B, while client 2 uses only
99 namespace: B. Therefore, data in the namespace: A is visible only to client 1
100 and data in namespace: B is shared between clients 1 and 2:
102 .. image:: ./_static/sdl_namespaces.png
104 :alt: SDL namespace concept example
106 Namespace management is planned to be moved under a managing entity which
107 enforces some control over how the namespaces are created. For now, however,
108 namespace naming needs to be manually coordinated between clients.
112 Clients save key-data pairs. Data is passed as byte vectors. SDL stores the
113 data as it is. Any structure that this data may have (e.g. a serialized JSON)
114 is meaningful only to the client itself. Clients are responsible for managing
115 the keys. As namespaces provide data isolation, keys in different namespaces
116 always access different data.
125 SDL provides currently following APIs:
127 * Asynchronous API for accessing SDL storage *shareddatalayer::AsyncStorage*
128 * Synchronous API for accessing SDL storage shareddatalayer::SyncStorage
130 Same SDL client can use one or more SDL APIs. There should rarely be need to
131 create several instances of the same SDL API though. All individual operations
132 done using SDL API functions are targeted to one namespace (accessing several
133 namespaces requires multiple operations).
135 SDL API functions are not thread-safe, meaning that same SDL instance must
136 not be shared between multiple threads without explicit locking in SDL client.
138 SDL API functions are atomic unless otherwise indicated. Indication of the
139 non-atomic behavior of certain function can be found from one or many of the
143 * Function parameters
144 * Function doxygen documentation (see below)
146 Refer to doxygen generated SDL API documentation below for further information
147 about SDL APIs and the functions they contain.
149 Doxygen Generated SDL API Documentation
150 =======================================
152 Pre-built online version of SDL API Doxygen documentation is not yet available.
154 Doxygen documentation can be generated manually. Follow instructions found from
155 :ref:`SDL developer guide <building_sdl_api_doc>`.
161 Building Clients Using SDL
162 **************************
164 SDL API functions can be used by including SDL public headers and by linking
167 The necessary compilation and linker flags can be acquired with the
170 pkg-config --cflags libsdl
171 pkg-config --libs libsdl
173 SDL internal implementation uses C++14, thus SDL clients need to be build
174 using a C++ compiler supporting C++14. However, SDL public API header files
175 contain only features which are available in C++11, thus SDL clients do not
176 need to be implemented (and compiled) using C++14 (C++11 is enough). The
177 compiler just needs to have support for C++14.
183 Using SDL in Application Pod
184 ****************************
186 SDL binary artifacts including Debian (.deb) and RPM Package Manager (.rpm)
187 packages are available in O-RAN-SC PackageCloud.io repository.
189 In runtime environment SDL needs also a database backend service, currently
190 SDL supports only Redis database. Recommended solution is to use DBaaS
191 component of the official RIC platform deployment.
193 **Deploying SDL database backend with DBaaS service in the RIC**
195 Download RIC deployment artifacts::
197 git clone "https://gerrit.o-ran-sc.org/r/it/dep"
199 The **ric-platform** directory contains Helm chart and scripts to deploy RIC
200 platform components, including also DBaaS component.
202 RIC DBaaS service must be running before starting application pod which is
203 using SDL API. DBaaS defines environment variables which are used to contact
204 DBaaS service (offering backend for SDL). Those environment variables are
205 exposed inside application container only if DBaaS service is running when
206 application container is started. Refer to
207 `Database Backend Configuration section <#database-backend-configuration>`_,
208 for information about available environment variables.
209 You may test SDL connectivity to its backend with the *sdltool* command inside
210 your application container::
212 sdltool test-connectivity
214 *sdltool* comes in SDL binary artifacts which are available in O-RAN-SC
215 PackageCloud.io repository.
217 For more information, see also `README <https://gerrit.o-ran-sc.org/r/gitweb?p=ric-plt/dbaas.git;a=blob;f=README.md;h=6391fc45ea762a5b606dcf9f867fac8087b1222f;hb=HEAD>`_
218 file of the *dbaas* O-RAN-SC gerrit repository.
227 Certain aspects in SDL functionality can be configured by using environment
230 Database Backend Configuration
231 ==============================
233 Database backend configuration can be used to configure, to which database
234 backend SDL instance connects. A list of available environment variables to
235 configure database backend:
239 * DBAAS_SERVICE_SENTINEL_PORT
242 After DBaaS service is installed, environment variables are exposed to
243 application containers. SDL library will automatically use these environment
244 variables. If DBaaS service is not used, above environment variables needs to
245 be set manually so that SDL backend can connect to correct database.
249 An example how environment variables can be set in bash shell, when standalone
250 Redis server is running in a Kubernetes Pod with k8s service name of *dbaas* and
253 export DBAAS_SERVICE_HOST=dbaas
254 export DBAAS_SERVICE_PORT=6379
256 Besides hostname, IPv4 and IPv6 addresses can be set to *DBAAS_SERVICE_HOST*.
258 An example how environment variables can be set in bash shell, when Redis
259 HA deployment is used. Please note that DBaaS does not support yet HA
260 deployment option. Below environment variables are only in the form of an
261 example to show how HA deployment would be configured::
263 export DBAAS_MASTER_NAME=my-master-sentinel
264 export DBAAS_SERVICE_HOST=dbaas
265 export DBAAS_SERVICE_SENTINEL_PORT=23550
274 `Doxygen generated SDL API documentation <#doxygen-generated-sdl-api-documentation>`_
275 describes which error codes are returned and which exceptions are thrown from
276 each SDL API function. Generally, asynchronous SDL APIs return error codes and
277 synchronous SDL APIs throw exceptions in error situations.
279 Handling Error Codes Returned From Asynchronous SDL APIs
280 ========================================================
282 Asynchronous SDL APIs return *std::error_code* based error codes in error
283 situations. Typically, error code is returned as a parameter in the related
286 Returned error code contains detailed information about the error which has
287 occurred. This information is valuable for SDL developers in case the issue
288 needs further investigation, but usually this information is too detailed for
289 SDL client error handling logic. For SDL client error handling purposes SDL
290 provides *shareddatalayer::error* constants and the returned *std::error_code*
291 can be compared against these constants.
293 Therefore SDL clients are recommended to store the returned *std::error_code*
294 somewhere (for example to the log) and implement the error handling logic based
295 on *shareddatalayer::error* constants. C++ code example below illustrates this:
301 log.error() << "SDL operation failed, error: " << error
302 << " message: " << error.message() << std::endl;
304 if (error == shareddatalayer::Error::NOT_CONNECTED)
305 // Error handling logic for shareddatalayer::Error::NOT_CONNECTED
306 else if (error == shareddatalayer::Error::OPERATION_INTERRUPTED)
307 // Error handling logic for shareddatalayer::Error::OPERATION_INTERRUPTED
308 else if (error == shareddatalayer::Error::BACKEND_FAILURE)
309 // Error handling logic for shareddatalayer::Error::BACKEND_FAILURE
310 else if (error == shareddatalayer::Error::REJECTED_BY_BACKEND)
311 // Error handling logic for shareddatalayer::Error::REJECTED_BY_BACKEND
314 *error* in the code block above is *std::error_code* type variable which is
315 returned from some asynchronous SDL API function. *log* is a logging service
316 what an SDL client is using. Note that this is a simple and incomplete example
317 for demonstration purposes and not meant to be used as such in real
318 environment. Complete error handling implementation depends on SDL client and
319 SDL API function which returned the error. For example, in some cases common
320 handling for several *shareddatalayer::error* constants might be sufficient.
322 **Instructions for Error Handling Logic Implementation**
324 Doxygen documentation contains detailed description for all
325 shareddatalayer::Error constants. This information helps to design error
326 handling logic for each shareddatalayer::Error constant. For example, following
327 information can be found from there:
330 * Is data modified in the backend data storage
331 * How to recover from error situation
334 Handling Exceptions Thrown by Synchronous SDL APIs
335 ==================================================
337 Synchronous SDL APIs throw exceptions in error situations. There are
338 corresponding exceptions for all *shareddatalayer::error* constants returned
339 by asynchronous APIs (see previous section). All exceptions thrown by SDL are
340 derived from *shareddatalayer::Exception*.
341 Therefore, a client can catch *shareddatalayer::Exception* in case the client
342 wants to implement common handling for some SDL originated exceptions. Note
343 that external services, which SDL uses, can throw exceptions which are not
344 derived from *shareddatalayer::Exception*.
346 Below is a C++ code example of a scenario where SDL client does common error
347 handling for all exceptions thrown from synchronous SDL API:
353 //Code which executes synchronous SDL API function
355 catch (const shareddatalayer::Exception& e)
357 log.error() << "SDL operation failed, error: " << e.what() << std::endl;
358 //Common error handling logic for all SDL errors
360 //Catch also non-SDL exceptions (like std::exception) if needed
362 Below C++ code example has separate handling for shareddatalayer::BackendError
363 exception and common handling for all other exceptions thrown by SDL:
369 //Code which executes synchronous SDL API function
371 catch (const shareddatalayer::BackendError& e)
373 log.error() << "SDL operation failed, error: " << e.what() << std::endl;
374 //Error handling logic for BackendError
376 catch (const shareddatalayer::Exception& e)
378 log.error() << "SDL operation failed, error: " << e.what() << std::endl;
379 //Common error handling logic for all other SDL errors than BackendError
381 //Catch also non-SDL exceptions (like std::exception) if needed
383 *log* is a logging service what an SDL client is using. Note that these are
384 simple and incomplete examples for demonstration purposes and they are not
385 meant to be used as such in real environment.
387 **Instructions for Error Handling Logic Implementation**
389 Doxygen documentation contains documentation for all exceptions thrown by SDL.
390 This documentation contains information which helps to design error handling
391 logic for each exception. For exceptions having corresponding error code,
392 exception documentation is usually a reference to corresponding error code
395 Each SDL API function, which throws exceptions, has a link to the documentation
396 of those exceptions. This link can be found from the Doxygen documentation of
397 given SDL API function.
406 This chapter discusses how certain general data storage related aspects work in
407 SDL. Discussed subjects include, for example, concurrency control and data
413 Production environments are typically deployed so that SDL backend data storage
414 and SDL clients are in different nodes (e.g. VM, container).
416 There are two different supported deployment modes for SDL backend data
419 * Standalone (single DB node without redundancy)
420 * Redundant (DB node pair working in master/slave redundancy model)
422 SDL does not currently have any intelligent logic (e.g. dynamic scaling) on
423 which storage node each namespace data is stored. This area might be developed
424 further in the future.
426 SDL does not prevent backend data storage to be deployed in the same node with
427 the SDL client. Such deployments are, however, typically used only in
428 development/testing type of environments.
433 SDL does not support transactions doing one or more units of work in ACID
434 manner (pessimistic concurrency control).
436 SDL supports optimistic concurrency control by providing Check and Set (CAS)
437 type conditional functions. These conditional functions provide possibility
438 to do certain data modification operations only if data value matches the SDL
439 client's last known value. Thus a SDL client can check that someone else has
440 not changed the data after it was read by the SDL client. If the data would
441 have been changed, SDL does not do the modification operation and this is
442 indicated to the SDL client. The SDL client can then decide how to handle the
443 situation (for example read the latest data and retry modification).
445 *AsyncStorage::setIfAsync* is an example of a conditional function discussed
446 above. Other conditional functions exist as well.
451 Currently all data stored to SDL is stored to in-memory backend data storage.
452 Meaning that, data is not preserved over DB node restart. DB node restart does
453 not necessarily cause data loss for SDL client though. Refer to
454 `SDL Deployment section <#sdl-deployment>`_, for information about SDL backend
455 data storage redundancy models.
464 This chapter gives recommendations on how to use SDL.
466 Building Clients Using SDL
467 ==========================
469 * Use *pkg-config* tool to acquire needed compilation and linking flags,
470 instead of hardcoding them. This ensures that flags are always up-to-date.
471 See more information from `here <#building-clients-using-sdl>`_.
472 * If you want to mock SDL APIs in unit testing, SDL provides helper classes
473 for that. By using these helper classes you need to implement mock
474 implementation only for those SDL API functions which you use in the unit
475 tests. See more information from `doxygen documentation <#doxygen-generated-sdl-api-documentation>`_
476 of the helper classes:
478 * *include/sdl/tst/mockableasyncstorage.hpp: MockableAsyncStorage*
479 * *include/sdl/tst/mockablesyncstorage.hpp: MockableSyncStorage*
484 * SDL APIs are not thread-safe. If same SDL API instance is shared between
485 multiple threads, SDL client has to use explicit locking to ensure that only
486 one thread at time executes SDL API functions.
487 * Each SDL instance establishes own connection to backend data storage, which
488 requires resources (how heavy this exactly is depends on used backend data
489 storage type). Thus, from performance point of view, only one SDL instance
490 per one SDL API should be used if reasonably possible. One SDL instance can
491 access multiple SDL namespaces when using *AsyncStorage* and *SyncStorage*
493 * Use waitReadyAsync() function before doing first operation via asynchronous
494 APIs to ensure that SDL and backend data storage are ready to handle
495 operations. See waitReadyAsync() function
496 `doxygen documentation <#doxygen-generated-sdl-api-documentation>`_
497 for corresponding asynchronous API for details.
498 * Avoid using heavy search functions (for example: *AsyncStorage::findKeys()*).
499 Rather define your keys so that you know which keys should be read.
504 * As namespace naming is currently on SDL client's responsibility, use enough
505 specific namespace names that same name is surely not used by someone else
506 (unless you want to share given namespace data with that someone else).
507 * Data entities related to each other should be placed under the same
508 namespace (unless there is a good reason not to). For example, accessing
509 multiple data entities with one SDL operation is possible only for data
510 entities belonging to same namespace.
511 * Identically named keys can be used in different namespaces. Creating own
512 namespaces for different use cases and unrelated data provides more freedom
513 into key name selection.
518 * Writing or reading one big junk of data at once is more efficient than
519 writing/reading the same amount of data in small steps. For example, create a
520 key list and read it once, rather than reading each key in a loop.
521 * If rolling upgrade needs to be supported, consider using Google Protocol
522 Buffers (or something similar) to make it possible to parse data which is
523 written by older or newer application version.