2 .. Copyright (c) 2019 AT&T Intellectual Property.
3 .. Copyright (c) 2019-2022 Nokia.
5 .. Licensed under the Creative Commons Attribution 4.0 International
6 .. Public License (the "License"); you may not use this file except
7 .. in compliance with the License. You may obtain a copy of the License at
9 .. https://creativecommons.org/licenses/by/4.0/
11 .. Unless required by applicable law or agreed to in writing, documentation
12 .. distributed under the License is distributed on an "AS IS" BASIS,
13 .. WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
15 .. See the License for the specific language governing permissions and
16 .. limitations under the License.
39 This is the user guide of O-RAN SC SDL C++ library.
40 Shared Data Layer (SDL) provides a lightweight, high-speed interface (API) for
41 accessing shared data storage. SDL can be used for storing and sharing any
42 data. Data can be shared at VNF level. One typical use case for SDL is sharing
43 the state data of stateful application processes. Thus enabling stateful
44 application processes to become stateless, conforming with, e.g., the
45 requirements of the fifth generation mobile networks.
47 Figure below illustrates some main points of SDL:
49 .. image:: ./_static/sdl_intro.png
51 :alt: SDL introduction
53 SDL has been implemented in many languages:
55 * C++ Linux shared library
59 This document focuses on C++ implementation of SDL but general principles are
60 the same in all implementations.
69 **Backend Data Storage**
71 Backend data storage refers to data storage technology behind SDL API which
72 handles the actual data storing. SDL API hides the backend data storage
73 implementation from SDL API clients, and therefore backend data storage
74 technology can be changed without affecting SDL API clients. Currently, Redis
75 database is the most commonly used backend data storage implementation.
77 Figure below illustrates how SDL API hides backend data storage technology
80 .. image:: ./_static/backend_data_storage.png
82 :alt: SDL API hides backend data storage technology from application
84 `SDL Deployment section <#sdl-deployment>`_ provides further information
85 about backend data storage deployment options.
89 Namespaces provide data isolation within SDL data storage. That is, data in
90 certain namespace is isolated from the data in other namespaces. Each SDL
91 client uses one or more namespaces.
93 Namespaces can be used, for example, to isolate data belonging to different
96 Figure below shows an example of the SDL namespace concept. There are two SDL
97 clients, both accessing SDL backend data storage using an SDL API instance
98 (C++ object). Client 1 uses both namespaces: A and B, while client 2 uses only
99 namespace: B. Therefore, data in the namespace: A is visible only to client 1
100 and data in namespace: B is shared between clients 1 and 2:
102 .. image:: ./_static/sdl_namespaces.png
104 :alt: SDL namespace concept example
106 Namespace management is planned to be moved under a managing entity which
107 enforces some control over how the namespaces are created. For now, however,
108 namespace naming needs to be manually coordinated between clients.
112 Clients save key-data pairs. Data is passed as byte vectors. SDL stores the
113 data as it is. Any structure that this data may have (e.g. a serialized JSON)
114 is meaningful only to the client itself. Clients are responsible for managing
115 the keys. As namespaces provide data isolation, keys in different namespaces
116 always access different data.
125 SDL provides currently following APIs:
127 * Asynchronous API for accessing SDL storage *shareddatalayer::AsyncStorage*
128 * Synchronous API for accessing SDL storage shareddatalayer::SyncStorage
130 Same SDL client can use one or more SDL APIs. There should rarely be need to
131 create several instances of the same SDL API though. All individual operations
132 done using SDL API functions are targeted to one namespace (accessing several
133 namespaces requires multiple operations).
135 SDL API functions are not thread-safe, meaning that same SDL instance must
136 not be shared between multiple threads without explicit locking in SDL client.
138 SDL API functions are atomic unless otherwise indicated. Indication of the
139 non-atomic behavior of certain function can be found from one or many of the
143 * Function parameters
144 * Function doxygen documentation (see below)
146 Refer to doxygen generated SDL API documentation below for further information
147 about SDL APIs and the functions they contain.
149 Doxygen Generated SDL API Documentation
150 =======================================
152 Pre-built online version of SDL API Doxygen documentation is not yet available.
154 Doxygen documentation can be generated manually. Follow instructions found from
155 :ref:`SDL developer guide <building_sdl_api_doc>`.
161 Building Clients Using SDL
162 **************************
164 SDL API functions can be used by including SDL public headers and by linking
167 The necessary compilation and linker flags can be acquired with the
170 pkg-config --cflags libsdl
171 pkg-config --libs libsdl
173 SDL internal implementation uses C++14, thus SDL clients need to be build
174 using a C++ compiler supporting C++14. However, SDL public API header files
175 contain only features which are available in C++11, thus SDL clients do not
176 need to be implemented (and compiled) using C++14 (C++11 is enough). The
177 compiler just needs to have support for C++14.
183 Using SDL in Application Pod
184 ****************************
186 SDL binary artifacts including Debian (.deb) and RPM Package Manager (.rpm)
187 packages are available in O-RAN-SC PackageCloud.io repository.
189 In runtime environment SDL needs also a database backend service, currently
190 SDL supports only Redis database. Recommended solution is to use DBaaS
191 component of the official RIC platform deployment.
193 **Deploying SDL database backend with DBaaS service in the RIC**
195 Download RIC deployment artifacts::
197 git clone "https://gerrit.o-ran-sc.org/r/it/dep"
199 The **ric-platform** directory contains Helm chart and scripts to deploy RIC
200 platform components, including also DBaaS component.
202 RIC DBaaS service must be running before starting application pod which is
203 using SDL API. DBaaS defines environment variables which are used to contact
204 DBaaS service (offering backend for SDL). Those environment variables are
205 exposed inside application container only if DBaaS service is running when
206 application container is started. Refer to
207 `Database Backend Configuration section <#database-backend-configuration>`_,
208 for information about available environment variables.
209 You may test SDL connectivity to its backend with the *sdltool* command inside
210 your application container::
212 sdltool test-connectivity
214 *sdltool* comes in SDL binary artifacts which are available in O-RAN-SC
215 PackageCloud.io repository.
217 For more information, see also `README <https://gerrit.o-ran-sc.org/r/gitweb?p=ric-plt/dbaas.git;a=blob;f=README.md;h=6391fc45ea762a5b606dcf9f867fac8087b1222f;hb=HEAD>`_
218 file of the *dbaas* O-RAN-SC gerrit repository.
227 Certain aspects in SDL functionality can be configured by using environment
230 Database Backend Configuration
231 ==============================
233 Database backend configuration can be used to configure, to which database
234 backend SDL instance connects. A list of available environment variables to
235 configure database backend:
239 * DBAAS_SERVICE_SENTINEL_PORT
242 * DBAAS_CLUSTER_ADDR_LIST
244 After DBaaS service is installed, environment variables are exposed to
245 application containers. SDL library will automatically use these environment
246 variables. If DBaaS service is not used, above environment variables needs to
247 be set manually so that SDL backend can connect to correct database.
249 When multiple Database (DB) service is used Nokia SEP deployments can have
250 comma separated list of DB ports, sentinel master group names and DB service
253 DBAAS_CLUSTER_ADDR_LIST=<comma separated list of DB services>
254 DBAAS_MASTER_NAME=<comma separated list of DB sentinel master names>
255 DBAAS_SERVICE_PORT=<comma separated list of DB service ports>
256 DBAAS_SERVICE_SENTINEL_PORT=<comma separated list of Redis Sentinel ports>
258 In RIC platform deployments above list type of environment variables will have
259 a single value, because only one Database (DB) service is supported in RIC.
263 An example how environment variables can be set in bash shell, when standalone
264 Redis server is running in a Kubernetes Pod with k8s service name of *dbaas* and
267 export DBAAS_SERVICE_HOST=dbaas
268 export DBAAS_SERVICE_PORT=6379
269 export DBAAS_NODE_COUNT=1
271 Besides hostname, IPv4 and IPv6 addresses can be set to *DBAAS_SERVICE_HOST*.
273 An example how environment variables can be set in bash shell, when Redis
274 HA deployment is used::
276 export DBAAS_MASTER_NAME=my-primary-sentinel
277 export DBAAS_SERVICE_HOST=dbaas
278 export DBAAS_SERVICE_SENTINEL_PORT=23550
279 export DBAAS_NODE_COUNT=3
281 An example how environment variables can be set in bash shell, when Redis
282 HA deployment with two DB service is used::
284 export DBAAS_CLUSTER_ADDR_LIST=dbaas-0,dbaas-1
285 export DBAAS_MASTER_NAME=my-dbaasmaster-0,my-dbaasmaster-1
286 export DBAAS_SERVICE_HOST=dbaas-0
287 export DBAAS_SERVICE_PORT=6379,6380
288 export DBAAS_SERVICE_SENTINEL_PORT=26379,26380
289 export DBAAS_NODE_COUNT=3
298 `Doxygen generated SDL API documentation <#doxygen-generated-sdl-api-documentation>`_
299 describes which error codes are returned and which exceptions are thrown from
300 each SDL API function. Generally, asynchronous SDL APIs return error codes and
301 synchronous SDL APIs throw exceptions in error situations.
303 Handling Error Codes Returned From Asynchronous SDL APIs
304 ========================================================
306 Asynchronous SDL APIs return *std::error_code* based error codes in error
307 situations. Typically, error code is returned as a parameter in the related
310 Returned error code contains detailed information about the error which has
311 occurred. This information is valuable for SDL developers in case the issue
312 needs further investigation, but usually this information is too detailed for
313 SDL client error handling logic. For SDL client error handling purposes SDL
314 provides *shareddatalayer::error* constants and the returned *std::error_code*
315 can be compared against these constants.
317 Therefore SDL clients are recommended to store the returned *std::error_code*
318 somewhere (for example to the log) and implement the error handling logic based
319 on *shareddatalayer::error* constants. C++ code example below illustrates this:
325 log.error() << "SDL operation failed, error: " << error
326 << " message: " << error.message() << std::endl;
328 if (error == shareddatalayer::Error::NOT_CONNECTED)
329 // Error handling logic for shareddatalayer::Error::NOT_CONNECTED
330 else if (error == shareddatalayer::Error::OPERATION_INTERRUPTED)
331 // Error handling logic for shareddatalayer::Error::OPERATION_INTERRUPTED
332 else if (error == shareddatalayer::Error::BACKEND_FAILURE)
333 // Error handling logic for shareddatalayer::Error::BACKEND_FAILURE
334 else if (error == shareddatalayer::Error::REJECTED_BY_BACKEND)
335 // Error handling logic for shareddatalayer::Error::REJECTED_BY_BACKEND
338 *error* in the code block above is *std::error_code* type variable which is
339 returned from some asynchronous SDL API function. *log* is a logging service
340 what an SDL client is using. Note that this is a simple and incomplete example
341 for demonstration purposes and not meant to be used as such in real
342 environment. Complete error handling implementation depends on SDL client and
343 SDL API function which returned the error. For example, in some cases common
344 handling for several *shareddatalayer::error* constants might be sufficient.
346 **Instructions for Error Handling Logic Implementation**
348 Doxygen documentation contains detailed description for all
349 shareddatalayer::Error constants. This information helps to design error
350 handling logic for each shareddatalayer::Error constant. For example, following
351 information can be found from there:
354 * Is data modified in the backend data storage
355 * How to recover from error situation
358 Handling Exceptions Thrown by Synchronous SDL APIs
359 ==================================================
361 Synchronous SDL APIs throw exceptions in error situations. There are
362 corresponding exceptions for all *shareddatalayer::error* constants returned
363 by asynchronous APIs (see previous section). All exceptions thrown by SDL are
364 derived from *shareddatalayer::Exception*.
365 Therefore, a client can catch *shareddatalayer::Exception* in case the client
366 wants to implement common handling for some SDL originated exceptions. Note
367 that external services, which SDL uses, can throw exceptions which are not
368 derived from *shareddatalayer::Exception*.
370 Below is a C++ code example of a scenario where SDL client does common error
371 handling for all exceptions thrown from synchronous SDL API:
377 //Code which executes synchronous SDL API function
379 catch (const shareddatalayer::Exception& e)
381 log.error() << "SDL operation failed, error: " << e.what() << std::endl;
382 //Common error handling logic for all SDL errors
384 //Catch also non-SDL exceptions (like std::exception) if needed
386 Below C++ code example has separate handling for shareddatalayer::BackendError
387 exception and common handling for all other exceptions thrown by SDL:
393 //Code which executes synchronous SDL API function
395 catch (const shareddatalayer::BackendError& e)
397 log.error() << "SDL operation failed, error: " << e.what() << std::endl;
398 //Error handling logic for BackendError
400 catch (const shareddatalayer::Exception& e)
402 log.error() << "SDL operation failed, error: " << e.what() << std::endl;
403 //Common error handling logic for all other SDL errors than BackendError
405 //Catch also non-SDL exceptions (like std::exception) if needed
407 *log* is a logging service what an SDL client is using. Note that these are
408 simple and incomplete examples for demonstration purposes and they are not
409 meant to be used as such in real environment.
411 **Instructions for Error Handling Logic Implementation**
413 Doxygen documentation contains documentation for all exceptions thrown by SDL.
414 This documentation contains information which helps to design error handling
415 logic for each exception. For exceptions having corresponding error code,
416 exception documentation is usually a reference to corresponding error code
419 Each SDL API function, which throws exceptions, has a link to the documentation
420 of those exceptions. This link can be found from the Doxygen documentation of
421 given SDL API function.
430 This chapter discusses how certain general data storage related aspects work in
431 SDL. Discussed subjects include, for example, concurrency control and data
437 Production environments are typically deployed so that SDL backend data storage
438 and SDL clients are in different nodes (e.g. VM, container).
440 There are two different supported deployment modes for SDL backend data
443 * Standalone (single DB node without redundancy)
444 * Redundant (DB node pair working in primary/replica redundancy model)
446 SDL supports also Redis sentinel based DB cluster where deployment has one or
447 more DBAAS Redis sentinel group. Different DBAAS Redis sentinel groups
448 can be used to distribute SDL DB operations to different SDL DB instances. When
449 more than one DBAAS Redis sentinel group exits the selection of SDL DB instance
450 is based on the namespace string hash calculation.
452 SDL does not prevent backend data storage to be deployed in the same node with
453 the SDL client. Such deployments are, however, typically used only in
454 development/testing type of environments.
459 SDL does not support transactions doing one or more units of work in ACID
460 manner (pessimistic concurrency control).
462 SDL supports optimistic concurrency control by providing Check and Set (CAS)
463 type conditional functions. These conditional functions provide possibility
464 to do certain data modification operations only if data value matches the SDL
465 client's last known value. Thus a SDL client can check that someone else has
466 not changed the data after it was read by the SDL client. If the data would
467 have been changed, SDL does not do the modification operation and this is
468 indicated to the SDL client. The SDL client can then decide how to handle the
469 situation (for example read the latest data and retry modification).
471 *AsyncStorage::setIfAsync* is an example of a conditional function discussed
472 above. Other conditional functions exist as well.
477 Currently all data stored to SDL is stored to in-memory backend data storage.
478 Meaning that, data is not preserved over DB node restart. DB node restart does
479 not necessarily cause data loss for SDL client though. Refer to
480 `SDL Deployment section <#sdl-deployment>`_, for information about SDL backend
481 data storage redundancy models.
490 This chapter gives recommendations on how to use SDL.
492 Building Clients Using SDL
493 ==========================
495 * Use *pkg-config* tool to acquire needed compilation and linking flags,
496 instead of hardcoding them. This ensures that flags are always up-to-date.
497 See more information from `here <#building-clients-using-sdl>`_.
498 * If you want to mock SDL APIs in unit testing, SDL provides helper classes
499 for that. By using these helper classes you need to implement mock
500 implementation only for those SDL API functions which you use in the unit
501 tests. See more information from `doxygen documentation <#doxygen-generated-sdl-api-documentation>`_
502 of the helper classes:
504 * *include/sdl/tst/mockableasyncstorage.hpp: MockableAsyncStorage*
505 * *include/sdl/tst/mockablesyncstorage.hpp: MockableSyncStorage*
510 * SDL APIs are not thread-safe. If same SDL API instance is shared between
511 multiple threads, SDL client has to use explicit locking to ensure that only
512 one thread at time executes SDL API functions.
513 * Each SDL instance establishes own connection to backend data storage, which
514 requires resources (how heavy this exactly is depends on used backend data
515 storage type). Thus, from performance point of view, only one SDL instance
516 per one SDL API should be used if reasonably possible. One SDL instance can
517 access multiple SDL namespaces when using *AsyncStorage* and *SyncStorage*
519 * Use waitReadyAsync() function before doing first operation via asynchronous
520 APIs to ensure that SDL and backend data storage are ready to handle
521 operations. See waitReadyAsync() function
522 `doxygen documentation <#doxygen-generated-sdl-api-documentation>`_
523 for corresponding asynchronous API for details.
524 * Use waitReady() function before doing first operation via synchronous
525 APIs to ensure that SDL and backend data storage are ready to handle
526 operations. See waitReady() function
527 `doxygen documentation <#doxygen-generated-sdl-api-documentation>`_
528 for corresponding synchronous API for details.
529 * Avoid using heavy search functions (for example: *AsyncStorage::findKeys()*).
530 Rather define your keys so that you know which keys should be read.
535 * As namespace naming is currently on SDL client's responsibility, use enough
536 specific namespace names that same name is surely not used by someone else
537 (unless you want to share given namespace data with that someone else).
538 * Data entities related to each other should be placed under the same
539 namespace (unless there is a good reason not to). For example, accessing
540 multiple data entities with one SDL operation is possible only for data
541 entities belonging to same namespace.
542 * Identically named keys can be used in different namespaces. Creating own
543 namespaces for different use cases and unrelated data provides more freedom
544 into key name selection.
549 * Writing or reading one big junk of data at once is more efficient than
550 writing/reading the same amount of data in small steps. For example, create a
551 key list and read it once, rather than reading each key in a loop.
552 * If rolling upgrade needs to be supported, consider using Google Protocol
553 Buffers (or something similar) to make it possible to parse data which is
554 written by older or newer application version.
563 There is a pre-installed *sdlcli* tool in DBaaS container. With this tool user
564 can see statistics of database backend (Redis), check healthiness of DBaaS
565 database backend, list database keys and get and set values into database.
566 For example to see statistics give below command inside DBaaS container::
570 To check healthiness of database::
574 Use *sdlcli* help to get more information about available commands::