2 .. Copyright (c) 2019 AT&T Intellectual Property.
3 .. Copyright (c) 2019 Nokia.
5 .. Licensed under the Creative Commons Attribution 4.0 International
6 .. Public License (the "License"); you may not use this file except
7 .. in compliance with the License. You may obtain a copy of the License at
9 .. https://creativecommons.org/licenses/by/4.0/
11 .. Unless required by applicable law or agreed to in writing, documentation
12 .. distributed under the License is distributed on an "AS IS" BASIS,
13 .. WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
15 .. See the License for the specific language governing permissions and
16 .. limitations under the License.
39 This is the user guide of O-RAN SC SDL C++ library.
40 Shared Data Layer (SDL) provides a lightweight, high-speed interface (API) for
41 accessing shared data storage. SDL can be used for storing and sharing any
42 data. Data can be shared at VNF level. One typical use case for SDL is sharing
43 the state data of stateful application processes. Thus enabling stateful
44 application processes to become stateless, conforming with, e.g., the
45 requirements of the fifth generation mobile networks.
47 Figure below illustrates some main points of SDL:
49 .. image:: ./_static/sdl_intro.png
51 :alt: SDL introduction
53 SDL has been implemented in many languages:
55 * C++ Linux shared library
59 This document focuses on C++ implementation of SDL but general principles are
60 the same in all implementations.
69 **Backend Data Storage**
71 Backend data storage refers to data storage technology behind SDL API which
72 handles the actual data storing. SDL API hides the backend data storage
73 implementation from SDL API clients, and therefore backend data storage
74 technology can be changed without affecting SDL API clients. Currently, Redis
75 database is the most commonly used backend data storage implementation.
77 Figure below illustrates how SDL API hides backend data storage technology
80 .. image:: ./_static/backend_data_storage.png
82 :alt: SDL API hides backend data storage technology from application
84 `SDL Deployment section <#sdl-deployment>`_ provides further information
85 about backend data storage deployment options.
89 Namespaces provide data isolation within SDL data storage. That is, data in
90 certain namespace is isolated from the data in other namespaces. Each SDL
91 client uses one or more namespaces.
93 Namespaces can be used, for example, to isolate data belonging to different
96 Figure below shows an example of the SDL namespace concept. There are two SDL
97 clients, both accessing SDL backend data storage using an SDL API instance
98 (C++ object). Client 1 uses both namespaces: A and B, while client 2 uses only
99 namespace: B. Therefore, data in the namespace: A is visible only to client 1
100 and data in namespace: B is shared between clients 1 and 2:
102 .. image:: ./_static/sdl_namespaces.png
104 :alt: SDL namespace concept example
106 Namespace management is planned to be moved under a managing entity which
107 enforces some control over how the namespaces are created. For now, however,
108 namespace naming needs to be manually coordinated between clients.
112 Clients save key-data pairs. Data is passed as byte vectors. SDL stores the
113 data as it is. Any structure that this data may have (e.g. a serialized JSON)
114 is meaningful only to the client itself. Clients are responsible for managing
115 the keys. As namespaces provide data isolation, keys in different namespaces
116 always access different data.
125 SDL provides currently following APIs:
127 * Asynchronous API for accessing SDL storage *shareddatalayer::AsyncStorage*
128 * Synchronous API for accessing SDL storage shareddatalayer::SyncStorage
130 Same SDL client can use one or more SDL APIs. There should rarely be need to
131 create several instances of the same SDL API though. All individual operations
132 done using SDL API functions are targeted to one namespace (accessing several
133 namespaces requires multiple operations).
135 SDL API functions are not thread-safe, meaning that same SDL instance must
136 not be shared between multiple threads without explicit locking in SDL client.
138 SDL API functions are atomic unless otherwise indicated. Indication of the
139 non-atomic behavior of certain function can be found from one or many of the
143 * Function parameters
144 * Function doxygen documentation (see below)
146 Refer to doxygen generated SDL API documentation below for further information
147 about SDL APIs and the functions they contain.
149 Doxygen Generated SDL API Documentation
150 =======================================
152 Pre-built online version of SDL API Doxygen documentation is not yet available.
154 Doxygen documentation can be generated manually. Follow instructions found from
155 :ref:`SDL developer guide <building_sdl_api_doc>`.
161 Building Clients Using SDL
162 **************************
164 SDL API functions can be used by including SDL public headers and by linking
167 The necessary compilation and linker flags can be acquired with the
170 pkg-config --cflags libsdl
171 pkg-config --libs libsdl
173 SDL internal implementation uses C++14, thus SDL clients need to be build
174 using a C++ compiler supporting C++14. However, SDL public API header files
175 contain only features which are available in C++11, thus SDL clients do not
176 need to be implemented (and compiled) using C++14 (C++11 is enough). The
177 compiler just needs to have support for C++14.
183 Using SDL in Application Pod
184 ****************************
186 SDL binary artifacts including Debian (.deb) and RPM Package Manager (.rpm)
187 packages are available in O-RAN-SC PackageCloud.io repository.
189 In runtime environment SDL needs also a database backend service, currently
190 SDL supports only Redis database. Recommended solution is to use DBaaS
191 component of the official RIC platform deployment.
193 **Deploying SDL database backend with DBaaS service in the RIC**
195 Download RIC deployment artifacts::
197 git clone "https://gerrit.o-ran-sc.org/r/it/dep"
199 The **ric-platform** directory contains Helm chart and scripts to deploy RIC
200 platform components, including also DBaaS component.
202 RIC DBaaS service must be running before starting application pod which is
203 using SDL API. DBaaS defines environment variables which are used to contact
204 DBaaS service (offering backend for SDL). Those environment variables are
205 exposed inside application container only if DBaaS service is running when
206 application container is started. Refer to
207 `Database Backend Configuration section <#database-backend-configuration>`_,
208 for information about available environment variables.
209 You may test SDL connectivity to its backend with the *sdltool* command inside
210 your application container::
212 sdltool test-connectivity
214 *sdltool* comes in SDL binary artifacts which are available in O-RAN-SC
215 PackageCloud.io repository.
217 For more information, see also `README <https://gerrit.o-ran-sc.org/r/gitweb?p=ric-plt/dbaas.git;a=blob;f=README.md;h=6391fc45ea762a5b606dcf9f867fac8087b1222f;hb=HEAD>`_
218 file of the *dbaas* O-RAN-SC gerrit repository.
227 Certain aspects in SDL functionality can be configured by using environment
230 Database Backend Configuration
231 ==============================
233 Database backend configuration can be used to configure, to which database
234 backend SDL instance connects. A list of available environment variables to
235 configure database backend:
239 * DBAAS_SERVICE_SENTINEL_PORT
243 After DBaaS service is installed, environment variables are exposed to
244 application containers. SDL library will automatically use these environment
245 variables. If DBaaS service is not used, above environment variables needs to
246 be set manually so that SDL backend can connect to correct database.
250 An example how environment variables can be set in bash shell, when standalone
251 Redis server is running in a Kubernetes Pod with k8s service name of *dbaas* and
254 export DBAAS_SERVICE_HOST=dbaas
255 export DBAAS_SERVICE_PORT=6379
256 export DBAAS_NODE_COUNT=1
258 Besides hostname, IPv4 and IPv6 addresses can be set to *DBAAS_SERVICE_HOST*.
260 An example how environment variables can be set in bash shell, when Redis
261 HA deployment is used::
263 export DBAAS_MASTER_NAME=my-primary-sentinel
264 export DBAAS_SERVICE_HOST=dbaas
265 export DBAAS_SERVICE_SENTINEL_PORT=23550
266 export DBAAS_NODE_COUNT=3
275 `Doxygen generated SDL API documentation <#doxygen-generated-sdl-api-documentation>`_
276 describes which error codes are returned and which exceptions are thrown from
277 each SDL API function. Generally, asynchronous SDL APIs return error codes and
278 synchronous SDL APIs throw exceptions in error situations.
280 Handling Error Codes Returned From Asynchronous SDL APIs
281 ========================================================
283 Asynchronous SDL APIs return *std::error_code* based error codes in error
284 situations. Typically, error code is returned as a parameter in the related
287 Returned error code contains detailed information about the error which has
288 occurred. This information is valuable for SDL developers in case the issue
289 needs further investigation, but usually this information is too detailed for
290 SDL client error handling logic. For SDL client error handling purposes SDL
291 provides *shareddatalayer::error* constants and the returned *std::error_code*
292 can be compared against these constants.
294 Therefore SDL clients are recommended to store the returned *std::error_code*
295 somewhere (for example to the log) and implement the error handling logic based
296 on *shareddatalayer::error* constants. C++ code example below illustrates this:
302 log.error() << "SDL operation failed, error: " << error
303 << " message: " << error.message() << std::endl;
305 if (error == shareddatalayer::Error::NOT_CONNECTED)
306 // Error handling logic for shareddatalayer::Error::NOT_CONNECTED
307 else if (error == shareddatalayer::Error::OPERATION_INTERRUPTED)
308 // Error handling logic for shareddatalayer::Error::OPERATION_INTERRUPTED
309 else if (error == shareddatalayer::Error::BACKEND_FAILURE)
310 // Error handling logic for shareddatalayer::Error::BACKEND_FAILURE
311 else if (error == shareddatalayer::Error::REJECTED_BY_BACKEND)
312 // Error handling logic for shareddatalayer::Error::REJECTED_BY_BACKEND
315 *error* in the code block above is *std::error_code* type variable which is
316 returned from some asynchronous SDL API function. *log* is a logging service
317 what an SDL client is using. Note that this is a simple and incomplete example
318 for demonstration purposes and not meant to be used as such in real
319 environment. Complete error handling implementation depends on SDL client and
320 SDL API function which returned the error. For example, in some cases common
321 handling for several *shareddatalayer::error* constants might be sufficient.
323 **Instructions for Error Handling Logic Implementation**
325 Doxygen documentation contains detailed description for all
326 shareddatalayer::Error constants. This information helps to design error
327 handling logic for each shareddatalayer::Error constant. For example, following
328 information can be found from there:
331 * Is data modified in the backend data storage
332 * How to recover from error situation
335 Handling Exceptions Thrown by Synchronous SDL APIs
336 ==================================================
338 Synchronous SDL APIs throw exceptions in error situations. There are
339 corresponding exceptions for all *shareddatalayer::error* constants returned
340 by asynchronous APIs (see previous section). All exceptions thrown by SDL are
341 derived from *shareddatalayer::Exception*.
342 Therefore, a client can catch *shareddatalayer::Exception* in case the client
343 wants to implement common handling for some SDL originated exceptions. Note
344 that external services, which SDL uses, can throw exceptions which are not
345 derived from *shareddatalayer::Exception*.
347 Below is a C++ code example of a scenario where SDL client does common error
348 handling for all exceptions thrown from synchronous SDL API:
354 //Code which executes synchronous SDL API function
356 catch (const shareddatalayer::Exception& e)
358 log.error() << "SDL operation failed, error: " << e.what() << std::endl;
359 //Common error handling logic for all SDL errors
361 //Catch also non-SDL exceptions (like std::exception) if needed
363 Below C++ code example has separate handling for shareddatalayer::BackendError
364 exception and common handling for all other exceptions thrown by SDL:
370 //Code which executes synchronous SDL API function
372 catch (const shareddatalayer::BackendError& e)
374 log.error() << "SDL operation failed, error: " << e.what() << std::endl;
375 //Error handling logic for BackendError
377 catch (const shareddatalayer::Exception& e)
379 log.error() << "SDL operation failed, error: " << e.what() << std::endl;
380 //Common error handling logic for all other SDL errors than BackendError
382 //Catch also non-SDL exceptions (like std::exception) if needed
384 *log* is a logging service what an SDL client is using. Note that these are
385 simple and incomplete examples for demonstration purposes and they are not
386 meant to be used as such in real environment.
388 **Instructions for Error Handling Logic Implementation**
390 Doxygen documentation contains documentation for all exceptions thrown by SDL.
391 This documentation contains information which helps to design error handling
392 logic for each exception. For exceptions having corresponding error code,
393 exception documentation is usually a reference to corresponding error code
396 Each SDL API function, which throws exceptions, has a link to the documentation
397 of those exceptions. This link can be found from the Doxygen documentation of
398 given SDL API function.
407 This chapter discusses how certain general data storage related aspects work in
408 SDL. Discussed subjects include, for example, concurrency control and data
414 Production environments are typically deployed so that SDL backend data storage
415 and SDL clients are in different nodes (e.g. VM, container).
417 There are two different supported deployment modes for SDL backend data
420 * Standalone (single DB node without redundancy)
421 * Redundant (DB node pair working in primary/replica redundancy model)
423 SDL supports also Redis sentinel based DB cluster where deployment has one or
424 more DBAAS Redis sentinel group. Different DBAAS Redis sentinel groups
425 can be used to distribute SDL DB operations to different SDL DB instances. When
426 more than one DBAAS Redis sentinel group exits the selection of SDL DB instance
427 is based on the namespace string hash calculation.
429 SDL does not prevent backend data storage to be deployed in the same node with
430 the SDL client. Such deployments are, however, typically used only in
431 development/testing type of environments.
436 SDL does not support transactions doing one or more units of work in ACID
437 manner (pessimistic concurrency control).
439 SDL supports optimistic concurrency control by providing Check and Set (CAS)
440 type conditional functions. These conditional functions provide possibility
441 to do certain data modification operations only if data value matches the SDL
442 client's last known value. Thus a SDL client can check that someone else has
443 not changed the data after it was read by the SDL client. If the data would
444 have been changed, SDL does not do the modification operation and this is
445 indicated to the SDL client. The SDL client can then decide how to handle the
446 situation (for example read the latest data and retry modification).
448 *AsyncStorage::setIfAsync* is an example of a conditional function discussed
449 above. Other conditional functions exist as well.
454 Currently all data stored to SDL is stored to in-memory backend data storage.
455 Meaning that, data is not preserved over DB node restart. DB node restart does
456 not necessarily cause data loss for SDL client though. Refer to
457 `SDL Deployment section <#sdl-deployment>`_, for information about SDL backend
458 data storage redundancy models.
467 This chapter gives recommendations on how to use SDL.
469 Building Clients Using SDL
470 ==========================
472 * Use *pkg-config* tool to acquire needed compilation and linking flags,
473 instead of hardcoding them. This ensures that flags are always up-to-date.
474 See more information from `here <#building-clients-using-sdl>`_.
475 * If you want to mock SDL APIs in unit testing, SDL provides helper classes
476 for that. By using these helper classes you need to implement mock
477 implementation only for those SDL API functions which you use in the unit
478 tests. See more information from `doxygen documentation <#doxygen-generated-sdl-api-documentation>`_
479 of the helper classes:
481 * *include/sdl/tst/mockableasyncstorage.hpp: MockableAsyncStorage*
482 * *include/sdl/tst/mockablesyncstorage.hpp: MockableSyncStorage*
487 * SDL APIs are not thread-safe. If same SDL API instance is shared between
488 multiple threads, SDL client has to use explicit locking to ensure that only
489 one thread at time executes SDL API functions.
490 * Each SDL instance establishes own connection to backend data storage, which
491 requires resources (how heavy this exactly is depends on used backend data
492 storage type). Thus, from performance point of view, only one SDL instance
493 per one SDL API should be used if reasonably possible. One SDL instance can
494 access multiple SDL namespaces when using *AsyncStorage* and *SyncStorage*
496 * Use waitReadyAsync() function before doing first operation via asynchronous
497 APIs to ensure that SDL and backend data storage are ready to handle
498 operations. See waitReadyAsync() function
499 `doxygen documentation <#doxygen-generated-sdl-api-documentation>`_
500 for corresponding asynchronous API for details.
501 * Use waitReady() function before doing first operation via synchronous
502 APIs to ensure that SDL and backend data storage are ready to handle
503 operations. See waitReady() function
504 `doxygen documentation <#doxygen-generated-sdl-api-documentation>`_
505 for corresponding synchronous API for details.
506 * Avoid using heavy search functions (for example: *AsyncStorage::findKeys()*).
507 Rather define your keys so that you know which keys should be read.
512 * As namespace naming is currently on SDL client's responsibility, use enough
513 specific namespace names that same name is surely not used by someone else
514 (unless you want to share given namespace data with that someone else).
515 * Data entities related to each other should be placed under the same
516 namespace (unless there is a good reason not to). For example, accessing
517 multiple data entities with one SDL operation is possible only for data
518 entities belonging to same namespace.
519 * Identically named keys can be used in different namespaces. Creating own
520 namespaces for different use cases and unrelated data provides more freedom
521 into key name selection.
526 * Writing or reading one big junk of data at once is more efficient than
527 writing/reading the same amount of data in small steps. For example, create a
528 key list and read it once, rather than reading each key in a loop.
529 * If rolling upgrade needs to be supported, consider using Google Protocol
530 Buffers (or something similar) to make it possible to parse data which is
531 written by older or newer application version.
540 There is a pre-installed *sdlcli* tool in DBaaS container. With this tool user
541 can see statistics of database backend (Redis), check healthiness of DBaaS
542 database backend, list database keys and get and set values into database.
543 For example to see statistics give below command inside DBaaS container::
547 To check healthiness of database::
551 Use *sdlcli* help to get more information about available commands::