*.py,cover
.hypothesis/
.pytest_cache/
+xunit-results.xml
# Translations
*.mo
--- /dev/null
+---
+version: 2
+
+formats:
+ - htmlzip
+
+build:
+ image: latest
+
+python:
+ version: 3.7
+ install:
+ - requirements: docs/requirements-docs.txt
+
+sphinx:
+ configuration: docs/conf.py
-# Python shareddatalayer library
+# Python Shared Data Layer (SDL) library in RAN Intelligent Controller (RIC)
+
+Please see the [README.md](./ricsdl-package/README.md) in `ricsdl-package` directory.
--- /dev/null
+from docs_conf.conf import *
+linkcheck_ignore = [
+ 'http://localhost.*',
+ 'http://127.0.0.1.*',
+ 'https://gerrit.o-ran-sc.org.*'
+]
--- /dev/null
+---
+project_cfg: oran
+project: ric-plt/sdlpy
--- /dev/null
+..
+.. Copyright (c) 2019 AT&T Intellectual Property.
+.. Copyright (c) 2019 Nokia.
+..
+.. Licensed under the Creative Commons Attribution 4.0 International
+.. Public License (the "License"); you may not use this file except
+.. in compliance with the License. You may obtain a copy of the License at
+..
+.. https://creativecommons.org/licenses/by/4.0/
+..
+.. Unless required by applicable law or agreed to in writing, documentation
+.. distributed under the License is distributed on an "AS IS" BASIS,
+.. WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+..
+.. See the License for the specific language governing permissions and
+.. limitations under the License.
+..
+
+Welcome to O-RAN Shared Data Layer (SDL) in Python Documentation
+================================================================
+
+.. toctree::
+ :maxdepth: 2
+ :caption: Contents:
+
+ overview.rst
+ release-notes.rst
+
+* :ref:`genindex`
+* :ref:`modindex`
+* :ref:`search`
--- /dev/null
+..
+.. Copyright (c) 2019 AT&T Intellectual Property.
+.. Copyright (c) 2019 Nokia.
+..
+.. Licensed under the Creative Commons Attribution 4.0 International
+.. Public License (the "License"); you may not use this file except
+.. in compliance with the License. You may obtain a copy of the License at
+..
+.. https://creativecommons.org/licenses/by/4.0/
+..
+.. Unless required by applicable law or agreed to in writing, documentation
+.. distributed under the License is distributed on an "AS IS" BASIS,
+.. WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+..
+.. See the License for the specific language governing permissions and
+.. limitations under the License.
+..
+
+Overview
+========
+
+Shared Data Layer (SDL) provides a lightweight, high-speed interface for
+accessing shared data storage. The purpose is to enable utilizing clients to
+become stateless, conforming with, e.g., the requirements of the fifth
+generation mobile networks.
--- /dev/null
+..
+.. Copyright (c) 2019 AT&T Intellectual Property.
+.. Copyright (c) 2019 Nokia.
+..
+.. Licensed under the Creative Commons Attribution 4.0 International
+.. Public License (the "License"); you may not use this file except
+.. in compliance with the License. You may obtain a copy of the License at
+..
+.. https://creativecommons.org/licenses/by/4.0/
+..
+.. Unless required by applicable law or agreed to in writing, documentation
+.. distributed under the License is distributed on an "AS IS" BASIS,
+.. WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+..
+.. See the License for the specific language governing permissions and
+.. limitations under the License.
+..
+
+
+Release-Notes
+=============
+
+
+This document provides the release notes for release 1.0.0 of ricsdl.
+
+.. contents::
+ :depth: 3
+ :local:
+
+
+Version history
+---------------
+
++--------------------+--------------------+--------------------+--------------------+
+| **Date** | **Ver.** | **Author** | **Comment** |
+| | | | |
++--------------------+--------------------+--------------------+--------------------+
+| 2019-12-05 | 1.0.0 | T Tietavainen | First version |
+| | | | |
++--------------------+--------------------+--------------------+--------------------+
+
+
+Summary
+-------
+
+This is the first version of this package.
+It implements RIC Shared Data Layer (SDL) library.
+
+
+
+
+Release Data
+------------
+This is the first version of this package.
+
+
+
+
+
+Feature Additions
+^^^^^^^^^^^^^^^^^
+
+
+Bug Corrections
+^^^^^^^^^^^^^^^
+
+
+Deliverables
+^^^^^^^^^^^^
+
+Software Deliverables
++++++++++++++++++++++
+
+This version provides Python package ricsdl.
+It can be retrieved from pypi.org.
+
+
+
+Documentation Deliverables
+++++++++++++++++++++++++++
+
+
+
+
+
+Known Limitations, Issues and Workarounds
+-----------------------------------------
+
+System Limitations
+^^^^^^^^^^^^^^^^^^
+
+
+
+Known Issues
+^^^^^^^^^^^^
+
+Workarounds
+^^^^^^^^^^^
+
+
+
+
+
+References
+----------
+
+
--- /dev/null
+sphinx
+sphinx-rtd-theme
+sphinxcontrib-httpdomain
+recommonmark
+lfdocs-conf
--- /dev/null
+<!--
+===============================LICENSE_START============================
+Copyright (c) 2019 AT&T Intellectual Property.
+Copyright (c) 2018-2019 Nokia.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+
+This source code is part of the near-RT RIC (RAN Intelligent Controller)
+platform project (RICP).
+===============================LICENSE_END============================== -->
+
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+ <groupId>org.o-ran-sc.ric-plt.ricsdl</groupId>
+ <artifactId>ricsdl</artifactId>
+ <version>0</version>
+ <properties>
+ <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
+ <sonar.sources>ricsdl-package</sonar.sources>
+ <sonar.junit.reportsPath>xunit-results.xml</sonar.junit.reportsPath>
+ <sonar.python.coverage.reportPath>coverage.xml</sonar.python.coverage.reportPath>
+ <sonar.python.xunit.reportPath>xunit-results.xml</sonar.python.xunit.reportPath>
+ <sonar.language>py</sonar.language>
+ <sonar.pluginname>python</sonar.pluginname>
+ <sonar.inclusions>ricsdl-package/*.py</sonar.inclusions>
+ <sonar.exclusions>ricsdl-package/tests/*,setup.py</sonar.exclusions>
+ </properties>
+</project>
--- /dev/null
+Unless otherwise specified, all software contained herein is licensed
+under the Apache License, Version 2.0 (the "Software License");
+you may not use this software except in compliance with the Software
+License. You may obtain a copy of the Software License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the Software License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the Software License for the specific language governing permissions
+and limitations under the Software License.
+
+
+
+Unless otherwise specified, all documentation contained herein is licensed
+under the Creative Commons License, Attribution 4.0 Intl. (the
+"Documentation License"); you may not use this documentation except in
+compliance with the Documentation License. You may obtain a copy of the
+Documentation License at
+
+https://creativecommons.org/licenses/by/4.0/
+
+Unless required by applicable law or agreed to in writing, documentation
+distributed under the Documentation License is distributed on an "AS IS"
+BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+implied. See the Documentation License for the specific language governing
+permissions and limitations under the Documentation License.
+
--- /dev/null
+RIC SDL
+=======
+
+Shared Data Layer in the RAN Intelligent Controller
+
+Shared Data Layer (SDL) provides a lightweight, high-speed interface for
+accessing shared data storage. The purpose is to enable utilizing clients to
+become stateless, conforming with, e.g., the requirements of the fifth
+generation mobile networks.
+
+
+Concepts
+--------
+
+Namespace
+
+Namespaces provide data isolation within SDL data storage. That is, data in
+certain namespace is isolated from the data in other namespaces. Each SDL
+client uses one or more namespaces. Namespaces can be used, for example, to
+isolate data belonging to different use cases.
+
+Keys and Data
+
+Clients save key-data pairs. Data is passed as a `bytes` type. SDL stores the
+data as it is. Any structure that this data may have (e.g. a data structure
+serialized by `pickle`) is meaningful only to the client itself. Clients are
+responsible for managing the keys. As namespaces provide data isolation,
+keys in different namespaces always access different data.
+
+Backend Data Storage
+
+Backend data storage refers to data storage technology behind SDL API, which
+handles the actual data storing. SDL API hides the backend data storage
+implementation from SDL API clients, and therefore backend data storage
+technology can be changed without affecting SDL API clients. Currently, Redis
+database is used as a backend data storage solution.
+
+
+Install
+-------
+
+Install from PyPi
+
+```
+python3 -m pip install ricsdl
+```
+
+Install using the source
+
+```
+python3 setup.py install
+```
+
+
+Usage
+-----
+
+Instructions how to use SDL can be found from O-RAN Software Community (SC)
+Documentation under Near Realtime RAN Intelligent Controller (RIC) section:
+[O-RAN SC Documentation Home](https://docs.o-ran-sc.org/en/latest/)
+
+
+Unit Testing
+------------
+
+To run the unit tests run the following command in the package directory:
+`
+python3 -m pytest
+`
+
+
+Examples
+--------
+
+See the ``examples`` directory.
+
+
+
+CI
+--
+
+The ci is done with the `tox` tool. See `tox.ini` file for details.
# See the License for the specific language governing permissions and
# limitations under the License.
-
-"""Shareddatalayer library."""
+#
+# This source code is part of the near-RT RIC (RAN Intelligent Controller)
+# platform project (RICP).
+#
--- /dev/null
+# Copyright (c) 2019 AT&T Intellectual Property.
+# Copyright (c) 2018-2019 Nokia.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+#
+# This source code is part of the near-RT RIC (RAN Intelligent Controller)
+# platform project (RICP).
+#
+
+
+"""
+Examples how to use synchronous API functions of the Shared Data Layer (SDL).
+Execution of these examples requires:
+ * Following Redis extension commands have been installed to runtime environment:
+ - MSETPUB
+ - SETIE
+ - SETIEPUB
+ - SETNXPUB
+ - DELPUB
+ - DELIE
+ - DELIEPUB
+ Redis v4.0 or greater is required. Older versions do not support extension modules.
+ Implementation of above commands is produced by RIC DBaaS:
+ https://gerrit.o-ran-sc.org/r/admin/repos/ric-plt/dbaas
+ In official RIC deployments these commands are installed by `dbaas` service to Redis
+ container(s).
+ In development environment you may want install commands manually to pod/container, which is
+ running Redis.
+ * Following environment variables are needed to set to the pod/container where the application
+ utilizing SDL is going to be run.
+ DBAAS_SERVICE_HOST = [redis server address]
+ DBAAS_SERVICE_PORT= [redis server port]
+ DBAAS_MASTER_NAME = [master Redis sentinel name]. Needed to set only if sentinel is in use.
+ DBAAS_SERVICE_SENTINEL_PORT = [Redis sentinel port number]. Needed to set only if sentinel
+ is in use.
+"""
+from ricsdl.syncstorage import SyncStorage
+from ricsdl.exceptions import RejectedByBackend, NotConnected, BackendError
+
+
+# Constants used in the examples below.
+MY_NS = 'my_ns'
+MY_GRP_NS = 'my_group_ns'
+MY_LOCK_NS = 'my_group_ns'
+
+
+def _try_func_return(func):
+ """
+ Generic wrapper function to call SDL API function and handle exceptions if they are raised.
+ """
+ try:
+ return func()
+ except RejectedByBackend as exp:
+ print(f'SDL function {func.__name__} failed: {str(exp)}')
+ # Permanent failure, just forward the exception
+ raise
+ except (NotConnected, BackendError) as exp:
+ print(f'SDL function {func.__name__} failed for a temporal error: {str(exp)}')
+ # Here we could have a retry logic
+
+
+# Creates SDL instance. The call creates connection to the SDL database backend.
+mysdl = _try_func_return(SyncStorage)
+
+
+# Sets a value 'my_value' for a key 'my_key' under given namespace. Note that value
+# type must be bytes and multiple key values can be set in one set function call.
+_try_func_return(lambda: mysdl.set(MY_NS, {'my_key': b'my_value'}))
+
+
+# Gets the value of 'my_value' under given namespace.
+# Note that the type of returned value is bytes.
+my_ret_dict = _try_func_return(lambda: mysdl.get(MY_NS, {'my_key', 'someting not existing'}))
+for key, val in my_ret_dict.items():
+ assert val.decode("utf-8") == u'my_value'
+
+
+# Sets a value 'my_value2' for a key 'my_key' under given namespace only if the old value is
+# 'my_value'.
+# Note that value types must be bytes.
+was_set = _try_func_return(lambda: mysdl.set_if(MY_NS, 'my_key', b'my_value', b'my_value2'))
+assert was_set is True
+# Try again. This time value 'my_value2' won't be set, because the key has already 'my_value2'
+# value.
+was_set = _try_func_return(lambda: mysdl.set_if(MY_NS, 'my_key', b'my_value', b'my_value2'))
+assert was_set is False
+
+
+# Sets a value 'my_value' for a key 'my_key2' under given namespace only if the key doesn't exist.
+# Note that value types must be bytes.
+was_set = _try_func_return(lambda: mysdl.set_if_not_exists(MY_NS, 'my_key2', b'my_value'))
+assert was_set is True
+# Try again. This time the key 'my_key2' already exists.
+was_set = _try_func_return(lambda: mysdl.set_if_not_exists(MY_NS, 'my_key2', b'my_value'))
+assert was_set is False
+
+
+# Removes a key 'my_key' under given namespace.
+_try_func_return(lambda: mysdl.remove(MY_NS, 'my_key'))
+my_ret_dict = _try_func_return(lambda: mysdl.get(MY_NS, 'my_key'))
+assert my_ret_dict == {}
+
+
+# Removes a key 'my_key' under given namespace only if the old value is 'my_value'.
+was_removed = _try_func_return(lambda: mysdl.remove_if(MY_NS, 'my_key2', b'my_value'))
+assert was_removed is True
+# Try again to remove not anymore existing key 'my_key'.
+was_removed = _try_func_return(lambda: mysdl.remove_if(MY_NS, 'my_key2', b'my_value'))
+assert was_removed is False
+
+
+# Removes all the keys under given namespace.
+_try_func_return(lambda: mysdl.set(MY_NS, {'my_key': b'something'}))
+my_ret_dict = _try_func_return(lambda: mysdl.get(MY_NS, {'my_key'}))
+assert my_ret_dict != {}
+
+_try_func_return(lambda: mysdl.remove_all(MY_NS))
+my_ret_dict = _try_func_return(lambda: mysdl.get(MY_NS, {'my_key'}))
+assert my_ret_dict == {}
+
+
+# Finds keys under given namespace that are matching to given key prefix 'my_k'.
+_try_func_return(lambda: mysdl.set(MY_NS, {'my_key': b'my_value'}))
+ret_keys = _try_func_return(lambda: mysdl.find_keys(MY_NS, ''))
+assert ret_keys == ['my_key']
+
+
+# Finds keys and their values under given namespace that are matching to given key prefix 'my_k'.
+# Note that the type of returned value is bytes.
+ret_key_values = _try_func_return(lambda: mysdl.find_and_get(MY_NS, '', atomic=True))
+assert ret_key_values == {'my_key': b'my_value'}
+
+_try_func_return(lambda: mysdl.remove_all(MY_NS))
+
+
+# Adds a member 'a' to a group 'my_group' under given namespace. A group is a unique collection of
+# members.
+# Note that member type must be bytes and multiple members can be set in one set function call.
+_try_func_return(lambda: mysdl.add_member(MY_GRP_NS, 'my_group', {b'a'}))
+# Try again to add a member 'a'. This time 'a' won't be added, because 'a' belongs already to
+# the group.
+_try_func_return(lambda: mysdl.add_member(MY_GRP_NS, 'my_group', {b'a'}))
+
+
+# Gets group 'my_group' members under given namespace.
+# Note that the type of returned member is bytes.
+ret_members = _try_func_return(lambda: mysdl.get_members(MY_GRP_NS, 'my_group'))
+assert ret_members == {b'a'}
+
+
+# Checks if 'a' is a member of the group 'my_group' under given namespace.
+was_member = _try_func_return(lambda: mysdl.is_member(MY_GRP_NS, 'my_group', b'a'))
+assert was_member is True
+was_member = _try_func_return(lambda: mysdl.is_member(MY_GRP_NS, 'my_group', b'not a member'))
+assert was_member is False
+
+
+# Returns the count of members of a group 'my_group' under given namespace.
+ret_count = _try_func_return(lambda: mysdl.group_size(MY_GRP_NS, 'my_group'))
+assert ret_count == 1
+
+
+# Removes the member 'a' of the group 'my_group' under given namespace.
+_try_func_return(lambda: mysdl.remove_member(MY_GRP_NS, 'my_group', {b'a', b'not exists'}))
+ret_count = _try_func_return(lambda: mysdl.group_size(MY_GRP_NS, 'my_group'))
+assert ret_count == 0
+
+
+# Removes the group 'my_group' under given namespace.
+_try_func_return(lambda: mysdl.add_member(MY_GRP_NS, 'my_group', {b'a', b'b', b'c'}))
+ret_count = _try_func_return(lambda: mysdl.group_size(MY_GRP_NS, 'my_group'))
+assert ret_count == 3
+
+_try_func_return(lambda: mysdl.remove_group(MY_GRP_NS, 'my_group'))
+ret_count = _try_func_return(lambda: mysdl.group_size(MY_GRP_NS, 'my_group'))
+ret_members = _try_func_return(lambda: mysdl.get_members(MY_GRP_NS, 'my_group'))
+assert ret_count == 0
+assert ret_members == set()
+
+
+# Gets a lock 'my_lock' resource under given namespace.
+# Note that this function does not take a lock, you need to call 'acquire' function to take
+# the lock to yourself.
+my_lock = _try_func_return(lambda: mysdl.get_lock_resource(MY_LOCK_NS, "my_lock", expiration=5.5))
+assert my_lock is not None
+
+
+# Acquires a lock from the lock resource. Return True if lock was taken within given retry limits.
+was_acquired = _try_func_return(lambda: my_lock.acquire(retry_interval=0.5, retry_timeout=2))
+assert was_acquired is True
+# Try again. This time a lock won't be acquired successfully, because we have a lock already.
+was_acquired = _try_func_return(lambda: my_lock.acquire(retry_interval=0.1, retry_timeout=0.2))
+assert was_acquired is False
+
+
+# Refreshs the remaining validity time of the existing lock back to the initial value.
+_try_func_return(my_lock.refresh)
+
+
+# Gets the remaining validity time of the lock.
+ret_time = _try_func_return(my_lock.get_validity_time)
+assert ret_time != 0
+
+
+# Releases the lock.
+_try_func_return(my_lock.release)
+
+
+# Locking example what utilizes python 'with' statement with SDL lock.
+# The lock is released automatically when we are out of the scope of
+# 'the with my_lock' statement.
+my_lock = _try_func_return(lambda: mysdl.get_lock_resource(MY_LOCK_NS, "my_lock", 2.5))
+with my_lock:
+ # Just an example how to use lock API
+ time_left = _try_func_return(my_lock.get_validity_time)
+
+ # Add here operations what needs to be done under a lock, for example some
+ # operations with a shared resources what needs to be done in a mutually
+ # exclusive way.
+
+# Lock is not anymore hold here
+
+
+# Closes the SDL connection.
+mysdl.close()
--- /dev/null
+# Copyright (c) 2019 AT&T Intellectual Property.
+# Copyright (c) 2018-2019 Nokia.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+#
+# This source code is part of the near-RT RIC (RAN Intelligent Controller)
+# platform project (RICP).
+#
+
+
+"""Shared Data Layer (SDL) library."""
+
+from .syncstorage import (SyncStorage, SyncLock)
+from .exceptions import (
+ SdlTypeError,
+ SdlException,
+ BackendError,
+ NotConnected,
+ RejectedByBackend
+)
+
+
+__version__ = '1.0.0'
+
+
+__all__ = [
+ 'SyncStorage',
+ 'SyncLock',
+ 'SdlTypeError',
+ 'SdlException',
+ 'BackendError',
+ 'NotConnected',
+ 'RejectedByBackend'
+]
--- /dev/null
+# Copyright (c) 2019 AT&T Intellectual Property.
+# Copyright (c) 2018-2019 Nokia.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+#
+# This source code is part of the near-RT RIC (RAN Intelligent Controller)
+# platform project (RICP).
+#
+
+
+"""Shared Data Layer (SDL) database backend module."""
+
+from importlib import import_module
+
+
+def get_backend_instance(configuration):
+ """
+ Select database backend solution and return and instance of it.
+ For now only Redis backend solution is supported.
+ """
+ backend_name = 'RedisBackend'
+ backend_module_name = 'redis'
+
+ package = __package__ or __name__
+ backend_module = import_module('.' + backend_module_name, package=package)
+ backend_class = getattr(backend_module, backend_name)
+ instance = backend_class(configuration)
+ return instance
+
+
+def get_backend_lock_instance(ns, name, expiration, backend):
+ """
+ Select database backend lock solution and return and instance of it.
+ For now only Redis backend lock solution is supported.
+ """
+ backend_lock_name = 'RedisBackendLock'
+ backend_module_name = 'redis'
+
+ package = __package__ or __name__
+ backend_module = import_module('.' + backend_module_name, package=package)
+ backend_lock_class = getattr(backend_module, backend_lock_name)
+ instance = backend_lock_class(ns, name, expiration, backend)
+ return instance
--- /dev/null
+# Copyright (c) 2019 AT&T Intellectual Property.
+# Copyright (c) 2018-2019 Nokia.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+#
+# This source code is part of the near-RT RIC (RAN Intelligent Controller)
+# platform project (RICP).
+#
+
+
+"""The module provides Shared Data Layer (SDL) database backend interface."""
+
+from typing import (Dict, Set, List, Union)
+from abc import ABC, abstractmethod
+
+
+class DbBackendAbc(ABC):
+ """An abstract Shared Data Layer (SDL) class providing database backend interface."""
+
+ @abstractmethod
+ def close(self):
+ """Close database backend connection."""
+ pass
+
+ @abstractmethod
+ def set(self, ns: str, data_map: Dict[str, bytes]) -> None:
+ """Write key value data mapping to database under a namespace."""
+ pass
+
+ @abstractmethod
+ def set_if(self, ns: str, key: str, old_data: bytes, new_data: bytes) -> bool:
+ """"Write key value to database under a namespace if the old value is expected one."""
+ pass
+
+ @abstractmethod
+ def set_if_not_exists(self, ns: str, key: str, data: bytes) -> bool:
+ """"Write key value to database under a namespace if key doesn't exist."""
+ pass
+
+ @abstractmethod
+ def get(self, ns: str, keys: List[str]) -> Dict[str, bytes]:
+ """"Return values of the keys under a namespace."""
+ pass
+
+ @abstractmethod
+ def find_keys(self, ns: str, key_prefix: str) -> List[str]:
+ """"Return all the keys matching search pattern under a namespace in database."""
+ pass
+
+ @abstractmethod
+ def find_and_get(self, ns: str, key_prefix: str, atomic: bool) -> Dict[str, bytes]:
+ """
+ Return all the keys with their values matching search pattern under a namespace in
+ database.
+ """
+ pass
+
+ @abstractmethod
+ def remove(self, ns: str, keys: List[str]) -> None:
+ """Remove keys and their data from database."""
+ pass
+
+ @abstractmethod
+ def remove_if(self, ns: str, key: str, data: bytes) -> bool:
+ """
+ Remove key and its data from database if if the current data value is expected
+ one.
+ """
+ pass
+
+ @abstractmethod
+ def add_member(self, ns: str, group: str, members: Set[bytes]) -> None:
+ """Add new members to a group under a namespace in database."""
+ pass
+
+ @abstractmethod
+ def remove_member(self, ns: str, group: str, members: Set[bytes]) -> None:
+ """Remove members from a group under a namespace in database."""
+ pass
+
+ @abstractmethod
+ def remove_group(self, ns: str, group: str) -> None:
+ """Remove a group under a namespace in database along with it's members."""
+ pass
+
+ @abstractmethod
+ def get_members(self, ns: str, group: str) -> Set[bytes]:
+ """Get all the members of a group under a namespace in database."""
+ pass
+
+ @abstractmethod
+ def is_member(self, ns: str, group: str, member: bytes) -> bool:
+ """Validate if a given member is in the group under a namespace in database."""
+ pass
+
+ @abstractmethod
+ def group_size(self, ns: str, group: str) -> int:
+ """Return the number of members in a group under a namespace in database."""
+ pass
+
+
+class DbBackendLockAbc(ABC):
+ """
+ An abstract Shared Data Layer (SDL) class providing database backend lock interface.
+ Args:
+ ns (str): Namespace under which this lock is targeted.
+ name (str): Lock name, identifies the lock key in a database backend.
+ """
+ def __init__(self, ns: str, name: str) -> None:
+ self._ns = ns
+ self._lock_name = name
+ super().__init__()
+
+ @abstractmethod
+ def acquire(self, retry_interval: Union[int, float] = 0.1,
+ retry_timeout: Union[int, float] = 10) -> bool:
+ """Acquire a database lock."""
+ pass
+
+ @abstractmethod
+ def release(self) -> None:
+ """Release a database lock."""
+ pass
+
+ @abstractmethod
+ def refresh(self) -> None:
+ """Refresh the remaining validity time of the database lock back to a initial value."""
+ pass
+
+ @abstractmethod
+ def get_validity_time(self) -> Union[int, float]:
+ """Return remaining validity time of the lock in seconds."""
+ pass
--- /dev/null
+# Copyright (c) 2019 AT&T Intellectual Property.
+# Copyright (c) 2018-2019 Nokia.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+#
+# This source code is part of the near-RT RIC (RAN Intelligent Controller)
+# platform project (RICP).
+#
+
+
+"""The module provides implementation of Shared Data Layer (SDL) database backend interface."""
+import contextlib
+from typing import (Dict, Set, List, Union)
+from redis import Redis
+from redis.sentinel import Sentinel
+from redis.lock import Lock
+from redis._compat import nativestr
+from redis import exceptions as redis_exceptions
+from ricsdl.configuration import _Configuration
+from ricsdl.exceptions import (
+ RejectedByBackend,
+ NotConnected,
+ BackendError
+)
+from .dbbackend_abc import DbBackendAbc
+from .dbbackend_abc import DbBackendLockAbc
+
+
+@contextlib.contextmanager
+def _map_to_sdl_exception():
+ """Translates known redis exceptions into SDL exceptions."""
+ try:
+ yield
+ except(redis_exceptions.ResponseError) as exc:
+ raise RejectedByBackend("SDL backend rejected the request: {}".
+ format(str(exc))) from exc
+ except(redis_exceptions.ConnectionError, redis_exceptions.TimeoutError) as exc:
+ raise NotConnected("SDL not connected to backend: {}".
+ format(str(exc))) from exc
+ except(redis_exceptions.RedisError) as exc:
+ raise BackendError("SDL backend failed to process the request: {}".
+ format(str(exc))) from exc
+
+
+class RedisBackend(DbBackendAbc):
+ """
+ A class providing an implementation of database backend of Shared Data Layer (SDL), when
+ backend database solution is Redis.
+
+ Args:
+ configuration (_Configuration): SDL configuration, containing credentials to connect to
+ Redis database backend.
+ """
+ def __init__(self, configuration: _Configuration) -> None:
+ super().__init__()
+ with _map_to_sdl_exception():
+ if configuration.get_params().db_sentinel_port:
+ sentinel_node = (configuration.get_params().db_host,
+ configuration.get_params().db_sentinel_port)
+ master_name = configuration.get_params().db_sentinel_master_name
+ self.__sentinel = Sentinel([sentinel_node])
+ self.__redis = self.__sentinel.master_for(master_name)
+ else:
+ self.__redis = Redis(host=configuration.get_params().db_host,
+ port=configuration.get_params().db_port,
+ db=0,
+ max_connections=20)
+ self.__redis.set_response_callback('SETIE', lambda r: r and nativestr(r) == 'OK' or False)
+ self.__redis.set_response_callback('DELIE', lambda r: r and int(r) == 1 or False)
+
+ def __del__(self):
+ self.close()
+
+ def __str__(self):
+ return str(
+ {
+ "Redis connection": repr(self.__redis)
+ }
+ )
+
+ def close(self):
+ self.__redis.close()
+
+ def set(self, ns: str, data_map: Dict[str, bytes]) -> None:
+ db_data_map = self._add_data_map_ns_prefix(ns, data_map)
+ with _map_to_sdl_exception():
+ self.__redis.mset(db_data_map)
+
+ def set_if(self, ns: str, key: str, old_data: bytes, new_data: bytes) -> bool:
+ db_key = self._add_key_ns_prefix(ns, key)
+ with _map_to_sdl_exception():
+ return self.__redis.execute_command('SETIE', db_key, new_data, old_data)
+
+ def set_if_not_exists(self, ns: str, key: str, data: bytes) -> bool:
+ db_key = self._add_key_ns_prefix(ns, key)
+ with _map_to_sdl_exception():
+ return self.__redis.setnx(db_key, data)
+
+ def get(self, ns: str, keys: List[str]) -> Dict[str, bytes]:
+ ret = dict()
+ db_keys = self._add_keys_ns_prefix(ns, keys)
+ with _map_to_sdl_exception():
+ values = self.__redis.mget(db_keys)
+ for idx, val in enumerate(values):
+ # return only key values, which has a value
+ if val:
+ ret[keys[idx]] = val
+ return ret
+
+ def find_keys(self, ns: str, key_prefix: str) -> List[str]:
+ escaped_key_prefix = self._escape_characters(key_prefix)
+ db_escaped_key_prefix = self._add_key_ns_prefix(ns, escaped_key_prefix + '*')
+ with _map_to_sdl_exception():
+ ret = self.__redis.keys(db_escaped_key_prefix)
+ return self._strip_ns_from_bin_keys(ns, ret)
+
+ def find_and_get(self, ns: str, key_prefix: str, atomic: bool) -> Dict[str, bytes]:
+ # todo: replace below implementation with redis 'NGET' module
+ ret = dict() # type: Dict[str, bytes]
+ with _map_to_sdl_exception():
+ matched_keys = self.find_keys(ns, key_prefix)
+ if matched_keys:
+ ret = self.get(ns, matched_keys)
+ return ret
+
+ def remove(self, ns: str, keys: List[str]) -> None:
+ db_keys = self._add_keys_ns_prefix(ns, keys)
+ with _map_to_sdl_exception():
+ self.__redis.delete(*db_keys)
+
+ def remove_if(self, ns: str, key: str, data: bytes) -> bool:
+ db_key = self._add_key_ns_prefix(ns, key)
+ with _map_to_sdl_exception():
+ return self.__redis.execute_command('DELIE', db_key, data)
+
+ def add_member(self, ns: str, group: str, members: Set[bytes]) -> None:
+ db_key = self._add_key_ns_prefix(ns, group)
+ with _map_to_sdl_exception():
+ self.__redis.sadd(db_key, *members)
+
+ def remove_member(self, ns: str, group: str, members: Set[bytes]) -> None:
+ db_key = self._add_key_ns_prefix(ns, group)
+ with _map_to_sdl_exception():
+ self.__redis.srem(db_key, *members)
+
+ def remove_group(self, ns: str, group: str) -> None:
+ db_key = self._add_key_ns_prefix(ns, group)
+ with _map_to_sdl_exception():
+ self.__redis.delete(db_key)
+
+ def get_members(self, ns: str, group: str) -> Set[bytes]:
+ db_key = self._add_key_ns_prefix(ns, group)
+ with _map_to_sdl_exception():
+ return self.__redis.smembers(db_key)
+
+ def is_member(self, ns: str, group: str, member: bytes) -> bool:
+ db_key = self._add_key_ns_prefix(ns, group)
+ with _map_to_sdl_exception():
+ return self.__redis.sismember(db_key, member)
+
+ def group_size(self, ns: str, group: str) -> int:
+ db_key = self._add_key_ns_prefix(ns, group)
+ with _map_to_sdl_exception():
+ return self.__redis.scard(db_key)
+
+ @classmethod
+ def _add_key_ns_prefix(cls, ns: str, key: str):
+ return '{' + ns + '},' + key
+
+ @classmethod
+ def _add_keys_ns_prefix(cls, ns: str, keylist: List[str]) -> List[str]:
+ ret_nskeys = []
+ for k in keylist:
+ ret_nskeys.append('{' + ns + '},' + k)
+ return ret_nskeys
+
+ @classmethod
+ def _add_data_map_ns_prefix(cls, ns: str, data_dict: Dict[str, bytes]) -> Dict[str, bytes]:
+ ret_nsdict = {}
+ for key, val in data_dict.items():
+ ret_nsdict['{' + ns + '},' + key] = val
+ return ret_nsdict
+
+ @classmethod
+ def _strip_ns_from_bin_keys(cls, ns: str, nskeylist: List[bytes]) -> List[str]:
+ ret_keys = []
+ for k in nskeylist:
+ nskey = k.decode("utf-8").split(',', 1)
+ if len(nskey) != 2:
+ msg = u'Illegal namespace %s key:%s' % (ns, nskey)
+ raise RejectedByBackend(msg)
+ ret_keys.append(nskey[1])
+ return ret_keys
+
+ @classmethod
+ def _escape_characters(cls, pattern: str) -> str:
+ return pattern.translate(str.maketrans(
+ {"(": r"\(",
+ ")": r"\)",
+ "[": r"\[",
+ "]": r"\]",
+ "*": r"\*",
+ "?": r"\?",
+ "\\": r"\\"}))
+
+ def get_redis_connection(self):
+ """Return existing Redis database connection."""
+ return self.__redis
+
+
+class RedisBackendLock(DbBackendLockAbc):
+ """
+ A class providing an implementation of database backend lock of Shared Data Layer (SDL), when
+ backend database solution is Redis.
+
+ Args:
+ ns (str): Namespace under which this lock is targeted.
+ name (str): Lock name, identifies the lock key in a Redis database backend.
+ expiration (int, float): Lock expiration time after which the lock is removed if it hasn't
+ been released earlier by a 'release' method.
+ redis_backend (RedisBackend): Database backend object containing connection to Redis
+ database.
+ """
+ lua_get_validity_time = None
+ # KEYS[1] - lock name
+ # ARGS[1] - token
+ # return < 0 in case of failure, otherwise return lock validity time in milliseconds.
+ LUA_GET_VALIDITY_TIME_SCRIPT = """
+ local token = redis.call('get', KEYS[1])
+ if not token then
+ return -10
+ end
+ if token ~= ARGV[1] then
+ return -11
+ end
+ return redis.call('pttl', KEYS[1])
+ """
+
+ def __init__(self, ns: str, name: str, expiration: Union[int, float],
+ redis_backend: RedisBackend) -> None:
+ super().__init__(ns, name)
+ self.__redis = redis_backend.get_redis_connection()
+ with _map_to_sdl_exception():
+ redis_lockname = '{' + ns + '},' + self._lock_name
+ self.__redis_lock = Lock(redis=self.__redis, name=redis_lockname, timeout=expiration)
+ self._register_scripts()
+
+ def __str__(self):
+ return str(
+ {
+ "lock namespace": self._ns,
+ "lock name": self._lock_name,
+ "lock status": self._lock_status_to_string()
+ }
+ )
+
+ def acquire(self, retry_interval: Union[int, float] = 0.1,
+ retry_timeout: Union[int, float] = 10) -> bool:
+ succeeded = False
+ self.__redis_lock.sleep = retry_interval
+ with _map_to_sdl_exception():
+ succeeded = self.__redis_lock.acquire(blocking_timeout=retry_timeout)
+ return succeeded
+
+ def release(self) -> None:
+ with _map_to_sdl_exception():
+ self.__redis_lock.release()
+
+ def refresh(self) -> None:
+ with _map_to_sdl_exception():
+ self.__redis_lock.reacquire()
+
+ def get_validity_time(self) -> Union[int, float]:
+ validity = 0
+ if self.__redis_lock.local.token is None:
+ msg = u'Cannot get validity time of an unlocked lock %s' % self._lock_name
+ raise RejectedByBackend(msg)
+
+ with _map_to_sdl_exception():
+ validity = self.lua_get_validity_time(keys=[self.__redis_lock.name],
+ args=[self.__redis_lock.local.token],
+ client=self.__redis)
+ if validity < 0:
+ msg = (u'Getting validity time of a lock %s failed with error code: %d'
+ % (self._lock_name, validity))
+ raise RejectedByBackend(msg)
+ ftime = validity / 1000.0
+ if ftime.is_integer():
+ return int(ftime)
+ return ftime
+
+ def _register_scripts(self):
+ cls = self.__class__
+ client = self.__redis
+ if cls.lua_get_validity_time is None:
+ cls.lua_get_validity_time = client.register_script(cls.LUA_GET_VALIDITY_TIME_SCRIPT)
+
+ def _lock_status_to_string(self) -> str:
+ try:
+ if self.__redis_lock.locked():
+ if self.__redis_lock.owned():
+ return 'locked'
+ return 'locked by someone else'
+ return 'unlocked'
+ except(redis_exceptions.RedisError) as exc:
+ return f'Error: {str(exc)}'
--- /dev/null
+# Copyright (c) 2019 AT&T Intellectual Property.
+# Copyright (c) 2018-2019 Nokia.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+#
+# This source code is part of the near-RT RIC (RAN Intelligent Controller)
+# platform project (RICP).
+#
+
+
+"""The module provides implementation of Shared Data Layer (SDL) configurability."""
+import os
+from collections import namedtuple
+
+
+class _Configuration():
+ """This class implements Shared Data Layer (SDL) configurability."""
+ Params = namedtuple('Params', ['db_host', 'db_port', 'db_sentinel_port',
+ 'db_sentinel_master_name'])
+
+ def __init__(self):
+ self.params = self._read_configuration()
+
+ def __str__(self):
+ return str(
+ {
+ "DB host": self.params.db_host,
+ "DB port": self.params.db_port,
+ "DB master sentinel": self.params.db_sentinel_master_name,
+ "DB sentinel port": self.params.db_sentinel_port
+ }
+ )
+
+ def get_params(self):
+ """Return SDL configuration."""
+ return self.params
+
+ @classmethod
+ def _read_configuration(cls):
+ return _Configuration.Params(db_host=os.getenv('DBAAS_SERVICE_HOST'),
+ db_port=os.getenv('DBAAS_SERVICE_PORT'),
+ db_sentinel_port=os.getenv('DBAAS_SERVICE_SENTINEL_PORT'),
+ db_sentinel_master_name=os.getenv('DBAAS_MASTER_NAME'))
# See the License for the specific language governing permissions and
# limitations under the License.
-"Exceptions raised by the shareddatalayer."
+#
+# This source code is part of the near-RT RIC (RAN Intelligent Controller)
+# platform project (RICP).
+#
+
+
+"Exceptions raised by the Shared Data Layer (SDL)."
+
class SdlTypeError(TypeError):
"""
"""
pass
+
class SdlException(Exception):
- """Base exception class for shareddatalayer exceptions."""
+ """Base exception class for Shared Data Layer (SDL) exceptions."""
pass
+
class NotConnected(SdlException):
"""
- Exception for not being connected to the database backend.
- Shareddatalayer is not connected to the backend data storage and therefore could not deliver the
- request to the backend data storage. Data in the backend data storage has not been altered.
+ Exception for SDL not being connected to the database backend.
+ SDL is not connected to the backend data storage and therefore could not deliver the request
+ to the backend data storage. Data in the backend data storage has not been altered.
Client is advised to try the operation again later.
"""
pass
+
class BackendError(SdlException):
"""
- Exception for request processing failure.
+ Exception for request processing failure in SDL database backend.
In case of a write type request, data in the backend data storage may or may not have been
altered. Client is advised to try the operation again later.
"""
pass
+
class RejectedByBackend(SdlException):
"""
- Exception for shareddatalayer rejecting the request.
+ Exception for SDL database backend rejecting the request.
Backend data storage rejected the request. In case of a write type request, data in the backend
data storage may or may not have been altered. It is likely that the same request will fail
repeatedly. It is advised to investigate the exact reason for the failure from the logs.
--- /dev/null
+# Copyright (c) 2019 AT&T Intellectual Property.
+# Copyright (c) 2018-2019 Nokia.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+#
+# This source code is part of the near-RT RIC (RAN Intelligent Controller)
+# platform project (RICP).
+#
+
+"""The module provides implementation of the syncronous Shared Data Layer (SDL) interface."""
+import builtins
+from typing import (Dict, Set, List, Union)
+from ricsdl.configuration import _Configuration
+from ricsdl.syncstorage_abc import (SyncStorageAbc, SyncLockAbc)
+import ricsdl.backend
+from ricsdl.backend.dbbackend_abc import DbBackendAbc
+from ricsdl.exceptions import SdlTypeError
+
+
+def func_arg_checker(exception, start_arg_idx, **types):
+ """Decorator to validate function arguments."""
+ def _check(func):
+ if not __debug__:
+ return func
+
+ def _validate(*args, **kwds):
+ for idx, arg in enumerate(args[start_arg_idx:], start_arg_idx):
+ if func.__code__.co_varnames[idx] in types and \
+ not isinstance(arg, types[func.__code__.co_varnames[idx]]):
+ raise exception(r"Wrong argument type: '{}'={}. Must be: {}".
+ format(func.__code__.co_varnames[idx], type(arg),
+ types[func.__code__.co_varnames[idx]]))
+ for kwdname, kwdval in kwds.items():
+ if kwdname in types and not isinstance(kwdval, types[kwdname]):
+ raise exception(r"Wrong argument type: '{}'={}. Must be: {}".
+ format(kwdname, type(kwdval), types[kwdname]))
+ return func(*args, **kwds)
+ _validate.__name__ = func.__name__
+ return _validate
+ return _check
+
+
+class SyncLock(SyncLockAbc):
+ """
+ This class implements Shared Data Layer (SDL) abstract 'SyncLockAbc' class.
+
+ A lock instance is created per namespace and it is identified by its `name` within a namespace.
+
+ Args:
+ ns (str): Namespace under which this lock is targeted.
+ name (str): Lock name, identifies the lock key in SDL storage.
+ expiration (int, float): Lock expiration time after which the lock is removed if it hasn't
+ been released earlier by a 'release' method.
+ storage (SyncStorage): Database backend object containing connection to a database.
+ """
+ @func_arg_checker(SdlTypeError, 1, ns=str, name=str, expiration=(int, float))
+ def __init__(self, ns: str, name: str, expiration: Union[int, float],
+ storage: 'SyncStorage') -> None:
+
+ super().__init__(ns, name, expiration)
+ self.__dbbackendlock = ricsdl.backend.get_backend_lock_instance(ns, name, expiration,
+ storage.get_backend())
+
+ def __str__(self):
+ return str(
+ {
+ "namespace": self._ns,
+ "name": self._name,
+ "expiration": self._expiration,
+ "backend lock": str(self.__dbbackendlock)
+ }
+ )
+
+ @func_arg_checker(SdlTypeError, 1, retry_interval=(int, float),
+ retry_timeout=(int, float))
+ def acquire(self, retry_interval: Union[int, float] = 0.1,
+ retry_timeout: Union[int, float] = 10) -> bool:
+ return self.__dbbackendlock.acquire(retry_interval, retry_timeout)
+
+ def release(self) -> None:
+ self.__dbbackendlock.release()
+
+ def refresh(self) -> None:
+ self.__dbbackendlock.refresh()
+
+ def get_validity_time(self) -> Union[int, float]:
+ return self.__dbbackendlock.get_validity_time()
+
+
+class SyncStorage(SyncStorageAbc):
+ """
+ This class implements Shared Data Layer (SDL) abstract 'SyncStorageAbc' class.
+
+ This class provides synchronous access to all the namespaces in SDL storage.
+ Data can be written, read and removed based on keys known to clients. Keys are unique within
+ a namespace, namespace identifier is passed as a parameter to all the operations.
+
+ Args:
+ None
+ """
+ def __init__(self) -> None:
+ super().__init__()
+ self.__configuration = _Configuration()
+ self.__dbbackend = ricsdl.backend.get_backend_instance(self.__configuration)
+
+ def __del__(self):
+ self.close()
+
+ def __str__(self):
+ return str(
+ {
+ "configuration": str(self.__configuration),
+ "backend": str(self.__dbbackend)
+ }
+ )
+
+ def close(self):
+ self.__dbbackend.close()
+
+ @func_arg_checker(SdlTypeError, 1, ns=str, data_map=dict)
+ def set(self, ns: str, data_map: Dict[str, bytes]) -> None:
+ self.__dbbackend.set(ns, data_map)
+
+ @func_arg_checker(SdlTypeError, 1, ns=str, key=str, old_data=bytes, new_data=bytes)
+ def set_if(self, ns: str, key: str, old_data: bytes, new_data: bytes) -> bool:
+ return self.__dbbackend.set_if(ns, key, old_data, new_data)
+
+ @func_arg_checker(SdlTypeError, 1, ns=str, key=str, data=bytes)
+ def set_if_not_exists(self, ns: str, key: str, data: bytes) -> bool:
+ return self.__dbbackend.set_if_not_exists(ns, key, data)
+
+ @func_arg_checker(SdlTypeError, 1, ns=str, keys=(str, builtins.set))
+ def get(self, ns: str, keys: Union[str, Set[str]]) -> Dict[str, bytes]:
+ return self.__dbbackend.get(ns, list(keys))
+
+ @func_arg_checker(SdlTypeError, 1, ns=str, key_prefix=str)
+ def find_keys(self, ns: str, key_prefix: str) -> List[str]:
+ return self.__dbbackend.find_keys(ns, key_prefix)
+
+ @func_arg_checker(SdlTypeError, 1, ns=str, key_prefix=str, atomic=bool)
+ def find_and_get(self, ns: str, key_prefix: str, atomic: bool) -> Dict[str, bytes]:
+ return self.__dbbackend.find_and_get(ns, key_prefix, atomic)
+
+ @func_arg_checker(SdlTypeError, 1, ns=str, keys=(str, builtins.set))
+ def remove(self, ns: str, keys: Union[str, Set[str]]) -> None:
+ self.__dbbackend.remove(ns, list(keys))
+
+ @func_arg_checker(SdlTypeError, 1, ns=str, key=str, data=bytes)
+ def remove_if(self, ns: str, key: str, data: bytes) -> bool:
+ return self.__dbbackend.remove_if(ns, key, data)
+
+ @func_arg_checker(SdlTypeError, 1, ns=str)
+ def remove_all(self, ns: str) -> None:
+ keys = self.__dbbackend.find_keys(ns, '')
+ if keys:
+ self.__dbbackend.remove(ns, keys)
+
+ @func_arg_checker(SdlTypeError, 1, ns=str, group=str, members=(bytes, builtins.set))
+ def add_member(self, ns: str, group: str, members: Union[bytes, Set[bytes]]) -> None:
+ self.__dbbackend.add_member(ns, group, members)
+
+ @func_arg_checker(SdlTypeError, 1, ns=str, group=str, members=(bytes, builtins.set))
+ def remove_member(self, ns: str, group: str, members: Union[bytes, Set[bytes]]) -> None:
+ self.__dbbackend.remove_member(ns, group, members)
+
+ @func_arg_checker(SdlTypeError, 1, ns=str, group=str)
+ def remove_group(self, ns: str, group: str) -> None:
+ self.__dbbackend.remove_group(ns, group)
+
+ @func_arg_checker(SdlTypeError, 1, ns=str, group=str)
+ def get_members(self, ns: str, group: str) -> Set[bytes]:
+ return self.__dbbackend.get_members(ns, group)
+
+ @func_arg_checker(SdlTypeError, 1, ns=str, group=str, member=bytes)
+ def is_member(self, ns: str, group: str, member: bytes) -> bool:
+ return self.__dbbackend.is_member(ns, group, member)
+
+ @func_arg_checker(SdlTypeError, 1, ns=str, group=str)
+ def group_size(self, ns: str, group: str) -> int:
+ return self.__dbbackend.group_size(ns, group)
+
+ @func_arg_checker(SdlTypeError, 1, ns=str, resource=str, expiration=(int, float))
+ def get_lock_resource(self, ns: str, resource: str, expiration: Union[int, float]) -> SyncLock:
+ return SyncLock(ns, resource, expiration, self)
+
+ def get_backend(self) -> DbBackendAbc:
+ """Return backend instance."""
+ return self.__dbbackend
# See the License for the specific language governing permissions and
# limitations under the License.
-"""The module provides synchronous shareddatalayer interface."""
+#
+# This source code is part of the near-RT RIC (RAN Intelligent Controller)
+# platform project (RICP).
+#
+
+
+"""The module provides synchronous Shared Data Layer (SDL) interface."""
from typing import (Dict, Set, List, Union)
from abc import ABC, abstractmethod
-
+from ricsdl.exceptions import (
+ RejectedByBackend
+)
__all__ = [
'SyncStorageAbc',
class SyncLockAbc(ABC):
"""
- An abstract synchronous lock class providing a shared, distributed locking mechanism, which can
- be utilized by clients to be able to operate with a shared resource in a mutually exclusive way.
+ An abstract synchronous Shared Data Layer (SDL) lock class providing a shared, distributed
+ locking mechanism, which can be utilized by clients to be able to operate with a shared
+ resource in a mutually exclusive way.
A lock instance is created per namespace and it is identified by its `name` within a
namespace.
Args:
ns (str): Namespace under which this lock is targeted.
- name (str): Lock name, identifies the lock key in shared data layer storage.
+ name (str): Lock name, identifies the lock key in SDL storage.
expiration (int, float): Lock expiration time after which the lock is removed if it hasn't
been released earlier by a 'release' method.
self._expiration = expiration
def __enter__(self, *args, **kwargs):
- self.acquire(*args, **kwargs)
- return self
+ if self.acquire(*args, **kwargs):
+ return self
+ raise RejectedByBackend("Unable to acquire lock within the time specified")
def __exit__(self, exception_type, exception_value, traceback):
self.release()
A lock can be used as a mutual exclusion locking entry for a shared resources.
All the exceptions except SdlTypeError are derived from SdlException base class. Client
- can catch only that exception if separate handling for different shareddatalayer error
- situations is not needed. Exception SdlTypeError is derived from build-in TypeError and it
- indicates misuse of the SDL API.
+ can catch only that exception if separate handling for different SDL error situations is
+ not needed. Exception SdlTypeError is derived from build-in TypeError and it indicates
+ misuse of the SDL API.
Args:
retry_interval (int, float): Lock acquiring retry interval in seconds. Supports both
Raises:
SdlTypeError: If function's argument is of an inappropriate type.
- NotConnected: If shareddatalayer is not connected to the backend data storage.
+ NotConnected: If SDL is not connected to the backend data storage.
RejectedByBackend: If backend data storage rejects the request.
BackendError: If the backend data storage fails to process the request.
"""
pass
-
def release(self) -> None:
"""
Release a lock atomically.
Release the already acquired lock.
Exceptions thrown are all derived from SdlException base class. Client can catch only that
- exception if separate handling for different shareddatalayer error situations is not
- needed.
+ exception if separate handling for different SDL error situations is not needed.
Args:
None
None
Raises:
- NotConnected: If shareddatalayer is not connected to the backend data storage.
+ NotConnected: If SDL is not connected to the backend data storage.
RejectedByBackend: If backend data storage rejects the request.
BackendError: If the backend data storage fails to process the request.
"""
pass
-
def refresh(self) -> None:
"""
Refresh the remaining validity time of the existing lock back to an initial value.
Exceptions thrown are all derived from SdlException base class. Client can catch only that
- exception if separate handling for different shareddatalayer error situations is not
- needed.
+ exception if separate handling for different SDL error situations is not needed.
Args:
None
None
Raises:
- NotConnected: If shareddatalayer is not connected to the backend data storage.
+ NotConnected: If SDL is not connected to the backend data storage.
RejectedByBackend: If backend data storage rejects the request.
BackendError: If the backend data storage fails to process the request.
"""
pass
-
def get_validity_time(self) -> Union[int, float]:
"""
Get atomically the remaining validity time of the lock in seconds.
Return atomically time in seconds until the lock expires.
Exceptions thrown are all derived from SdlException base class. Client can catch only that
- exception if separate handling for different shareddatalayer error situations is not
- needed.
+ exception if separate handling for different SDL error situations is not needed.
Args:
None
(int, float): Validity time of the lock in seconds.
Raises:
- NotConnected: If shareddatalayer is not connected to the backend data storage.
+ NotConnected: If SDL is not connected to the backend data storage.
RejectedByBackend: If backend data storage rejects the request.
BackendError: If the backend data storage fails to process the request.
"""
class SyncStorageAbc(ABC):
"""
- An abstract class providing synchronous access to shared data layer storage.
+ An abstract class providing synchronous access to Shared Data Layer (SDL) storage.
- This class provides synchronous access to all the namespaces in shared data layer storage.
+ This class provides synchronous access to all the namespaces in SDL storage.
Data can be written, read and removed based on keys known to clients. Keys are unique within
a namespace, namespace identifier is passed as a parameter to all the operations.
A concrete implementation subclass 'SyncStorage' derives from this abstract class.
"""
+ @abstractmethod
+ def close(self):
+ """
+ Close the connection to SDL storage.
+
+ Args:
+ None
+
+ Returns:
+ None
+
+ Raises:
+ NotConnected: If SDL is not connected to the backend data storage.
+ RejectedByBackend: If backend data storage rejects the request.
+ BackendError: If the backend data storage fails to process the request.
+ """
+ pass
+
@abstractmethod
def set(self, ns: str, data_map: Dict[str, bytes]) -> None:
"""
- Write data to shared data layer storage.
+ Write data to SDL storage.
Writing is done atomically, i.e. either all succeeds, or all fails.
All the exceptions except SdlTypeError are derived from SdlException base class. Client
- can catch only that exception if separate handling for different shareddatalayer error
- situations is not needed. Exception SdlTypeError is derived from build-in TypeError and it
- indicates misuse of the SDL API.
+ can catch only that exception if separate handling for different SDL error situations is
+ not needed. Exception SdlTypeError is derived from build-in TypeError and it indicates
+ misuse of the SDL API.
Args:
ns (str): Namespace under which this operation is targeted.
Raises:
SdlTypeError: If function's argument is of an inappropriate type.
- NotConnected: If shareddatalayer is not connected to the backend data storage.
+ NotConnected: If SDL is not connected to the backend data storage.
RejectedByBackend: If backend data storage rejects the request.
BackendError: If the backend data storage fails to process the request.
"""
pass
-
@abstractmethod
def set_if(self, ns: str, key: str, old_data: bytes, new_data: bytes) -> bool:
"""
user's last known value.
All the exceptions except SdlTypeError are derived from SdlException base class. Client
- can catch only that exception if separate handling for different shareddatalayer error
- situations is not needed. Exception SdlTypeError is derived from build-in TypeError and it
- indicates misuse of the SDL API.
+ can catch only that exception if separate handling for different SDL error situations is
+ not needed. Exception SdlTypeError is derived from build-in TypeError and it indicates
+ misuse of the SDL API.
Args:
ns (str): Namespace under which this operation is targeted.
Raises:
SdlTypeError: If function's argument is of an inappropriate type.
- NotConnected: If shareddatalayer is not connected to the backend data storage.
+ NotConnected: If SDL is not connected to the backend data storage.
RejectedByBackend: If backend data storage rejects the request.
BackendError: If the backend data storage fails to process the request.
"""
pass
-
@abstractmethod
def set_if_not_exists(self, ns: str, key: str, data: bytes) -> bool:
"""
- Write data to shared data layer storage if key does not exist.
+ Write data to SDL storage if key does not exist.
Conditionally set the value of a key. If key already exists, then its value is not
modified. Checking the key existence and potential set operation is done as a one atomic
operation.
All the exceptions except SdlTypeError are derived from SdlException base class. Client
- can catch only that exception if separate handling for different shareddatalayer error
- situations is not needed. Exception SdlTypeError is derived from build-in TypeError and it
- indicates misuse of the SDL API.
+ can catch only that exception if separate handling for different SDL error situations is
+ not needed. Exception SdlTypeError is derived from build-in TypeError and it indicates
+ misuse of the SDL API.
Args:
ns (str): Namespace under which this operation is targeted.
Raises:
SdlTypeError: If function's argument is of an inappropriate type.
- NotConnected: If shareddatalayer is not connected to the backend data storage.
+ NotConnected: If SDL is not connected to the backend data storage.
RejectedByBackend: If backend data storage rejects the request.
BackendError: If the backend data storage fails to process the request.
"""
pass
-
@abstractmethod
def get(self, ns: str, keys: Union[str, Set[str]]) -> Dict[str, bytes]:
"""
- Read data from shared data layer storage.
+ Read data from SDL storage.
Only those entries that are found will be returned.
All the exceptions except SdlTypeError are derived from SdlException base class. Client
- can catch only that exception if separate handling for different shareddatalayer error
- situations is not needed. Exception SdlTypeError is derived from build-in TypeError and it
- indicates misuse of the SDL API.
+ can catch only that exception if separate handling for different SDL error situations is
+ not needed. Exception SdlTypeError is derived from build-in TypeError and it indicates
+ misuse of the SDL API.
Args:
ns (str): Namespace under which this operation is targeted.
Raises:
SdlTypeError: If function's argument is of an inappropriate type.
- NotConnected: If shareddatalayer is not connected to the backend data storage.
+ NotConnected: If SDL is not connected to the backend data storage.
RejectedByBackend: If backend data storage rejects the request.
BackendError: If the backend data storage fails to process the request.
"""
pass
-
@abstractmethod
def find_keys(self, ns: str, key_prefix: str) -> List[str]:
"""
No prior knowledge about the keys in the given namespace exists, thus operation is not
guaranteed to be atomic or isolated.
All the exceptions except SdlTypeError are derived from SdlException base class. Client
- can catch only that exception if separate handling for different shareddatalayer error
- situations is not needed. Exception SdlTypeError is derived from build-in TypeError and it
- indicates misuse of the SDL API.
+ can catch only that exception if separate handling for different SDL error situations is
+ not needed. Exception SdlTypeError is derived from build-in TypeError and it indicates
+ misuse of the SDL API.
Args:
ns (str): Namespace under which this operation is targeted.
Raises:
SdlTypeError: If function's argument is of an inappropriate type.
- NotConnected: If shareddatalayer is not connected to the backend data storage.
+ NotConnected: If SDL is not connected to the backend data storage.
RejectedByBackend: If backend data storage rejects the request.
BackendError: If the backend data storage fails to process the request.
"""
pass
-
@abstractmethod
def find_and_get(self, ns: str, key_prefix: str, atomic: bool) -> Dict[str, bytes]:
"""
- Find keys and get their respective data from shared data layer storage.
+ Find keys and get their respective data from SDL storage.
Only those entries that are matching prefix will be returned.
NOTE: In atomic action, if the prefix produces huge number of matches, that can have
a severe impact on system performance, due to DB is blocked for long time.
All the exceptions except SdlTypeError are derived from SdlException base class. Client
- can catch only that exception if separate handling for different shareddatalayer error
- situations is not needed. Exception SdlTypeError is derived from build-in TypeError and it
- indicates misuse of the SDL API.
+ can catch only that exception if separate handling for different SDL error situations is
+ not needed. Exception SdlTypeError is derived from build-in TypeError and it indicates
+ misuse of the SDL API.
Args:
ns (str): Namespace under which this operation is targeted.
Raises:
SdlTypeError: If function's argument is of an inappropriate type.
- NotConnected: If shareddatalayer is not connected to the backend data storage.
+ NotConnected: If SDL is not connected to the backend data storage.
RejectedByBackend: If backend data storage rejects the request.
BackendError: If the backend data storage fails to process the request.
"""
pass
-
@abstractmethod
def remove(self, ns: str, keys: Union[str, Set[str]]) -> None:
"""
- Remove data from shared data layer storage. Existing keys are removed.
+ Remove data from SDL storage. Existing keys are removed.
Removing is done atomically, i.e. either all succeeds, or all fails.
All the exceptions except SdlTypeError are derived from SdlException base class. Client
- can catch only that exception if separate handling for different shareddatalayer error
- situations is not needed. Exception SdlTypeError is derived from build-in TypeError and it
- indicates misuse of the SDL API.
+ can catch only that exception if separate handling for different SDL error situations is
+ not needed. Exception SdlTypeError is derived from build-in TypeError and it indicates
+ misuse of the SDL API.
Args:
ns (str): Namespace under which this operation is targeted.
Raises:
SdlTypeError: If function's argument is of an inappropriate type.
- NotConnected: If shareddatalayer is not connected to the backend data storage.
+ NotConnected: If SDL is not connected to the backend data storage.
RejectedByBackend: If backend data storage rejects the request.
BackendError: If the backend data storage fails to process the request.
"""
pass
-
@abstractmethod
def remove_if(self, ns: str, key: str, data: bytes) -> bool:
"""
- Conditionally remove data from shared data layer storage if the current data value matches
- the user's last known value.
+ Conditionally remove data from SDL storage if the current data value matches the user's
+ last known value.
All the exceptions except SdlTypeError are derived from SdlException base class. Client
- can catch only that exception if separate handling for different shareddatalayer error
- situations is not needed. Exception SdlTypeError is derived from build-in TypeError and it
- indicates misuse of the SDL API.
+ can catch only that exception if separate handling for different SDL error situations is
+ not needed. Exception SdlTypeError is derived from build-in TypeError and it indicates
+ misuse of the SDL API.
Args:
ns (str): Namespace under which this operation is targeted.
Raises:
SdlTypeError: If function's argument is of an inappropriate type.
- NotConnected: If shareddatalayer is not connected to the backend data storage.
+ NotConnected: If SDL is not connected to the backend data storage.
RejectedByBackend: If backend data storage rejects the request.
BackendError: If the backend data storage fails to process the request.
"""
pass
-
@abstractmethod
def remove_all(self, ns: str) -> None:
"""
No prior knowledge about the keys in the given namespace exists, thus operation is not
guaranteed to be atomic or isolated.
All the exceptions except SdlTypeError are derived from SdlException base class. Client
- can catch only that exception if separate handling for different shareddatalayer error
- situations is not needed. Exception SdlTypeError is derived from build-in TypeError and it
- indicates misuse of the SDL API.
+ can catch only that exception if separate handling for different SDL error situations is
+ not needed. Exception SdlTypeError is derived from build-in TypeError and it indicates
+ misuse of the SDL API.
Args:
ns (str): Namespace under which this operation is targeted.
Raises:
SdlTypeError: If function's argument is of an inappropriate type.
- NotConnected: If shareddatalayer is not connected to the backend data storage.
+ NotConnected: If SDL is not connected to the backend data storage.
RejectedByBackend: If backend data storage rejects the request.
BackendError: If the backend data storage fails to process the request.
"""
pass
-
@abstractmethod
def add_member(self, ns: str, group: str, members: Union[bytes, Set[bytes]]) -> None:
"""
- Add new members to a shared data layer group under the namespace.
+ Add new members to a SDL group under the namespace.
- Shared data layer groups are identified by their name, which is a key in storage. Shared
- data layer groups are unordered collections of members where each member is unique. If
- a member to be added is already a member of the group, its addition is silently ignored. If
- the group does not exist, it is created, and specified members are added to the group.
+ SDL groups are identified by their name, which is a key in storage. SDL groups are
+ unordered collections of members where each member is unique. If a member to be added is
+ already a member of the group, its addition is silently ignored. If the group does not
+ exist, it is created, and specified members are added to the group.
All the exceptions except SdlTypeError are derived from SdlException base class. Client
- can catch only that exception if separate handling for different shareddatalayer error
- situations is not needed. Exception SdlTypeError is derived from build-in TypeError and it
- indicates misuse of the SDL API.
+ can catch only that exception if separate handling for different SDL error situations is
+ not needed. Exception SdlTypeError is derived from build-in TypeError and it indicates
+ misuse of the SDL API.
Args:
ns (str): Namespace under which this operation is targeted.
Raises:
SdlTypeError: If function's argument is of an inappropriate type.
- NotConnected: If shareddatalayer is not connected to the backend data storage.
+ NotConnected: If SDL is not connected to the backend data storage.
RejectedByBackend: If backend data storage rejects the request.
BackendError: If the backend data storage fails to process the request.
"""
pass
-
@abstractmethod
def remove_member(self, ns: str, group: str, members: Union[bytes, Set[bytes]]) -> None:
"""
- Remove members from a shared data layer group.
+ Remove members from a SDL group.
- Shared data layer groups are unordered collections of members where each member is unique.
- If a member to be removed does not exist in the group, its removal is silently ignored. If
- a group does not exist, it is treated as an empty group and hence members removal is
- silently ignored.
+ SDL groups are unordered collections of members where each member is unique. If a member to
+ be removed does not exist in the group, its removal is silently ignored. If a group does
+ not exist, it is treated as an empty group and hence members removal is silently ignored.
All the exceptions except SdlTypeError are derived from SdlException base class. Client
- can catch only that exception if separate handling for different shareddatalayer error
- situations is not needed. Exception SdlTypeError is derived from build-in TypeError and it
- indicates misuse of the SDL API.
+ can catch only that exception if separate handling for different SDL error situations is
+ not needed. Exception SdlTypeError is derived from build-in TypeError and it indicates
+ misuse of the SDL API.
Args:
ns (str): Namespace under which this operation is targeted.
Raises:
SdlTypeError: If function's argument is of an inappropriate type.
- NotConnected: If shareddatalayer is not connected to the backend data storage.
+ NotConnected: If SDL is not connected to the backend data storage.
RejectedByBackend: If backend data storage rejects the request.
BackendError: If the backend data storage fails to process the request.
"""
pass
-
@abstractmethod
def remove_group(self, ns: str, group: str) -> None:
"""
- Remove a shared data layer group along with its members.
+ Remove a SDL group along with its members.
- Shared data layer groups are unordered collections of members where each member is unique.
- If a group to be removed does not exist, its removal is silently ignored.
+ SDL groups are unordered collections of members where each member is unique. If a group to
+ be removed does not exist, its removal is silently ignored.
All the exceptions except SdlTypeError are derived from SdlException base class. Client
- can catch only that exception if separate handling for different shareddatalayer error
- situations is not needed. Exception SdlTypeError is derived from build-in TypeError and it
- indicates misuse of the SDL API.
+ can catch only that exception if separate handling for different SDL error situations is
+ not needed. Exception SdlTypeError is derived from build-in TypeError and it indicates
+ misuse of the SDL API.
Args:
ns (str): Namespace under which this operation is targeted.
Raises:
SdlTypeError: If function's argument is of an inappropriate type.
- NotConnected: If shareddatalayer is not connected to the backend data storage.
+ NotConnected: If SDL is not connected to the backend data storage.
RejectedByBackend: If backend data storage rejects the request.
BackendError: If the backend data storage fails to process the request.
"""
pass
-
@abstractmethod
def get_members(self, ns: str, group: str) -> Set[bytes]:
"""
- Get all the members of a shared data layer group.
+ Get all the members of a SDL group.
- Shared data layer groups are unordered collections of members where each member is unique.
- If the group does not exist, empty set is returned.
+ SDL groups are unordered collections of members where each member is unique. If the group
+ does not exist, empty set is returned.
All the exceptions except SdlTypeError are derived from SdlException base class. Client
- can catch only that exception if separate handling for different shareddatalayer error
- situations is not needed. Exception SdlTypeError is derived from build-in TypeError and it
- indicates misuse of the SDL API.
+ can catch only that exception if separate handling for different SDL error situations is
+ not needed. Exception SdlTypeError is derived from build-in TypeError and it indicates
+ misuse of the SDL API.
Args:
ns (str): Namespace under which this operation is targeted.
Raises:
SdlTypeError: If function's argument is of an inappropriate type.
- NotConnected: If shareddatalayer is not connected to the backend data storage.
+ NotConnected: If SDL is not connected to the backend data storage.
RejectedByBackend: If backend data storage rejects the request.
BackendError: If the backend data storage fails to process the request.
"""
pass
-
@abstractmethod
def is_member(self, ns: str, group: str, member: bytes) -> bool:
"""
- Validate if a given member is in the shared data layer group.
+ Validate if a given member is in the SDL group.
- Shared data layer groups are unordered collections of members where each member is unique.
- If the group does not exist, false is returned.
+ SDL groups are unordered collections of members where each member is unique. If the group
+ does not exist, false is returned.
All the exceptions except SdlTypeError are derived from SdlException base class. Client
- can catch only that exception if separate handling for different shareddatalayer error
- situations is not needed. Exception SdlTypeError is derived from build-in TypeError and it
- indicates misuse of the SDL API.
+ can catch only that exception if separate handling for different SDL error situations is
+ not needed. Exception SdlTypeError is derived from build-in TypeError and it indicates
+ misuse of the SDL API.
Args:
ns (str): Namespace under which this operation is targeted.
Raises:
SdlTypeError: If function's argument is of an inappropriate type.
- NotConnected: If shareddatalayer is not connected to the backend data storage.
+ NotConnected: If SDL is not connected to the backend data storage.
RejectedByBackend: If backend data storage rejects the request.
BackendError: If the backend data storage fails to process the request.
"""
pass
-
@abstractmethod
def group_size(self, ns: str, group: str) -> int:
"""
Return the number of members in a group.
- Shared data layer groups are unordered collections of members where each member is unique.
- If the group does not exist, value 0 is returned.
+ SDL groups are unordered collections of members where each member is unique. If the group
+ does not exist, value 0 is returned.
All the exceptions except SdlTypeError are derived from SdlException base class. Client
- can catch only that exception if separate handling for different shareddatalayer error
- situations is not needed. Exception SdlTypeError is derived from build-in TypeError and it
- indicates misuse of the SDL API.
+ can catch only that exception if separate handling for different SDL error situations is
+ not needed. Exception SdlTypeError is derived from build-in TypeError and it indicates
+ misuse of the SDL API.
Args:
ns (str): Namespace under which this operation is targeted.
Raises:
SdlTypeError: If function's argument is of an inappropriate type.
- NotConnected: If shareddatalayer is not connected to the backend data storage.
+ NotConnected: If SDL is not connected to the backend data storage.
RejectedByBackend: If backend data storage rejects the request.
BackendError: If the backend data storage fails to process the request.
"""
pass
-
@abstractmethod
def get_lock_resource(self, ns: str, resource: str,
expiration: Union[int, float]) -> SyncLockAbc:
"""
- Return a lock resource for shared data layer.
+ Return a lock resource for SDL.
A lock resource instance is created per namespace and it is identified by its `name` within
a namespace. A `get_lock_resource` returns a lock resource instance, it does not acquire
a lock. Lock resource provides lock handling methods such as acquiring a lock, extend
expiration time and releasing a lock.
All the exceptions except SdlTypeError are derived from SdlException base class. Client
- can catch only that exception if separate handling for different shareddatalayer error
- situations is not needed. Exception SdlTypeError is derived from build-in TypeError and it
- indicates misuse of the SDL API.
+ can catch only that exception if separate handling for different SDL error situations is
+ not needed. Exception SdlTypeError is derived from build-in TypeError and it indicates
+ misuse of the SDL API.
Args:
ns (str): Namespace under which this operation is targeted.
- resource (str): Resource is used within namespace as a key for a lock entry in
- shareddatalayer.
+ resource (str): Resource is used within namespace as a key for a lock entry in SDL.
expiration (int, float): Expiration time of a lock
Returns:
Raises:
SdlTypeError: If function's argument is of an inappropriate type.
- NotConnected: If shareddatalayer is not connected to the backend data storage.
+ NotConnected: If SDL is not connected to the backend data storage.
RejectedByBackend: If backend data storage rejects the request.
BackendError: If the backend data storage fails to process the request.
"""
--- /dev/null
+# Copyright (c) 2019 AT&T Intellectual Property.
+# Copyright (c) 2018-2019 Nokia.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+#
+# This source code is part of the near-RT RIC (RAN Intelligent Controller)
+# platform project (RICP).
+#
+
+"""Setup script of Shared Data Layer (SDL) package."""
+
+from os.path import dirname, abspath, join as path_join
+from setuptools import setup, find_packages
+from ricsdl import __version__
+
+SETUP_DIR = abspath(dirname(__file__))
+
+
+def _long_descr():
+ """Yields the content of documentation files for the long description"""
+ try:
+ doc_path = path_join(SETUP_DIR, "README.md")
+ with open(doc_path) as file:
+ return file.read()
+ except FileNotFoundError: # this happens during unit testing, we don't need it
+ return ""
+
+
+setup(
+ name="ricsdl",
+ version=__version__,
+ packages=find_packages(exclude=["tests.*", "tests"]),
+ author="Timo Tietavainen",
+ author_email='timo.tietavainen@nokia.com',
+ license='Apache 2.0',
+ description="Shared Data Layer (SDL) provides a high-speed interface to access shared storage",
+ url="https://gerrit.o-ran-sc.org/r/admin/repos/ric-plt/sdlpy",
+ classifiers=[
+ "Development Status :: 4 - Beta",
+ "Intended Audience :: Telecommunications Industry",
+ "Programming Language :: Python :: 3",
+ "Programming Language :: Python :: 3.7",
+ "License :: OSI Approved :: Apache Software License",
+ "Operating System :: POSIX :: Linux",
+ ],
+ python_requires=">=3.7",
+ keywords="RIC SDL",
+ install_requires=[
+ 'setuptools',
+ 'redis'
+ ],
+ long_description=_long_descr(),
+ long_description_content_type="text/markdown",
+)
--- /dev/null
+# Copyright (c) 2019 AT&T Intellectual Property.
+# Copyright (c) 2018-2019 Nokia.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+#
+# This source code is part of the near-RT RIC (RAN Intelligent Controller)
+# platform project (RICP).
+#
--- /dev/null
+# Copyright (c) 2019 AT&T Intellectual Property.
+# Copyright (c) 2018-2019 Nokia.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+#
+# This source code is part of the near-RT RIC (RAN Intelligent Controller)
+# platform project (RICP).
+#
--- /dev/null
+# Copyright (c) 2019 AT&T Intellectual Property.
+# Copyright (c) 2018-2019 Nokia.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+#
+# This source code is part of the near-RT RIC (RAN Intelligent Controller)
+# platform project (RICP).
+#
+
+
+from unittest.mock import patch, Mock
+import pytest
+from redis import exceptions as redis_exceptions
+import ricsdl.backend
+from ricsdl.backend.redis import (RedisBackendLock, _map_to_sdl_exception)
+from ricsdl.configuration import _Configuration
+import ricsdl.exceptions
+
+
+@pytest.fixture()
+def redis_backend_fixture(request):
+ request.cls.ns = 'some-ns'
+ request.cls.dl_redis = [b'1', b'2']
+ request.cls.dm = {'a': b'1', 'b': b'2'}
+ request.cls.dm_redis = {'{some-ns},a': b'1', '{some-ns},b': b'2'}
+ request.cls.key = 'a'
+ request.cls.key_redis = '{some-ns},a'
+ request.cls.keys = ['a', 'b']
+ request.cls.keys_redis = ['{some-ns},a', '{some-ns},b']
+ request.cls.data = b'123'
+ request.cls.old_data = b'1'
+ request.cls.new_data = b'3'
+ request.cls.keyprefix = 'x?'
+ request.cls.keyprefix_redis = r'{some-ns},x\?*'
+ request.cls.matchedkeys = ['x1', 'x2', 'x3', 'x4', 'x5']
+ request.cls.matchedkeys_redis = [b'{some-ns},x1', b'{some-ns},x2', b'{some-ns},x3',
+ b'{some-ns},x4', b'{some-ns},x5']
+ request.cls.matcheddata_dl_redis = [b'10', b'11', b'12', b'13', b'14']
+ request.cls.matcheddata_dm = {'x1': b'10', 'x2': b'11', 'x3': b'12',
+ 'x4': b'13', 'x5': b'14'}
+ request.cls.group = 'some-group'
+ request.cls.group_redis = '{some-ns},some-group'
+ request.cls.groupmembers = set([b'm1', b'm2'])
+ request.cls.groupmember = b'm1'
+ request.cls.is_atomic = True
+
+ request.cls.configuration = Mock()
+ mock_conf_params = _Configuration.Params(db_host=None,
+ db_port=None,
+ db_sentinel_port=None,
+ db_sentinel_master_name=None)
+ request.cls.configuration.get_params.return_value = mock_conf_params
+ with patch('ricsdl.backend.redis.Redis') as mock_redis:
+ db = ricsdl.backend.get_backend_instance(request.cls.configuration)
+ request.cls.mock_redis = mock_redis.return_value
+ request.cls.db = db
+
+ yield
+
+
+@pytest.mark.usefixtures('redis_backend_fixture')
+class TestRedisBackend:
+ def test_set_function_success(self):
+ self.db.set(self.ns, self.dm)
+ self.mock_redis.mset.assert_called_once_with(self.dm_redis)
+
+ def test_set_function_can_map_redis_exception_to_sdl_exception(self):
+ self.mock_redis.mset.side_effect = redis_exceptions.ResponseError('redis error!')
+ with pytest.raises(ricsdl.exceptions.RejectedByBackend):
+ self.db.set(self.ns, self.dm)
+
+ def test_set_if_function_success(self):
+ self.mock_redis.execute_command.return_value = True
+ ret = self.db.set_if(self.ns, self.key, self.old_data, self.new_data)
+ self.mock_redis.execute_command.assert_called_once_with('SETIE', self.key_redis,
+ self.new_data, self.old_data)
+ assert ret is True
+
+ def test_set_if_function_returns_false_if_same_data_already_exists(self):
+ self.mock_redis.execute_command.return_value = False
+ ret = self.db.set_if(self.ns, self.key, self.old_data, self.new_data)
+ self.mock_redis.execute_command.assert_called_once_with('SETIE', self.key_redis,
+ self.new_data, self.old_data)
+ assert ret is False
+
+ def test_set_if_function_can_map_redis_exception_to_sdl_exception(self):
+ self.mock_redis.execute_command.side_effect = redis_exceptions.ResponseError('redis error!')
+ with pytest.raises(ricsdl.exceptions.RejectedByBackend):
+ self.db.set_if(self.ns, self.key, self.old_data, self.new_data)
+
+ def test_set_if_not_exists_function_success(self):
+ self.mock_redis.setnx.return_value = True
+ ret = self.db.set_if_not_exists(self.ns, self.key, self.new_data)
+ self.mock_redis.setnx.assert_called_once_with(self.key_redis, self.new_data)
+ assert ret is True
+
+ def test_set_if_not_exists_function_returns_false_if_key_already_exists(self):
+ self.mock_redis.setnx.return_value = False
+ ret = self.db.set_if_not_exists(self.ns, self.key, self.new_data)
+ self.mock_redis.setnx.assert_called_once_with(self.key_redis, self.new_data)
+ assert ret is False
+
+ def test_set_if_not_exists_function_can_map_redis_exception_to_sdl_exception(self):
+ self.mock_redis.setnx.side_effect = redis_exceptions.ResponseError('redis error!')
+ with pytest.raises(ricsdl.exceptions.RejectedByBackend):
+ self.db.set_if_not_exists(self.ns, self.key, self.new_data)
+
+ def test_get_function_success(self):
+ self.mock_redis.mget.return_value = self.dl_redis
+ ret = self.db.get(self.ns, self.keys)
+ self.mock_redis.mget.assert_called_once_with(self.keys_redis)
+ assert ret == self.dm
+
+ def test_get_function_returns_empty_dict_when_no_key_values_exist(self):
+ self.mock_redis.mget.return_value = [None, None]
+ ret = self.db.get(self.ns, self.keys)
+ self.mock_redis.mget.assert_called_once_with(self.keys_redis)
+ assert ret == dict()
+
+ def test_get_function_returns_dict_only_with_found_key_values_when_some_keys_exist(self):
+ self.mock_redis.mget.return_value = [self.data, None]
+ ret = self.db.get(self.ns, self.keys)
+ self.mock_redis.mget.assert_called_once_with(self.keys_redis)
+ assert ret == {self.key: self.data}
+
+ def test_get_function_can_map_redis_exception_to_sdl_exception(self):
+ self.mock_redis.mget.side_effect = redis_exceptions.ResponseError('redis error!')
+ with pytest.raises(ricsdl.exceptions.RejectedByBackend):
+ self.db.get(self.ns, self.keys)
+
+ def test_find_keys_function_success(self):
+ self.mock_redis.keys.return_value = self.matchedkeys_redis
+ ret = self.db.find_keys(self.ns, self.keyprefix)
+ self.mock_redis.keys.assert_called_once_with(self.keyprefix_redis)
+ assert ret == self.matchedkeys
+
+ def test_find_keys_function_returns_empty_list_when_no_matching_keys_found(self):
+ self.mock_redis.keys.return_value = []
+ ret = self.db.find_keys(self.ns, self.keyprefix)
+ self.mock_redis.keys.assert_called_once_with(self.keyprefix_redis)
+ assert ret == []
+
+ def test_find_keys_function_can_map_redis_exception_to_sdl_exception(self):
+ self.mock_redis.keys.side_effect = redis_exceptions.ResponseError('redis error!')
+ with pytest.raises(ricsdl.exceptions.RejectedByBackend):
+ self.db.find_keys(self.ns, self.keyprefix)
+
+ def test_find_and_get_function_success(self):
+ self.mock_redis.keys.return_value = self.matchedkeys_redis
+ self.mock_redis.mget.return_value = self.matcheddata_dl_redis
+ ret = self.db.find_and_get(self.ns, self.keyprefix, self.is_atomic)
+ self.mock_redis.keys.assert_called_once_with(self.keyprefix_redis)
+ self.mock_redis.mget.assert_called_once_with([i.decode() for i in self.matchedkeys_redis])
+ assert ret == self.matcheddata_dm
+
+ def test_find_and_get_function_returns_empty_dict_when_no_matching_keys_exist(self):
+ self.mock_redis.keys.return_value = list()
+ ret = self.db.find_and_get(self.ns, self.keyprefix, self.is_atomic)
+ self.mock_redis.keys.assert_called_once_with(self.keyprefix_redis)
+ assert not self.mock_redis.mget.called
+ assert ret == dict()
+
+ def test_remove_function_success(self):
+ self.db.remove(self.ns, self.keys)
+ self.mock_redis.delete.assert_called_once_with(*self.keys_redis)
+
+ def test_remove_function_can_map_redis_exception_to_sdl_exception(self):
+ self.mock_redis.delete.side_effect = redis_exceptions.ResponseError('redis error!')
+ with pytest.raises(ricsdl.exceptions.RejectedByBackend):
+ self.db.remove(self.ns, self.keys)
+
+ def test_remove_if_function_success(self):
+ self.mock_redis.execute_command.return_value = True
+ ret = self.db.remove_if(self.ns, self.key, self.new_data)
+ self.mock_redis.execute_command.assert_called_once_with('DELIE', self.key_redis,
+ self.new_data)
+ assert ret is True
+
+ def test_remove_if_function_returns_false_if_data_does_not_match(self):
+ self.mock_redis.execute_command.return_value = False
+ ret = self.db.remove_if(self.ns, self.key, self.new_data)
+ self.mock_redis.execute_command.assert_called_once_with('DELIE', self.key_redis,
+ self.new_data)
+ assert ret is False
+
+ def test_remove_if_function_can_map_redis_exception_to_sdl_exception(self):
+ self.mock_redis.execute_command.side_effect = redis_exceptions.ResponseError('redis error!')
+ with pytest.raises(ricsdl.exceptions.RejectedByBackend):
+ self.db.remove_if(self.ns, self.key, self.new_data)
+
+ def test_add_member_function_success(self):
+ self.db.add_member(self.ns, self.group, self.groupmembers)
+ self.mock_redis.sadd.assert_called_once_with(self.group_redis, *self.groupmembers)
+
+ def test_add_member_function_can_map_redis_exception_to_sdl_exception(self):
+ self.mock_redis.sadd.side_effect = redis_exceptions.ResponseError('redis error!')
+ with pytest.raises(ricsdl.exceptions.RejectedByBackend):
+ self.db.add_member(self.ns, self.group, self.groupmembers)
+
+ def test_remove_member_function_success(self):
+ self.db.remove_member(self.ns, self.group, self.groupmembers)
+ self.mock_redis.srem.assert_called_once_with(self.group_redis, *self.groupmembers)
+
+ def test_remove_member_function_can_map_redis_exception_to_sdl_exception(self):
+ self.mock_redis.srem.side_effect = redis_exceptions.ResponseError('redis error!')
+ with pytest.raises(ricsdl.exceptions.RejectedByBackend):
+ self.db.remove_member(self.ns, self.group, self.groupmembers)
+
+ def test_remove_group_function_success(self):
+ self.db.remove_group(self.ns, self.group)
+ self.mock_redis.delete.assert_called_once_with(self.group_redis)
+
+ def test_remove_group_function_can_map_redis_exception_to_sdl_exception(self):
+ self.mock_redis.delete.side_effect = redis_exceptions.ResponseError('redis error!')
+ with pytest.raises(ricsdl.exceptions.RejectedByBackend):
+ self.db.remove_group(self.ns, self.group)
+
+ def test_get_members_function_success(self):
+ self.mock_redis.smembers.return_value = self.groupmembers
+ ret = self.db.get_members(self.ns, self.group)
+ self.mock_redis.smembers.assert_called_once_with(self.group_redis)
+ assert ret is self.groupmembers
+
+ def test_get_members_function_can_map_redis_exception_to_sdl_exception(self):
+ self.mock_redis.smembers.side_effect = redis_exceptions.ResponseError('redis error!')
+ with pytest.raises(ricsdl.exceptions.RejectedByBackend):
+ self.db.get_members(self.ns, self.group)
+
+ def test_is_member_function_success(self):
+ self.mock_redis.sismember.return_value = True
+ ret = self.db.is_member(self.ns, self.group, self.groupmember)
+ self.mock_redis.sismember.assert_called_once_with(self.group_redis, self.groupmember)
+ assert ret is True
+
+ def test_is_member_function_can_map_redis_exception_to_sdl_exception(self):
+ self.mock_redis.sismember.side_effect = redis_exceptions.ResponseError('redis error!')
+ with pytest.raises(ricsdl.exceptions.RejectedByBackend):
+ self.db.is_member(self.ns, self.group, self.groupmember)
+
+ def test_group_size_function_success(self):
+ self.mock_redis.scard.return_value = 100
+ ret = self.db.group_size(self.ns, self.group)
+ self.mock_redis.scard.assert_called_once_with(self.group_redis)
+ assert ret == 100
+
+ def test_group_size_function_can_map_redis_exception_to_sdl_exception(self):
+ self.mock_redis.scard.side_effect = redis_exceptions.ResponseError('Some redis error!')
+ with pytest.raises(ricsdl.exceptions.RejectedByBackend):
+ self.db.group_size(self.ns, self.group)
+
+ def test_get_redis_connection_function_success(self):
+ ret = self.db.get_redis_connection()
+ assert ret is self.mock_redis
+
+ def test_redis_backend_object_string_representation(self):
+ str_out = str(self.db)
+ assert str_out is not None
+
+
+class MockRedisLock:
+ def __init__(self, redis, name, timeout=None, sleep=0.1,
+ blocking=True, blocking_timeout=None, thread_local=True):
+ self.redis = redis
+ self.name = name
+ self.timeout = timeout
+ self.sleep = sleep
+ self.blocking = blocking
+ self.blocking_timeout = blocking_timeout
+ self.thread_local = bool(thread_local)
+
+
+@pytest.fixture(scope="module")
+def mock_redis_lock():
+ def _mock_redis_lock(name, timeout=None, sleep=0.1,
+ blocking=True, blocking_timeout=None, thread_local=True):
+ return MockRedisLock(name, timeout, sleep, blocking, blocking_timeout, thread_local)
+ return _mock_redis_lock
+
+
+@pytest.fixture()
+def redis_backend_lock_fixture(request, mock_redis_lock):
+ request.cls.ns = 'some-ns'
+ request.cls.lockname = 'some-lock-name'
+ request.cls.lockname_redis = '{some-ns},some-lock-name'
+ request.cls.expiration = 10
+ request.cls.retry_interval = 0.1
+ request.cls.retry_timeout = 1
+
+ request.cls.mock_lua_get_validity_time = Mock()
+ request.cls.mock_lua_get_validity_time.return_value = 2000
+
+ request.cls.mock_redis = Mock()
+ request.cls.mock_redis.register_script = Mock()
+ request.cls.mock_redis.register_script.return_value = request.cls.mock_lua_get_validity_time
+
+ mocked_dbbackend = Mock()
+ mocked_dbbackend.get_redis_connection.return_value = request.cls.mock_redis
+ with patch('ricsdl.backend.redis.Lock') as mock_redis_lock:
+ lock = ricsdl.backend.get_backend_lock_instance(request.cls.ns, request.cls.lockname,
+ request.cls.expiration, mocked_dbbackend)
+ request.cls.mock_redis_lock = mock_redis_lock.return_value
+ request.cls.lock = lock
+ yield
+ RedisBackendLock.lua_get_validity_time = None
+
+
+@pytest.mark.usefixtures('redis_backend_lock_fixture')
+class TestRedisBackendLock:
+ def test_acquire_function_success(self):
+ self.lock.acquire(self.retry_interval, self.retry_timeout)
+ self.mock_redis_lock.acquire.assert_called_once_with(blocking_timeout=self.retry_timeout)
+
+ def test_acquire_function_can_map_redis_exception_to_sdl_exception(self):
+ self.mock_redis_lock.acquire.side_effect = redis_exceptions.LockError('redis lock error!')
+ with pytest.raises(ricsdl.exceptions.BackendError):
+ self.lock.acquire(self.retry_interval, self.retry_timeout)
+
+ def test_release_function_success(self):
+ self.lock.release()
+ self.mock_redis_lock.release.assert_called_once()
+
+ def test_release_function_can_map_redis_exception_to_sdl_exception(self):
+ self.mock_redis_lock.release.side_effect = redis_exceptions.LockError('redis lock error!')
+ with pytest.raises(ricsdl.exceptions.BackendError):
+ self.lock.release()
+
+ def test_refresh_function_success(self):
+ self.lock.refresh()
+ self.mock_redis_lock.reacquire.assert_called_once()
+
+ def test_refresh_function_can_map_redis_exception_to_sdl_exception(self):
+ self.mock_redis_lock.reacquire.side_effect = redis_exceptions.LockError('redis lock error!')
+ with pytest.raises(ricsdl.exceptions.BackendError):
+ self.lock.refresh()
+
+ def test_get_validity_time_function_success(self):
+ self.mock_redis_lock.name = self.lockname_redis
+ self.mock_redis_lock.local.token = 123
+
+ ret = self.lock.get_validity_time()
+ self.mock_lua_get_validity_time.assert_called_once_with(
+ keys=[self.lockname_redis], args=[123], client=self.mock_redis)
+ assert ret == 2
+
+ def test_get_validity_time_function_can_raise_exception_if_lock_is_unlocked(self):
+ self.mock_redis_lock.name = self.lockname_redis
+ self.mock_redis_lock.local.token = None
+
+ with pytest.raises(ricsdl.exceptions.RejectedByBackend) as excinfo:
+ self.lock.get_validity_time()
+ assert f"Cannot get validity time of an unlocked lock {self.lockname}" in str(excinfo.value)
+
+ def test_get_validity_time_function_can_raise_exception_if_lua_script_fails(self):
+ self.mock_redis_lock.name = self.lockname_redis
+ self.mock_redis_lock.local.token = 123
+ self.mock_lua_get_validity_time.return_value = -10
+
+ with pytest.raises(ricsdl.exceptions.RejectedByBackend) as excinfo:
+ self.lock.get_validity_time()
+ assert f"Getting validity time of a lock {self.lockname} failed with error code: -10" in str(excinfo.value)
+
+ def test_redis_backend_lock_object_string_representation(self):
+ str_out = str(self.lock)
+ assert str_out is not None
+
+
+def test_redis_response_error_exception_is_mapped_to_rejected_by_backend_sdl_exception():
+ with pytest.raises(ricsdl.exceptions.RejectedByBackend) as excinfo:
+ with _map_to_sdl_exception():
+ raise redis_exceptions.ResponseError('Some redis error!')
+ assert "SDL backend rejected the request: Some redis error!" in str(excinfo.value)
+
+
+def test_redis_connection_error_exception_is_mapped_to_not_connected_sdl_exception():
+ with pytest.raises(ricsdl.exceptions.NotConnected) as excinfo:
+ with _map_to_sdl_exception():
+ raise redis_exceptions.ConnectionError('Some redis error!')
+ assert "SDL not connected to backend: Some redis error!" in str(excinfo.value)
+
+
+def test_rest_redis_exceptions_are_mapped_to_backend_error_sdl_exception():
+ with pytest.raises(ricsdl.exceptions.BackendError) as excinfo:
+ with _map_to_sdl_exception():
+ raise redis_exceptions.RedisError('Some redis error!')
+ assert "SDL backend failed to process the request: Some redis error!" in str(excinfo.value)
+
+
+def test_system_error_exceptions_are_not_mapped_to_any_sdl_exception():
+ with pytest.raises(SystemExit):
+ with _map_to_sdl_exception():
+ raise SystemExit('Fatal error')
--- /dev/null
+# Copyright (c) 2019 AT&T Intellectual Property.
+# Copyright (c) 2018-2019 Nokia.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+#
+# This source code is part of the near-RT RIC (RAN Intelligent Controller)
+# platform project (RICP).
+#
+
+
+from unittest.mock import patch, Mock
+import pytest
+from ricsdl.syncstorage import SyncStorage
+from ricsdl.syncstorage import SyncLock
+from ricsdl.syncstorage import func_arg_checker
+from ricsdl.exceptions import SdlTypeError
+
+
+@pytest.fixture()
+def sync_storage_fixture(request):
+ request.cls.ns = 'some-ns'
+ request.cls.key = 'a'
+ request.cls.keys = {'a', 'b'}
+ request.cls.dm = {'a': b'1', 'b': b'2'}
+ request.cls.old_data = b'1'
+ request.cls.new_data = b'3'
+ request.cls.keyprefix = 'x'
+ request.cls.matchedkeys = ['x1', 'x2', 'x3', 'x4', 'x5']
+ request.cls.is_atomic = True
+ request.cls.group = 'some-group'
+ request.cls.groupmembers = set([b'm1', b'm2'])
+ request.cls.groupmember = b'm1'
+ request.cls.lock_name = 'some-lock-name'
+ request.cls.lock_int_expiration = 10
+ request.cls.lock_float_expiration = 1.1
+
+ with patch('ricsdl.backend.get_backend_instance') as mock_db_backend:
+ storage = SyncStorage()
+ request.cls.mock_db_backend = mock_db_backend.return_value
+ request.cls.storage = storage
+ yield
+
+
+@pytest.mark.usefixtures('sync_storage_fixture')
+class TestSyncStorage:
+ def test_set_function_success(self):
+ self.storage.set(self.ns, self.dm)
+ self.mock_db_backend.set.assert_called_once_with(self.ns, self.dm)
+
+ def test_set_function_can_raise_exception_for_wrong_argument(self):
+ with pytest.raises(SdlTypeError):
+ self.storage.set(123, {'a': b'v1'})
+ with pytest.raises(SdlTypeError):
+ self.storage.set('ns', [1, 2])
+
+ def test_set_if_function_success(self):
+ self.mock_db_backend.set_if.return_value = True
+ ret = self.storage.set_if(self.ns, self.key, self.old_data, self.new_data)
+ self.mock_db_backend.set_if.assert_called_once_with(self.ns, self.key, self.old_data,
+ self.new_data)
+ assert ret is True
+
+ def test_set_if_function_can_return_false_if_same_data_already_exists(self):
+ self.mock_db_backend.set_if.return_value = False
+ ret = self.storage.set_if(self.ns, self.key, self.old_data, self.new_data)
+ self.mock_db_backend.set_if.assert_called_once_with(self.ns, self.key, self.old_data,
+ self.new_data)
+ assert ret is False
+
+ def test_set_if_function_can_raise_exception_for_wrong_argument(self):
+ with pytest.raises(SdlTypeError):
+ self.storage.set_if(0xbad, 'key', b'v1', b'v2')
+ with pytest.raises(SdlTypeError):
+ self.storage.set_if('ns', 0xbad, b'v1', b'v2')
+ with pytest.raises(SdlTypeError):
+ self.storage.set_if('ns', 'key', 0xbad, b'v2')
+ with pytest.raises(SdlTypeError):
+ self.storage.set_if('ns', 'key', b'v1', 0xbad)
+
+ def test_set_if_not_exists_function_success(self):
+ self.mock_db_backend.set_if_not_exists.return_value = True
+ ret = self.storage.set_if_not_exists(self.ns, self.key, self.new_data)
+ self.mock_db_backend.set_if_not_exists.assert_called_once_with(self.ns, self.key,
+ self.new_data)
+ assert ret is True
+
+ def test_set_if_not_exists_function_can_return_false_if_key_already_exists(self):
+ self.mock_db_backend.set_if_not_exists.return_value = False
+ ret = self.storage.set_if_not_exists(self.ns, self.key, self.new_data)
+ self.mock_db_backend.set_if_not_exists.assert_called_once_with(self.ns, self.key,
+ self.new_data)
+ assert ret is False
+
+ def test_set_if_not_exists_function_can_raise_exception_for_wrong_argument(self):
+ with pytest.raises(SdlTypeError):
+ self.storage.set_if_not_exists(0xbad, 'key', b'v1')
+ with pytest.raises(SdlTypeError):
+ self.storage.set_if_not_exists('ns', 0xbad, b'v1')
+ with pytest.raises(SdlTypeError):
+ self.storage.set_if_not_exists('ns', 'key', 0xbad)
+
+ def test_get_function_success(self):
+ self.mock_db_backend.get.return_value = self.dm
+ ret = self.storage.get(self.ns, self.keys)
+ self.mock_db_backend.get.assert_called_once()
+ call_args, _ = self.mock_db_backend.get.call_args
+ assert call_args[0] == self.ns
+ assert len(call_args[1]) == len(self.keys)
+ assert all(k in call_args[1] for k in self.keys)
+ assert ret == self.dm
+
+ def test_get_function_can_return_empty_dict_when_no_key_values_exist(self):
+ self.mock_db_backend.get.return_value = dict()
+ ret = self.storage.get(self.ns, self.keys)
+ self.mock_db_backend.get.assert_called_once()
+ call_args, _ = self.mock_db_backend.get.call_args
+ assert call_args[0] == self.ns
+ assert len(call_args[1]) == len(self.keys)
+ assert all(k in call_args[1] for k in self.keys)
+ assert ret == dict()
+
+ def test_get_function_can_raise_exception_for_wrong_argument(self):
+ with pytest.raises(SdlTypeError):
+ self.storage.get(0xbad, self.key)
+ with pytest.raises(SdlTypeError):
+ self.storage.get(self.ns, 0xbad)
+
+ def test_find_keys_function_success(self):
+ self.mock_db_backend.find_keys.return_value = self.matchedkeys
+ ret = self.storage.find_keys(self.ns, self.keyprefix)
+ self.mock_db_backend.find_keys.assert_called_once_with(self.ns, self.keyprefix)
+ assert ret == self.matchedkeys
+
+ def test_find_keys_function_can_return_empty_list_when_no_keys_exist(self):
+ self.mock_db_backend.find_keys.return_value = list()
+ ret = self.storage.find_keys(self.ns, self.keyprefix)
+ self.mock_db_backend.find_keys.assert_called_once_with(self.ns, self.keyprefix)
+ assert ret == list()
+
+ def test_find_keys_function_can_raise_exception_for_wrong_argument(self):
+ with pytest.raises(SdlTypeError):
+ self.storage.find_keys(0xbad, self.keyprefix)
+ with pytest.raises(SdlTypeError):
+ self.storage.find_keys(self.ns, 0xbad)
+
+ def test_find_and_get_function_success(self):
+ self.mock_db_backend.find_and_get.return_value = self.dm
+ ret = self.storage.find_and_get(self.ns, self.keyprefix, self.is_atomic)
+ self.mock_db_backend.find_and_get.assert_called_once_with(self.ns, self.keyprefix,
+ self.is_atomic)
+ assert ret == self.dm
+
+ def test_find_and_get_function_can_return_empty_dict_when_no_keys_exist(self):
+ self.mock_db_backend.find_and_get.return_value = dict()
+ ret = self.storage.find_and_get(self.ns, self.keyprefix, self.is_atomic)
+ self.mock_db_backend.find_and_get.assert_called_once_with(self.ns, self.keyprefix,
+ self.is_atomic)
+ assert ret == dict()
+
+ def test_find_and_get_function_can_raise_exception_for_wrong_argument(self):
+ with pytest.raises(SdlTypeError):
+ self.storage.find_and_get(0xbad, self.keyprefix, self.is_atomic)
+ with pytest.raises(SdlTypeError):
+ self.storage.find_and_get(self.ns, 0xbad, self.is_atomic)
+ with pytest.raises(SdlTypeError):
+ self.storage.find_and_get(self.ns, self.keyprefix, 0xbad)
+
+ def test_remove_function_success(self):
+ self.storage.remove(self.ns, self.keys)
+ self.mock_db_backend.remove.assert_called_once()
+ call_args, _ = self.mock_db_backend.remove.call_args
+ assert call_args[0] == self.ns
+ assert isinstance(call_args[1], list)
+ assert len(call_args[1]) == len(self.keys)
+ assert all(k in call_args[1] for k in self.keys)
+
+ def test_remove_function_can_raise_exception_for_wrong_argument(self):
+ with pytest.raises(SdlTypeError):
+ self.storage.remove(0xbad, self.keys)
+ with pytest.raises(SdlTypeError):
+ self.storage.remove(self.ns, 0xbad)
+
+ def test_remove_if_function_success(self):
+ self.mock_db_backend.remove_if.return_value = True
+ ret = self.storage.remove_if(self.ns, self.key, self.new_data)
+ self.mock_db_backend.remove_if.assert_called_once_with(self.ns, self.key, self.new_data)
+ assert ret is True
+
+ def test_remove_if_function_can_return_false_if_data_does_not_match(self):
+ self.mock_db_backend.remove_if.return_value = False
+ ret = self.storage.remove_if(self.ns, self.key, self.old_data)
+ self.mock_db_backend.remove_if.assert_called_once_with(self.ns, self.key, self.old_data)
+ assert ret is False
+
+ def test_remove_if_function_can_raise_exception_for_wrong_argument(self):
+ with pytest.raises(SdlTypeError):
+ self.storage.remove_if(0xbad, self.keys, self.old_data)
+ with pytest.raises(SdlTypeError):
+ self.storage.remove_if(self.ns, 0xbad, self.old_data)
+ with pytest.raises(SdlTypeError):
+ self.storage.remove_if(self.ns, self.keys, 0xbad)
+
+ def test_remove_all_function_success(self):
+ self.mock_db_backend.find_keys.return_value = ['a1']
+ self.storage.remove_all(self.ns)
+ self.mock_db_backend.find_keys.assert_called_once_with(self.ns, '')
+ self.mock_db_backend.remove.assert_called_once_with(self.ns,
+ self.mock_db_backend.find_keys.return_value)
+
+ def test_remove_all_function_can_raise_exception_for_wrong_argument(self):
+ with pytest.raises(SdlTypeError):
+ self.storage.remove_all(0xbad)
+
+ def test_add_member_function_success(self):
+ self.storage.add_member(self.ns, self.group, self.groupmembers)
+ self.mock_db_backend.add_member.assert_called_once_with(self.ns,
+ self.group, self.groupmembers)
+
+ def test_add_member_function_can_raise_exception_for_wrong_argument(self):
+ with pytest.raises(SdlTypeError):
+ self.storage.add_member(0xbad, self.group, self.groupmembers)
+ with pytest.raises(SdlTypeError):
+ self.storage.add_member(self.ns, 0xbad, self.groupmembers)
+ with pytest.raises(SdlTypeError):
+ self.storage.add_member(self.ns, self.group, 0xbad)
+
+ def test_remove_member_function_success(self):
+ self.storage.remove_member(self.ns, self.group, self.groupmembers)
+ self.mock_db_backend.remove_member.assert_called_once_with(self.ns, self.group,
+ self.groupmembers)
+
+ def test_remove_member_function_can_raise_exception_for_wrong_argument(self):
+ with pytest.raises(SdlTypeError):
+ self.storage.remove_member(0xbad, self.group, self.groupmembers)
+ with pytest.raises(SdlTypeError):
+ self.storage.remove_member(self.ns, 0xbad, self.groupmembers)
+ with pytest.raises(SdlTypeError):
+ self.storage.remove_member(self.ns, self.group, 0xbad)
+
+ def test_remove_group_function_success(self):
+ self.storage.remove_group(self.ns, self.group)
+ self.mock_db_backend.remove_group.assert_called_once_with(self.ns, self.group)
+
+ def test_remove_group_function_can_raise_exception_for_wrong_argument(self):
+ with pytest.raises(SdlTypeError):
+ self.storage.remove_group(0xbad, self.group)
+ with pytest.raises(SdlTypeError):
+ self.storage.remove_group(self.ns, 0xbad)
+
+ def test_get_members_function_success(self):
+ self.mock_db_backend.get_members.return_value = self.groupmembers
+ ret = self.storage.get_members(self.ns, self.group)
+ self.mock_db_backend.get_members.assert_called_once_with(self.ns, self.group)
+ assert ret == self.groupmembers
+
+ def test_get_members_function_can_raise_exception_for_wrong_argument(self):
+ with pytest.raises(SdlTypeError):
+ self.storage.get_members(0xbad, self.group)
+ with pytest.raises(SdlTypeError):
+ self.storage.get_members(self.ns, 0xbad)
+
+ def test_is_member_function_success(self):
+ self.mock_db_backend.is_member.return_value = True
+ ret = self.storage.is_member(self.ns, self.group, self.groupmember)
+ self.mock_db_backend.is_member.assert_called_once_with(self.ns, self.group,
+ self.groupmember)
+ assert ret is True
+
+ def test_is_member_function_can_raise_exception_for_wrong_argument(self):
+ with pytest.raises(SdlTypeError):
+ self.storage.is_member(0xbad, self.group, self.groupmember)
+ with pytest.raises(SdlTypeError):
+ self.storage.is_member(self.ns, 0xbad, self.groupmember)
+ with pytest.raises(SdlTypeError):
+ self.storage.is_member(self.ns, self.group, 0xbad)
+
+ def test_group_size_function_success(self):
+ self.mock_db_backend.group_size.return_value = 100
+ ret = self.storage.group_size(self.ns, self.group)
+ self.mock_db_backend.group_size.assert_called_once_with(self.ns, self.group)
+ assert ret == 100
+
+ def test_group_size_function_can_raise_exception_for_wrong_argument(self):
+ with pytest.raises(SdlTypeError):
+ self.storage.group_size(0xbad, self.group)
+ with pytest.raises(SdlTypeError):
+ self.storage.group_size(self.ns, 0xbad)
+
+ @patch('ricsdl.syncstorage.SyncLock')
+ def test_get_lock_resource_function_success_when_expiration_time_is_integer(self, mock_db_lock):
+ ret = self.storage.get_lock_resource(self.ns, self.lock_name, self.lock_int_expiration)
+ mock_db_lock.assert_called_once_with(self.ns, self.lock_name, self.lock_int_expiration,
+ self.storage)
+ assert ret == mock_db_lock.return_value
+
+ @patch('ricsdl.syncstorage.SyncLock')
+ def test_get_lock_resource_function_success_when_expiration_time_is_float_number(self,
+ mock_db_lock):
+ ret = self.storage.get_lock_resource(self.ns, self.lock_name, self.lock_float_expiration)
+ mock_db_lock.assert_called_once_with(self.ns, self.lock_name, self.lock_float_expiration,
+ self.storage)
+ assert ret == mock_db_lock.return_value
+
+ def test_get_lock_resource_function_can_raise_exception_for_wrong_argument(self):
+ with pytest.raises(SdlTypeError):
+ self.storage.get_lock_resource(0xbad, self.lock_name, self.lock_int_expiration)
+ with pytest.raises(SdlTypeError):
+ self.storage.get_lock_resource(self.ns, 0xbad, self.lock_int_expiration)
+ with pytest.raises(SdlTypeError):
+ self.storage.get_lock_resource(self.ns, self.lock_name, 'bad')
+
+ def test_get_backend_function_success(self):
+ ret = self.storage.get_backend()
+ assert ret == self.mock_db_backend
+
+ def test_storage_object_string_representation(self):
+ str_out = str(self.storage)
+ assert str_out is not None
+
+
+@pytest.fixture()
+def lock_fixture(request):
+ request.cls.ns = 'some-ns'
+ request.cls.lockname = 'some-lock-name'
+ request.cls.expiration = 10
+ request.cls.retry_interval = 0.1
+ request.cls.retry_timeout = 1
+
+ with patch('ricsdl.backend.get_backend_lock_instance') as mock_db_backend_lock:
+ lock = SyncLock('test-ns', 'test-lock-name', request.cls.expiration, Mock())
+ request.cls.mock_db_backend_lock = mock_db_backend_lock.return_value
+ request.cls.lock = lock
+ yield
+
+
+@pytest.mark.usefixtures('lock_fixture')
+class TestSyncLock:
+ def test_acquire_function_success_when_timeout_and_interval_are_integers(self):
+ self.lock.acquire(self.retry_interval, self.retry_timeout)
+ self.mock_db_backend_lock.acquire.assert_called_once_with(self.retry_interval,
+ self.retry_timeout)
+
+ def test_acquire_function_success_when_timeout_and_interval_are_float_numbers(self):
+ self.lock.acquire(float(self.retry_interval), float(self.retry_timeout))
+ self.mock_db_backend_lock.acquire.assert_called_once_with(float(self.retry_interval),
+ float(self.retry_timeout))
+
+ def test_acquire_function_can_raise_exception_for_wrong_argument(self):
+ with pytest.raises(SdlTypeError):
+ self.lock.acquire('bad', self.retry_timeout)
+ with pytest.raises(SdlTypeError):
+ self.lock.acquire(self.retry_interval, 'bad')
+
+ def test_release_function_success(self):
+ self.lock.release()
+ self.mock_db_backend_lock.release.assert_called_once()
+
+ def test_refresh_function_success(self):
+ self.lock.refresh()
+ self.mock_db_backend_lock.refresh.assert_called_once()
+
+ def test_get_validity_time_function_success(self):
+ self.mock_db_backend_lock.get_validity_time.return_value = self.expiration
+ ret = self.lock.get_validity_time()
+ self.mock_db_backend_lock.get_validity_time.assert_called_once()
+ assert ret == self.expiration
+
+ def test_get_validity_time_function_success_when_returned_time_is_float(self):
+ self.mock_db_backend_lock.get_validity_time.return_value = float(self.expiration)
+ ret = self.lock.get_validity_time()
+ self.mock_db_backend_lock.get_validity_time.assert_called_once()
+ assert ret == float(self.expiration)
+
+ def test_lock_object_string_representation(self):
+ str_out = str(self.lock)
+ assert str_out is not None
+
+
+def test_function_arg_validator():
+ @func_arg_checker(SdlTypeError, 0, a=str, b=(int, float), c=set, d=(dict, type(None)))
+ def _my_func(a='abc', b=1, c={'x', 'y'}, d={'x': b'1'}):
+ pass
+ with pytest.raises(SdlTypeError, match=r"Wrong argument type: 'a'=<class 'NoneType'>. "
+ r"Must be: <class 'str'>"):
+ _my_func(None)
+
+ with pytest.raises(SdlTypeError, match=r"Wrong argument type: 'b'=<class 'str'>. "):
+ _my_func('abc', 'wrong type')
+
+ with pytest.raises(SdlTypeError, match=r"Wrong argument type: 'c'=<class 'str'>. "
+ r"Must be: <class 'set'>"):
+ _my_func('abc', 1.0, 'wrong type')
+
+ with pytest.raises(SdlTypeError, match=r"Wrong argument type: 'd'=<class 'str'>. "):
+ _my_func('abc', 1.0, {'x', 'y'}, 'wrong type')
+
+
+def test_function_kwarg_validator():
+ @func_arg_checker(SdlTypeError, 0, a=str, b=(int, float), c=set, d=(dict, type(None)))
+ def _my_func(a='abc', b=1, c={'x', 'y'}, d={'x': b'1'}):
+ pass
+ with pytest.raises(SdlTypeError, match=r"Wrong argument type: 'a'=<class 'NoneType'>. "
+ r"Must be: <class 'str'>"):
+ _my_func(a=None)
+
+ with pytest.raises(SdlTypeError, match=r"Wrong argument type: 'b'=<class 'str'>. "):
+ _my_func(b='wrong type')
+
+ with pytest.raises(SdlTypeError, match=r"Wrong argument type: 'c'=<class 'str'>. "
+ r"Must be: <class 'set'>"):
+ _my_func(c='wrong type')
+
+ with pytest.raises(SdlTypeError, match=r"Wrong argument type: 'd'=<class 'str'>. "):
+ _my_func(d='wrong type')
--- /dev/null
+# Copyright (c) 2019 AT&T Intellectual Property.
+# Copyright (c) 2018-2019 Nokia.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+#
+# This source code is part of the near-RT RIC (RAN Intelligent Controller)
+# platform project (RICP).
+#
+
+
+[tox]
+envlist = py37,flake8
+
+[testenv]
+deps =
+ pytest
+ coverage
+ pytest-cov
+setenv =
+ DBAAS_SERVICE_HOST=localhost
+ DBAAS_SERVICE_PORT=6379
+commands =
+ pytest --junitxml xunit-results.xml --cov ricsdl --cov-report xml --cov-report term-missing --cov-report html --cov-fail-under=70
+ coverage xml -i
+
+[testenv:flake8]
+basepython = python3.7
+skip_install = true
+deps = flake8
+commands = flake8 setup.py ricsdl
+
+[flake8]
+#Do not warn about line lengths more than 79 characters
+ignore = E501
+
+[pytest]
+junit_family=legacy
--- /dev/null
+# documentation only
+[tox]
+minversion = 2.0
+envlist =
+ docs,
+ docs-linkcheck,
+skipsdist = true
+
+[testenv:docs]
+basepython = python3
+deps =
+ sphinx
+ sphinx-rtd-theme
+ sphinxcontrib-httpdomain
+ recommonmark
+ lfdocs-conf
+
+commands =
+ sphinx-build -W -b html -n -d {envtmpdir}/doctrees ./docs/ {toxinidir}/docs/_build/html
+ echo "Generated docs available in {toxinidir}/docs/_build/html"
+whitelist_externals = echo
+
+[testenv:docs-linkcheck]
+basepython = python3
+deps = sphinx
+ sphinx-rtd-theme
+ sphinxcontrib-httpdomain
+ recommonmark
+ lfdocs-conf
+commands = sphinx-build -W -b linkcheck -d {envtmpdir}/doctrees ./docs/ {toxinidir}/docs/_build/linkcheck
Code Review / ric-plt / sdlpy.git / commitdiff
