Patch set 2: Cherry picks added and squashed.
Issue-ID: NONRTRIC-757
Change-Id: I47b9b6c322ebdd8f6f29aa5f92c1b3847976efb9
Signed-off-by: halil.cakal <halil.cakal@est.tech>
.coverage
coverage.xml
htmlcov/
+
+# Python virtual env
+venv/
--- /dev/null
+ Apache License
+ Version 2.0, January 2004
+ http://www.apache.org/licenses/
+
+ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+ 1. Definitions.
+
+ "License" shall mean the terms and conditions for use, reproduction,
+ and distribution as defined by Sections 1 through 9 of this document.
+
+ "Licensor" shall mean the copyright owner or entity authorized by
+ the copyright owner that is granting the License.
+
+ "Legal Entity" shall mean the union of the acting entity and all
+ other entities that control, are controlled by, or are under common
+ control with that entity. For the purposes of this definition,
+ "control" means (i) the power, direct or indirect, to cause the
+ direction or management of such entity, whether by contract or
+ otherwise, or (ii) ownership of fifty percent (50%) or more of the
+ outstanding shares, or (iii) beneficial ownership of such entity.
+
+ "You" (or "Your") shall mean an individual or Legal Entity
+ exercising permissions granted by this License.
+
+ "Source" form shall mean the preferred form for making modifications,
+ including but not limited to software source code, documentation
+ source, and configuration files.
+
+ "Object" form shall mean any form resulting from mechanical
+ transformation or translation of a Source form, including but
+ not limited to compiled object code, generated documentation,
+ and conversions to other media types.
+
+ "Work" shall mean the work of authorship, whether in Source or
+ Object form, made available under the License, as indicated by a
+ copyright notice that is included in or attached to the work
+ (an example is provided in the Appendix below).
+
+ "Derivative Works" shall mean any work, whether in Source or Object
+ form, that is based on (or derived from) the Work and for which the
+ editorial revisions, annotations, elaborations, or other modifications
+ represent, as a whole, an original work of authorship. For the purposes
+ of this License, Derivative Works shall not include works that remain
+ separable from, or merely link (or bind by name) to the interfaces of,
+ the Work and Derivative Works thereof.
+
+ "Contribution" shall mean any work of authorship, including
+ the original version of the Work and any modifications or additions
+ to that Work or Derivative Works thereof, that is intentionally
+ submitted to Licensor for inclusion in the Work by the copyright owner
+ or by an individual or Legal Entity authorized to submit on behalf of
+ the copyright owner. For the purposes of this definition, "submitted"
+ means any form of electronic, verbal, or written communication sent
+ to the Licensor or its representatives, including but not limited to
+ communication on electronic mailing lists, source code control systems,
+ and issue tracking systems that are managed by, or on behalf of, the
+ Licensor for the purpose of discussing and improving the Work, but
+ excluding communication that is conspicuously marked or otherwise
+ designated in writing by the copyright owner as "Not a Contribution."
+
+ "Contributor" shall mean Licensor and any individual or Legal Entity
+ on behalf of whom a Contribution has been received by Licensor and
+ subsequently incorporated within the Work.
+
+ 2. Grant of Copyright License. Subject to the terms and conditions of
+ this License, each Contributor hereby grants to You a perpetual,
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+ copyright license to reproduce, prepare Derivative Works of,
+ publicly display, publicly perform, sublicense, and distribute the
+ Work and such Derivative Works in Source or Object form.
+
+ 3. Grant of Patent License. Subject to the terms and conditions of
+ this License, each Contributor hereby grants to You a perpetual,
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+ (except as stated in this section) patent license to make, have made,
+ use, offer to sell, sell, import, and otherwise transfer the Work,
+ where such license applies only to those patent claims licensable
+ by such Contributor that are necessarily infringed by their
+ Contribution(s) alone or by combination of their Contribution(s)
+ with the Work to which such Contribution(s) was submitted. If You
+ institute patent litigation against any entity (including a
+ cross-claim or counterclaim in a lawsuit) alleging that the Work
+ or a Contribution incorporated within the Work constitutes direct
+ or contributory patent infringement, then any patent licenses
+ granted to You under this License for that Work shall terminate
+ as of the date such litigation is filed.
+
+ 4. Redistribution. You may reproduce and distribute copies of the
+ Work or Derivative Works thereof in any medium, with or without
+ modifications, and in Source or Object form, provided that You
+ meet the following conditions:
+
+ (a) You must give any other recipients of the Work or
+ Derivative Works a copy of this License; and
+
+ (b) You must cause any modified files to carry prominent notices
+ stating that You changed the files; and
+
+ (c) You must retain, in the Source form of any Derivative Works
+ that You distribute, all copyright, patent, trademark, and
+ attribution notices from the Source form of the Work,
+ excluding those notices that do not pertain to any part of
+ the Derivative Works; and
+
+ (d) If the Work includes a "NOTICE" text file as part of its
+ distribution, then any Derivative Works that You distribute must
+ include a readable copy of the attribution notices contained
+ within such NOTICE file, excluding those notices that do not
+ pertain to any part of the Derivative Works, in at least one
+ of the following places: within a NOTICE text file distributed
+ as part of the Derivative Works; within the Source form or
+ documentation, if provided along with the Derivative Works; or,
+ within a display generated by the Derivative Works, if and
+ wherever such third-party notices normally appear. The contents
+ of the NOTICE file are for informational purposes only and
+ do not modify the License. You may add Your own attribution
+ notices within Derivative Works that You distribute, alongside
+ or as an addendum to the NOTICE text from the Work, provided
+ that such additional attribution notices cannot be construed
+ as modifying the License.
+
+ You may add Your own copyright statement to Your modifications and
+ may provide additional or different license terms and conditions
+ for use, reproduction, or distribution of Your modifications, or
+ for any such Derivative Works as a whole, provided Your use,
+ reproduction, and distribution of the Work otherwise complies with
+ the conditions stated in this License.
+
+ 5. Submission of Contributions. Unless You explicitly state otherwise,
+ any Contribution intentionally submitted for inclusion in the Work
+ by You to the Licensor shall be under the terms and conditions of
+ this License, without any additional terms or conditions.
+ Notwithstanding the above, nothing herein shall supersede or modify
+ the terms of any separate license agreement you may have executed
+ with Licensor regarding such Contributions.
+
+ 6. Trademarks. This License does not grant permission to use the trade
+ names, trademarks, service marks, or product names of the Licensor,
+ except as required for reasonable and customary use in describing the
+ origin of the Work and reproducing the content of the NOTICE file.
+
+ 7. Disclaimer of Warranty. Unless required by applicable law or
+ agreed to in writing, Licensor provides the Work (and each
+ Contributor provides its Contributions) on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+ implied, including, without limitation, any warranties or conditions
+ of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+ PARTICULAR PURPOSE. You are solely responsible for determining the
+ appropriateness of using or redistributing the Work and assume any
+ risks associated with Your exercise of permissions under this License.
+
+ 8. Limitation of Liability. In no event and under no legal theory,
+ whether in tort (including negligence), contract, or otherwise,
+ unless required by applicable law (such as deliberate and grossly
+ negligent acts) or agreed to in writing, shall any Contributor be
+ liable to You for damages, including any direct, indirect, special,
+ incidental, or consequential damages of any character arising as a
+ result of this License or out of the use or inability to use the
+ Work (including but not limited to damages for loss of goodwill,
+ work stoppage, computer failure or malfunction, or any and all
+ other commercial damages or losses), even if such Contributor
+ has been advised of the possibility of such damages.
+
+ 9. Accepting Warranty or Additional Liability. While redistributing
+ the Work or Derivative Works thereof, You may choose to offer,
+ and charge a fee for, acceptance of support, warranty, indemnity,
+ or other liability obligations and/or rights consistent with this
+ License. However, in accepting such obligations, You may act only
+ on Your own behalf and on Your sole responsibility, not on behalf
+ of any other Contributor, and only if You agree to indemnify,
+ defend, and hold each Contributor harmless for any liability
+ incurred by, or claims asserted against, such Contributor by reason
+ of your accepting any such warranty or additional liability.
+
+ END OF TERMS AND CONDITIONS
+
+ APPENDIX: How to apply the Apache License to your work.
+
+ To apply the Apache License to your work, attach the following
+ boilerplate notice, with the fields enclosed by brackets "[]"
+ replaced with your own identifying information. (Don't include
+ the brackets!) The text should be enclosed in the appropriate
+ comment syntax for the file format. We also recommend that a
+ file or class name and description of purpose be included on the
+ same "printed page" as the copyright notice for easier
+ identification within third-party archives.
+
+ Copyright [yyyy] [name of copyright owner]
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
--- /dev/null
+.. This work is licensed under a Creative Commons Attribution 4.0 International License.
+.. SPDX-License-Identifier: CC-BY-4.0
+.. Copyright (C) 2022 Nordix
+
+.. |nbsp| unicode:: 0xA0
+ :trim:
+
+.. |nbh| unicode:: 0x2011
+ :trim:
+
+.. |yaml-icon| image:: ./images/yaml_logo.png
+ :width: 40px
+
+.. _calloutserver:
+
+=====================
+Callout Server
+=====================
+
+API Documentation
+=================
+
+The O-RAN SC external call-out server allows behavioral extensions to be added to the A1 Simulator. It creates an external call-out server, which provides a RESTful API. A1 Policy operations, for some A1 Policy Types, can then be redirected to the external call-out server, supporting supplemental simulator behavior for those A1 Policy Types.
+
+**Note:** call-out server functionality is only available for *'STD_2.0.0'* version simulators.
+
+The external call-out server exposes a 'Callout server API' REST API. This internal API is invoked directly by the A1 Simulator, and is not intended to be used by any other client. The 'Callout Server API' is documented in `Callout Server API <./EXT_SRV_api.html>`_ and in OpenAPI YAML format:
+
+.. csv-table::
+ :header: "API name", "|yaml-icon|"
+ :widths: 10,5
+
+ "Callout Server API", ":download:`link <../near-rt-ric-simulator/test/EXT_SRV/api/EXT_SRV_api.yaml>`"
+
+External call-out servers also expose an 'Admin API' to manipulate the behavior of the call-out server itself. The 'Callout Server Admin API' is documented below:
+
+Admin Functions
+================
+
+Health Check
+------------
+
+GET
++++
+
+Returns the status of the external server.
+
+**URL path:**
+ /
+
+**Parameters:**
+ None.
+
+**Responses:**
+ 200:
+ OK
+
+**Examples:**
+
+**Call**: ::
+
+ curl -X GET "http://localhost:9095/"
+
+**Result**:
+
+200: ::
+
+ OK
+
+
+Delete all policy instances in external server
+----------------------------------------------
+
+POST
+++++
+
+Delete all policy instances.
+
+**URL path:**
+
+/serveradmin/deleteinstances
+
+**Parameters:**
+
+None.
+
+**Responses:**
+
+200:
+
+All a1 policy instances deleted
+
+**Examples:**
+
+**Call**: ::
+
+ curl -X POST "http://localhost:9095/serveradmin/deleteinstances"
+
+**Result**:
+
+200: ::
+
+ All a1 policy instances deleted
+
+
+Response manipulation
+---------------------
+It is possible to manipulate the response of all operations on the external server
+
+POST
+++++
+
+Force a specific response code for the all (the next) external server operation. Unless it is reset, it will always respond the same response code back.
+
+**URL path:**
+
+/serveradmin/forceresponse?code=<http-response-code>
+
+**Parameters:**
+
+code: (*Required*)
+
+The HTTP response code to return.
+
+**Responses:**
+
+200:
+
+Force response code: <expected code> set for all external server response until it is resetted
+
+**Examples:**
+
+**Call**: ::
+
+ curl -X POST "http://localhost:9095/serveradmin/forceresponse?code=500"
+
+**Result**:
+
+200: ::
+
+ Force response code: 500 set for all external server response until it is resetted
+
+
+Reset response-manipulation
+---------------------------
+It is possible to reset the response manipulation on the external server
+
+POST
+++++
+
+Clears specific response code for all (the next) external server operation.
+
+**URL path:**
+
+/serveradmin/forceresponse?code=<http-response-code>
+
+**Parameters:**
+
+code: (*Required*)
+
+The HTTP response code to return.
+
+**Responses:**
+
+200:
+
+Force response code has been resetted for all external server responses
+
+**Examples:**
+
+**Call**: ::
+
+ curl -X POST "http://localhost:9095/serveradmin/forceresponse?code=500"
+
+**Result**:
+
+200: ::
+
+ Force response code has been resetted for all external server responses
+
+
+Response time manipulation
+--------------------------
+It is possible to set a period of time to delay response time.
+
+POST
+++++
+
+Force delayed response of all A1 responses. The setting will remain until the delay is set to '0'
+
+**URL path:**
+
+/serveradmin/forcedelay?delay=<delay-time-seconds>
+
+**Parameters:**
+
+delay: (*Required*)
+
+The time in seconds to delay all responses.
+
+**Responses:**
+
+200:
+
+Force delay: <expected_delay> sec set for all external server responses until it is resetted
+
+**Examples:**
+
+**Call**: ::
+
+ curl -X POST "http://localhost:9095/serveradmin/forcedelay?delay=5"
+
+**Result**:
+
+200: ::
+
+ Force delay: 5 sec set for all external server responses until it is resetted
+
+
+Reset response time manipulation
+--------------------------------
+It is also possible to reset delay response time.
+
+POST
+++++
+
+The setting will clear the delay.
+
+**URL path:**
+
+/serveradmin/forcedelay
+
+**Parameters:**
+
+None.
+
+The time in seconds to delay all responses.
+
+**Responses:**
+
+200:
+
+Force delay has been resetted for all external server responses
+
+**Examples:**
+
+**Call**: ::
+
+ curl -X POST "http://localhost:9095/serveradmin/forcedelay"
+
+**Result**:
+
+200: ::
+
+ Force delay has been resetted for all external server responses
from docs_conf.conf import *
+branch = 'latest'
+
+language = 'en'
+
linkcheck_ignore = [
'http://localhost.*',
'http://127.0.0.1.*',
- 'https://gerrit.o-ran-sc.org.*'
+ 'https://gerrit.o-ran-sc.org.*',
+ './KAFKA_DISPATCHER_api.html',
+ './EXT_SRV_api.html', #Generated file that doesn't exist at link check.
]
+
+extensions = ['sphinxcontrib.redoc']
+
+redoc = [
+ {
+ 'name': 'CALLOUT SERVER',
+ 'page': 'EXT_SRV_api',
+ 'spec': '../near-rt-ric-simulator/test/EXT_SRV/api/EXT_SRV_api.yaml',
+ 'embed': True,
+ },
+ {
+ 'name': 'Kafka Message Dispatcher',
+ 'page': 'KAFKA_DISPATCHER_api',
+ 'spec': '../near-rt-ric-simulator/test/KAFKA_DISPATCHER/api/KAFKA_DISPATCHER_api.yaml',
+ 'embed': True,
+ },
+ ]
+
+redoc_uri = 'https://cdn.jsdelivr.net/npm/redoc@next/bundles/redoc.standalone.js'
.. This work is licensed under a Creative Commons Attribution 4.0 International License.
.. SPDX-License-Identifier: CC-BY-4.0
-.. Copyright (C) 2021 Nordix
+.. Copyright (C) 2021-2022 Nordix
.. |nbsp| unicode:: 0xA0
:trim:
./overview.rst
./simulator-api.rst
+ ./callout-server.rst
+ ./kafka-message-dispatcher.rst
./release-notes.rst
--- /dev/null
+.. This work is licensed under a Creative Commons Attribution 4.0 International License.
+.. SPDX-License-Identifier: CC-BY-4.0
+.. Copyright (C) 2022 Nordix
+
+.. |nbsp| unicode:: 0xA0
+ :trim:
+
+.. |nbh| unicode:: 0x2011
+ :trim:
+
+.. |yaml-icon| image:: ./images/yaml_logo.png
+ :width: 40px
+
+========================
+Kafka Message Dispatcher
+========================
+
+API Documentation
+=================
+
+The O-RAN SC Kafka Message Dispatcher is a specific implementation of an A1 Simulator ref:`calloutserver`, which further redirects A1 Policy operations to a Kafka message topic, to be consumed by an external function.
+
+A1 Policy are redirected as Kafka messages to a configured Kafka Topic to an external receiver, then responses from the external receiver are collected from another configured Kafka Topic. This provides a Kafka-based request-response abstraction for adding supplemental simulator behavior for particular A1 Policy Types. After a request message is sent, a response message will be expected within some configurable timeout interval (default: 30 sec). The topics to be used for particular A1 Policy Types is configured using a JSON map (Example: `policytype_to_topicmap.json <../near-rt-ric-simulator/test/KAFKA_DISPATCHER/resources/policytype_to_topicmap.json>`
+
+**Note:** As with other A1 Simulator call-out servers, the Kafka message dispatcher functionality is only available for *'STD_2.0.0'* version simulators.
+
+The Kafka message dispatcher exposes a 'Kafka Message Dispatcher' REST API. This internal API is invoked directly by the A1 Simulator, and is not intended to be used by any other client. This API is documented in `Kafka Message Dispatcher API <./KAFKA_DISPATCHER_api.html>`_ and in OpenAPI YAML format:
+
+.. csv-table::
+ :header: "API name", "|yaml-icon|"
+ :widths: 10,5
+
+ "Kafka Message Dispatcher API", ":download:`link <../near-rt-ric-simulator/test/KAFKA_DISPATCHER/api/KAFKA_DISPATCHER_api.yaml>`"
+
+The Kafka message dispatcher also exposes an 'Admin API' to manipulate the behavior of the Kafka message dispather itself. The 'Kafka Message Dispatcher Admin API' is documented below:
+
+Admin Functions
+================
+
+Health Check
+------------
+
+GET
++++
+
+Returns the status of the Kafka Message Dispatcher.
+
+**URL path:**
+ /
+
+**Parameters:**
+ None.
+
+**Responses:**
+ 200:
+ OK
+
+**Examples:**
+
+**Call**: ::
+
+ curl -X GET "http://localhost:7075/"
+
+**Result**:
+
+200: ::
+
+ OK
+
+
+Response manipulation
+---------------------
+It is possible to manipulate the response of all operations on the Kafka Message Dispatcher module
+
+POST
+++++
+
+Force a specific response code for the all (the next) Kafka Message Dispatcher module operations. Unless it is reset, it will always respond the same response code back.
+
+**URL path:**
+
+/dispatcheradmin/forceresponse?code=<http-response-code>
+
+**Parameters:**
+
+code: (*Required*)
+
+The HTTP response code to return.
+
+**Responses:**
+
+200:
+
+Force response code: <expected code> set for all all dispatcher response until it is resetted
+
+**Examples:**
+
+**Call**: ::
+
+ curl -X POST "http://localhost:7075/dispatcheradmin/forceresponse?code=500"
+
+**Result**:
+
+200: ::
+
+ Force response code: 500 set for all dispatcher response until it is resetted
+
+
+Reset response-manipulation
+---------------------------
+It is possible to reset the response manipulation on the Kafka Message Dispatcher module
+
+POST
+++++
+
+Clears specific response code for all (the next) Kafka Message Dispatcher module operation.
+
+**URL path:**
+
+/dispatcheradmin/forceresponse
+
+**Parameters:**
+
+code: (*Required*)
+
+The HTTP response code to return.
+
+**Responses:**
+
+200:
+
+Force response code has been resetted for dispatcher responses
+
+**Examples:**
+
+**Call**: ::
+
+ curl -X POST "http://localhost:7075/dispatcheradmin/forceresponse"
+
+**Result**:
+
+200: ::
+
+ Force response code has been resetted for dispatcher responses
+
+
+Response time manipulation
+--------------------------
+It is possible to set a period of time to delay response time.
+
+POST
+++++
+
+Force delayed response of all dispatcher responses. The setting will remain until the delay is cleared.
+
+**URL path:**
+
+/dispatcheradmin/forcedelay?delay=<delay-time-seconds>
+
+**Parameters:**
+
+delay: (*Required*)
+
+The time in seconds to delay all responses.
+
+**Responses:**
+
+200:
+
+Force delay: <expected_delay> sec set for all dispatcher responses until it is resetted
+
+**Examples:**
+
+**Call**: ::
+
+ curl -X POST "http://localhost:7075/dispatcheradmin/forcedelay?delay=5"
+
+**Result**:
+
+200: ::
+
+ Force delay: 5 sec set for all dispatcher responses until it is resetted
+
+
+Reset response time manipulation
+--------------------------------
+It is also possible to reset delay response time.
+
+POST
+++++
+
+The setting will clear the delay.
+
+**URL path:**
+
+/dispatcheradmin/forcedelay
+
+**Parameters:**
+
+None.
+
+The time in seconds to delay all responses.
+
+**Responses:**
+
+200:
+
+Force delay has been resetted for all dispatcher responses
+
+**Examples:**
+
+**Call**: ::
+
+ curl -X POST "http://localhost:7075/dispatcheradmin/forcedelay"
+
+**Result**:
+
+200: ::
+
+ Force delay has been resetted for all dispatcher responses
.. This work is licensed under a Creative Commons Attribution 4.0 International License.
.. http://creativecommons.org/licenses/by/4.0
-.. Copyright (C) 2021 Nordix
+.. Copyright (C) 2021-2022 Nordix
=============
Release-Notes
=============
-This document provides the release notes for the release of the Near-RT RIC A1 Interface Simulator.
+This document provides the release notes for the release of the A1 Simulator (previously called Near-RT RIC A1 Interface).
.. contents::
:depth: 3
:local:
-Version history Near-RT RIC A1 Interface Simulator
-==================================================
+Version history A1 Simulator (previously called Near-RT RIC A1 Interface)
+=========================================================================
+------------+----------+------------------+----------------+
| **Date** | **Ver.** | **Author** | **Comment** |
sphinx
sphinx-rtd-theme
+sphinx-bootstrap-theme
sphinxcontrib-httpdomain
+sphinxcontrib-redoc
+sphinxcontrib-needs
+sphinxcontrib-swaggerdoc
recommonmark
lfdocs-conf
-# O-RAN-SC Near-RealTime RIC Simulator
+# O-RAN-SC A1 Simulator
-The O-RAN SC Near-RealTime RIC simulates the A1 as an generic REST API which can receive and send northbound messages. The simulator validates the payload and applies policy.
+The O-RAN SC A1 simulator simulates the A1 as an generic REST API which can receive and send northbound messages. The simulator validates the payload and applies policy.
The simulator supports multiple A1 interface versions (version of the open API yaml file\):
Note that test can be performed both using the nonsecure http port and the secure https port.
-Build and start the simulator container using:
+Build and start the simulator containers: STD_1.1.3 and OSC_2.1.0, using:
-./build\_and\_start.sh duplicate-check|ignore-duplicate
+./build_and_start.sh duplicate-check|ignore-duplicate
+Build and start the simulator container version STD_2.0.0, using two alternatives: ext-srv or kafka-srv. However, both can not be used at the same time to start A1 sim.
+
+In order to start with ext-srv:
+./build_and_start.sh duplicate-check|ignore-duplicate ext-srv|ext-srv-secure|ignore-ext-srv
+
+In order to start with kafka-srv:
+./build_and_start.sh duplicate-check|ignore-duplicate kafka-srv|kafka-srv-secure publish-resp|ignore-publish
+
+STD_2.0.0 version is now including an external server that is a Python server building RESTful API. The external server supports HTTP/HTTPS protocols.
+The description of the start parameters are explained below:
+ext-srv: Runs external server that supports HTTP protocol only.
+ext-srv-secure: Runs external server that supports HTTPS protocol as well.
+ignore-ext-srv: Ignores external server to run.
+
+STD_2.0.0 version also includes an kafka message dispatcher that is a Python server building RESTful APIs. The kafka server supports HTTP/HTTPS protocols.
+The description of the start parameters are explained below:
+kafka-srv: Runs kafka server that supports HTTP protocol only.
+kafka-srv-secure: Runs kafka server that supports HTTPS protocol as well.
+publish-resp: The flag controls the dispatcher module to decide auto responding to each requests for test purposes only.
+ignore-publish: If the A1 sim is being started using ignore flag, then the dispatcher module will look for a respone message published by south-bound module.
This will build and start the container in interactive mode. The built container only resides in the local docker repository.
Note, the default port is 8085 for http and 8185 for https. When running the simulator as a container, the defualt ports can be re-mapped to any port on the localhost.
-In a second terminal, go to the same folder and run the basic test script, basic\_test.sh nonsecure|secure or commands.sh nonsecure|secure duplicate-check|ignore-duplicate
+In a second terminal, go to the same folder and run the basic test script, basic_test.sh nonsecure|secure or commands.sh nonsecure|secure duplicate-check|ignore-duplicate for STD_1.1.3 and OSC_2.1.0 versions.
+
+For the STD_2.0.0 version, in a second terminal, go to the same folder and run the basic test script for external server activated case:
+./basic_test.sh nonsecure|secure duplicate-check|ignore-duplicate ext-srv|ext-srv-secure|ignore-ext-srv
+The description of the test script parameters are explained below:
+nonsecure|secure: Runs test cases with either support of HTTP/HTTPS protocol.
+duplicate-check|ignore-duplicate: Runs test cases with either support of duplicate/ignore-duplicate flag for the policies.
+ext-srv|ext-srv-secure|ignore-ext-srv: If the simulator started with ext-srv or ext-srv-secure parameter, then one of these options can be used. Otherwise, ignore-ext-srv parameter should be used.
+
+For the STD_2.0.0 version, in a second terminal, go to the same folder and run the basic test script for kafka dispatcher server activated case:
+./basic_test.sh nonsecure|secure duplicate-check|ignore-duplicate ext-srv|ext-srv-secure|ignore-ext-srv
+The description of the test script parameters are explained below:
+nonsecure|secure: Runs test cases with either support of HTTP/HTTPS protocol.
+duplicate-check|ignore-duplicate: Runs test cases with either support of duplicate/ignore-duplicate flag in accordance with the one which used while starting A1 sim.
+ext-srv|ext-srv-secure|ignore-ext-srv: If the simulator started with kafka-srv or kafka-srv-secure parameter, then ignore-ext-srv option should be used.
Note that the arg for duplicate check must match in both scripts.
This script runs a number of tests towards the simulator to make sure it works properply.
Navigate to 'near-rt-ric-simulator/tests'. Choose the version to test and use that file for test.
-Use 'python3 -m pytest \<filename>' to run unit test only with no coverage check
+Use 'python3 -m pytest \<filename>' to run unit test only with no coverage check. Before running that command, the dependencies which are pytest and connexion should be installed in your virtual environment. If the latest connexion version arises DeprecationWarning, you may try to install connexion with version 2.6.0.
Or use 'coverage run -m pytest \<filename>' to run unit test and produce coverage data.
content:
application/problem+json:
schema:
- "$ref": "#/components/schemas/ProblemDetails"
\ No newline at end of file
+ "$ref": "#/components/schemas/ProblemDetails"
---
-tag: 2.3.0
+tag: 2.3.1
APPL_PROB_JSON='application/problem+json'
EXT_SRV_URL=os.getenv('EXT_SRV_URL')
-
+KAFKA_DISPATCHER_URL=os.getenv('KAFKA_DISPATCHER_URL')
# API Function: Get all policy type ids
def get_all_policy_types():
pjson=create_problem_json(None, "Duplicate, the policy json already exists.", 400, None, policy_id)
return Response(json.dumps(pjson), 400, mimetype=APPL_PROB_JSON)
+ #Callout hooks for kafka dispatcher
+ if (KAFKA_DISPATCHER_URL is not None):
+ resp = callout_kafka_dispatcher(policy_type_id, policy_id, data, retcode)
+ if (resp != 200):
+ pjson=create_error_response(resp)
+ return Response(json.dumps(pjson), 500, mimetype=APPL_PROB_JSON)
+
#Callout hooks for external server
#When it fails, break and return 419 HTTP status code
if (EXT_SRV_URL is not None):
pjson=create_problem_json(None, "The requested policy does not exist.", 404, None, policy_id)
return Response(json.dumps(pjson), 404, mimetype=APPL_PROB_JSON)
+ #Callout hooks for kafka dispatcher
+ if (KAFKA_DISPATCHER_URL is not None):
+ resp = callout_kafka_dispatcher(policy_type_id, policy_id, None, 204)
+ if (resp != 200):
+ pjson=create_error_response(resp)
+ return Response(json.dumps(pjson), 500, mimetype=APPL_PROB_JSON)
+
#Callout hooks for external server
#When it fails, break and return 419 HTTP status code
if (EXT_SRV_URL is not None):
pjson=create_problem_json(None, "The requested policy does not exist.", 404, None, policy_id)
return Response(json.dumps(pjson), 404, mimetype=APPL_PROB_JSON)
+ #Callout hooks for kafka dispatcher
+ if (KAFKA_DISPATCHER_URL is not None):
+ resp = callout_kafka_dispatcher(policy_type_id, policy_id, None, 202)
+ if (resp != 200):
+ pjson=create_error_response(resp)
+ return Response(json.dumps(pjson), 500, mimetype=APPL_PROB_JSON)
+
return Response(json.dumps(policy_status[policy_id]), status=200, mimetype=APPL_JSON)
+
+# Helper: Callout kafka dispatcher server to notify it for policy operations
+def callout_kafka_dispatcher(policy_type_id, policy_id, payload, retcode):
+
+ target_url = KAFKA_DISPATCHER_URL + "/policytypes/" + policy_type_id + "/kafkadispatcher/" + policy_id
+ try:
+ # create operation, publish with payload
+ if (retcode == 201):
+ resp=requests.put(target_url, json=payload, timeout=30, verify=False)
+ return resp.status_code
+ # update operation, publish with payload
+ elif (retcode == 200):
+ # add headers an update-flag
+ headers = {'updateoper' : 'yes'}
+ resp=requests.put(target_url, json=payload, headers=headers, timeout=30, verify=False)
+ return resp.status_code
+ # delete operation, publish without payload
+ elif (retcode == 204):
+ resp=requests.delete(target_url, timeout=30, verify=False)
+ return resp.status_code
+ # get policy status operation, publish without payload
+ elif (retcode == 202):
+ # update endpoint
+ target_url = target_url + "/status"
+ resp=requests.get(target_url, timeout=30, verify=False)
+ return resp.status_code
+ except Exception:
+ return 419
+
+
# Helper: Callout external server to notify it for policy operations
# Returns 200, 201 and 204 for the success callout hooks, for the others returns 419
def callout_external_server(policy_id, payload, operation):
--- /dev/null
+# Documentation
+.idea/
+.tox
+docs/_build/
+.DS_STORE
+
+# IDE
+.project
+.vscode
+
+.coverage
+coverage.xml
+htmlcov/
+
+# Python virtual env
+venv/
--- /dev/null
+# O-RAN-SC External Server extension for A1 Simulator
+
+The O-RAN SC external server is an extension for A1 simulator. It creates an external web server building RESTful API. It is capable of recieving Rest calls from the northbound simulator version and responses back to it.
+
+The external server supports GET, PUT and DELETE operations (version of the open API yaml file\):
+
+| Yaml file version | Version id |
+| --------------------- | ------------------- |
+| EXT_SRV_api.yaml | 0.0.1 |
+
+The overall folder structure is \(relative to the location of this README file\):
+
+| Dir | Description |
+| ---------------- | ----------- |
+|. |Dockerfile, tox.ini and README |
+|api |The open api yaml |
+|src |Python source code |
+|certificate |A self-signed certificate and a key |
+|docs |Auto generated API descriptions in HTML format |
+
+The external server handles the requests that are defined in the open API yaml file. All these requests are implemented in the server.py file in the src folder. In addition, a number of administrative functions are also supported and implemented by the main.py in the source folder.
+
+The section below outlines the supported open api rest-based operations as well as the adminstrative operations.
+
+# Ports and certificates
+
+The external server normally opens the port 9095 for http. If a certificate and a key are provided the external server will open port 9195 for https instead. The port 9195 is only opened if a valid certificate and key is found.
+The certificate and key shall be placed in the same directory and the directory shall be mounted to /usr/src/app/cert in the container.
+
+| Port | Protocol |
+| -------- | ----- |
+| 9095 | http |
+| 9195 | https |
+
+The directory certificate contains a self-signed cert. Use the script generate_cert_and_key.sh to generate a new certificate and key. The password of the certificate must be set 'test'.
+The same urls are availables on both the http port 9095 and the https port 9195. If using curl and https, the flag -k shall be given to make curl ignore checking the certificate.
+
+# Supported operations in External Server 0.0.1
+
+
+For the complete yaml specification, see [openapi.yaml](../api/EXT_SRV_api.yaml)
+
+URIs for server:
+
+| Function | Path and parameters |
+| --------------------- | ------------------- |
+| GET, Get all A1 policy ids | localhost:9095/a1policies |
+| GET, Query for an A1 policy | localhost:9095/a1policy/{a1policyId} |
+| PUT, Create an A1 policy | localhost:9095/a1policy/{a1policyId} |
+| DELETE, Delete an A1 policy | localhost:9095/a1policy/{a1policyId} |
+
+URIs for admin operations:
+
+| Function | Path and parameters |
+| --------------------- | ------------------- |
+| POST, Delete all A1 policy instances | localhost:9095/serveradmin/deleteinstances |
+| POST, Force a specific response code for all A1 operation | localhost:9095/serveradmin/forceresponse?code=500 |
+| POST, Reset force response code | localhost:9095/serveradmin/forceresponse |
+| POST, Force delayed response of all A1 operations | localhost:9095/serveradmin/forcedelay?delay=5 |
+| POST, Reset force delayed response | localhost:9095/serveradmin/forcedelay |
+
+
+## License
+
+Copyright (C) 2022 Nordix Foundation.
+Licensed under the Apache License, Version 2.0 (the "License")
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+
+For more information about license please see the [LICENSE](LICENSE.txt) file for details.
openapi: 3.0.0
info:
title: 'External Server for A1 simulator'
- version: 2.0.0
+ version: 0.0.1
description: |
External test server.
© 2022, O-RAN Alliance.
All rights reserved.
+ license:
+ name: Copyright (C) 2022 Nordix Foundation. Licensed under the Apache License.
+ url: http://www.apache.org/licenses/LICENSE-2.0
externalDocs:
description: 'An external server building CRUD RestFUL APIs which is provisioned by A1 simulator. It will be a refrence point for the callouts'
- url: 'https://www.testserver/specifications'
+ url: 'https://docs.o-ran-sc.org/projects/o-ran-sc-sim-a1-interface/en/latest/EXT_SRV_api.html'
servers:
- url: '{apiRoot}'
variables:
apiRoot:
- default: 'https://testserver.com'
+ default: 'http://www.example.com'
paths:
'/a1policies':
get:
--- /dev/null
+from docs_conf.conf import *
+
+language = 'en'
+
+extensions = [
+ 'sphinxcontrib.openapi',
+]
+
+linkcheck_ignore = [
+ 'http://localhost.*',
+ 'http://127.0.0.1.*',
+ 'https://gerrit.o-ran-sc.org.*'
+]
--- /dev/null
+---
+project_cfg: oran
+project: sim-a1-interface
--- /dev/null
+.. This work is licensed under a Creative Commons Attribution 4.0 International License.
+.. SPDX-License-Identifier: CC-BY-4.0
+.. Copyright (C) 2022 Nordix
+
+.. _ext-srv-api:
+
+==========================
+External Server API
+==========================
+
+.. Generates content from EXT_SRV_api.yaml
+.. openapi:: ../api/EXT_SRV_api.yaml
--- /dev/null
+.. This work is licensed under a Creative Commons Attribution 4.0 International License.
+.. SPDX-License-Identifier: CC-BY-4.0
+.. Copyright (C) 2022 Nordix
+
+.. |nbsp| unicode:: 0xA0
+ :trim:
+
+.. |nbh| unicode:: 0x2011
+ :trim:
+
+.. _a1-interface-ext-srv:
+
+======================
+A1 Interface EXT SRV
+======================
+
+.. toctree::
+ :maxdepth: 2
+ :caption: Contents:
+
+ ./overview.rst
+ ./ext-srv-api.rst
+ ./release-notes.rst
+
--- /dev/null
+.. This work is licensed under a Creative Commons Attribution 4.0 International License.
+.. SPDX-License-Identifier: CC-BY-4.0
+.. Copyright (C) 2020 Nordix
+
+.. |nbsp| unicode:: 0xA0
+ :trim:
+
+.. |nbh| unicode:: 0x2011
+ :trim:
+
+
+A1 Simulator EXT SRV
+=====================
+
+The A1 Simulator terminates the A1 interface and provides a way to test Non-RT RIC services without the need to deploy Near |nbh| RT |nbsp| RICs.
+
+Apart from providing the A1 API, the simulator also provides an administrative API to manage policy types and manipulate
+the simulator, see ":ref:`ext-srv-api`".
+
+The A1 Simulator supports running multiple simulations using different versions of the A1 Application protocol, and supports realistic stateful simulation of A1 Enrichment Information and A1 Policy behaviours.
+
+For information on how to run the simulator, see the *README.md* file in the repository.
--- /dev/null
+.. This work is licensed under a Creative Commons Attribution 4.0 International License.
+.. http://creativecommons.org/licenses/by/4.0
+.. Copyright (C) 2022 Nordix
+
+=============
+Release-Notes
+=============
+
+
+This document provides the release notes for the release of the Near-RT RIC A1 Interface Simulator.
+
+.. contents::
+ :depth: 3
+ :local:
+
+
+Version history Near-RT RIC A1 Interface Simulator
+==================================================
+
++------------+----------+------------------+----------------+
+| **Date** | **Ver.** | **Author** | **Comment** |
+| | | | |
++------------+----------+------------------+----------------+
+| 2022-06-29 | 2.3.0 | Halil Cakal | F Release |
+| | | | |
++------------+----------+------------------+----------------+
+
+Release Data
+============
+
+F Release
+---------
++-----------------------------+-------------------------------------------------------+
+| **Project** | Non-RT RIC |
+| | |
++-----------------------------+-------------------------------------------------------+
+| **Repo/commit-ID** | a1-interface/595506e290356d26b8eebfab32ef8d3f625cbb0a |
+| | |
++-----------------------------+-------------------------------------------------------+
+| **Release designation** | F |
+| | |
++-----------------------------+-------------------------------------------------------+
+| **Release date** | 2022-06-29 |
+| | |
++-----------------------------+-------------------------------------------------------+
+| **Purpose of the delivery** | Added Callout hooks towards external server for |
+| | create and delete operations |
+| | |
++-----------------------------+-------------------------------------------------------+
--- /dev/null
+# ==================================================================================
+# Copyright (c) 2022 Nordix
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# ==================================================================================
+
+[tox]
+envlist = docs,docs-linkcheck
+minversion = 2.0
+skipsdist = true
+
+# doc jobs
+[testenv:docs]
+whitelist_externals = echo
+# Version 3.8 is required, otherwise AttributeError will arise
+basepython = python3.8
+deps =
+ sphinx
+ sphinx-rtd-theme
+ sphinxcontrib-httpdomain
+ # Version 0.6.0 is required, otherwise BlockGrammer error will arise
+ sphinxcontrib-openapi==0.6.0
+ recommonmark
+ lfdocs-conf
+commands =
+ sphinx-build -W -b html -n -d {envtmpdir}/doctrees ./docs/ {toxinidir}/docs/_build/html
+ echo "Generated docs available in {toxinidir}/docs/_build/html"
+
+[testenv:docs-linkcheck]
+skipsdist = true
+basepython = python3.8
+deps = sphinx
+ sphinx-rtd-theme
+ sphinxcontrib-httpdomain
+ # Version 0.6.0 is required, otherwise BlockGrammer error will arise
+ sphinxcontrib-openapi==0.6.0
+ recommonmark
+ lfdocs-conf
+commands = sphinx-build -W -b linkcheck -d {envtmpdir}/doctrees ./docs/ {toxinidir}/docs/_build/linkcheck
--- /dev/null
+# Documentation
+.idea/
+.tox
+docs/_build/
+.DS_STORE
+
+# IDE
+.project
+.vscode
+
+.coverage
+coverage.xml
+htmlcov/
+
+# Python virtual env
+venv/
--- /dev/null
+# ============LICENSE_START===============================================
+# Copyright (C) 2022 Nordix Foundation. All rights reserved.
+# ========================================================================
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# ============LICENSE_END=================================================
+#
+
+FROM python:3.8-slim-buster
+
+RUN pip install connexion[swagger-ui]
+RUN pip install kafka-python
+
+#install nginx and curl
+RUN apt-get update && apt-get install -y nginx=1.14.* nginx-extras curl
+
+WORKDIR /usr/src/app
+
+COPY api api
+COPY nginx.conf nginx.conf
+COPY certificate /usr/src/app/cert
+COPY src src
+COPY resources resources
+
+ARG user=nonrtric
+ARG group=nonrtric
+
+RUN groupadd $user && \
+ useradd -r -g $group $user
+RUN chown -R $user:$group /usr/src/app
+RUN chown -R $user:$group /var/log/nginx
+RUN chown -R $user:$group /var/lib/nginx
+RUN chown -R $user:$group /etc/nginx/conf.d
+RUN touch /var/run/nginx.pid
+RUN chown -R $user:$group /var/run/nginx.pid
+
+USER ${user}
+
+RUN chmod +x src/start.sh
+CMD src/start.sh
--- /dev/null
+# O-RAN-SC Kafka Dispatcher Module extension for A1 Simulator
+
+The O-RAN SC dispatcher module is an extension for A1 simulator. It creates a web server building RESTful APIs. It is capable of recieving Rest calls from the northbound simulator version and respond back to it.
+
+The kafka dispatcher module supports GET, PUT and DELETE operations (version of the open API yaml file\):
+
+| Yaml file version | Version id |
+| -------------------------- |-------------------- |
+| KAFKA_DISPATCHER_api.yaml | 0.0.1 |
+
+The overall folder structure is \(relative to the location of this README file\):
+
+| Dir | Description |
+| ---------------- | ----------- |
+|. |Dockerfile, tox.ini and README |
+|api |The open api yaml |
+|src |Python source code |
+|certificate |A self-signed certificate and a key |
+|docs |Auto generated API descriptions in HTML format |
+
+The dispatcher module handles the requests that are defined in the open API yaml file. All these requests are implemented in the dispatcher.py file in the src folder. In addition, a number of administrative functions are also supported and implemented by the main.py in the source folder.
+
+The section below outlines the supported open api rest-based operations as well as the adminstrative operations.
+
+# Ports and certificates
+
+The dispatcher module normally opens the port 7075 for http. If a certificate and a key are provided the kafka dispatcher module will open port 7175 for https instead. The port 7175 is only opened if a valid certificate and key is found.
+The certificate and key shall be placed in the same directory and the directory shall be mounted to /usr/src/app/cert in the container.
+
+| Port | Protocol |
+| -------- | ----- |
+| 7075 | http |
+| 7175 | https |
+
+The directory certificate contains a self-signed cert. Use the script generate_cert_and_key.sh to generate a new certificate and key. The password of the certificate must be set 'test'.
+The same urls are availables on both the http port 7075 and the https port 7175. If using curl and https, the flag -k shall be given to make curl ignore checking the certificate.
+
+# Supported operations in Kafka Dispatcher Module 0.0.1
+
+
+For the complete yaml specification, see [openapi.yaml](../api/KAFKA_DISPATCHER_api.yaml)
+
+URIs for server:
+
+| Function | Path and parameters |
+| --------------------- | ------------------- |
+| GET, Get the kafka request and response topic map corresponding to policy type | localhost:7075/policytypetotopicmapping/ANR |
+| GET, Dispatch policy status query opertion as kafka message to kafka cluster | localhost:7075/policytypes/ANR/kafkadispatcher/alpha/status |
+| PUT, Dispatch create and update operation as kafka message to kafka cluster | localhost:7075/policytypes/ANR/kafkadispatcher/alpha |
+| DELETE, Dispatch policy delete opertion as kafka message to kafka cluster | localhost:7075/policytypes/ptype1/kafkadispatcher/alpha |
+
+URIs for admin operations:
+
+| Function | Path and parameters |
+| --------------------- | ------------------- |
+| GET, Get status of dispatcher module | http://localhost:7075/ |
+| POST, Force a specific response code for all dispatcher module operations | localhost:7075/dispatcheradmin/forceresponse?code=500 |
+| POST, Reset force response code | localhost:7075/dispatcheradmin/forceresponse |
+| POST, Force delayed response of all dispatcher module operations | localhost:7075/dispatcheradmin/forcedelay?delay=5 |
+| POST, Reset force delayed response | localhost:7075/dispatcheradmin/forcedelay |
+
+
+## License
+
+Copyright (C) 2022 Nordix Foundation.
+Licensed under the Apache License, Version 2.0 (the "License")
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+
+For more information about license please see the [LICENSE](LICENSE.txt) file for details.
--- /dev/null
+openapi: 3.0.0
+info:
+ title: 'Kafka message dispatcher for A1 interface'
+ version: 0.0.1
+ description: |
+ Kafka message dispatcher server.
+ license:
+ name: Copyright (C) 2022 Nordix Foundation. Licensed under the Apache License.
+ url: http://www.apache.org/licenses/LICENSE-2.0
+externalDocs:
+ description: 'RestFUL APIs that create and dispatch Kafka messages to Kafka brokers'
+ url: 'https://docs.o-ran-sc.org/projects/o-ran-sc-sim-a1-interface/en/latest/index.html'
+servers:
+ - url: '{apiRoot}'
+ variables:
+ apiRoot:
+ default: 'https://example.com'
+paths:
+ '/policytypetotopicmapping/{policyTypeId}':
+ parameters:
+ - name: policyTypeId
+ in: path
+ required: true
+ schema:
+ "$ref": "#/components/schemas/PolicyTypeId"
+ get:
+ operationId: dispatcher.get_policy_type_to_topic_mapping
+ description: 'Get the kafka request and response topic map corresponding to policy type'
+ tags:
+ - The mapping from policy type to kafka topic request and response object
+ responses:
+ 200:
+ description: 'The policy type to topic map schemas'
+ content:
+ application/json:
+ schema:
+ "$ref": "#/components/schemas/PolicyTypeToTopicMap"
+ 404:
+ "$ref": "#/components/responses/404-NotFound"
+ 429:
+ "$ref": "#/components/responses/429-TooManyRequests"
+ 503:
+ "$ref": "#/components/responses/503-ServiceUnavailable"
+
+ '/policytypes/{policyTypeId}/kafkadispatcher/{policyId}':
+ parameters:
+ - name: policyTypeId
+ in: path
+ required: true
+ schema:
+ "$ref": "#/components/schemas/PolicyTypeId"
+ - name: policyId
+ in: path
+ required: true
+ schema:
+ "$ref": "#/components/schemas/A1PolicyId"
+ put:
+ operationId: dispatcher.put_policy
+ description: 'Dispatch create and update operation as kafka message to kafka cluster'
+ tags:
+ - Individual policy Object
+ requestBody:
+ required: true
+ content:
+ application/json:
+ schema:
+ "$ref": "#/components/schemas/A1PolicyObject"
+ responses:
+ 200:
+ description: 'Create or update operation dispatched'
+ 400:
+ "$ref": "#/components/responses/400-BadRequest"
+ 408:
+ "$ref": "#/components/responses/408-RequestTimeout"
+ 419:
+ "$ref": "#/components/responses/419-KafkaMessagePublishFailed"
+ 429:
+ "$ref": "#/components/responses/429-TooManyRequests"
+ 503:
+ "$ref": "#/components/responses/503-ServiceUnavailable"
+ 507:
+ "$ref": "#/components/responses/507-InsufficientStorage"
+ delete:
+ operationId: dispatcher.delete_policy
+ description: 'Dispatch policy delete opertion as kafka message to kafka cluster'
+ responses:
+ 200:
+ description: 'Delete operation dispatched'
+ 408:
+ "$ref": "#/components/responses/408-RequestTimeout"
+ 419:
+ "$ref": "#/components/responses/419-KafkaMessagePublishFailed"
+ 429:
+ "$ref": "#/components/responses/429-TooManyRequests"
+ 503:
+ "$ref": "#/components/responses/503-ServiceUnavailable"
+
+ '/policytypes/{policyTypeId}/kafkadispatcher/{policyId}/status':
+ parameters:
+ - name: policyTypeId
+ in: path
+ required: true
+ schema:
+ "$ref": "#/components/schemas/PolicyTypeId"
+ - name: policyId
+ in: path
+ required: true
+ schema:
+ "$ref": "#/components/schemas/A1PolicyId"
+ get:
+ operationId: dispatcher.get_policy_status
+ description: 'Dispatch policy status query opertion as kafka message to kafka cluster'
+ tags:
+ - Individual A1 Policy Status Object
+ responses:
+ 200:
+ description: 'Query policy status operation dispatched'
+ 429:
+ "$ref": "#/components/responses/429-TooManyRequests"
+ 503:
+ "$ref": "#/components/responses/503-ServiceUnavailable"
+
+components:
+ schemas:
+ #
+ # Representation objects
+ #
+ A1PolicyObject:
+ description: 'A generic policy object'
+ type: object
+
+ A1Policy:
+ description: 'A generic policy string'
+ type: string
+
+ PolicyTypeToTopicMap:
+ description: 'Request and response topic map for each policy type'
+ type: object
+ properties:
+ policy_type:
+ type: object
+ properties:
+ request_topic:
+ type: string
+ example: kafkatopicreq
+ response_topic:
+ type: string
+ example: kafkatopicres
+
+ ProblemDetails:
+ description: 'A problem detail to carry details in a HTTP response according to RFC 7807'
+ type: object
+ properties:
+ type:
+ type: string
+ title:
+ type: string
+ status:
+ type: number
+ detail:
+ type: string
+ instance:
+ type: string
+
+ #
+ # Simple data types
+ #
+ JsonSchema:
+ description: 'A JSON schema following http://json-schema.org/draft-07/schema'
+ type: object
+
+ A1PolicyId:
+ description: 'A1 policy identifier.'
+ type: string
+
+ PolicyTypeId:
+ description: 'Policy type identifier assigned by the A1-P Provider'
+ type: string
+
+ responses:
+ 400-BadRequest:
+ description: 'A1 policy not properly formulated or not related to the method'
+ content:
+ application/problem+json:
+ schema:
+ "$ref": "#/components/schemas/ProblemDetails"
+
+ 404-NotFound:
+ description: 'No resource found at the URI'
+ content:
+ application/problem+json:
+ schema:
+ "$ref": "#/components/schemas/ProblemDetails"
+
+ 405-MethodNotAllowed:
+ description: 'Method not allowed for the URI'
+ content:
+ application/problem+json:
+ schema:
+ "$ref": "#/components/schemas/ProblemDetails"
+
+ 408-RequestTimeout:
+ description: 'Request could not be processed in given amount of time'
+ content:
+ application/problem+json:
+ schema:
+ "$ref": "#/components/schemas/ProblemDetails"
+
+ 409-Conflict:
+ description: 'Request could not be processed in the current state of the resource'
+ content:
+ application/problem+json:
+ schema:
+ "$ref": "#/components/schemas/ProblemDetails"
+
+ 419-KafkaMessagePublishFailed:
+ description: 'Publishing the kafka message to the broker gets fail'
+ content:
+ application/problem+json:
+ schema:
+ "$ref": "#/components/schemas/ProblemDetails"
+
+ 429-TooManyRequests:
+ description: 'Too many requests have been sent in a given amount of time'
+ content:
+ application/problem+json:
+ schema:
+ "$ref": "#/components/schemas/ProblemDetails"
+
+ 503-ServiceUnavailable:
+ description: 'The provider is currently unable to handle the request due to a temporary overload'
+ content:
+ application/problem+json:
+ schema:
+ "$ref": "#/components/schemas/ProblemDetails"
+
+ 507-InsufficientStorage:
+ description: 'The method could not be performed on the resource because the provider is unable to store the representation needed to successfully complete the request'
+ content:
+ application/problem+json:
+ schema:
+ "$ref": "#/components/schemas/ProblemDetails"
--- /dev/null
+-----BEGIN CERTIFICATE-----
+MIID+zCCAuOgAwIBAgIUWy7JHRvA2GfOU5op4Xcc/8wQF18wDQYJKoZIhvcNAQEL
+BQAwgYwxCzAJBgNVBAYTAklFMRIwEAYDVQQIDAlXRVNUTUVBVEgxEDAOBgNVBAcM
+B0FUSExPTkUxETAPBgNVBAoMCEVyaWNzc29uMQwwCgYDVQQLDANFU1QxETAPBgNV
+BAMMCGVzdC50ZWNoMSMwIQYJKoZIhvcNAQkBFhRoYWxpbC5jYWthbEBlc3QudGVj
+aDAeFw0yMjA2MTcwOTEwMDNaFw00OTExMDEwOTEwMDNaMIGMMQswCQYDVQQGEwJJ
+RTESMBAGA1UECAwJV0VTVE1FQVRIMRAwDgYDVQQHDAdBVEhMT05FMREwDwYDVQQK
+DAhFcmljc3NvbjEMMAoGA1UECwwDRVNUMREwDwYDVQQDDAhlc3QudGVjaDEjMCEG
+CSqGSIb3DQEJARYUaGFsaWwuY2FrYWxAZXN0LnRlY2gwggEiMA0GCSqGSIb3DQEB
+AQUAA4IBDwAwggEKAoIBAQCqwDVZ7txWX/FaiRiSVa2jnBcV7KN6eqwcKtP3cNP+
+3VTm4YtcY6yp/dPXTYqkAX1qmp5i8USFPnbCstAijI5Uy8kl63dYbirHMPwt9AOL
+TXrFRrJ/sev3ULJWKB1IOGt2rFhoUXA23Hv1hagyvjx2upbnVmhrz5qBOT1wuzwN
+U2PjFaCFHBs0XphFS/UDEQlvpbNz/jxwHVrEdO8Jr951OFlUBczDDGk0jJ3hRc0p
+iM5LNGH02yDvE6pCqqY5Fo5aaj9Vi0Kztv1D/NClWcr3Yh3IuMyZkfS+S8nPd7Nu
+VHKWo7cPv9QpeziWiqx0fZcSAh6tUF52hrwGrMONuEojAgMBAAGjUzBRMB0GA1Ud
+DgQWBBRiX8NchgUa825PcmhA0b+BOnkx5TAfBgNVHSMEGDAWgBRiX8NchgUa825P
+cmhA0b+BOnkx5TAPBgNVHRMBAf8EBTADAQH/MA0GCSqGSIb3DQEBCwUAA4IBAQAi
+p6UMfLeTROQJNS4ZAkpQKEtrhGaE8DpnZW+tCLWaxkTk3Mo3ybE3rBW2zugmXFIU
+K2PRKivxHmVxY/YQL3sxzQRnzRPl0wuEmm+r0LpGR6VXfTaPrT6YzKM0TU/xKqJI
+INk6JiBx8v9dp5LWGoJEs0e4wvoV8Bd05u+73cbhIUisO7KmCp/u1D1BHYtBNDCp
+sVH6y9mAlvMIAOzy4wOWqoAxcW2ES8JbesbLOxt/IaQO9DQFPUIjTZURG+62lNCS
++2+lb1ihNqzEECuyw1GQRt88HrSWuj99bCBRRiBij900anadoKIDHdWSEkzhBa0K
+zJ5KoQK/o4szZ3VPByfh
+-----END CERTIFICATE-----
--- /dev/null
+#!/bin/bash
+
+# ============LICENSE_START===============================================
+# Copyright (C) 2022 Nordix Foundation. All rights reserved.
+# ========================================================================
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# ============LICENSE_END=================================================
+#
+
+# This will generate a self-signed certificate with password 'test'
+
+SUBJECT="/C=IE/ST=WESTMEATH/L=ATHLONE/O=Ericsson/OU=EST/CN=est.tech/emailAddress=halil.cakal@est.tech"
+PW=test
+echo $PW > pass
+
+openssl req -x509 -passout file:pass -newkey rsa:2048 -keyout key.crt -subj "$SUBJECT" -out cert.crt -days 9999
--- /dev/null
+-----BEGIN ENCRYPTED PRIVATE KEY-----
+MIIFHDBOBgkqhkiG9w0BBQ0wQTApBgkqhkiG9w0BBQwwHAQI/PWiKXnGNAMCAggA
+MAwGCCqGSIb3DQIJBQAwFAYIKoZIhvcNAwcECPdSVeuyJFIwBIIEyGmpW5lUwpKF
+tlcwBj2WvKF/7GJFw9LEuIBnkm0Ya8LV5Fu5XS9OGg877OkLVr1P152YjDQ+h1nD
+ZZSOEqjAGGSlEshie2DSr9bdJdBlr8MTog7SlHQsrU0QLxPRBBwqseI5cyu1Vb41
+4V5+l0iWUoDvXH3mUeH460A43GU4ZUTimkWpP9M9LHyrofjMg7sLzwvOns207DXK
+0hffc5293t85CQPau1WenHsfXn65tkPFblYILpJyU4sa1kLa4f5Ynt7RH2mNztfI
+26kViIuOBDXcmLAxL3WGZaR9u71qpl0wC1umXWxWNt69DRVyOf23mHHSuuDSEyM3
++x0rrbj/QaLNnoEjAFvijEAkYdp/jPKKP3kp6LpSVvVVOGLP7srrIrOe4q6bPFfK
+d5u1Vnh3/PVEf8xPbJe8UJ5cfx3mWhT+saZOIpEpQvom5GwSYt869P1xyAaa59cx
+TcT9KC/Ytg7fSHwXwTclIvucD+cJvbEZNFAwxMkL94a60LNfZ/odq1Bu304Shm/V
+DSNeDi1HjfoC3aca7bjsXE8Xj82JQLaSGt7+AuY3gICA5cnJxxWv5VoyRZVPsiRj
+Z6ykP2ikjkaLQaqDJmpbnx2ZK/lfrkJI1yI0kYK0xApUQdx9ks8c/AeEcUby2z+H
+qPJZuuh1NlEv8jSFn1CO3a5Bpq53EtQlxonKzJYHdoKm1oIEbIUE60K+1oxyXt9S
++l8PgH0T2QlM2lvipy5XegPrTuMYqDywEt4cf1Yk/8RSYEeLzcfKzSuNuy556YRd
+Of9nJOKPkVr1cp5si9Vyt+t4cD826WWV1ZgEcIK0i8uhEQxHb+y88DDWAV1DJGog
+M0qPGm95lWj4ESiv7A/+AbXY3rJRMp/JB1jmGTNfa3jQ9P29cTwhlpvPnhqviLRH
+1YfIOJfghrF55e83bLmJcfwVktMwO8Wzovw7U/1Rzy1j/fyUNkhOEbziIu97ScD+
+7AQDfRqBZZXWREOylAmfWMqZxwVn+CytAFyTLP5CpjSKgBV+qJ5xc3GqNJxaKGK5
+ULXfJKtTheJV3YYnvCuDySFuCv/dkUVjaA3/TqTikKm4THxS+DjtFJey+KWTx57p
+X+8ky/E3zAuZzP7r6Dszhp+CAfvXp4CwkLitqfbwja2lcxQ+hbpHYjLN6zvHmmOZ
+o0ZeoNpsWj00jAc9NrJt08DfcNomlUz9CZgpvE64gXX7wPCw5zG/c5dbFuoImEeI
+h+fKwGh2KdJwazvZ/B20/TRUn1WaQzyRiCJMIgc3aO3AiuffeurliO93iTeVCjJD
+d+Bdt4Ub0zPRikTqY4PUDiyfM+vRHOkJ39dY39XaZEsLLXB6XeACWcWT6iBF56Pe
+UAi+C4IwPLBHMsIVVSvH8/Rg0IFBHVmkGIi0q7gIyVTcR7FQeCzDZVAjJn4aBgXo
+S57v2AbMdD3pwie4ou91NkiM/SnSimLA0BxEh1UEhZk2BEW2Yy6OuvcLn3EfeZVu
+M9UNmwIFaq9jWM/qcRQa7MbXTZUTUIPsOvMOsZwPIurqtXWZZTWZ8D0eu2Hu6Vui
+FGyY1xIVcIpGXesADsYwy+CSW/EjpV1s+/LGDsUcqZpMeWmj84Zh1Tjt+Fet+967
+dSViTwISZ+O8F2uq0MaiPg==
+-----END ENCRYPTED PRIVATE KEY-----
--- /dev/null
+# user www-data;
+worker_processes auto;
+pid /run/nginx.pid;
+include /etc/nginx/modules-enabled/*.conf;
+
+env ALLOW_HTTP;
+
+events {
+ worker_connections 768;
+ # multi_accept on;
+}
+
+http {
+
+ ##
+ # Basic Settings
+ ##
+
+ sendfile on;
+ tcp_nopush on;
+ tcp_nodelay on;
+ keepalive_timeout 65;
+ types_hash_max_size 2048;
+ # server_tokens off;
+
+ # server_names_hash_bucket_size 64;
+ # server_name_in_redirect off;
+
+ include /etc/nginx/mime.types;
+ default_type application/octet-stream;
+
+ perl_set $allow_http 'sub { return $ENV{"ALLOW_HTTP"}; }';
+
+ server { # simple reverse-proxy
+ listen 7075;
+ listen [::]:7075;
+ server_name localhost;
+ if ($allow_http != true) {
+ return 444;
+ }
+
+ # serve dynamic requests
+ location / {
+ proxy_set_header Host $host;
+ proxy_set_header X-Real-IP $remote_addr;
+ proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+ proxy_pass http://localhost:7777;
+ }
+ }
+
+ server { # simple reverse-proxy
+ listen 7175 ssl;
+ listen [::]:7175 ssl;
+ server_name localhost;
+ ssl_certificate /usr/src/app/cert/cert.crt;
+ ssl_certificate_key /usr/src/app/cert/key.crt;
+ ssl_password_file /usr/src/app/cert/pass;
+
+ # serve dynamic requests
+ location / {
+ proxy_set_header Host $host;
+ proxy_set_header X-Real-IP $remote_addr;
+ proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+ proxy_pass http://localhost:7777;
+ }
+ }
+ ##
+ # SSL Settings
+ ##
+
+ ssl_protocols TLSv1 TLSv1.1 TLSv1.2; # Dropping SSLv3, ref: POODLE
+ ssl_prefer_server_ciphers on;
+
+ ##
+ # Logging Settings
+ ##
+
+ access_log /var/log/nginx/access.log;
+ error_log /var/log/nginx/error.log;
+
+ ##
+ # Gzip Settings
+ ##
+
+ gzip on;
+
+ # gzip_vary on;
+ # gzip_proxied any;
+ # gzip_comp_level 6;
+ # gzip_buffers 16 8k;
+ # gzip_http_version 1.1;
+ # gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript;
+}
--- /dev/null
+{
+ "ANR": {
+ "request_topic": "kafkatopicreq",
+ "response_topic": "kafkatopicres"
+ },
+ "STD_1": {
+ "request_topic": "kafkatopicreq2",
+ "response_topic": "kafkatopicres2"
+ },
+ "STD_2": {
+ "request_topic": "kafkatopicreq3",
+ "response_topic": "kafkatopicres3"
+ }
+}
--- /dev/null
+# ============LICENSE_START===============================================
+# Copyright (C) 2022 Nordix Foundation. All rights reserved.
+# ========================================================================
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# ============LICENSE_END=================================================
+#
+
+import os
+import json
+import time
+import math
+
+from flask import request, Response
+from datetime import datetime
+from kafka.consumer.fetcher import ConsumerRecord
+from kafka import TopicPartition
+from var_declaration import forced_settings
+from maincommon import create_kafka_producer, create_kafka_consumer, create_kafka_event, create_kafka_response_event, byte_to_str, get_random_string
+
+
+MSG_BROKER_URL=os.getenv('MSG_BROKER_URL')
+
+TIME_OUT=os.getenv('TIME_OUT')
+publishresponse=os.getenv('PUBLISH_RESP')
+
+#Constsants
+APPL_JSON='application/json'
+TEXT_PLAIN='text/plain'
+APPL_PROB_JSON='application/problem+json'
+
+# API Function: Dispatch create or update events to Kafka cluster
+def put_policy(policyTypeId, policyId):
+
+ if ((r := check_modified_response()) is not None):
+ return r
+
+ policy_type_id = str(policyTypeId)
+ policy_id = str(policyId)
+
+ try:
+ # Error based unit test rel only, for more info please check basic_test_with_cust_header
+ req_id_from_header = request.headers.get('requestid')
+ # Differentiate if the PUT is update or create operation since the endpoint is the same
+ update_oper_from_header = request.headers.get('updateoper')
+ data = request.data
+ data = json.loads(data)
+ except Exception:
+ pjson=create_problem_json(None, "The a1policy is corrupt or missing.", 400, None, policy_id)
+ return Response(json.dumps(pjson), 400, mimetype=APPL_PROB_JSON)
+
+ # Decide if the operation is update or create
+ if (update_oper_from_header is not None):
+ kafka_event = create_kafka_event(policy_type_id, policy_id, data, 'UPDATE')
+ else:
+ kafka_event = create_kafka_event(policy_type_id, policy_id, data, 'CREATE')
+
+ # Synch callout hooks towards kafka broker
+ if (MSG_BROKER_URL is not None):
+ return publish_and_consume(kafka_event, req_id_from_header, policy_type_id)
+
+ return Response('', 200, mimetype=TEXT_PLAIN)
+
+
+# API Function: Dispatch delete events to south Kafka cluster
+def delete_policy(policyTypeId, policyId):
+
+ if ((r := check_modified_response()) is not None):
+ return r
+
+ policy_type_id = str(policyTypeId)
+ policy_id = str(policyId)
+
+ req_id_from_header = request.headers.get('requestid')
+ print('req_id_from_header', req_id_from_header)
+
+ # Synch callout hooks towards kafka broker
+ kafka_event = create_kafka_event(policy_type_id, policy_id, None, 'DELETE')
+ if (MSG_BROKER_URL is not None):
+ return publish_and_consume(kafka_event, req_id_from_header, policy_type_id)
+
+ return Response('', 200, mimetype=TEXT_PLAIN)
+
+
+# API Function: Get status for a policy
+def get_policy_status(policyTypeId, policyId):
+
+ if ((r := check_modified_response()) is not None):
+ return r
+
+ policy_type_id=str(policyTypeId)
+ policy_id=str(policyId)
+
+ req_id_from_header = request.headers.get('requestid')
+ print('req_id_from_header', req_id_from_header)
+
+ # Synch callout hooks towards kafka broker
+ kafka_event = create_kafka_event(policy_type_id, policy_id, None, 'GET')
+ if (MSG_BROKER_URL is not None):
+ return publish_and_consume(kafka_event, req_id_from_header, policy_type_id)
+
+ return Response('', 200, mimetype=TEXT_PLAIN)
+
+
+def get_policy_type_to_topic_mapping(policyTypeId):
+
+ if ((r := check_modified_response()) is not None):
+ return r
+
+ policy_type_id = str(policyTypeId)
+
+ m_file = open('../resources/policytype_to_topicmap.json')
+ map_in_dict = json.load(m_file)
+
+ if policy_type_id in map_in_dict.keys():
+ topic_address = map_in_dict[policy_type_id]
+ return Response(json.dumps(topic_address), 200, mimetype=APPL_JSON)
+ else:
+ pjson=create_problem_json(None, "The policy type to topic mapping does not exist.", 404, None, policy_type_id)
+ return Response(json.dumps(pjson), 404, mimetype=APPL_PROB_JSON)
+
+
+# Helper: Publishes and consumes (to/from) the target broker and the topic in two-way synch
+def publish_and_consume(kafka_event, req_id_from_header, pol_type_id):
+
+ # Instantiate KafkaProducer with keyword arguments
+ producer = create_kafka_producer()
+
+ # Assigns an id to each request that is supposed to get a result
+ # if a req_id already exists in req headers, it means that test generated req_id is in use for testing only
+ if (req_id_from_header is None):
+ req_id = get_random_string(16)
+ else:
+ req_id = req_id_from_header
+
+ try:
+
+ resp = get_policy_type_to_topic_mapping(pol_type_id)
+ # if the policy type to topic mapping could not be found, then returns 404
+ # else gets target topic to publish the message to
+ if (resp.status_code == 404):
+ return resp
+ else:
+ data = json.loads(resp.data)
+ target_topic_req = data['request_topic']
+ target_topic_res = data['response_topic']
+
+ # synch-publish
+ # KafkaProducer.send(topicname, value=broker_message, key=req_id, headers=None, partition=None, timestamp_ms=None)
+ fut_rec_metadata = producer.send(target_topic_req, kafka_event, req_id)
+ record_metadata = fut_rec_metadata.get()
+ print('Future:', record_metadata)
+ publish_time_in_ms = record_metadata.timestamp
+
+ # For test purposes only triggered from A1 sim
+ # Publish the success response event with no error-info to response topic
+ # It is obvious that non of the requests will have a request id in the header except the test scripts: basic_test and timeout_test
+ if (publishresponse is not None and req_id_from_header is None):
+ kafka_response_event = create_kafka_response_event(200, "")
+ producer.send(target_topic_res, kafka_response_event, req_id)
+
+ # synch-consume
+ consumer_record = consume_record_for(req_id, target_topic_res)
+ if (isinstance(consumer_record, ConsumerRecord)):
+
+ print("Consumer Record:", consumer_record)
+ cons_rec_value = consumer_record.value
+ cons_rec_val_in_dict = json.loads(cons_rec_value)
+ resp_code = cons_rec_val_in_dict['response-code']
+
+ # if response code success, then check for time-out
+ if (int(resp_code) == 200):
+ # time-out control block, default time-out duration is thirty seconds
+ consume_time_in_ms = consumer_record.timestamp
+ elapsed_time_in_ms = consume_time_in_ms - publish_time_in_ms
+ print('Elapsed time in ms:', elapsed_time_in_ms)
+ if (elapsed_time_in_ms < int(TIME_OUT) * 1000):
+ return Response('', 200, mimetype=APPL_JSON)
+ else:
+ # returns time-out response code
+ pjson=create_error_response(408)
+ return Response(json.dumps(pjson), 408, mimetype=APPL_PROB_JSON)
+ else:
+ # for all other responses returns special error of this module by wrapping actual resp code
+ pjson=create_error_response(419)
+ return Response(json.dumps(pjson), 419, mimetype=APPL_PROB_JSON)
+
+ elif (isinstance(consumer_record, Response)):
+ # Returns time-out response
+ return consumer_record
+ else:
+ # returns special error of this module
+ pjson=create_error_response(419)
+ return Response(json.dumps(pjson), 419, mimetype=APPL_PROB_JSON)
+
+ except Exception as err:
+ print('Error while publish and consume', err)
+ pjson=create_error_response(419)
+ return Response(json.dumps(pjson), 419, mimetype=APPL_PROB_JSON)
+ finally:
+ producer.close()
+
+
+# Helper: Searches for req_id by seeking every five seconds up to thirty seconds
+# Helper: If the req_id is found, then ConsumerRecord will be returned
+# Helper: If the req_id is not found, then Response Request Timeout will be returned
+def consume_record_for(req_id, target_topic_res):
+
+ try:
+ print ('req_id looking for in consumer: ' + target_topic_res, req_id)
+ consumer = create_kafka_consumer()
+ topic_partition = TopicPartition(target_topic_res, 0)
+ consumer.assign([topic_partition])
+
+ sleep_period_in_sec = 5
+ poll_cycle_threshold = calc_pollcycle_threshold(sleep_period_in_sec)
+ poll_retries = 0
+
+ while (poll_retries < poll_cycle_threshold):
+ for consumer_record in consumer:
+ # Get req_id as msg_key and converts it from byte to str for each consumer record
+ msg_key = byte_to_str(consumer_record.key)
+ print ('msg_key in a consumer_record:', msg_key)
+ if (req_id == msg_key):
+ print ('req_id is found in consumer records', req_id)
+ return consumer_record
+
+ print('Sleeping for ' + str(sleep_period_in_sec) + ' seconds...')
+ time.sleep(sleep_period_in_sec)
+ poll_retries += 1
+
+ # Returns time-out response
+ pjson=create_error_response(408)
+ return Response(json.dumps(pjson), 408, mimetype=APPL_PROB_JSON)
+
+ except Exception as err:
+ print('Error while consume record for req_id', err)
+ pjson=create_error_response(419)
+ return Response(json.dumps(pjson), 419, mimetype=APPL_PROB_JSON)
+ finally:
+ consumer.close()
+
+# Helper: calculates poll cycle threshold
+def calc_pollcycle_threshold(sleep_period_in_sec):
+
+ poll_cycle_threshold = int(TIME_OUT) / sleep_period_in_sec
+ poll_cycle_threshold = math.floor(poll_cycle_threshold)
+ return poll_cycle_threshold
+
+# Helper: Create a response object if forced http response code is set
+def get_forced_response():
+
+ if (forced_settings['code'] is not None):
+ resp_code=forced_settings['code']
+ pjson=create_error_response(int(resp_code))
+ return Response(json.dumps(pjson), pjson['status'], mimetype=APPL_PROB_JSON)
+ return None
+
+
+# Helper: Delay if delayed response code is set
+def do_delay():
+
+ if (forced_settings['delay'] is not None):
+ try:
+ val=int(forced_settings['delay'])
+ time.sleep(val)
+ except Exception:
+ return
+
+
+# Helper: Check if response shall be delayed or a forced response shall be sent
+def check_modified_response():
+
+ do_delay()
+ return get_forced_response()
+
+
+# Helper: Create a problem json object
+def create_problem_json(type_of, title, status, detail, instance):
+
+ error = {}
+ if type_of is not None:
+ error["type"] = type_of
+ if title is not None:
+ error["title"] = title
+ if status is not None:
+ error["status"] = status
+ if detail is not None:
+ error["detail"] = detail
+ if instance is not None:
+ error["instance"] = instance
+ return error
+
+
+# Helper: Create a problem json based on a generic http response code
+def create_error_response(code):
+
+ if code == 400:
+ return(create_problem_json(None, "Bad request", 400, "Object in payload not properly formulated or not related to the method", None))
+ elif code == 404:
+ return(create_problem_json(None, "Not found", 404, "No resource found at the URI", None))
+ elif code == 405:
+ return(create_problem_json(None, "Method not allowed", 405, "Method not allowed for the URI", None))
+ elif code == 408:
+ return(create_problem_json(None, "Request timeout", 408, "Request timeout", None))
+ elif code == 409:
+ return(create_problem_json(None, "Conflict", 409, "Request could not be processed in the current state of the resource", None))
+ elif (code == 419):
+ return(create_problem_json(None, "Kafka message publish failed", 419, "Publishing the event could not be processed on the Kafka cluster", None))
+ elif code == 429:
+ return(create_problem_json(None, "Too many requests", 429, "Too many requests have been sent in a given amount of time", None))
+ elif code == 507:
+ return(create_problem_json(None, "Insufficient storage", 507, "The method could not be performed on the resource because the provider is unable to store the representation needed to successfully complete the request", None))
+ elif code == 503:
+ return(create_problem_json(None, "Service unavailable", 503, "The provider is currently unable to handle the request due to a temporary overload", None))
+ else:
+ return(create_problem_json(None, "Unknown", code, "Not implemented response code", None))
--- /dev/null
+# ============LICENSE_START===============================================
+# Copyright (C) 2022 Nordix Foundation. All rights reserved.
+# ========================================================================
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# ============LICENSE_END=================================================
+#
+
+import json
+import sys
+import requests
+
+
+from flask import request, Response, Flask, json
+from var_declaration import forced_settings, app
+from maincommon import check_timeout, check_apipath
+
+#Constants
+TEXT_PLAIN='text/plain'
+
+check_apipath()
+check_timeout()
+
+# app is created in var_declarations
+
+import payload_logging # app var need to be initialized
+
+#Check alive function
+@app.route('/', methods=['GET'])
+def test():
+ return Response("OK", 200, mimetype=TEXT_PLAIN)
+
+#Set|Reset force response to be returned from dispatcher
+#/dispatcheradmin/forceresponse?code=<responsecode>
+@app.route('/dispatcheradmin/forceresponse', methods=['POST'])
+def forceresponse():
+
+ query_param=request.args.get('code')
+ forced_settings['code']=query_param
+
+ if (query_param is None):
+ return Response("Force response code has been resetted for dispatcher responses", 200, mimetype=TEXT_PLAIN)
+ else:
+ return Response("Force response code: " + str(forced_settings['code']) + " set for all dispatcher response until it is resetted", 200, mimetype=TEXT_PLAIN)
+
+#Set|Reset force delay response, in seconds, for all external server responses
+#/a1policy/forcedelay?delay=<seconds>
+@app.route('/dispatcheradmin/forcedelay', methods=['POST'])
+def forcedelay():
+
+ query_param=request.args.get('delay')
+ forced_settings['delay']=query_param
+
+ if (query_param is None):
+ return Response("Force delay has been resetted for all dispatcher responses ", 200, mimetype=TEXT_PLAIN)
+ else:
+ return Response("Force delay: " + str(forced_settings['delay']) + " sec set for all dispatcher responses until it is resetted ", 200, mimetype=TEXT_PLAIN)
+
+port_number = 7777
+if len(sys.argv) >= 2:
+ if isinstance(sys.argv[1], int):
+ port_number = sys.argv[1]
+
+#Import base RestFUL API functions from Open API
+app.add_api('KAFKA_DISPATCHER_api.yaml')
+
+if __name__ == '__main__':
+ app.run(port=port_number, host="127.0.0.1", threaded=False)
--- /dev/null
+# ============LICENSE_START===============================================
+# Copyright (C) 2022 Nordix Foundation. All rights reserved.
+# ========================================================================
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# ============LICENSE_END=================================================
+#
+
+import os
+import sys
+import json
+from pathlib import Path
+from flask import Response
+import socket
+import ssl
+import random
+import string
+
+from kafka import KafkaProducer, KafkaConsumer
+
+#Must exist
+apipath=os.environ['APIPATH']
+timeout=os.getenv('TIME_OUT')
+
+MSG_BROKER_URL=os.getenv('MSG_BROKER_URL')
+
+
+# Make sure the api path is set, otherwise exit
+def check_apipath():
+ if (apipath is None):
+ print("Env APIPATH not set. Exiting....")
+ sys.exit(1)
+
+# Make sure the timeout is set and greater than zero, otherwise exit
+def check_timeout():
+ if (timeout is None):
+ print("Env TIME_OUT not set. Exiting....")
+ sys.exit(1)
+ elif (int(timeout) < 0):
+ print("Env TIME_OUT must be greater than zero. Exiting....")
+ sys.exit(1)
+
+# Instantiate KafkaProducer with keyword arguments
+# https://kafka-python.readthedocs.io/en/master/apidoc/KafkaProducer.html
+def create_kafka_producer():
+
+ producer = KafkaProducer(
+ bootstrap_servers = [MSG_BROKER_URL],
+ key_serializer = str.encode,
+ value_serializer = lambda m: json.dumps(m).encode('ascii'),
+ )
+ return producer
+
+
+# Instantiate KafkaConsumer with keyword arguments
+# https://kafka-python.readthedocs.io/en/master/apidoc/KafkaConsumer.html
+def create_kafka_consumer():
+ consumer = KafkaConsumer(
+ # kafka cluster endpoint
+ bootstrap_servers = MSG_BROKER_URL,
+ # move to the earliest or latest available message
+ auto_offset_reset = 'earliest',
+ # number of milliseconds to block during message iteration
+ # if no new message available during this period of time, iteration through a for-loop will stop automatically
+ consumer_timeout_ms = 100,
+ value_deserializer = lambda m: json.loads(m.decode('ascii')),
+ #enable_auto_commit=False
+ )
+ return consumer
+
+
+# Helper: Builds a Kafka event
+def create_kafka_event(policy_type_id, policy_id, payload, operation):
+
+ kafka_event_format = {'action': operation_to_action(operation), 'payload': payload, 'policy_type_id': policy_type_id, 'policy_id': policy_id}
+ # converts dict to str
+ kafka_event_json = json.dumps(kafka_event_format)
+ return kafka_event_json
+
+# Helper: Builds a Kafka event
+def create_kafka_response_event(response_code, error_info):
+
+ kafka_response_event_format = {'response-code': response_code, 'error-info': error_info}
+ # converts dict to str
+ kafka_response_event_json = json.dumps(kafka_response_event_format)
+ return kafka_response_event_json
+
+# Helper: Converts a HTTP operation to an explanation
+def operation_to_action(argument):
+
+ switcher = {
+ 'CREATE': "CreatePolicy",
+ 'UPDATE': "UpdatePolicy",
+ 'DELETE': "DeletePolicy",
+ 'GET': "GetPolicyStatus",
+ }
+ return switcher.get(argument, None)
+
+
+# Helper: Converts a byte array to a str
+def byte_to_str(byte_arr):
+
+ if (byte_arr is not None):
+ return byte_arr.decode('utf-8')
+ else:
+ return None
+
+
+# Helper: Creates random string
+def get_random_string(length):
+
+ characters = string.ascii_letters + string.digits + string.punctuation
+ password = ''.join(random.choice(characters) for i in range(length))
+ return password
--- /dev/null
+# ============LICENSE_START===============================================
+# Copyright (C) 2022 Nordix Foundation. All rights reserved.
+# ========================================================================
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# ============LICENSE_END=================================================
+#
+
+from var_declaration import app
+from flask import Flask, request, Response
+
+#Constants
+TEXT_PLAIN='text/plain'
+
+#Vars
+payload_log=True
+
+#Function to activate/deactivate http header and payload logging
+@app.route('/payload_logging/<state>', methods=['POST', 'PUT'])
+def set_payload_logging(state):
+ global payload_log
+ if (state == "on"):
+ payload_log=True
+ elif (state == "off"):
+ payload_log=False
+ else:
+ return Response("Unknown state: "+state+" - use 'on' or 'off'", 400, mimetype=TEXT_PLAIN)
+
+ return Response("Payload and header logging set to: "+state, 200, mimetype=TEXT_PLAIN)
+
+# Generic function to log http header and payload - called before the request
+@app.app.before_request
+def log_request_info():
+ if (payload_log is True):
+ print('')
+ print('-----Request-----')
+ print('Req Headers: ', request.headers)
+ print('Req Body: ', request.get_data())
+
+# Generic function to log http header and payload - called after the response
+@app.app.after_request
+def log_response_info(response):
+ if (payload_log is True):
+ print('-----Response-----')
+ print('Resp Headers: ', response.headers)
+ print('Resp Body: ', response.get_data())
+ return response
+
+# Helper function to check loggin state
+def is_payload_logging():
+ return payload_log
--- /dev/null
+#!/bin/bash
+
+# ============LICENSE_START===============================================
+# Copyright (C) 2022 Nordix Foundation. All rights reserved.
+# ========================================================================
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# ============LICENSE_END=================================================
+#
+
+#Set path to open api
+export APIPATH=$PWD/api
+echo "APIPATH set to: "$APIPATH
+
+cd src
+
+#start nginx
+nginx -c /usr/src/app/nginx.conf
+
+#start Kafka message dispatcher
+echo "Path to main.py: "$PWD
+python -u main.py
--- /dev/null
+# ============LICENSE_START===============================================
+# Copyright (C) 2022 Nordix Foundation. All rights reserved.
+# ========================================================================
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# ============LICENSE_END=================================================
+#
+
+from maincommon import apipath
+import connexion
+
+#Main app
+app = connexion.App(__name__, specification_dir=apipath)
+
+forced_settings = {}
+forced_settings['code']=None
+forced_settings['delay']=None
--- /dev/null
+#!/bin/bash
+
+# ============LICENSE_START===============================================
+# Copyright (C) 2022 Nordix Foundation. All rights reserved.
+# ========================================================================
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# ============LICENSE_END=================================================
+#
+
+# Script for basic test of the Kafka message dispatcher.
+# Run the build_and_start with the same arg, except arg 'nonsecure|secure', as this script
+
+print_usage() {
+ echo "Usage: ./basic_test.sh nonsecure|secure "
+ exit 1
+}
+
+if [ $# -ne 1 ]; then
+ print_usage
+fi
+if [ "$1" != "nonsecure" ] && [ "$1" != "secure" ]; then
+ print_usage
+fi
+
+if [ $1 == "nonsecure" ]; then
+ #Default http port for the simulator
+ PORT=7075
+ # Set http protocol
+ HTTPX="http"
+else
+ #Default https port for the simulator
+ PORT=7175
+ # Set https protocol
+ HTTPX="https"
+fi
+
+. ../common/test_common.sh
+. ../common/elapse_time_curl.sh
+
+echo "=== Kafka message dispatcher hello world ==="
+RESULT="OK"
+do_curl GET / 200
+
+echo "=== Reset force delay ==="
+RESULT="Force delay has been resetted for all dispatcher responses"
+do_curl POST /dispatcheradmin/forcedelay 200
+
+echo "=== API: Get policy type to topic mapping of type: ANR ==="
+res=$(cat jsonfiles/ANR_to_topic_map.json)
+RESULT="json:$res"
+do_curl GET /policytypetotopicmapping/ANR 200
+
+echo "=== Put policy: shall publish and consume for put policy operation ==="
+req_id=$(get_random_number)
+RESULT=""
+do_curl PUT /policytypes/ANR/kafkadispatcher/alpha 200 jsonfiles/alpha_policy.json $req_id &
+proc_id=$!
+publish_response_event $req_id kafkatopicres
+wait $proc_id
+
+echo "=== Get policy status: shall publish and consume for get policy status operation ==="
+req_id=$(get_random_number)
+RESULT=""
+do_curl GET /policytypes/ANR/kafkadispatcher/alpha/status 200 jsonfiles/alpha_policy.json $req_id &
+proc_id=$!
+publish_response_event $req_id kafkatopicres
+wait $proc_id
+
+echo "=== Put policy: shall publish and consume for put policy operation for alpha ==="
+req_id=$(get_random_number)
+RESULT=""
+do_curl PUT /policytypes/STD_1/kafkadispatcher/alpha 200 jsonfiles/alpha_policy.json $req_id &
+proc_id=$!
+publish_response_event $req_id kafkatopicres2
+wait $proc_id
+
+echo "=== Delete policy: shall publish and consume for delete policy operation for alpha ==="
+req_id=$(get_random_number)
+RESULT=""
+do_curl DELETE /policytypes/STD_1/kafkadispatcher/alpha 200 jsonfiles/alpha_policy.json $req_id &
+proc_id=$!
+publish_response_event $req_id kafkatopicres2
+wait $proc_id
+
+echo "=== Set force delay 5 sec ==="
+RESULT="Force delay: 5 sec set for all dispatcher responses until it is resetted"
+do_curl POST '/dispatcheradmin/forcedelay?delay=5' 200
+
+echo "=== Hello world: shall wait at least <delay-time> sec and then respond while hello world ==="
+RESULT="OK"
+do_elapsetime_curl GET / 200 jsonfiles/alpha_policy.json 5
+
+echo "=== Reset force delay ==="
+RESULT="Force delay has been resetted for all dispatcher responses"
+do_curl POST /dispatcheradmin/forcedelay 200
+
+echo "=== Put policy: shall publish and consume for put policy operation for beta ==="
+req_id=$(get_random_number)
+RESULT=""
+do_curl PUT /policytypes/STD_1/kafkadispatcher/beta 200 jsonfiles/beta_policy.json $req_id &
+proc_id=$!
+publish_response_event $req_id kafkatopicres2
+wait $proc_id
+
+echo "=== Get policy status: shall publish and consume for get policy status operation ==="
+req_id=$(get_random_number)
+RESULT=""
+do_curl GET /policytypes/ANR/kafkadispatcher/alpha/status 200 jsonfiles/beta_policy.json $req_id &
+proc_id=$!
+publish_response_event $req_id kafkatopicres
+wait $proc_id
+
+echo "=== Put policy: shall publish and consume for put policy operation for alpha ==="
+req_id=$(get_random_number)
+RESULT=""
+do_curl PUT /policytypes/STD_2/kafkadispatcher/alpha 200 jsonfiles/alpha_policy.json $req_id &
+proc_id=$!
+publish_response_event $req_id kafkatopicres3
+wait $proc_id
+
+echo "=== Set force response code: 500 ==="
+RESULT="Force response code: 500 set for all dispatcher response until it is resetted"
+do_curl POST '/dispatcheradmin/forceresponse?code=500' 200
+
+echo "=== Put policy: shall not publish and consume for put policy operation for alpha ==="
+req_id=$(get_random_number)
+res=$(cat jsonfiles/forced_response.json)
+RESULT="json:$res"
+do_curl PUT /policytypes/ANR/kafkadispatcher/alpha 500 jsonfiles/alpha_policy.json $req_id &
+proc_id=$!
+publish_response_event $req_id kafkatopicres
+wait $proc_id
+
+echo "=== Reset force response code ==="
+RESULT="Force response code has been resetted for dispatcher responses"
+do_curl POST /dispatcheradmin/forceresponse 200
+
+echo "=== Get policy status: shall publish and consume for get policy status operation ==="
+req_id=$(get_random_number)
+RESULT=""
+do_curl GET /policytypes/ANR/kafkadispatcher/alpha/status 200 jsonfiles/alpha_policy.json $req_id &
+proc_id=$!
+publish_response_event $req_id kafkatopicres
+wait $proc_id
+
+echo "=== Delete policy: shall publish and consume for delete policy operation for alpha ==="
+req_id=$(get_random_number)
+RESULT=""
+do_curl DELETE /policytypes/STD_1/kafkadispatcher/alpha 200 jsonfiles/alpha_policy.json $req_id &
+proc_id=$!
+publish_response_event $req_id kafkatopicres2
+wait $proc_id
+
+echo "********************"
+echo "*** All tests ok ***"
+echo "********************"
--- /dev/null
+#!/bin/bash
+
+# ============LICENSE_START===============================================
+# Copyright (C) 2022 Nordix Foundation. All rights reserved.
+# ========================================================================
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# ============LICENSE_END=================================================
+#
+
+# Script to build and start the kafka dispatcher container
+# Make sure to run the simulator including args as is this script
+
+print_usage() {
+ echo "Usage: ./build_and_start.sh publish-resp|ignore-publish"
+ exit 1
+}
+
+if [ $# -ne 1 ]; then
+ print_usage
+fi
+
+if [ $1 == "publish-resp" ]; then
+ PUBLISH_RESP="-e PUBLISH_RESP=1"
+elif [ $1 == "ignore-publish" ]; then
+ PUBLISH_RESP=""
+else
+ print_usage
+fi
+
+echo "Building Kafka message dispatcher image..."
+cd ../KAFKA_DISPATCHER/
+
+#Build the image
+docker build -t kafka_dispatcher .
+
+docker stop kafkamessagedispatcher > /dev/null 2>&1
+docker rm -f kafkamessagedispatcher > /dev/null 2>&1
+
+echo "Starting Kafka message dispatcher..."
+echo "PWD path: "$PWD
+
+#Run the container in interactive mode with host networking driver which allows docker to access localhost, unsecure port 7075, secure port 7175, TIME_OUT must be in seconds, PUBLISH_RESP decides auto responding for testing that run by A1 sim
+docker run --network host --rm -it -p 7075:7075 -p 7175:7175 -e ALLOW_HTTP=true -e MSG_BROKER_URL=localhost:9092 -e TIME_OUT=30 $PUBLISH_RESP --volume "$PWD/certificate:/usr/src/app/cert" --name kafkamessagedispatcher kafka_dispatcher
--- /dev/null
+{
+ "request_topic": "kafkatopicreq",
+ "response_topic": "kafkatopicres"
+}
--- /dev/null
+{
+
+ "title": "A1 policy external server",
+ "description": "A1 policies notifying external server",
+ "type": "object",
+ "properties": {
+ "a1policyType": "alpha_test_policy",
+ "url" : "http://www.com"
+ }
+
+}
--- /dev/null
+{
+
+ "title": "A1 policy external server",
+ "description": "A1 policies notifying external server",
+ "type": "object",
+ "properties": {
+ "a1policyType": "beta_test_policy",
+ "url" : "http://www.com"
+ }
+
+}
--- /dev/null
+{
+ "title": "Unknown",
+ "status": 500,
+ "detail": "Not implemented response code"
+}
--- /dev/null
+{
+ "title": "Request timeout",
+ "status": 408,
+ "detail": "Request timeout"
+}
--- /dev/null
+#!/bin/bash
+
+# ============LICENSE_START===============================================
+# Copyright (C) 2022 Nordix Foundation. All rights reserved.
+# ========================================================================
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# ============LICENSE_END=================================================
+#
+
+# Script for error testing of the Kafka message dispatcher
+# The timeout value should be equal to TIME_OUT param that exist in the start script
+# Run the script with the args: nonsecure|secure timeout=30
+
+print_usage() {
+ echo "Usage: ./basic_test.sh nonsecure|secure timeout=30"
+ exit 1
+}
+
+if [ $# -ne 2 ]; then
+ print_usage
+fi
+if [ "$1" != "nonsecure" ] && [ "$1" != "secure" ]; then
+ print_usage
+fi
+
+timeout=$(echo "$2" | cut -d'=' -f2)
+regexp_for_number='^[0-9]+$'
+
+if ! [[ $timeout =~ $regexp_for_number ]] ; then
+ echo "error:"$timeout" Not a number"
+ exit 1
+else
+ if [ $timeout -le 0 ]; then
+ echo "Timeout value must be greater than zero"
+ exit 1
+ fi
+fi
+
+if [ $1 == "nonsecure" ]; then
+ # Default http port for the simulator
+ PORT=7075
+ # Set http protocol
+ HTTPX="http"
+else
+ #Default https port for the simulator
+ PORT=7175
+ # Set https protocol
+ HTTPX="https"
+fi
+
+. ../common/test_common.sh
+
+echo "=== Kafka message dispatcher hello world ==="
+RESULT="OK"
+do_curl GET / 200
+
+echo "=== Reset force delay ==="
+RESULT="Force delay has been resetted for all dispatcher responses"
+do_curl POST /dispatcheradmin/forcedelay 200
+
+# asynch error test case
+echo "=== Put policy: shall publish and consume time-out ==="
+req_id=$(get_random_number)
+res=$(cat jsonfiles/timeout_response.json)
+RESULT="json:$res"
+# asynch callout
+do_curl PUT /policytypes/ANR/kafkadispatcher/alpha 408 jsonfiles/alpha_policy.json $req_id &
+proc_id=$!
+sleep $timeout
+# after time out duration, publish the event
+publish_response_event $req_id kafkatopicres
+# wait until the main process to be completed
+wait $proc_id
+
+# asynch success test case after 10s
+echo "=== Put policy: shall publish and consume success at least 10 secs later ==="
+req_id=$(get_random_number)
+RESULT=""
+# asynch callout
+do_curl PUT /policytypes/STD_1/kafkadispatcher/alpha 200 jsonfiles/alpha_policy.json $req_id &
+proc_id=$!
+sleep 10
+# after 10s, publish the event
+publish_response_event $req_id kafkatopicres2
+# wait until the main process to be completed
+wait $proc_id
+
+# asynch error test case
+echo "=== Get policy status: shall publish and consume time-out ==="
+req_id=$(get_random_number)
+res=$(cat jsonfiles/timeout_response.json)
+RESULT="json:$res"
+# asynch callout
+do_curl GET /policytypes/STD_2/kafkadispatcher/alpha/status 408 jsonfiles/alpha_policy.json $req_id &
+proc_id=$!
+sleep $timeout
+# after time out duration, publish the event
+publish_response_event $req_id kafkatopicres3
+# wait until the main process to be completed
+wait $proc_id
+
+# asynch success test case after 10s
+echo "=== Get policy status: shall publish and consume success at least 15 secs later ==="
+req_id=$(get_random_number)
+RESULT=""
+# asynch callout
+do_curl GET /policytypes/ANR/kafkadispatcher/alpha/status 200 jsonfiles/alpha_policy.json $req_id &
+proc_id=$!
+sleep 15
+# after 15s, publish the event
+publish_response_event $req_id kafkatopicres
+# wait until the main process to be completed
+wait $proc_id
+
+# asynch success test case without any delay
+echo "=== Delete policy: shall publish and consume success ==="
+req_id=$(get_random_number)
+RESULT=""
+# asynch callout
+do_curl DELETE /policytypes/STD_1/kafkadispatcher/alpha 200 jsonfiles/alpha_policy.json $req_id &
+proc_id=$!
+publish_response_event $req_id kafkatopicres2
+# wait until the main process to be completed
+wait $proc_id
+
+
+echo "********************"
+echo "*** All tests ok ***"
+echo "********************"
--- /dev/null
+#!/bin/bash
+
+# ============LICENSE_START===============================================
+# Copyright (C) 2022 Nordix Foundation. All rights reserved.
+# ========================================================================
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# ============LICENSE_END=================================================
+#
+
+# Script to build and start the container
+# Make sure to run the simulator with the same arg as this script
+
+print_usage() {
+ echo "Usage: ./build_and_start.sh duplicate-check|ignore-duplicate kafka-srv|kafka-srv-secure publish-resp|ignore-publish"
+ exit 1
+}
+
+if [ $# -ne 3 ]; then
+ print_usage
+fi
+
+if [ $1 == "duplicate-check" ]; then
+ DUP_CHECK=1
+elif [ $1 == "ignore-duplicate" ]; then
+ DUP_CHECK=0
+else
+ print_usage
+fi
+
+if [ $2 == "kafka-srv" ]; then
+ URL="http://localhost:7075"
+elif [ $2 == "kafka-srv-secure" ]; then
+ URL="https://localhost:7175"
+else
+ print_usage
+fi
+
+if [ $3 == "publish-resp" ]; then
+ PUBLISH_RESP="-e PUBLISH_RESP=1"
+elif [ $3 == "ignore-publish" ]; then
+ PUBLISH_RESP=""
+else
+ print_usage
+fi
+
+URL_FLAG=""
+if [ ! -z "$URL" ]; then
+ URL_FLAG="-e KAFKA_DISPATCHER_URL=$URL"
+fi
+
+# Stop and remove container images if they run
+
+echo "Stopping A1 simulator image..."
+docker stop a1StdSimulator > /dev/null 2>&1
+docker rm -f a1StdSimulator > /dev/null 2>&1
+
+echo "Stopping kafka dispatcher server image..."
+docker stop kafkamessagedispatcher > /dev/null 2>&1
+docker rm -f kafkamessagedispatcher > /dev/null 2>&1
+
+# Initialize path variables for certificate and build operations
+
+dirstd2=$PWD
+
+cd ../../
+dirnrtsim=$PWD
+
+cd test/KAFKA_DISPATCHER/
+dirkafkasrv=$PWD
+
+# Build containers
+
+cd $dirnrtsim
+echo "Building A1 simulator image..."
+docker build -t a1test .
+
+if [ ! -z "$URL" ]; then
+ cd $dirkafkasrv
+ echo "Building kafka server image..."
+ docker build -t kafka_dispatcher .
+fi
+
+# Run containers
+
+# Runs kafka server in detached mode
+# In order to tail logs use:: docker logs -f kafkamessagedispatcher
+if [ ! -z "$URL" ]; then
+ docker run -d --network host --rm -it -p 7075:7075 -p 7175:7175 -e ALLOW_HTTP=true -e MSG_BROKER_URL=localhost:9092 -e TIME_OUT=30 $PUBLISH_RESP --volume "$dirkafkasrv/certificate:/usr/src/app/cert" --name kafkamessagedispatcher kafka_dispatcher
+fi
+
+# Runs A1 simulator
+docker run --network host --rm -it -p 8085:8085 -p 8185:8185 -e A1_VERSION=STD_2.0.0 -e ALLOW_HTTP=true -e REMOTE_HOSTS_LOGGING=1 -e DUPLICATE_CHECK=$DUP_CHECK $URL_FLAG --volume "$dirnrtsim/certificate:/usr/src/app/cert" --name a1StdSimulator a1test
--- /dev/null
+# ============LICENSE_START===============================================
+# Copyright (C) 2022 Nordix Foundation. All rights reserved.
+# ========================================================================
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# ============LICENSE_END=================================================
+#
+
+# This is a script for test-purposes only
+# It consumes a response-event from a kafka bus with different apporaches
+# In order to use this script, you must have an venv for Python and kafka-python libs has to be installed
+# To instal kafka-python please use: pip install kafka-python
+# Example of an response-event json
+#{
+ #"response-code": "400",
+ #"error-info": "Bad format"
+#}
+
+
+import os
+import json
+import sys
+import math
+import time
+
+from kafka import KafkaConsumer, TopicPartition
+from threading import RLock
+
+# Response string with JSON format
+response_data_JSON = """
+{
+ "response-code": 200,
+ "error-info": ""
+}
+"""
+
+# in seconds
+TIME_OUT=30
+target_topic_res='kafkatopicres'
+MSG_BROKER_URL='localhost:9092'
+
+# Instantiate KafkaConsumer with keyword arguments
+# https://kafka-python.readthedocs.io/en/master/apidoc/KafkaConsumer.html
+def create_kafka_consumer():
+ consumer = KafkaConsumer(
+ # kafka cluster endpoint
+ bootstrap_servers = MSG_BROKER_URL,
+ # move to the earliest or latest available message
+ auto_offset_reset = 'earliest',
+ # number of milliseconds to block during message iteration
+ # if no new message available during this period of time, iteration through a for-loop will stop automatically
+ consumer_timeout_ms = 100,
+ value_deserializer = lambda m: json.loads(m.decode('ascii')),
+ #enable_auto_commit=False
+ )
+ return consumer
+
+# Helper: Searches for req_id by seeking every five seconds up to thirty seconds
+# Helper: If the req_id is found, then ConsumerRecord will be returned
+# Helper: If the req_id is not found, then Response Request Timeout will be returned
+def consume(req_id):
+
+ try:
+ print ('req_id looking for in consumer:', req_id)
+ consumer = create_kafka_consumer()
+ # Latch to target topic and partition
+ topic_partition = TopicPartition(target_topic_res, 0)
+ consumer.assign([topic_partition])
+
+ sleep_period_in_sec = 5
+ poll_cycle_threshold = calc_pollcycle_threshold(sleep_period_in_sec)
+ poll_retries = 0
+
+ while (poll_retries < poll_cycle_threshold):
+ for consumer_record in consumer:
+ # Get req_id as msg_key and converts it from byte to str for each consumer record
+ msg_key = byte_to_str(consumer_record.key)
+ print ('msg_key in a consumer_record:', msg_key)
+ if (req_id == msg_key):
+ print ('req_id is found in consumer records', req_id)
+ return consumer_record
+
+ print('Sleeping for ' + str(sleep_period_in_sec) + ' seconds...')
+ time.sleep(sleep_period_in_sec)
+ poll_retries += 1
+
+ return 1
+ except Exception as err:
+ print('Error while consume record for req_id', err)
+ return 1
+ finally:
+ consumer.close()
+
+# Helper: calculates poll cycle threshold
+def calc_pollcycle_threshold(sleep_period_in_sec):
+
+ poll_cycle_threshold = int(TIME_OUT) / sleep_period_in_sec
+ poll_cycle_threshold = math.floor(poll_cycle_threshold)
+ return poll_cycle_threshold
+
+# Helper: Converts a byte array to a str
+def byte_to_str(byte_arr):
+
+ if (byte_arr is not None):
+ return byte_arr.decode('utf-8')
+ else:
+ return None
+
+if __name__ == '__main__':
+ try:
+ requestid = sys.argv[1]
+ future = consume(requestid)
+ except Exception as err:
+ print('Error in __main__', err)
+ print (1)
+ sys.exit()
--- /dev/null
+# ============LICENSE_START===============================================
+# Copyright (C) 2022 Nordix Foundation. All rights reserved.
+# ========================================================================
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# ============LICENSE_END=================================================
+#
+
+# This script publishes a response-event to a kafka bus
+# In order to use this script, you must have an venv for Python and kafka-python libs has to be installed
+# To instal kafka-python please use: pip install kafka-python
+# Example of an response-event json
+#{
+ #"response-code": "400",
+ #"error-info": "Bad format"
+#}
+
+
+import os
+import json
+import sys
+
+from kafka import KafkaProducer
+
+# Response string with JSON format
+response_data_JSON = """
+{
+ "response-code": 200,
+ "error-info": ""
+}
+"""
+
+# Instantiate KafkaProducer with keyword arguments
+# https://kafka-python.readthedocs.io/en/master/apidoc/KafkaProducer.html
+def create_kafka_producer():
+
+ producer = KafkaProducer(
+ bootstrap_servers = ['localhost:9092'],
+ key_serializer = str.encode,
+ value_serializer = lambda m: json.dumps(m).encode('ascii'),
+ )
+ return producer
+
+# Helper: Publishes (to) the target broker and the topic in synch
+def publish(kafka_evet, req_id, targettopic):
+
+ # Instantiate KafkaProducer with keyword arguments
+ producer = create_kafka_producer()
+ # Assigns an id to each request that is supposed to get a result
+ # req_id = 'Hll1EsycKLNRric7'
+
+ try:
+
+ # synch-publish
+ # KafkaProducer.send(topicname, value=broker_message, key=req_id, headers=None, partition=None, timestamp_ms=None)
+ fut_rec_metadata = producer.send(targettopic, kafka_evet, req_id)
+ return fut_rec_metadata.get()
+
+ except Exception as err:
+ print('Error while publish', err)
+ finally:
+ producer.close()
+
+if __name__ == '__main__':
+ try:
+
+ requestid = sys.argv[1]
+ targettopic = sys.argv[2]
+ # response_data_JSON is str
+ future = publish(response_data_JSON, requestid, targettopic)
+
+ if (future is not None):
+ print (0)
+ else:
+ print (1)
+
+ except Exception:
+ print (1)
+ sys.exit()
#!/bin/bash
# ============LICENSE_START===============================================
-# Copyright (C) 2020 Nordix Foundation. All rights reserved.
+# Copyright (C) 2020-2022 Nordix Foundation. All rights reserved.
# ========================================================================
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
#Expects the env $RESULT to contain the expected RESULT.
#If json, the RESULT shall begin with 'json:'.
#Any json parameter with unknown value shall be given as "????" to skip checking the value.
+#The requestid parameter is being introduced in the fifth order.
do_curl() {
if [ $# -lt 3 ]; then
echo "Need 3 or more parameters, <http-operation> <url> <response-code> [file]: "$@
exit 1
fi
curlstr="curl -X "$1" -skw %{http_code} $HTTPX://localhost:"${PORT}${2}" -H accept:*/*"
- if [ $# -gt 3 ]; then
+ if [ $# -eq 4 ]; then
curlstr=$curlstr" -H Content-Type:application/json --data-binary @"$4
fi
+ if [ $# -ge 5 ]; then
+ curlstr=$curlstr" -H Content-Type:application/json --data-binary @"$4" -H requestid:"$5
+ fi
echo " CMD (${BASH_LINENO[0]}):"$curlstr
res=$($curlstr)
status=${res:${#res}-3}
fi
}
+# Triggers publish_event_to_kafka_bus.py script to send msg to Kafka broker
+# The aim of this function is to realize error related test cases only
+# The request_id for the Kafka msg, should be passed here as a function parameter
+publish_response_event() {
+ if [ $# -ne 2 ]; then
+ echo "Need 2 parameter, <request_id> <target_topic>"
+ echo "Exiting test script....."
+ exit 1
+ fi
+ res=$(python ../common/publish_response_event_to_kafka_bus.py "$1" "$2")
+ if [ $res -eq 0 ]; then
+ echo " Result as expected "
+ else
+ echo " Result not expected "
+ echo " Exiting..... "
+ exit 1
+ fi
+}
+
+# Creates 16 digits random number using letters and numbers only
+get_random_number() {
+ r_num=$(tr -dc A-Za-z0-9 < /dev/urandom | head -c 16)
+ echo $r_num
+}
+
+# It is being used to cross-test-cases in between A1 sim and external server
+# The parameter it holds all with regards to External Server relates e.g. HTTPX_EXT_SRV and PORT_EXT_SRV
do_curl_ext_srv() {
if [ $# -lt 3 ]; then
echo "Need 3 or more parameters, <http-operation> <url> <response-code> [file]: "$@
# ==================================================================================
-# Copyright (c) 2020 Nordix
+# Copyright (c) 2020-2022 Nordix
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# doc jobs
[testenv:docs]
whitelist_externals = echo
-basepython = python3
-deps =
- sphinx
- sphinx-rtd-theme
- sphinxcontrib-httpdomain
- recommonmark
- lfdocs-conf
+basepython = python3.8
+deps = -r{toxinidir}/docs/requirements-docs.txt
commands =
sphinx-build -W -b html -n -d {envtmpdir}/doctrees ./docs/ {toxinidir}/docs/_build/html
echo "Generated docs available in {toxinidir}/docs/_build/html"
[testenv:docs-linkcheck]
skipsdist = true
-basepython = python3
-deps = sphinx
- sphinx-rtd-theme
- sphinxcontrib-httpdomain
- recommonmark
- lfdocs-conf
-commands = sphinx-build -W -b linkcheck -d {envtmpdir}/doctrees ./docs/ {toxinidir}/docs/_build/linkcheck
+basepython = python3.8
+deps = -r{toxinidir}/docs/requirements-docs.txt
+commands =
+ sphinx-build -W -b linkcheck -d {envtmpdir}/doctrees ./docs/ {toxinidir}/docs/_build/linkcheck
Code Review / sim / a1-interface.git / commitdiff
