+ Delete procedure cannot fail from xApp point of view. Subscription Manager always responds with successful REST Subscription Delete Response to xApp.
+
+ * Timeout in Subscription Manager
+
+ Timeout can only happen in route delete to Routing manager. Subscription Manager always responds with successful REST Subscription Delete Response to xApp.
+
+ * Timeout in xApp
+
+ xApp can resend the same REST Delete Request if request timeouts.
+
+ * xApp restart
+
+ When xApp is restarted for any reason it may resend REST subscription requests for subscriptions which have already been subscribed. If REPORT or INSERT type
+ subscription already exists and RMR endpoint of requesting xApp is attached to subscription then successful response is sent to xApp directly without
+ updating Routing Manager and E2 Node. If POLICY type subscription already exists, request is forwarded to E2 Node and successful response is sent to xApp.
+ E2 Node is expected to accept duplicate POLICY type requests. In restart IP address of the xApp may change but domain service address name does not.
+ RMR message routing uses domain service address name.
+
+ * Subscription Manager restart
+
+ Subscription Manager stores REST request ids, E2 subscriptions and their mapping to REST request ids in db (SDL). In start up Subscription Manager restores REST request
+ ids, E2 subscriptions and their mapping from db. For E2 subscriptions which were not successfully completed, Subscription Manager sends delete request to E2 Node and
+ removes routes created for those. In restart case xApp may need to resend the same REST request to get all E2 subscriptions completed.
+
+ Restoring subscriptions from db can be disabled via submgr-config.yaml file by setting "readSubsFromDb": "false".
+
+ * E2 connection break
+
+ Subscription Manager subscribes E2 connection status notifications from RNIB. Whenever E2 interface goes up or down Subscription Manager gets notifies. When interface is down
+ subscription is not possible. Subscription Manager rejects new request for the E2 node. Http Reject cause is 503 Subscribe Service Unavailable. When interface goes down
+ Subscription Manager deletes all subscriptions related to the RanName from its memory and database. E2 node and XApp are expected to do the same.
+
+Metrics
+-------
+ Subscription Manager adds following statistic counters:
+
+ Subscription create counters:
+ - SubReqFromXapp: The total number of SubscriptionRequest messages received from xApp
+ - SubRespToXapp: The total number of SubscriptionResponse messages sent to xApp
+ - SubFailToXapp: The total number of SubscriptionFailure messages sent to xApp
+ - RestSubReqFromXapp: The total number of Rest SubscriptionRequest messages received from xApp,
+ - RestSubRespToXapp: The total number of Rest SubscriptionResponse messages sent to xApp,
+ - RestSubFailToXapp: The total number of Rest SubscriptionFailure messages sent to xApp
+ - RestReqRejDueE2Down: The total number of Rest SubscriptionRequest messages rejected due E2 Interface down
+ - RestSubNotifToXapp: The total number of successful Rest SubscriptionNotification messages sent to xApp
+ - RestSubFailNotifToXapp: The total number of failure Rest SubscriptionNotification messages sent to xApp
+ - SubReqToE2: The total number of SubscriptionRequest messages sent to E2Term
+ - SubReReqToE2: The total number of SubscriptionRequest messages resent to E2Term
+ - SubRespFromE2: The total number of SubscriptionResponse messages from E2Term
+ - PartialSubRespFromE2: The total number of partial SubscriptionResponse messages from E2Term
+ - SubFailFromE2: The total number of SubscriptionFailure messages from E2Term
+ - SubReqTimerExpiry: The total number of SubscriptionRequest timer expires
+ - RouteCreateFail: The total number of subscription route create failure
+ - RouteCreateUpdateFail: The total number of subscription route create update failure
+ - MergedSubscriptions: The total number of merged Subscriptions
+ - DuplicateE2SubReq: The total number of same E2 SubscriptionRequest messages from same xApp,
+
+ Subscription delete counters:
+ - SubDelReqFromXapp: The total number of SubscriptionDeleteResponse messages received from xApp
+ - SubDelRespToXapp: The total number of SubscriptionDeleteResponse messages sent to xApp
+ - RestSubDelReqFromXapp: The total number of Rest SubscriptionDeleteRequest messages received from xApp
+ - RestSubDelRespToXapp: The total number of Rest SubscriptionDeleteResponse messages sent to xApp
+ - RestSubDelFailToXapp: The total number of Rest SubscriptionDeleteFailure messages sent to xApp
+ - SubDelReqToE2: The total number of SubscriptionDeleteRequest messages sent to E2Term
+ - SubDelReReqToE2: The total number of SubscriptionDeleteRequest messages resent to E2Term
+ - SubDelRespFromE2: The total number of SubscriptionDeleteResponse messages from E2Term
+ - SubDelFailFromE2: The total number of SubscriptionDeleteFailure messages from E2Term
+ - SubDelReqTimerExpiry: The total number of SubscriptionDeleteRequest timer expires
+ - RouteDeleteFail: The total number of subscription route delete failure
+ - RouteDeleteUpdateFail: The total number of subscription route delete update failure
+ - UnmergedSubscriptions: The total number of unmerged Subscriptions
+
+ SDL failure counters:
+ - SDLWriteFailure: The total number of SDL write failures
+ - SDLReadFailure: The total number of SDL read failures
+ - SDLRemoveFailure: The total number of SDL remove failures
+
+ E2 interface state counters
+ - E2StateChangedToUp: The total number of E2 interface change connected state
+ - E2StateChangedToDown: The total number of E2 interface change disconnected state
+
+Configurable parameters
+-----------------------
+ Subscription Manager has following configurable parameters.
+ - Retry timeout for RIC Subscription Request message
+ - e2tSubReqTimeout_ms: 2000 is the default value
+
+ - Retry timeout for RIC Subscription Delete Request message
+ - e2tSubDelReqTime_ms: 2000 is the default value
+
+ - Waiting time for RIC Subscription Response and RIC Subscription Delete Response messages
+ - e2tRecvMsgTimeout_ms: 2000 is the default value
+
+ - Try count for RIC Subscription Request message
+ - e2tMaxSubReqTryCount: 2 is the default value
+
+ - Try count for RIC Subscription Delete Request message
+ - e2tMaxSubDelReqTryCount: 2 is the default value
+
+ - Shall Subscription Manager check is E2 interface up before new SubscriptionRequest is processed
+ - checkE2State: true is the default value
+
+ - Are subscriptions read from database in Subscription Manager startup
+ - readSubsFromDb: "true" is the default value
+
+ - How many times Subscription Manager tries to read database in start up before it continues startup procedure
+ - dbTryCount: 200 is the default value
+
+ - Shall Subscription Manager try to read data base forever in start up before it continues startup procedure
+ - dbRetryForever: true is the default value
+
+
+ The parameters can be changed on the fly via Kubernetes Configmap. Default parameters values are defined in Helm chart
+
+ Use following command to open Subscription Manager's Configmap in editor. First change parameter and then store the
+ change. The change is visible in Subscription Manager's log after some 20 - 30 seconds. Note that some of the parameters
+ maybe be useful only for testing purpose.
+
+ .. code-block:: none
+
+ Open in default editor:
+
+ kubectl edit cm configmap-ricplt-submgr-submgrcfg -n ricplt
+
+ Open in Nano editor:
+
+ KUBE_EDITOR="nano" kubectl edit cm configmap-ricplt-submgr-submgrcfg -n ricplt
+
+
+REST interface for debugging and testing
+----------------------------------------
+ Give following commands to get Subscription Manager pod's IP address
+
+ .. code-block:: none