add kubespray to the XTesting as it provides newer version of kubenetes and can be...
[it/test.git] / XTesting / kubespray / docs / kube-router.md
1 # Kube-router
2
3 Kube-router is a L3 CNI provider, as such it will setup IPv4 routing between
4 nodes to provide Pods' networks reachability.
5
6 See [kube-router documentation](https://www.kube-router.io/).
7
8 ## Verifying kube-router install
9
10 Kube-router runs its pods as a `DaemonSet` in the `kube-system` namespace:
11
12 * Check the status of kube-router pods
13
14 ```ShellSession
15 # From the CLI
16 kubectl get pod --namespace=kube-system -l k8s-app=kube-router -owide
17
18 # output
19 NAME                READY     STATUS    RESTARTS   AGE       IP               NODE                   NOMINATED NODE
20 kube-router-4f679   1/1       Running   0          2d        192.168.186.4    mykube-k8s-node-nf-2   <none>
21 kube-router-5slf8   1/1       Running   0          2d        192.168.186.11   mykube-k8s-node-nf-3   <none>
22 kube-router-lb6k2   1/1       Running   0          20h       192.168.186.14   mykube-k8s-node-nf-6   <none>
23 kube-router-rzvrb   1/1       Running   0          20h       192.168.186.17   mykube-k8s-node-nf-4   <none>
24 kube-router-v6n56   1/1       Running   0          2d        192.168.186.6    mykube-k8s-node-nf-1   <none>
25 kube-router-wwhg8   1/1       Running   0          20h       192.168.186.16   mykube-k8s-node-nf-5   <none>
26 kube-router-x2xs7   1/1       Running   0          2d        192.168.186.10   mykube-k8s-master-1    <none>
27 ```
28
29 * Peek at kube-router container logs:
30
31 ```ShellSession
32 # From the CLI
33 kubectl logs --namespace=kube-system -l k8s-app=kube-router | grep Peer.Up
34
35 # output
36 time="2018-09-17T16:47:14Z" level=info msg="Peer Up" Key=192.168.186.6 State=BGP_FSM_OPENCONFIRM Topic=Peer
37 time="2018-09-17T16:47:16Z" level=info msg="Peer Up" Key=192.168.186.11 State=BGP_FSM_OPENCONFIRM Topic=Peer
38 time="2018-09-17T16:47:46Z" level=info msg="Peer Up" Key=192.168.186.10 State=BGP_FSM_OPENCONFIRM Topic=Peer
39 time="2018-09-18T19:12:24Z" level=info msg="Peer Up" Key=192.168.186.14 State=BGP_FSM_OPENCONFIRM Topic=Peer
40 time="2018-09-18T19:12:28Z" level=info msg="Peer Up" Key=192.168.186.17 State=BGP_FSM_OPENCONFIRM Topic=Peer
41 time="2018-09-18T19:12:38Z" level=info msg="Peer Up" Key=192.168.186.16 State=BGP_FSM_OPENCONFIRM Topic=Peer
42 [...]
43 ```
44
45 ## Gathering kube-router state
46
47 Kube-router Pods come bundled with a "Pod Toolbox" which provides very
48 useful internal state views for:
49
50 * IPVS: via `ipvsadm`
51 * BGP peering and routing info: via `gobgp`
52
53 You need to `kubectl exec -it ...` into a kube-router container to use these, see
54 <https://www.kube-router.io/docs/pod-toolbox/> for details.
55
56 ## Kube-router configuration
57
58 You can change the default configuration by overriding `kube_router_...` variables
59 (as found at `roles/network_plugin/kube-router/defaults/main.yml`),
60 these are named to follow `kube-router` command-line options as per
61 <https://www.kube-router.io/docs/user-guide/#try-kube-router-with-cluster-installers>.
62
63 ## Advanced BGP Capabilities
64
65 <https://github.com/cloudnativelabs/kube-router#advanced-bgp-capabilities>
66
67 If you have other networking devices or SDN systems that talk BGP, kube-router will fit in perfectly.
68 From a simple full node-to-node mesh to per-node peering configurations, most routing needs can be attained.
69 The configuration is Kubernetes native (annotations) just like the rest of kube-router.
70
71 For more details please refer to the <https://github.com/cloudnativelabs/kube-router/blob/master/docs/bgp.md.>
72
73 Next options will set up annotations for kube-router, using `kubectl annotate` command.
74
75 ```yml
76 kube_router_annotations_master: []
77 kube_router_annotations_node: []
78 kube_router_annotations_all: []
79 ```