Skip to main content

Kubernetes Management

To management kubernetes clusters in OEC we need to use jump servers/vms

Connect to VM/jump servers using MobaXterm or any ssh client from Amazon Workspaces (use either of below IPs)

  • 10.134.25.241
  • 10.134.196.126 (this does not work on Ireland OEC)

Login with your OEC credentials (same as windows username) once logged in we can connect to dpcli to use kubectl

dpcli commands can be found in LastPass under the entry OEC DPCLI kubernetes

You have to login to different environments depending on where you are developing:

  • OEC
  • Tiger
  • AkerBP

Example for OEC2 CLI:​

sudo docker run -it -e TOKEN="<Token>" -v $(pwd):/root/app distservices-docker-release.repo.openearth.io/distarch/dpcli-oec-prod-dsif2:4.0

dpcli for TIGER​

Here we dont have access to listing namespaces. We only have access to dwp and dwp-abp namespaces so after logging in with sudo docker run ... we need use kubectl like this

kubectl -ndwp-abp get pods

Namespaces​

  • dwp-dev (for containers, deployed versions of feature branches)
  • dwp-system (for base microservices of dwp, middleware etc.)
  • dsis-system (for services such as EDM, DSIF etc.)
  • wf-system (web-framework)

for AkerBP it's a bit different. The namespaces here are tagged with release number e.g akerbp242. Feature branches are deployed to dwp-system-akerbp242-dev while OEC service are in dwp-system-akerbp242-dev

To see all relevant namespaces for release 242​

kubectl get ns | grep akerbp242

Useful commands​

Using dwp-dev as an example here and b4c-1166-hss-d-microservice-holesection-summary which is a microservice based on b4c-116-hss-deliver branch 15 first characters of branch + microservice name

kubectl get pod -n dwp-dev | grep b4c-116-hss
#Output
b4c-1166-hss-d-microservice-holesection-summary-6474bdfbb8d8d9l   2/2     Running                      0                  20m
b4c-1166-hss-d-webapp-dwp-webapp-6c795f8f9d-tn92b                 2/2     Running                      0                  34h

To inspect pod/containers​

kubectl -n dwp-dev describe b4c-1166-hss-d-microservice-holesection-summary-6474bdfbb8d8d9l

Logs for specific container within pod​

kubectl -n dwp-dev logs b4c-1166-hss-d-microservice-holesection-summary-6474bdfbb8d8d9l -c <container-name>

Logs for all containers​

kubectl -n dwp-dev logs b4c-1166-hss-d-microservice-holesection-summary-6474bdfbb8d8d9l -c --all-containers=true

To see service status for various services like EDM, BPM, DSIS​

kubectl -n dsis-system get pod
#Output
NAME                                  READY   STATUS    RESTARTS        AGE
conf-6fb84bcdd5-jpdwq                 2/2     Running   0               20d
dsbpm-868dbf9bf-4lrx5                 2/2     Running   0               41h
dsbpm-server-664f88ff96-lnxff         2/2     Running   0               4h13m
dsbpm-server-664f88ff96-lv8sv         2/2     Running   0               4h13m
dsdata-7d869f8799-pxdw6               2/2     Running   0               41h
dsdata-7d869f8799-v4c7c               2/2     Running   0               41h
dsdatadwp-67b4798f8d-qpzcd            2/2     Running   0               46m
dsdatadwp-67b4798f8d-xvvn8            2/2     Running   0               46m
dsdq-7bb984d45d-cql2f                 2/2     Running   0               44m
dsdq-7bb984d45d-zzdxc                 0/2     Error     3               43h
dsdsedm-5c9649c574-n4p42              2/2     Running   0               19h
dsdsedmdwp-64fdd995f4-c7hlb           2/2     Running   0               18h
dsdsedmdwp-64fdd995f4-fqpxl           2/2     Running   0               10h
dsdsedmdwp-64fdd995f4-pz4v7           2/2     Running   0               18h
dsdsedmdwp-64fdd995f4-x6cjp           2/2     Running   0               18h
dsdsow-6cb8d5dd8b-4sqfp               2/2     Running   0               17m
dsdsow-6cb8d5dd8b-p6xc9               2/2     Running   0               28m
dsdsowalt-55cdb68f4-ftt4w             2/2     Running   0               74m
dsmessaging-logger-5d66ccddf5-xzdsc   2/2     Running   0               10h
dsmessaging-master-0                  1/1     Running   0               19h
dsnotification-b988f7955-kqldp        2/2     Running   0               19h
dssearch-57c6dbf866-zrsx4             2/2     Running   0               28h
dssearchserverv1-0                    2/2     Running   4 (5d10h ago)   7d16h
dssearchserverv1-1                    2/2     Running   4 (5d10h ago)   6d14h
dssearchserverv1-zookeeper-0          1/1     Running   0               5d10h
dstransfer-6887d65b7d-zbzqh           2/2     Running   0               14h
dstransferzain-f748fb468-gh5f5        2/2     Running   0               50m
dtworkflow-d6f54b5c9-xwxlz            2/2     Running   0               43h
mgmtconsole-778459b964-shrc8          2/2     Running   1               43h
sdm-mapsync-tool-6976b479d4-cgrf9     2/2     Running   0               2d7h
sdm-npdsync-tool-7b899897dd-h9w4k     2/2     Running   0               10d
sdm-user-sync-tool-dc756fdc5-tkzjt    2/2     Running   0               77m

Starting all pods in a namespace​

Halliburton/Landmark have policies in place that scales down over the weekend.

kubectl scale deployment --all --replicas=1 -n <namespace_name>

or we can use this script https://github.com/AkerBP/pathfinders/blob/main/scripts/k8s/scale-namespaces.sh

# Open a text editor, paste the script, save as scale-namespaces.sh
# Script will only scale empty namespaces
chmod +x ./scale-namespaces.sh
./scale-namespaces.sh "-akerbp242"

Restarting pods in case of issues​

kubectl delete pod <pod-name> -n <namespace>
Connecting...