akka Service Deployment on Kubernetes
Introduction
There are a lot of good blog posts about akka and how to develop modern services with this great framework. Most of these tutorials don’t show you afterwards how you can package and deploy your services on a kubernetes cluster or minikube. In this blog post I will show you how you can package your akka services with sbt and the sbt-native-packager and then deploy it on minikube with the help of akka-management (> 1.0.1).
The Demo architecture
Our demo application is an advanced greeter application. You can greet a person via http interface and your akka-cluster tells you how often the person you have called was greeted yet.
In our simple demo case we have two services which communicate inside the k8s cluster network over gRPC and the akka-grpc module. Service 1 is the backend service which runs an akka cluster on several pods with a kubernetes replicaset. Service 1 uses the akka-management kubernetes-api discovery to create and join an akka cluster on your replicaset. akka-management also contains health und readiness checks for your services on port 8558. Service 1 has also an gRPC interface which can be accessed from service 2 via gRPC on port 8081. Service 2 is a demo for a frontend service. It has a HTTP API which can be accessed outside the cluster as a service. In our case it is a NodePort on minikube. In a real world application it would be an ingress.
Remember: This demo app does not put a lot of value on code quality. So ServiceDiscovery for gRPC endpoint is also made with akka discovery and a bit dirty, because it doesn’t use the cluster ip of service 1 for it’s gRPC calls. It just took the first resolution address from the pod list with selector app=service1 and blocks service2 while searching with an Await. For a production use-case this should be done with map in a future and the get calls should be checked if they are defined. Logging is also very dirty and there are some println. Originally I created the app for one of our student who writes his thesis about “advanced logging, monitoring and tracing in akka”. When he finishes his work I will add this in this example and we’ll publish another blogpost. The println should be gone then.
Prepare the kubernetes cluster / minikube
For our demo case we’ll use minikube. On your Mac your installation is really simple. brew cask install minikube. After you run minikube (minikube start), you have to configure your environment with eval $(minikube docker-env).
For the service discovery with kubernetes-api we also need some rbac RoleBindings for the default account in the default namespace. This script comes from lightbend. You can also find it in the official akka-management docs.
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: pod-reader
rules:
- apiGroups: [""]
resources: ["pods"]
verbs: ["get", "watch", "list"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: read-pods
subjects:
- kind: User
name: system:serviceaccount:default:default
roleRef:
kind: Role
name: pod-reader
apiGroup: rbac.authorization.k8s.ioBuilding Docker containers
The repository contains a multi sbt project with some handy commands. So there is a sbt alias command for running and stopping all services in the right order with the sbt-revolver plugin. Start all services locally with runAlland stop them with stopAll. To build 2 docker containers for the 2 services you can use the command buildAll. In the k8s folder is a script which uses two shell subprocesses for building the containers directly with docker:publishLocal.
The configuration is very simply. It is set in the build.sbt file for each service. For more information look up the official documentation of sbt-native-packager.
enablePlugins(JavaAppPackaging)
enablePlugins(DockerPlugin)packageName in Docker := "innfactory-test/service1"
version in Docker := "0.1"
dockerExposedPorts := Seq(2552, 8558)
and
enablePlugins(JavaAppPackaging)
enablePlugins(DockerPlugin)
packageName in Docker := "innfactory-test/service2"
version in Docker := "0.1"
dockerExposedPorts := Seq(2552, 8558, 8090)Up and Running (The Deployment)
When your build is successful you can use the yaml files for deployment with kubectl apply -f. Be careful and deploy service 1 first kubectl apply -f service1.yaml, because we use Await in service 2 and we need a guaranteed answer from the kubernetes-api! When the akka cluster is ready and all pods are healthy you can deploy service 2 kubectl apply -f service2.yaml
The deployment uses livenessProbe and readinessProbe from akka-management so we need the containerPort 8558.
livenessProbe:
httpGet:
path: /alive
port: 8558
readinessProbe:
httpGet:
path: /ready
port: 8558apiVersion: v1
kind: Service
metadata:
labels:
app: service2
name: service2
spec:
ports:
- name: "http-api"
port: 8090
protocol: TCP
targetPort: 8090
selector:
app: service2
type: NodePort
When you use minikube you can access this service directly. You can access the endpoint url from minikube minikube service service2. When you open this URL in your favorite browser it should response with 200:"Service 2 is ok".
Both services create a deployment and a service. service 1 is a cluster IP and service 2 is a node port.
apiVersion: v1
kind: Service
metadata:
labels:
app: service2
name: service2
spec:
ports:
- name: "http-api"
port: 8090
protocol: TCP
targetPort: 8090
selector:
app: service2
type: NodePortThen you can greet people and you also get the count how often the given person was greeted yet.
e.g http://192.168.99.100:32462/greet/innFactorywould respond with: Hello innFactory - The actorsystem greeted you 1 times!
Remember: We have deployed a cluster ip service, but we don’t use it yet. If you are in a real world environment you should use this cluster ip for gRPC communication. We also use the NodePort to access service 2. In a real world example you should create also an ingress.
# e.g
apiVersion: "extensions/v1beta1"
kind: Ingress
metadata:
name: service2
annotations:
"ingress.kubernetes.io/rewrite-target": "/"
spec:
rules:
- http:
paths:
- backend:
serviceName: service2
servicePort: 8090