This week ePages hosted the Hamburg Kubernetes/Cloud Native Meetup. With 46 participants actually showing up, the room was full but not yet packed with Kubernauts being eager to learn and discuss.
Smooth sailing with Kubernetes?
In a very entertaining talk, Christian shared experiences they learned the hard way.
Why would you want to do that?
Getting project-specific computing resources at universities is usually causing headaches, and lots of paperwork. Even if you eventually get what you needed, well then, it was certainly too late.
The HAW Hamburg wanted to provide their students, professors, and researchers an accessible cloud computing infrastructure, where they could learn essential skills for today’s software industry, and experiment with various technology and demanding algorithms.
The Kubernetes cluster was designed to support approx. 1000 concurrent users. The hard facts are:
- 6 storage nodes, 8 compute nodes, 1 GPU node
- Overall resources: 6 10-core Xeon processors, 4 Tesla V100 GPUs, ~1 TB RAM, 4 TB SSD storage, 33 TB HDD storage and one big Cisco switch 🙂
To solve the AAA-Problem (Authentication, Authorization and Admission) they relied on GitLab as being the single source of truth, mapping GitLab resources to Kubernetes Namespaces, and RoleBindings.
Using a simple utility script, students or HAW employees are obtaining
kubectl credentials using their LDAP accounts.
Christian and the HAW Hamburg did not solve the Admission problem, yet. So currently all resources are free-for-all. In the future, they intend to introduce a leasing mechanism, and let tenants self-service resources.
They highlighted additional challenges like losing access to the cluster (
kubectl timing out), because one student made extensive use of the Kubernetes API server.
etcd running out of memory, causing a sychronization cascade that kept the whole cluster busy for nearly a day.
Also they struggled with multi-tenant logging, since all available open-source solutions are either you-see-everything or you-see-nothing.
Eat & greet
Just in time when the presentation was finished, pizza arrived, generously sponsored by Loodse. Combined with the usual drinks and snacks provided by ePages, the networking and discussion part of the meetup brought very interesting conversations. It even lasted more than twice as long as the talk itself, which made it a real community event.