SpringOne 2020 Tomas Pizarro Moreno: Member of Technical Staff, VMware

1. Confidential │ ©2020 VMware, Inc.
Distribute Your App and
Engage Your
Community with a Helm
Repository
Tomas Pizarro Moreno
SpringOne, 2020

2. Confidential │ ©2020 VMware, Inc. 2
Telecommunications Engineer, University of Seville
6 years at Bitnami, now part of VMWare
Previously focused on creating production ready assets for several
platforms (Single VMs, Multi Tier apps, containers, Helm charts, …)
Now, part of the Tanzu Application Catalog team
Where you can find me:
• https://github.com/tompizmor
• https://www.linkedin.com/in/tompizmor/
• @tompizmor in Kubernetes slack
Who am I?

3. 3Confidential │ ©2020 VMware, Inc.
Distributed repositories

4. Confidential │ ©2020 VMware, Inc. 4
Distributed repositories
• Repositories in Helm were designed to be distributed.
• The stable repository was created to kick start charts. This made people think
on it as “THE” repository to discover charts.
• Helm v3 removed the stable repository by default. It was the first step to move
to distributed repositories.
• The stable repository did not accept new charts since 13th Nov, 2019 and it will
be marked obsolete at 13th Nov, 2020.
• It seems useful to share our experience maintaining a repository with more
than 70 charts.

5. Confidential │ ©2020 VMware, Inc. 5
Chart repository maintenance process
Store
You need a place to
store your charts
Test
Pass tests to
guarantee the
quality
Maintain
Feedback loop and
update components
Publish
Make the charts
available to
everyone

6. Confidential │ ©2020 VMware, Inc. 6
Store your charts

7. Confidential │ ©2020 VMware, Inc. 7
Store your charts
Requirements to store and serve charts are cheap.
Just a web server with ability to serve a yaml file
and gzipped tarballs.
Some of the most common options are AWS S3
bucket, GitHub Pages, Google Cloud Storage, Jfrog
Artifactory and also ordinary web servers like
Apache or Nginx.
If you want to self-host your charts you can also use
ChartMuseum or Harbor.

8. Confidential │ ©2020 VMware, Inc. 8
Harbor
Serve your charts
Other great features:
• Is also a docker image registry
• It can scan your images for security vulnerabilities
with different engines (clair and trivy)
• It supports signed docker images via notary
• First OCI-compliant open source registry.

9. Confidential │ ©2020 VMware, Inc. 9
Test your charts

10. Confidential │ ©2020 VMware, Inc. 10
Running helm template path/to/local/chart can be useful to identify syntax errors without
having to install the chart.
Another option is to run helm install with the --dry-run option.
Helm template
Test your charts

11. Confidential │ ©2020 VMware, Inc. 11
A step further would be to run a linter. For example, the chart-testing tool is a great way
to lint and test your chart locally using a Kind cluster.
https://github.com/helm/chart-testing
Some of the things checked by the linter:
• Version checking
• YAML schema validation in Chart.yaml
• YAML linting on Chart.yaml and values.yaml
Helm lint
Test your charts

12. Confidential │ ©2020 VMware, Inc. 12
But rendering the template, installing with --dry-run or running a linter does not
guarantee that the Kubernetes manifests will be properly deployed into the cluster.
It seems that if we want to ensure our chart works properly we will need to install it.
Helm install
Test your charts

13. Confidential │ ©2020 VMware, Inc. 13
Some charts requires previous configuration or specify certain properties from the values
to be properly deployed.
Even if it can be deployed by default, it might be interesting to test a specific configuration
of the chart.
Examples:
• MongoDB Standalone vs MongoDB Replica set
• WordPress chart with different kinds of services (LoadBalancer, Ingress, …)
• Deploy a chart with or without persistent volumes
Helm install with custom values
Test your charts

14. Confidential │ ©2020 VMware, Inc. 14
Test your charts
Verification and functional tests
Apart from checking that pods are running, it is important to verify that the application is
properly configured.
To verify the application is properly configured we run two different kind of tests:
Verification: Important files and binaries exists, permissions properly configured,
binaries basic functionality works, etc
Functional: Automatic navigation through the web page to verify it properly
works.

15. Confidential │ ©2020 VMware, Inc. 15
Verification tests
Test your charts

16. Confidential │ ©2020 VMware, Inc. 16
Test your charts
Functional tests

17. Confidential │ ©2020 VMware, Inc. 17
Functional tests
Test your charts

18. Confidential │ ©2020 VMware, Inc. 18
It is important to guarantee upgradability
between chart releases for minor and patch
new versions.
It is expected that a major change in the
chart will require manual steps before or
after run the helm upgrade command.
Test your charts
Helm upgrades

19. Confidential │ ©2020 VMware, Inc. 19
Test your charts
Helm upgrade
Install base
chart
Install WordPress chart
version 7.0.0
Populate some
data
Create a post, upload
an image, add a user, …
Upgrade to
latest version
Run helm upgrade to
the latest version.
Check previous
data
Verify previous post,
image, user still exists and
regular tests keeps passing
1 2 3 4

20. 20Confidential │ ©2020 VMware, Inc.
Deploy to several clusters
Different Kubernetes clusters, different
environments…

21. Confidential │ ©2020 VMware, Inc. 21
Different services
Test your charts
• TMC (VMWare Tanzu Mission Control)
• GKE (Google Kubernetes Engine)
• AKS (Azure Kubernetes Service)
• EKS (Amazon Elastic Container Service for Kubernetes)
• IKS (IBM Cloud Kubernetes Service)

22. Confidential │ ©2020 VMware, Inc. 22
Test your charts
Different services, different requirements
- Changing permissions on default AKS persistent volumes was slow for some
applications.
- IKS does not support Kubernetes securityContext
- Some Kubernetes platforms run containers as non-root by default

23. Confidential │ ©2020 VMware, Inc. 23
Maintain your charts

24. Confidential │ ©2020 VMware, Inc. 24
It is important to maintain the docker images used in your charts up-to-date.
Not only to get the new features and bugs fixed, but for security.
Bitnami also test all the images used by the Helm charts before they are released.
Keep your charts up-to-date

25. Confidential │ ©2020 VMware, Inc. 25
Apply user feedback
Listen to your users.
Keep the feedback loop and short as
possible.
Increase the quality of the helm charts. Bug
fixing, new features, new best practices in
the industry, helm identify and testing
corner cases in different scenarios,…
Do
Adjust
Learn

26. 26Confidential │ ©2020 VMware, Inc.
Other tips

27. Confidential │ ©2020 VMware, Inc. 27
• Avoid using mutable or rolling tags. Otherwise your helm chart won’t be immutable and
an update of the underlying docker image can break your deployment.
• Document every major change in the README
• Document how to access the chart using each type of Kubernetes service
• Validate user inputs as much as you can
• Create a checklist for new helm charts development
Other tips

28. Confidential │ ©2020 VMware, Inc. 28
Make your charts
available to everyone

29. Confidential │ ©2020 VMware, Inc. 29
Make them available to everyone
Helm Hub

30. Confidential │ ©2020 VMware, Inc. 30
Make them available to everyone
Helm Hub
Adding your repository to the Helm Hub is super easy. You just need to send a pull request
to the https://github.com/helm/hub/ repository with the following information:
1. Add your repository name and base URL to the file config/repo-values.yaml
2. Add your contact information to the file repos.yaml
Additionally, the charts from your repository should fulfill the next expectations:
1. Should have a maintainer
2. Should pass the Helm lint and be installable and upgradable in all community
supported version of Kubernetes
3. Should have a NOTES.txt template with useful information
4. Charts versions should be immutable

31. Confidential │ ©2020 VMware, Inc. 31
If you don’t want to make your charts available to everyone but to the users of your
Kubernetes cluster you can do it with Kubeapps, a web-based UI for deploying and
managing applications in your own Kubernetes cluster.
Evolution of the Helm Hub
Make them available to everyone
Kubeapps
There is also a public hub from Kubeapps where you can
submit your charts so they are available.
https://hub.kubeapps.com/

32. Confidential │ ©2020 VMware, Inc. 32
Make them available to everyone
Kubeapps

33. Confidential │ ©2020 VMware, Inc. 33
Make them available to everyone
Artifact Hub

34. Confidential │ ©2020 VMware, Inc. 34
Make them available to everyone
Artifact Hub
Hub for finding, installing and publishing packages and configurations for CNCF projects.
Currently in alpha state with support for Helm charts, Falco configurations, OPA policies
and OLM operators in development.
As Kubeapps, it can be installed in cluster.

35. Confidential │ ©2020 VMware, Inc. 35
Make them available to everyone
jFrog ChartCenter

36. Confidential │ ©2020 VMware, Inc. 36
Make them available to everyone
jFrog ChartCenter
Another Web UI to discover Helm packages from different Helm chart repositories.
It shows chart dependencies and vulnerability information.
It is also possible to publish your chart repository if charts meets these requirements

37. Confidential │ ©2020 VMware, Inc. 37
Make them available to everyone
Cloud Providers Marketplaces

38. Confidential │ ©2020 VMware, Inc. 38
Conclusions
• Store:
• Be aware of the features of each option to make a choice
• Test:
• Test as much as you can
• Maintain:
• Invest time updating the images and listen to the community
• Publish:
• Add your repo to Helm Hub

39. 39Confidential │ ©2020 VMware, Inc.
Thanks