This document discusses using reprepro to create and manage an APT repository for hosting custom packages and configurations. Reprepro allows syncing packages from external repositories, resigning packages with a custom key, and distributing packages to different environments like development, staging, and production. Configurations can be packaged and deployed per-environment to simplify management across suites. Integrating the custom repository with configuration management tools like Ansible promotes conformity.
Deploying with Super Cow Powers (Hosting your own APT repository with reprepro)
1. Deploying with Super
Cow Powers
Hosting your own APT repository
with reprepro
Simon Boulet
Consultant, Deployment and Automation
simon@nostalgeek.com
DevOps Montréal
February 2015
1
2. Challenge of Modern Application
You want:
- Nginx 1.7
- Node.js 0.11
- MongoDB 2.6
- Consul
But latest Ubuntu has:
- Nginx 1.4.6
- Node.js 0.10.25
- MongoDB 2.4.9
- Consul N/A
2
3. /etc/apt/sources.list
3
How do you turn this:
deb http://downloads-distro.mongodb.org/repo/debian-sysvinit dist 10gen
deb https://deb.nodesource.com/node-devel wheezy main
deb http://nginx.org/packages/mainline/debian/ wheezy nginx
deb http://ppa.launchpad.net/bcandrea/consul/ubuntu/ trusty main
Into this:
deb http://apt.devops.quebec/ dev main
6. Reprepro
- Manage your own APT repository
- Allow for syncing external repos
- Can do signatures checks and resign
- Does NOT package .deb for you
- Does NOT make your repository externally
accessible
6
9. Reprepro: conf/distributions
Codename: dev
Suite: unstable
Architectures: amd64
Components: main
Tracking: minimal
Update: mongodb nodesource nginx consul debian-20141003
SignWith: ABCD1234
Codename: prod
Suite: stable
Architectures: amd64
Components: main
Tracking: minimal
SignWith: ABCD1234
reprepro update
9
10. Packaging Configurations Tricks
- Rebuild config packages simultaneously for
all environments
- Bump config package version on each build
- Don’t store secrets in packages
- Use conf.d directories when available
- Setup diversion if you really need to update
configurations files provided by other
packages
10
11. Config Package: debian/control
Source: superapp-config
Section: unknown
Priority: extra
Maintainer: Simon Boulet <simon@nostalgeek.com>
Build-Depends: debhelper (>= 8.0.0)
Standards-Version: 3.9.3
Package: superapp-config-dev
Architecture: all
Provides: superapp-config
Description: Super App Config (dev)
Package: superapp-config-prod
Architecture: all
Provides: superapp-config
Description: Super App Config (prod)
11
13. Multiple Environments
- Use per-environment config package
- Each environment to have their own suite:
deb http://apt.devops.quebec/ dev main
deb http://apt.devops.quebec/ staging main
deb http://apt.devops.quebec/ prod main
- Always add package to dev, and use copy to
promote from dev to staging or prod
13
14. Promoting Dev > Staging > Prod
Adding to dev:
reprepro includedeb dev <.deb file>
Promoting from dev to prod:
reprepro copy prod dev <packages...>
14
15. Integrating with CM Tools
Ansible:
- Add your repository (apt_repository)
- Import your signing key (apt_key)
- Ensure conformity (ansible --check)
15
16. Export your Repository
Using your favorite web server:
- Make /dist and /pool folders available
- Use .htaccess (or other method) for limiting
access
Using SSH:
deb ssh://repo@apt.devops.quebec/path/to/repo dev main
16
17. Going Large Scale
- Sync your repository to an Object Store
(Amazon S3, Rackspace Cloud Files, etc.)
- Use CDN service in front of your repository
(CloudFront, CloudFlare, etc.)
17
18. Notes on using Amazon S3
- S3 treats “+” in filename as space
characters. Packages with “+” in their
version numbers won’t work [1]
- No HTTP authentication on S3. See apt-
transport-s3 [2] for private repo.
18
[1] https://forums.aws.amazon.com/message.jspa?messageID=208095
[2] https://github.com/kyleshank/apt-transport-s3
19. Deploying with Super Cow Powers
- Control versions of packages in different
environments (enforces deployment
pipeline)
- Simplifies repo and key management by
having a centralized repo
- Ease config management by packaging
application configuration
19