Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Dependency Management With Pinto
1. Dependency Management
With Pinto
Jeffrey Thalhammer
thaljef@cpan.org
YAPC::NA
June 15, 2012
2. CPAN Is Heaven
Tens of thousands of
modules for every
purpose imaginable
Lots of ser vices for
testing, bug tracking,
ratings, forums
Awesome tool chain
for building, testing,
installing
3. CPAN Is Hell
The CPAN is not
stable
Developers
constantly adding
and removing stuff
Sometimes the code
is broken
Tool chain expects
backward
compatibility
8. More Shops Are Writing CPAN-
style Dists
• M::B
and EU::MM more
commonplace
• Dist::Zilla
makes the
development fun
• cpanmmakes the
deployment easy
• DarkPANs proliferate
47. Injecting Your Own Distribution
pinto add some/dir/My-App-1.0.tar.gz
pinto list
48. Injecting Your Own Distribution
pinto add some/dir/My-App-1.0.tar.gz
pinto list
rl My::App 1.0 YOU/My-App-1.0.tar.gz
rf URI 1.59 GAAS/URI-1.59.tar.gz
rf URI::Escape 3.31 GAAS/URI-1.59.tar.gz
...
49. Installing With cpan
$ cpan
cpan[1]> o conf urllist file://$HOME/mypan
cpan[2]> install My::App
50. Installing With cpan
$ cpan
cpan[1]> o conf urllist file://$HOME/mypan
cpan[2]> install My::App
point to your repository
51. Installing With cpan
$ cpan
cpan[1]> o conf urllist file://$HOME/mypan
cpan[2]> install My::App
69. What Is A Stack?
A named mapping from package
names to distribution archives
Conceptually equivalent to the
02packages.details.txt file
In a CPAN, there is only one
“stack”, and it usually contains
only latest package versions
But a Pinto repository can have
multiple stacks, that contain
arbitrary package versions
70. The Default Stack
Every Pinto repository has a
built-in stack called “init”
It is the default stack for all
operations
You can change the default
stack
102. Merging Stacks
pinto merge upgrades init
rf init URI 1.62 GAAS/URI-1.62.tar.gz
rf upgrades URI 1.62 GAAS/URI-1.62.tar.gz
...
103. Why Use Stacks?
• Stacks for upgrades
• Stacks for each feature
• Stacks for dev/qa/prod
• Stacks for each product
• Stacks for each perl version
• Stacks for each customer
111. Pinning A Package
pinto pin URI
pinto list
rf+ URI 1.59 GAAS/URI-1.59.tar.gz
rf+ URI::Escape 3.31 GAAS/URI-1.59.tar.gz
rf+ URI::Heuristic 4.20 GAAS/URI-1.59.tar.gz
112. Pinning A Package
pinto pin URI
pinto list
rf+ URI 1.59 GAAS/URI-1.59.tar.gz
rf+ URI::Escape 3.31 GAAS/URI-1.59.tar.gz
rf+ URI::Heuristic 4.20 GAAS/URI-1.59.tar.gz
“+” indicates a pin
113. Pinning A Package
pinto pin URI
pinto list
rf+ URI 1.59 GAAS/URI-1.59.tar.gz
rf+ URI::Escape 3.31 GAAS/URI-1.59.tar.gz
rf+ URI::Heuristic 4.20 GAAS/URI-1.59.tar.gz
117. Pinning A Package
Suppose you want to use Catalyst
And Catalyst requires Plack 0.99
And Plack 0.99 requires HTTP::Request 6.03
118. Pinning A Package
Suppose you want to use Catalyst
And Catalyst requires Plack 0.99
And Plack 0.99 requires HTTP::Request 6.03
And HTTP::Request requires ...
119. Pinning A Package
Suppose you want to use Catalyst
And Catalyst requires Plack 0.99
And Plack 0.99 requires HTTP::Request 6.03
And HTTP::Request requires ...
...requires URI 1.62
136. Experiment With Upgrades
You have a “prod” stack with DBIx::Class 1.6
Make a copy of the “prod” stack called “test”
137. Experiment With Upgrades
You have a “prod” stack with DBIx::Class 1.6
Make a copy of the “prod” stack called “test”
Upgrade to DBIx::Class 1.7 on “test” stack
138. Experiment With Upgrades
You have a “prod” stack with DBIx::Class 1.6
Make a copy of the “prod” stack called “test”
Upgrade to DBIx::Class 1.7 on “test” stack
Build & test application using the “test” stack
139. Experiment With Upgrades
You have a “prod” stack with DBIx::Class 1.6
Make a copy of the “prod” stack called “test”
Upgrade to DBIx::Class 1.7 on “test” stack
Build & test application using the “test” stack
If the tests fail pin DBIx::Class on the “prod” stack
142. Making Local Patches
You find a bug in Plack-0.98
You patch and re-package as Plack-0.98_01.tar.gz
143. Making Local Patches
You find a bug in Plack-0.98
You patch and re-package as Plack-0.98_01.tar.gz
Put Plack-0.98_01.tar.gz in the “prod” stack
144. Making Local Patches
You find a bug in Plack-0.98
You patch and re-package as Plack-0.98_01.tar.gz
Put Plack-0.98_01.tar.gz in the “prod” stack
Pin Plack on the “prod” stack
145. Making Local Patches
You find a bug in Plack-0.98
You patch and re-package as Plack-0.98_01.tar.gz
Put Plack-0.98_01.tar.gz in the “prod” stack
Pin Plack on the “prod” stack
Remove pin when the author fixes bug
146. Making Local Patches
You find a bug in Plack-0.98
You patch and re-package as Plack-0.98_01.tar.gz
Put Plack-0.98_01.tar.gz in the “prod” stack
Pin Plack on the “prod” stack
Remove pin when the author fixes bug
Pull Plack-0.99 into the “prod” stack
149. Pinto And Legacy Code
Don’t always know what has been installed.
So use Dist::Sur veyor to discover dependencies.
Stash dependency list in a text file.
Then feed dependencies into a Pinto repository.
150. Pinto And Legacy Code
Don’t always know what has been installed.
So use Dist::Sur veyor to discover dependencies.
Stash dependency list in a text file.
Then feed dependencies into a Pinto repository.
pinto pull --norecurse < dependencies.txt
151. Pinto And Legacy Code
Don’t always know what has been installed.
So use Dist::Sur veyor to discover dependencies.
Stash dependency list in a text file.
Then feed dependencies into a Pinto repository.
pinto pull --norecurse < dependencies.txt
do not automatically fetch deps
152. Pinto And Legacy Code
Don’t always know what has been installed.
So use Dist::Sur veyor to discover dependencies.
Stash dependency list in a text file.
Then feed dependencies into a Pinto repository.
pinto pull --norecurse < dependencies.txt
157. Pinto And The Development Cycle
Usually don’t know dependencies ahead of time.
158. Pinto And The Development Cycle
Usually don’t know dependencies ahead of time.
Might install several modules before choosing.
159. Pinto And The Development Cycle
Usually don’t know dependencies ahead of time.
Might install several modules before choosing.
This process might take several days or weeks.
160. Pinto And The Development Cycle
Usually don’t know dependencies ahead of time.
Might install several modules before choosing.
This process might take several days or weeks.
By the time we decide, CPAN might changed.
161. Pinto And The Development Cycle
Usually don’t know dependencies ahead of time.
Might install several modules before choosing.
This process might take several days or weeks.
By the time we decide, CPAN might changed.
So there’s a hole in the development process.
162. Pinto And The Development Cycle
pinto install -L ~/myperl5 --pull Catalyst
163. Pinto And The Development Cycle
pinto install -L ~/myperl5 --pull Catalyst
“install” command
164. Pinto And The Development Cycle
pinto install -L ~/myperl5 --pull Catalyst
install into local::lib
165. Pinto And The Development Cycle
pinto install -L ~/myperl5 --pull Catalyst
package name
166. Pinto And The Development Cycle
pinto install -L ~/myperl5 --pull Catalyst
awesome --pull flag
167. Pinto And The Development Cycle
pinto install -L ~/myperl5 --pull Catalyst
168. Pinto And The Development Cycle
pinto install -L ~/myperl5 --pull Catalyst
First, pulls Catalyst and any missing
dependencies into the repository.
169. Pinto And The Development Cycle
pinto install -L ~/myperl5 --pull Catalyst
First, pulls Catalyst and any missing
dependencies into the repository.
Dependencies must not violate pin constraints
or command fails before installing anything.
170. Pinto And The Development Cycle
pinto install -L ~/myperl5 --pull Catalyst
First, pulls Catalyst and any missing
dependencies into the repository.
Dependencies must not violate pin constraints
or command fails before installing anything.
Then, installs Catalyst and dependencies (from
the repository) into your local::lib directory.
171. Gotchas
Pinto does not index exactly like PAUSE
May not work for old or oddly packaged code
Pinto does not enforce permissions
Pinto is strict about version numbers
172. Pinto And Teamwork
Repository on shared storage area
Easy to setup
Performance may suck on NFS
Usual permission issues
173. Pinto And Teamwork
Repository on a remote host via HTTP
pintod on remote host
pinto on local host
Can utilize fast storage
Can use HTTP authentication
180. Using pintod For Installation
$ cpan
cpan[1]> o conf urllist http://remotehost/dev
cpan[2]> install My::App
181. Using pintod For Installation
$ cpan
cpan[1]> o conf urllist http://remotehost/dev
cpan[2]> install My::App
append stack name to url
182. Using pintod For Installation
$ cpan
cpan[1]> o conf urllist http://remotehost/dev
cpan[2]> install My::App
cpanm --mirror http://remotehost/dev
--mirror-only My::App
183. Using pintod For Installation
$ cpan
cpan[1]> o conf urllist http://remotehost/dev
cpan[2]> install My::App
cpanm --mirror http://remotehost/dev
--mirror-only My::App
append stack name to url
184. Using pintod For Installation
$ cpan
cpan[1]> o conf urllist http://remotehost/dev
cpan[2]> install My::App
cpanm --mirror http://remotehost/dev
--mirror-only My::App
pinto --root=http://remotehost install
--stack=dev My::App
185. Using pintod For Installation
$ cpan
cpan[1]> o conf urllist http://remotehost/dev
cpan[2]> install My::App
cpanm --mirror http://remotehost/dev
--mirror-only My::App
pinto --root=http://remotehost install
--stack=dev My::App
point to pinto ser ver
186. Using pintod For Installation
$ cpan
cpan[1]> o conf urllist http://remotehost/dev
cpan[2]> install My::App
cpanm --mirror http://remotehost/dev
--mirror-only My::App
pinto --root=http://remotehost install
--stack=dev My::App
specify the stack
187. Using pintod For Installation
$ cpan
cpan[1]> o conf urllist http://remotehost/dev
cpan[2]> install My::App
cpanm --mirror http://remotehost/dev
--mirror-only My::App
pinto --root=http://remotehost install
--stack=dev My::App
195. Pinto Net works
Might not be a good idea
No obvious way to resolve
namespace conflicts
196. Pinto Net works
Might not be a good idea
No obvious way to resolve
namespace conflicts
We’ll have to wait and see
if this pans out
197. Pinto And OS Packaging
• UsePinto to build up your
application in some
directory
• Packagethe directory with
your favorite OS tools
• Add dependencies on non-
perl stuff
• Deploy as usual
198. Pinto And Other Tools
• Still
just a pile of directories
and files with an index
• AnnoCPAN, MetaCPAN,
CPAN::Mini::Webser ver, etc.
should still work
199. Pinto versus Carton
Pinto Carton
Light weight
VCS-Friendly
Local Patches
Stack Support
Enterprise
Strength
200. Some Odds & Ends
• Dist::Zilla::Plugin::Pinto::Add - release your dist to a
Pinto repository with dzil
• Dist::Zilla::Chef
- use Pinto to manage your project
dependencies within the dzil workflow
202. Extending Pinto
• Create
an Action subclass and override the “execute”
method
• YourAction has access to the configuration, logger, and
repository objects
• Your Action can do whatever it wants
• Like extract POD or ack source code in a distribution
• But the API is not really stable yet
203. Future Plans
More VCS-behaviors: revert, diffs
Report upstream impacts of upgrades
Check stack for unsatisfiable dependencies
Visualize dependency graphs
Faster