2. Introductions - Travis McBee / Lance Rochelle
Travis McBee is the manager for PCF, PKS, and Redis Engineering with Enterprise Platform Services
at Wells Fargo. Currently he is accountable for the automation first mentality within Cloud
Engineering for PCF, PKS, and Redis. His role allows him to work closely with other engineering
teams within Wells Fargo and guide the successful migration of application to our internal cloud
environments. His leadership and guidance have been paramount in the success of the platform
engineering.
Prior to Wells Fargo Travis attended Virginia Tech and holds a Bachelor of Science degree in
Management Science and Information Systems.
2
Lance Rochelle is a Principal Engineer within Enterprise Platform Services at Wells Fargo. Currently
he is accountable for engineering solutions within Wells Fargo in regards to cloud environments and
works closely with other teams in the bank to make sure the project is successful. With over 20 years
of experience in security, infrastructure and platform engineering, and leadership in the financial
industry, Lance has a proven record within Wells Fargo for delivering successful projects.
Prior to Wells Fargo, Lance held various contract positions doing security engineering for
Department of Defense.
3. What can be done to create a world class platform team?
Ask similar companies within your vertical what works for them and why it works and what does not work.
Find out from the platform partner if they have insight to how they would run it.
Ask your contracting companies and HR business partners to do market research on qualified candidates, be open minded as much as possible.
Explain to Senior Leaders and HR business partners that the platform engineering team are not tied to a specific skillset.
Participate in the rest of this presentation.
3
4. 30+ years in the making
In [almost] the beginning.
• Everything in the technology industry is manually installed, configured, and maintained.
• This made it very difficult when attempting to configure two components similarly, humans can cause quite a bit of errors.
• Most engineers are tied to a very specific skillset.
Here comes the World Wide Web. The industry starts to change drastically.
• Companies start deploying servers at a faster rate than ever before.
• Skillsets started to diversify based off necessity, but for the most part people would focus on one or two disciplines.
The era of pseudo automation begins.
• Each component type begins to deploy automation for their technology stack. This started the divergence into component level skillsets (i.e.
Operating System, Middleware, and Database concentrations).
• This forces engineers and administrators to make a decision; continue doing things manually or use automation tools for their technology
stack. Problem: each technology stack used their own nearly proprietary way to do automation.
Enterprise automation solutions arise out of necessity.
4
5. Recent engineering responsibility changes
Component counts increase at an alarming rate.
• With the arrival of containers and an automation first mindset, the number of devices on the network are going to continue to grow to an
unprecedented rate.
Engineers need to know more today than ever before.
• With proper enterprise automation tooling engineers and administrators need to have an extensive background in the entire component
stack.
• The scope of what an engineer is required to know grows as the platforms become more complex.
• Several additional contributing factors.
– IT Engineers have moved into different roles over the past several years.
– Engineering tasks are hidden by automation.
– Candidate pool is shrinking due to increased demand.
5
6. What to look for in a platform engineer?
Senior administrator / engineer
Strong scripting experience
Automation mindset
Experience with automation tools
Teachable
Diversity
Experience working with app development teams
Intermediate networking skills
Basic understanding of security
Understanding of Cloud technologies
6
7. Automate everything / Tools of the trade
While there are many tools of the trade; not one tool solves all the problems.
Standard DevOps Engineering tools are very useful. (https://landscape.cncf.io/) (1,229) (retrieved 10/4/2019 09:15 ET)
The platform team may have to create some home grown tools, but as you can see there are quite a few tools to choose from.
7
8. Question Time?
When people work together do they ?
a) Get the job done faster
b) Take longer to finish
c) Not get it done
d) None of the above
e) It depends
8
9. Question Time?
When people work together do they ?
a) Get the job done faster
b) Take longer to finish
c) Not get it done
d) None of the above
e) It depends
In the scenario with software development / engineering the case diminishing returns / Brooks’s Law “adding human resources to a late software
project makes it later“, comes into play more often than not.
Software and Platform Engineers require time to get up to speed and understand the problem, just throwing human resources at this type of
problem at the last minute will cause more harm than good.
9
10. Two Pizza Teams -- Agile
Two Pizza team was coined nearly a decade ago, with the two driving factors of Efficiency and Scalability
Efficiency -- The ratio of tested and delivered updates rated against the energy put into the work.
Scalability -- Being able to upgrade on demand, however; at some point the team becomes to large and needs to be broken into smaller teams.
Links -- People need to make connections with their team members the larger the team the more connections they need to make and maintain.
Keeping teams smaller eliminates miscommunication, mismanagement, and misinterpretation.
Agile --
Iterative
– 2 to 3 week sprints depending on what the teams focus
Efficient Communication
– Some people require face-to-face communication; but that might not be possible with the skillsets platform engineering teams require
Utilize tools at your disposal
– Train in communication, coaching conversation can be hard.
– Create a personal connection with each person on the team.
– Be clear about expectations, be genuine, and be authentic within your communication delivery methods.
Short Feedback loop
– Utilizing small sprints and scrum ceremonies creates a short feedback loop
Focus
– Quality over Quantity
10
11. Paired Programming / Paired Engineering
11
How much is to much?
We have seen that pairing 100% of the time is mentally draining, our goal is about 20~24 hours per week; this leaves time for meetings and
also allows us to work with a geographically dispersed team during their peak work time. This may vary depending on location and team
personalities.
How can it be done remotely?
Utilizing your favorite collaboration tools is extremely helpful; gamifying the tools use is also beneficial.
What does it solve?
– At first glance to the number crunchers it appears to just be added head count; it is a hard sell.
– It seems counter intuitive to do this, it is not!!!
– Reduces the number of errors from the start
– Increases learning
– Allows for engagement of team members
– Increases communication
What do the ceremonies look like?
– Weekly iteration planning / Sprint planning
– Daily standups
– Sprint review
– Weekly retrospective / Sprint retrospective
12. Training / Lab Environments
12
Listen to the needs of the engineering team -- Engineers can be subtle when it comes to training requests; listen with the intent of listening
during coaching and review sessions
Regular consistent training -- As engineers are hired to the team make sure their onboarding is consistent and they are trained.
Utilize vendors -- Several vendors have classes online and through in person training.
Conferences -- Sometimes it is a hard sell to senior leaders to be able to send several folks to a conference; budget cuts can cut into this along
with other concerns. However; it is one of the best ways to learn from other companies in other verticals and our peers on technological best
practices.
Cross Training -- Going back to the paired programing allow folks to run in different circuits within the team; allow novice-novice, senior-novice,
and senior-senior pairs to work together during appropriate times.
Lab Environments – Critical to the overall success of any platform is the ability to test changes in a controlled environment.
– Testing automation
– Software delivery certification
– Being able to teach in environments within your facility is very valuable
13. How many people does it take to run the platform
After talking with several companies their biggest problem wasn’t “running” the platform, it was keeping up with the amount of change from
the vendor. Some vendors update their software weekly (if not more).
With an automation first mindset the number of foundations or infrastructure environments isn’t the key to how many engineers are needed to
run the platform.
Managing all of the updates for all of the components is not an skill that everyone has to start with.
13