3. #evolverocks
A way of thinking, behaving or working
that exists in an organization
CULTURE IS KEY
3
4. #evolverocks
ITERATE EVERYTHING, FEAR NOTHING!
TRY STUFF!
• Hypothetical: lets just say you are asked to start a major project
initiative where they would like multiple sites migrated to AEM in
parallel and quickly…..
• What do you do?
4
5. #evolverocks
MVP – Minimal Viable Product, the smallest useful set of
capabilities that will meet the business need in production
MVP AND RELEASE PLANNING
5
6. #evolverocks
Abstraction:
Story review components ensures multi-use lego approach
Story Mappings aids in MVP creation
CSS Strategy shared plus specific
ABSTRACTION LEADS TO FREEDOM
DOESN’T HURT OPTIMIZATION EITHER
6
7. #evolverocks
Bigger projects vs smaller projects
Mature team including solid Product Owners
New team, learning curve/ramp up requirements
Pairing!
Time zones & communications
SCRUM VS KANBAN
CHOICES CHOICES
7
8. #evolverocks
Obvious answer: when communication goes to *$@#;
When standups take more than 15 minutes
BUT (behold the underlying truth!)
Sometimes you need to try something to bring along more people to the agile way
• New to agile, new to AEM, new to processes
• Pair people so they function as one – only one speaks at Standup
• Make each team as cross functional and complete as possible – include UX,
PO, QA, Analytics, Devs, SM
SCRUM TEAM SIZING
HOW BIG IS TOO BIG?
8
9. #evolverocks
If the process is nailed AUTOMATE IT! (why not?)
• Releases
• Deploys
• Testing
• Go/NoGo
• Trouble-shooting
AUTOMATION FOCUS
FREE THE HUMANS – MAKE THE MACHINES DO IT!
9
10. #evolverocks
SHARED PAIN IS A GREAT MOTIVATOR:
UAT Handoff
• Provide UAT Wikis
• Review Wikis (UAT Team)
Beta Release
• Cut Code and Create Release Candidate
• Deploy to Beta (Test) Environment
• Mark tickets for handoff
PRODUCTION SUPPORT
• Handle issues incoming from production
DOTS: delivery of the sprint duties with sprint cycle rotation
40%
60%
10
11. #evolverocks
Focus one team on DOTS and live the pain
HOW TO MAKE DOTS LESS SCARY
NOTHING TO FEAR BUT FEAR ITSELF (OR JUST ELIMINATE THAT NASTY
FEAR STUFF)
Ooohh
Noooo
I’m on
DOTS!
11
12. #evolverocks
• Github Webhook to kick off build (Jenkins)
• Build includes Unit/Integration/Functional Tests
• Karma Tests
• Deploy to CI
• Validation Scripts -> Check bundle status and that right ones are there
• SmokeTest (author and publish are different) ensure app is up and running properly
CONTINUOUS INTEGRATION UPLIFT
First Stop: CI/CD
12
13. #evolverocks
• Content Syncs from Prod to Test Envs (Prod -> DW -> TestEnv) (remember Grabbit?)
https://docs.adobe.com/docs/en/aem/6-2/administer/content/assets/best-practices-for-
assets/assets-migration-guide.html
• Release Cut (merge master into Release branch)
• Update Jira Tickets with ToProd Info (tagging tickets that will be deployed in next
release)
• Update UAT Handoff Wikis -> Letting business and UAT teams know which tickets are
ready for UAT and their status
• Deploy latest release to Betas for testing.
RELEASE PROCESS AUTOMATION
THINGS WE USED TO DO MANUALLY THAT ARE NOW CRON JOBS
Second Stop: Build & Release
13
14. #evolverocks
Utility Jobs - actions that used to be 100% manual now able to be triggered by
Jenkins Job:
• Server Restarts
• Server Clean UP (Remove Zombie Bundles)
• Clear Dispatcher Cache
• Clear JSP Cache
• Cherry-Pick (needed commits onto next release branch)
• Build and Deploy Random Branch Jobs
Our success: we extrapolated this entire process for our analytics code stream; mirroring
the entire automated process for this additional code stream. Puppet makes it all work
locally!
RELEASE PROCESS - ROCKSTAR
14
15. #evolverocks
How to find out before the business team that something is wrong in PROD???
Spiked a story on identifying key transactions/metrics
Spiked a story to learn with the experts on how to use our tool: AppD
Focused on Instrumenting Business Transactions
First Focus: FOLLOW THE $$$$$$$ -- do CHECKOUT END TO END!
Collaborative team approach
Tried all of it out in the BETA environment – Learned the hard way that we could
impact production with our instrumentation so testing it is a good thing!
NOW FOR THE HEAVY LIFTING:
TROUBLE-SHOOTING AUTOMATION
15
16. #evolverocks
• Instrumentation For Checkout
- Metrics on Avg Load, Response time, #Errors for every page in the flow
- Counts on Splash Page Errors (specific reasons the flow fails or crashes)
- Counts on flow entry and completion (rough look at close rate)
• Instrumentation For Business Class
- Metrics on Avg Load, Response time, # Errors on Home page vs other pages
- Monitoring on Geolocation service
- Number of times served up from "FingerPrint" (on network) availablityh
- Number of times served up from fallback
UPLIFT IN METRICS FOR THE DEV TEAM:
ITS NOT JUST ABOUT ANALYTICS FOR THE BUSINESS
16
17. #evolverocks
Graphing the Metrics
- Trending information on Error counts (Error/min VS Exception/min)
- Dashboarding for DOTS (support) team
- Traffic to nodes and datacenters
- Response times, errors, load
VISUALS
17
19. #evolverocks
Current Alerts
- Average response times get too high
- End User Monitoring times get too high
- Disk space gets too low
- Error rate gets too high
Axioms
- Only send alerts containing actionable steps
to solve the issue!
- Notices only go to people who can fix
the problem
NOTIFICATIONS
19
20. #evolverocks
FUTURE DREAMS
Can actually be configured to trigger Jenkins jobs (like restarts)
A great example would be when Disk space gets too low (due to AEM needing
compaction)- AppD could alert and in the alert trigger a job that:
1. Emails support teams
2. Takes specific node out of node balancer
3. Performs compaction (including restarts)
4. Validates node
5. Puts back in load balancer
20
21. #evolverocks
DEVOPS: THE KEYS TO THE KINGDOM
ITS HARD WORK, BUT WELL WORTH THE EFFORT!
• For the 6.1 upgrade, we tackled improving our delivery & tooling with
the migration to the cloud
• We wanted elasticity, consistency that had eluded us in our VM env
• VAGRANT image made it so ALL DEVS matched PROD! QUALITY
• Configuring any new environment, be it author, pub can be done
almost instantly with our “infrastructure as code”
• Fully redundant across data centers, GSLB
• =#FREEDOM!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
21
22. TOOLS USED TO HELP AUTOMATE THE
AGILE DEV/TEST/DELIVERY CYCLE
#evolverocks 22