1. The Art of the Upgrade
Kyle Adams
Solutions Engineer at Alfresco
@kylefadams
#SummitNow

2. Objectives
• Arm you with the strategy and tactics to
perform a successful upgrade of the core
Alfresco repository.
#SummitNow
#SummitNow

3. Strategy and Tactics
"Strategy without tactics is the slowest route
to victory.”
“Tactics without strategy is the noise before
defeat.”
– Sun Tzu, The Art of War
#SummitNow
#SummitNow

4. Strategy and Tactics
“...tactics are the actual means used to gain
an objective, while strategy is the overall
campaign plan…”
[1]
[1] http://en.wikipedia.org/wiki/Tactic_(method)
#SummitNow
#SummitNow

5. Upgrade Strategy
Strategy: Accurately predict how my Alfresco instance
will behave throughout the upgrade process.
Tactic:
Validate
Tactic:
Monitor
Tactic:
Execute
Tactic:
Prepare
#SummitNow
#SummitNow

6. Upgrade Strategy
Strategy: Accurately predict how my Alfresco instance
will behave throughout the upgrade process.
Tactic:
Validate
Tactic:
Monitor
80% of Work
Tactic:
Execute
Tactic:
Prepare
#SummitNow
#SummitNow

7. Prepare
"To ... not prepare is the greatest of crimes;
to be prepared beforehand for any
contingency is the greatest of virtues.”
[1]
[1] Sun Tzu, The Art of War
#SummitNow
#SummitNow

8. Prepare – Current State
•
•
•
•
•
•
•
Count of all database tables
Size and number of files in content store
Size of lucene or Solr indexes
Custom Content Models
Custom Process definitions
Installed modules
Customizations
#SummitNow
#SummitNow

9. Prepare – Upgrade Path
•
http://docs.alfresco.com/4.2/topic/com.alfresco.enterprise.doc/concepts/up
grade-path.html
#SummitNow
#SummitNow

10. Prepare – Backup/Restore
• Backup Prod Database
• Restore into new instance
• Backup Prod lucene/solr indices
• Restore into new instance
• Backup Prod ContentStore
• Restore into new instance
#SummitNow
#SummitNow

11. Prepare – Backup/Restore
• Backup Prod ContentStore
• What if your ContentStore is too large to
restoring into a lower environment for
testing?
#SummitNow
#SummitNow

12. Prepare – Backup/Restore
• No excuse!
• We can mock up the ContentStore
against the database using small sample
files.
• https://github.com/kylefernandadams/alfre
sco-mock-content-generator
• CAUTION: Only use this for testing
purposes.
#SummitNow
#SummitNow

13. Prepare – Alfresco Instances
• Install an out-of-the-box Alfresco instance for
each upgrade segment
• Current Version: 3.3.4
• Target Version: 4.2.0
• Path: 3.3.4 -> 3.3.5 -> 4.2.0
• Don’t forget to request license files for each
minor version from Alfresco support!
• Point each instance to common content store,
database, and indices.
#SummitNow
• Do not perform an in-place upgrade!
#SummitNow

14. Prepare – Alfresco Instances
• Do not perform an in-place upgrade!
#SummitNow
#SummitNow

15. Prepare – Patches
• Know exactly which patches will run
during each segment of the upgrade.
• alfresco/WEBINF/classes/alfresco/patch/patchservices-context.xml
• Patch Java
docs:http://dev.alfresco.com/resource/docs/java/reposi
tory/org/alfresco/repo/admin/patch/impl/packagesummary.html
#SummitNow
#SummitNow

16. Prepare – Patches
• Still unsure?
• Perform a dry run of the upgrade with
OOTB installation.
• Set logging to debug for the following
packages
• org.alfresco.repo.admin.patch
• org.alfresco.repo.domain.patch
• Review the logging output
#SummitNow
#SummitNow

17. Prepare – Customizations
• Only include bare minimum extensions
and overrides (Example: content model)
• Restore a Prod backup into the
extensionRoot
#SummitNow
#SummitNow

18. Prepare – Tools for Monitoring
•
•
•
Logs
• tail, text editor, Splunk
JVM
• YourKit, JProfiler
DB Queries
• Vendor-specific tools (MySQL Workbench, SQL
Developer, PgAdmin, SQL Server Mgmt Studio)
• Quickly toggle log4j logging for java.sql.*
#SummitNow
#SummitNow

19. Prepare – Tools for Monitoring
• Hardware & OS (CPU, Memory, Disk, etc)
• top, nmon, ProcessExplorer
#SummitNow
#SummitNow

20. Upgrade Strategy
Strategy: Accurately predict how my Alfresco instance
will behave throughout the upgrade process.
Tactic:
Validate
Tactic:
Monitor
Tactic:
Execute
Tactic:
Prepare
#SummitNow
#SummitNow

21. Execute
• Validate your newly restored instance is
working as expected.
• Example: 3.3.4
• Shutdown the instance
• Startup the instance for the first segment
of the upgrade.
• Example: 3.3.4 -> 3.3.5
#SummitNow
#SummitNow

22. Upgrade Strategy
Strategy: Accurately predict how my Alfresco instance
will behave throughout the upgrade process.
Tactic:
Validate
Tactic:
Monitor
Tactic:
Execute
Tactic:
Prepare
#SummitNow
#SummitNow

23. Monitor - Logs
• Based on logging output from your dry
run, you should be able to determine the
overall progress of your upgrade.
• What if the logging output appears to be
stalled?
#SummitNow
#SummitNow

24. Monitor - Logs
• Stalled?
#SummitNow
#SummitNow

25. Monitor - JVM
• Leverage a Java profiling tool such as
YourKit
• YourKit will tell you exactly which
class and methods are being called.
• What if it appears that the upgrade has
stalled on a given method call?
#SummitNow
#SummitNow

26. Monitor – JVM
• Still Stalled???
#SummitNow
#SummitNow

27. Monitor – SQL Queries
• Use your DB vendor-specific tools to monitor
the queries being executed
• Don’t have MySQL Workbench, SQL Developer,
PgAdmin, or SQL Server Mgmt Studio?
• Quickly toggle log4j logger for java.sql.*
• You must add java.sql to customlog4j.properties
#SummitNow
#SummitNow

28. Upgrade Strategy
Strategy: Accurately predict how my Alfresco instance
will behave throughout the upgrade process.
Tactic:
Validate
Tactic:
Monitor
Tactic:
Execute
Tactic:
Prepare
#SummitNow
#SummitNow

29. Validate – Smoke Tests
• Test basic Alfresco functionality.
• Test one or more of your major use cases.
• For the latest 4.x versions, leverage
WebDrone.
#SummitNow
#SummitNow

30. Upgrade Strategy
Strategy: Accurately predict how my Alfresco instance
will behave throughout the upgrade process.
Tactic:
Validate
Repeat for
each segment
Tactic:
Monitor
Tactic:
Execute
Tactic:
Prepare
#SummitNow
#SummitNow

31. Gotchas – Cyclical ACL’s
• FixAclInheritancePatch could fail with
StackOverflowException during 3.4
upgrade.
• Added cyclical ACL detection in 3.4.13
• Jira: MNT-2580
#SummitNow
#SummitNow

32. Gotchas – Orphaned Nodes
• Orphaned Nodes and Associations to
deleted nodes
• Reindex could fail with
“InvalidNodeRefException: Node does not
exist”
#SummitNow
#SummitNow

33. Gotchas – Orphaned Nodes
• Find orphaned nodes:
SELECT n.*
FROM alf_node n, alf_qname q, alf_namespace ns
where q.id = n.type_qname_id
and ns.id = q.ns_id
and not (ns.uri = 'http://www.alfresco.org/model/system/1.0' AND q.local_name =
'deleted')
AND NOT EXISTS
(SELECT 1 FROM alf_store where root_node_id = n.id)
AND NOT EXISTS
(SELECT 1 FROM alf_child_assoc where child_node_id = n.id)
#SummitNow
#SummitNow

34. Gotchas – Orphaned Nodes
• Find associations to deleted nodes:
SELECT a.*
FROM alf_child_assoc a, alf_node n, alf_qname q, alf_namespace ns
where (n.id = a.child_node_id or n.id = a.parent_node_id)
and q.id = n.type_qname_id
and ns.id = q.ns_id
and ns.uri = 'http://www.alfresco.org/model/system/1.0' AND q.local_name =
'deleted'
#SummitNow
#SummitNow

35. Gotchas – Orphaned Nodes
• CAUTION: Do not attempt to fix these by
hacking the database!
• Detection and fix via Java has been
added as of 4.1.4
• Jira: MNT-8531
#SummitNow
#SummitNow

36. Gotchas – Nodes without Txn’s
• May cause schema updates to fail with
*ContraintViolationExceptions.
#SummitNow
#SummitNow

37. Gotchas – Nodes without Txn’s
• Find nodes without transactions:
SELECT *
FROM alf_node node
WHERE node.transaction_id
NOT IN (
SELECT id FROM alf_transaction
);
#SummitNow
#SummitNow

38. Gotchas – Nodes without Txn’s
• CAUTION: Do not alter the database
without the consent or the assistance of
Alfresco Support or Alfresco Consulting.
• Fix:
UPDATE alf_node set transaction_id = (SELECT MIN(id) from
alf_transaction)
WHERE transaction_id NOT IN (SELECT id FROM
alf_transaction);
#SummitNow
#SummitNow

39. Gotchas – Long-running
Patches
• MigrateVersionStorePatch (2.x  3.x)
• Run asynchronously as a scheduled job
#
Never:
* * * * * ? 2099
#
Once every thirty minutes: 0 0/30 * * * ?
version.store.migrateVersionStore.cronExpression=* * * * * ? 2099
# Limit number of version histories to migrate per job cycle, where -1 =
unlimited. Note: if limit > 0 then need to schedule job to run regularly in order to
complete the migration.
version.store.migrateVersionStore.limitPerJobCycle=-1
version.store.migrateVersionStore.runAsScheduledJob=false
#SummitNow
#SummitNow

40. Gotchas – Long-running
Patches
• ContentUrlConverterPatch (...  3.2.x)
• Run asynchronously as a scheduled job
#
Never:
* * * * * ? 2099
#
Once every thirty minutes: 0 0/30 * * * ?
system.content.contentUrlConverter.cronExpression=* * * * * ? 2099
system.content.contentUrlConverter.threadCount=2
system.content.contentUrlConverter.batchSize=500
system.content.contentUrlConverter.runAsScheduledJob=false
#SummitNow
#SummitNow

41. Gotchas – Long-running
Patches
• NodeDeleted.sql (…  4.1.x)
• Renames alf_node to t_alf_node
• Recreate the alf_node table without the
node_deleted column
• Insert all rows from t_alf_node into
alf_node
• Drop the temporary t_alf_node table
#SummitNow
#SummitNow

42. Gotchas – Database Statistics
• Don’t forget to recompute your database
statistics!
• Without histograms if you’re using
Oracle
#SummitNow
#SummitNow

43. Summary
• Be meticulous!
• Have attention to detail!
• Leave nothing up to mystery!
• There should be no surprises.
• “Know thy self, know thy enemy. A
thousand battles, a thousand victories”
•
[1] Sun Tzu, The Art of War
[1]
#SummitNow
#SummitNow

44. Summary
• “The supreme art of war is to subdue the
enemy without fighting.”
[1]
•
[1] Sun Tzu – The Art of War
#SummitNow
#SummitNow

45. Contact Information
•
•
•
•
Kyle Adams
Solutions Engineer at Alfresco
Email: kyle.adams@alfresco.com
Twitter: @kylefadams
#SummitNow
#SummitNow

46. #SummitNow