AEM provides a gazillion out of the box services ready for your command; it's hard to organize them and be aware of all their possibilities. The scattered web of knowledge doesn't help when you have to perform repetitive tasks without remembering all the technical details each time. Based on CQ Unix Toolkit (https://github.com/Cognifide/CQ-Unix-Toolkit) there is an alternative approach for CQ REST knowledge stack: The idea is very straightforward and simple: 1. Explore CQ REST services knowledge on daily basis 2. Wrap it in simple and handy tool by hiding unnecessary technical details 3. Share them on GitHub and make them useful for everyone 4. Use it and boost your personal and team productivity as well as performance Be creative! Write tools, don't read twice!

1. Organizing The World
of
Adobe CQ REST infinitive possibilities
Arkadiusz.Kita@Cognifide.com
9th April 2014

2. Arkadiusz Kita CQ Developer

3. Wednesday morning, 9th April 2014...
...another beautiful day at work!
There is time for my favourite
coffee!

4. Wednesday morning, 9th April 2014...
...another beautiful day at work!
...but wait - there is an important
e-mail in my inbox!

5. 1 The client needs PoC ASAP

6. 1 The client needs PoC ASAP
2 Next week 1 mln hits campaign!

7. 1 The client needs PoC ASAP
2 Next week 1 mln hits campaign!
3 Agency is ready for content entry!

8. 1 The client needs PoC ASAP
2 Next week 1 mln hits campaign!
3 Agency is ready for content entry!
Mainframe
4 Ad-hoc infrastructure!

9. 1 The client needs PoC ASAP
2 Next week 1 mln hits campaign!
3 Agency is ready for content entry!
Mainframe
4 Ad-hoc infrastructure!
5 Monitoring/maintenance required!

10. It seems like huge pressure and a lot of work to do...

11. ...or maybe rather a lot of CQ clicks!

12. Managing a cloud of...
...thousands of AEM/CQ instances

13. Managing a cloud of...
...thousands of AEM/CQ instances
...hundreds of dev, test, qa, uat, live environments

14. Managing a cloud of...
...thousands of AEM/CQ instances
...hundreds of dev, test, qa, uat, live environments
...tens of client deployments

15. Managing a cloud of...
...thousands of AEM/CQ instances
...hundreds of dev, test, qa, uat, live environments
...tens of client deployments
in a consistent way!

16. Providing condensed...
...for developers/administrators

17. Providing condensed...
checklists
...for developers/administrators

18. Providing condensed...
checklists knowledge
...for developers/administrators

19. Providing condensed...
checklists knowledge know-how
...for developers/administrators

20. Providing condensed...
checklists knowledge know-how
...for developers/administrators
in a reusable way!

21. Last but not least!
1 Automating repetetive tasks

22. Last but not least!
1 Automating repetetive tasks
2 Improving safety and security by not touching anything twice

23. Last but not least!
1 Automating repetetive tasks
2 Improving safety and security by not touching anything twice
3 Focusing on real challenges!!! Discovering new possibilities!

24. Last but not least!
1 Automating repetetive tasks
2 Improving safety and security by not touching anything twice
3 Focusing on real challenges!!! Discovering new possibilities!
4 Being happy, not bored!

25. Last but not least!
1 Automating repetetive tasks
2 Improving safety and security by not touching anything twice
3 Focusing on real challenges!!! Discovering new possibilities!
4 Being happy, not bored!
5 Loving what you do!

26. Use AEM/CQ REST infinitive possibilities!
UNIXUNIX
Try the github.com/Cognifide/CQ-Unix-Toolkit!
Start using it!
Help us develop it!

27. What is the CQ Unix Toolkit?
+ + = UNIXUNIX

28. What is the CQ Unix Toolkit?
+ + = UNIXUNIX
open source user-friendly curl wrapper

29. What is the CQ Unix Toolkit?
+ + = UNIXUNIX
open source user-friendly curl wrapper
set of independent simple micro tools

30. What is the CQ Unix Toolkit?
+ + = UNIXUNIX
open source user-friendly curl wrapper
set of independent simple micro tools
AEM/CQ HTTP REST semi-code/semi-documentation

31. What is the CQ Unix Toolkit?
+ + = UNIXUNIX
open source user-friendly curl wrapper
set of independent simple micro tools
AEM/CQ HTTP REST semi-code/semi-documentation
allows to control CQ/dispatcher by HTTP

32. What is the CQ Unix Toolkit?
+ + = UNIXUNIX
open source user-friendly curl wrapper
set of independent simple micro tools
AEM/CQ HTTP REST semi-code/semi-documentation
allows to control CQ/dispatcher by HTTP
Commons are extracted into one cqapi tool.

33. Use cases

34. Case 1 — distributed REST knowledge
No common place for AEM/CQ REST documentation!

35. Case 2 — hacking packages

36. Case 2 — hacking packages
1 Installing hotfixes/releases automatically
cqput; cqrun

37. Case 2 — hacking packages
1 Installing hotfixes/releases automatically
cqput; cqrun
2 Synchronizing content between instances
cqpkg; cqput; cqbld; cqcp; cqrun; cqsnp

38. Case 2 — hacking packages
1 Installing hotfixes/releases automatically
cqput; cqrun
2 Synchronizing content between instances
cqpkg; cqput; cqbld; cqcp; cqrun; cqsnp
3 Making path-based backups
cqpkg; cqput; cqbld; cqcp

39. Case 2 — hacking packages
1 Installing hotfixes/releases automatically
cqput; cqrun
2 Synchronizing content between instances
cqpkg; cqput; cqbld; cqcp; cqrun; cqsnp
3 Making path-based backups
cqpkg; cqput; cqbld; cqcp
4 Uninstalling deleted packages
cqsnp

40. Case 2 — hacking packages
1 Installing hotfixes/releases automatically
cqput; cqrun
2 Synchronizing content between instances
cqpkg; cqput; cqbld; cqcp; cqrun; cqsnp
3 Making path-based backups
cqpkg; cqput; cqbld; cqcp
4 Uninstalling deleted packages
cqsnp
5 Clearing content paths
cqpkg; cqput; cqrun

41. Case 2 — hacking packages
1 Installing hotfixes/releases automatically
cqput; cqrun
2 Synchronizing content between instances
cqpkg; cqput; cqbld; cqcp; cqrun; cqsnp
3 Making path-based backups
cqpkg; cqput; cqbld; cqcp
4 Uninstalling deleted packages
cqsnp
5 Clearing content paths
cqpkg; cqput; cqrun
6 Moving paths from repository externally
cqpkg; cqput; cqrun; cqsnp; cqsnp

42. Case 2 — hacking packages
1 Installing hotfixes/releases automatically
cqput; cqrun
2 Synchronizing content between instances
cqpkg; cqput; cqbld; cqcp; cqrun; cqsnp
3 Making path-based backups
cqpkg; cqput; cqbld; cqcp
4 Uninstalling deleted packages
cqsnp
5 Clearing content paths
cqpkg; cqput; cqrun
6 Moving paths from repository externally
cqpkg; cqput; cqrun; cqsnp; cqsnp
7 Deleting unknown/garbage packages
cqls; cqdel; cqsnp

43. Case 3 — monitoring/maintenance

44. Case 3 — monitoring/maintenance
1 cqmon – Sling JMX monitoring

45. Case 3 — monitoring/maintenance
1 cqmon – Sling JMX monitoring
2 cqwfl – running/stale workflows status

46. Case 3 — monitoring/maintenance
1 cqmon – Sling JMX monitoring
2 cqwfl – running/stale workflows status
3 cqchk – repository consistency checks

47. Case 3 — monitoring/maintenance
1 cqmon – Sling JMX monitoring
2 cqwfl – running/stale workflows status
3 cqchk – repository consistency checks
4 cqtpm – TarPM compaction

48. Case 3 — monitoring/maintenance
1 cqmon – Sling JMX monitoring
2 cqwfl – running/stale workflows status
3 cqchk – repository consistency checks
4 cqtpm – TarPM compaction
5 cqmrg – Lucene indexes merge

49. Case 3 — monitoring/maintenance
1 cqmon – Sling JMX monitoring
2 cqwfl – running/stale workflows status
3 cqchk – repository consistency checks
4 cqtpm – TarPM compaction
5 cqmrg – Lucene indexes merge
6 cqgc – DataStore garbage collection

50. Case 3 — monitoring/maintenance
1 cqmon – Sling JMX monitoring
2 cqwfl – running/stale workflows status
3 cqchk – repository consistency checks
4 cqtpm – TarPM compaction
5 cqmrg – Lucene indexes merge
6 cqgc – DataStore garbage collection
7 cqdam – AEM/CQ DAM Health Tool

51. Case 3 — monitoring/maintenance
1 cqmon – Sling JMX monitoring
2 cqwfl – running/stale workflows status
3 cqchk – repository consistency checks
4 cqtpm – TarPM compaction
5 cqmrg – Lucene indexes merge
6 cqgc – DataStore garbage collection
7 cqdam – AEM/CQ DAM Health Tool
8 cqosgi + diff – bundle states (by using patterns)

52. Case 3 — monitoring/maintenance
1 cqmon – Sling JMX monitoring
2 cqwfl – running/stale workflows status
3 cqchk – repository consistency checks
4 cqtpm – TarPM compaction
5 cqmrg – Lucene indexes merge
6 cqgc – DataStore garbage collection
7 cqdam – AEM/CQ DAM Health Tool
8 cqosgi + diff – bundle states (by using patterns)
$ cqosgi -m > pattern-file;
$ cqosgi -s org.apache.sling.jcr.davex
Stopping bundle: org.apache.sling.jcr.davex (#109)
$ cqosgi -m | diff -q - pattern-file
Files - and pattern-file differ

53. Case 4 — common devops tasks

54. Case 4 — common devops tasks
1 cqosgi – automated bundle management

55. Case 4 — common devops tasks
1 cqosgi – automated bundle management
2 cqpkg – create ad-hoc empty packages

56. Case 4 — common devops tasks
1 cqosgi – automated bundle management
2 cqpkg – create ad-hoc empty packages
3 cqrepkg – fast offline repackaging for your content snapshots

57. Case 4 — common devops tasks
1 cqosgi – automated bundle management
2 cqpkg – create ad-hoc empty packages
3 cqrepkg – fast offline repackaging for your content snapshots
4 cqclr – clear dispatcher cache

58. Case 4 — common devops tasks
1 cqosgi – automated bundle management
2 cqpkg – create ad-hoc empty packages
3 cqrepkg – fast offline repackaging for your content snapshots
4 cqclr – clear dispatcher cache
5 cqcfg – change bundle configuration remotely
configure SMTP client
tune Sling Main servlet

59. Case 4 — common devops tasks
1 cqosgi – automated bundle management
2 cqpkg – create ad-hoc empty packages
3 cqrepkg – fast offline repackaging for your content snapshots
4 cqclr – clear dispatcher cache
5 cqcfg – change bundle configuration remotely
configure SMTP client
tune Sling Main servlet
6 cqjcr – list JCR nodes, change and add them
dynamically-scoped backups:
cqjcr; cqpkg; cqbld; cqcp; cqdel
change Sling redirect/disable ContentFinder

60. Development

61. Development
Create new tools with us!
Organize the AEM/CQ REST World!

62. Rule 1 — Capture day to day requests

63. Rule 2 — Write user-friendly usage messages
$ cqbld
Usage: cqbld [OPTION...] package-name
Build (rebuild) already uploaded package by group id and name in CQ Package
Manager using instance URL.
Examples:
cqbld -u admin pack # Build package named pack
cqbld -u admin -g GRP pack # Build package named pack in group GRP
cqbld -i http://localhost:5510 # Build package for localhost instance on tcp
-g com.group stuff # port 5510 named stuff in group:com.group
-p secret # with password provided: secret
Options:
-u use specified usernamed for connection
-p use provided password for authentication
-i use specified instance URL to connect
-g locate package by additional group ID

64. Rule 3 — Choose descriptive code over comments
# Invoke curl tool with current referer and
# instance URL and credentials to get list
# of packages
STATUS=`$CURLBIN -s --write-out "%{http_code}"
-u "$u:$p" -H "Referer: $url/crx/packmgr"
-F cmd=ls "$url/crx/packmgr/service.jsp"`

65. Rule 3 — Choose descriptive code over comments
# Invoke curl tool with current referer and
# instance URL and credentials to get list
# of packages
STATUS=`$CURLBIN -s --write-out "%{http_code}"
-u "$u:$p" -H "Referer: $url/crx/packmgr"
-F cmd=ls "$url/crx/packmgr/service.jsp"`

66. Rule 3 — Choose descriptive code over comments
PACKAGE_MANAGER_PATH="/crx/packmgr"
REFERER="${instance}${PACKAGE_MANAGER_PATH}"
AUTH="${username}:${password}"
FILEPATH="${PACKAGE_MANAGER_PATH}/service.jsp"
FULL_URL="${instance}${FILEPATH}"
REFERER_HEADER= "Referer: ${REFERER}"
COMMAND="cmd=ls"
Use shellcheck.net to validate POSIX shell syntax!

67. Rule 3 — Choose descriptive code over comments
STATUS=$("${CURLBIN}"
-s
--write-out "%{http_code}"
-u "${AUTH}"
-H "${REFERER_HEADER}"
-F "${COMMAND}"
"${FULL_URL}")

68. Rule 3 — Choose descriptive code over comments
STATUS=$("${CURLBIN}"
-s
--write-out "%{http_code}"
-u "${AUTH}"
-H "${REFERER_HEADER}"
-F "${COMMAND}"
"${FULL_URL}")

69. Rule 4 — Use POSIX getopts for custom parameters
while getopts ":abc${apigetopts}" opt
do
case "${opt}" in
...
?)
echo "Invalid option: -${OPTARG}" >&2; _usage;;
:)
echo "Option -${OPTARG} requires an argument." >&2;
_usage;;
esac
done

70. Rule 5 — Use cqapi to detect/handle curl command
cqapi
shared code, status, parsing common options, error handling

71. Rule 6 — Use the most usable REST HTTP
pick!
rate!
choose!

72. Rule 7 — Reserve API cmdline options for CQ/AEM connectivity
-u username
-i instance URL
-p password

73. Rule 8 — Avoid including external custom dependencies
This can be harmful,
hard in use and not
portable!

74. Rule 9 — Keep It Simple Stupid
KiSS
Keep toolkit files
as small as possible
up to 400 LOC

75. Rule 9 — Keep It Simple Stupid
KiSS
Keep toolkit files
as small as possible
up to 400 LOC
Manage complexity!
Simplify logic!

76. Thank you!
Q&A

77. Resources
https://github.com/Cognifide/CQ-Unix-Toolkit
Presentation symbols provided by Tango Desktop Project
blog entry on Cognifide website