The CloudStack usage service is used to track consumption of resources in Apache CloudStack for reporting and billing purposes. This talk will give an overview of the service before diving deeper into how data is processed from the CloudStack database into the different usage types before being aggregated into billable units or time slices in the usage database.
2. C l i c k t o e d i t
The Cloud Specialists
ShapeBlue.com @ShapeBlue
Ab ou t m e
• Cloud Architect @ ShapeBlue
• Background:
• Cloud and virtualization architect with 20 years experience
from the service provider, financial and manufacturing
industries.
• Specialize in:
• Cloud infrastructure architecture and engineering.
• Virtualization - VMware vSphere, Citrix XenServer, KVM.
• Automationand configurationmanagement, love Ansible.
• Involvedwith CloudStacksince version 2.1.
• Downtime: traveland motorbikes.
3. C l i c k t o e d i t
The Cloud Specialists ShapeBlue.com @ShapeBlue
“ShapeBlue are expert builders of public & private
clouds. They are the leading global CloudStack
services company.”
Ab o u t S h ap eB l u e
4. C l i c k t o e d i t
The Cloud Specialists
ShapeBlue.com @ShapeBlue
S h a p e B l u e c u s t o m e r s
5. C l i c k t o e d i t
The Cloud Specialists
ShapeBlue.com @ShapeBlue
S h a p e B l u e c u s t o m e r s
6. C l i c k t o e d i t
The Cloud Specialists
ShapeBlue.com @ShapeBlue
S h a p e B l u e c u s t o m e r s
7. C l i c k t o e d i t
The Cloud Specialists
ShapeBlue.com @ShapeBlue
CloudStack usage service
8. C l i c k t o e d i t
The Cloud Specialists
ShapeBlue.com @ShapeBlue
O ver vi ew
CloudStack usage is a complimentary service
which:
• tracks end user consumption of CloudStack
resources
• summarizes all data in a separate database for
reporting or billing
The usage database can be:
• queried directly
• queried through the CloudStack API
• or it can be integrated into external billing or reporting systems.
9. C l i c k t o e d i t
The Cloud Specialists
ShapeBlue.com @ShapeBlue
H ow d o I i n s t al l i t ?
CentOS:
# yum install cloudstack-usage
# chkconfig cloudstack-usage on
# service cloudstack-usage on
Ubuntu:
# apt-get install cloudstack-usage
# update-rc.d cloudstack-usage defaults
# service cloudstack-usage on
Configuration files:
# ls -l /etc/cloudstack/usage/
total 4
lrwxrwxrwx. 1 root root 40 Sep 8 08:18 db.properties -> /etc/cloudstack/management/db.properties
lrwxrwxrwx. 1 root root 30 Sep 8 08:18 key -> /etc/cloudstack/management/key
-rw-r--r--. 1 root root 2968 Jul 12 10:36 log4j-cloud.xml
10. C l i c k t o e d i t
The Cloud Specialists
ShapeBlue.com @ShapeBlue
G l ob al s et t i n g s
Setting Comment
usage.stats.job.aggregation.range The range of time for aggregating the user statistics specified in
minutes (e.g. 1440 for daily (default), 60 for hourly)
usage.stats.job.exec.time The time at which the usage statistics aggregation job will run
(default 00:15). This is utilized together with the
usage.stats.job.aggregation.range to determine service scheduling.
enable.usage.server Switches usage service on/off
usage.aggregation.timezone Timezone used for usage aggregation (default “GMT”)
usage.execution.timezone Timezone for usage job execution
usage.sanity.check.interval Interval (in days) to check sanity of usage data
usage.snapshot.virtualsize.select Set the value to true if snapshot usage need to consider virtual size,
else physical size is considered (default “false”)
11. C l i c k t o e d i t
The Cloud Specialists
ShapeBlue.com @ShapeBlue
U s ag e even t s an d t y p es
• All lifecycle events for resources in
CloudStack are tracked in the
cloud.usage_event table.
• This table lists all timestamps for:
• Create / destroy
• Start / stop
• Request / release
• These events are processed by the
usage service and categorized as usage
types.
Create
Start
Request
Release
Stop
Destroy
12. C l i c k t o e d i t
The Cloud Specialists
ShapeBlue.com @ShapeBlue
U s ag e t y p es
# cloudmonkey list usagetypes
count = 21
usagetype:
+-------------+--------------------------------------+
| usagetypeid | description |
+-------------+--------------------------------------+
| 1 | Running Vm Usage |
| 2 | Allocated Vm Usage |
| 3 | IP Address Usage |
| 4 | Network Usage (Bytes Sent) |
| 5 | Network Usage (Bytes Received) |
| 6 | Volume Usage |
| 7 | Template Usage |
| 8 | ISO Usage |
| 9 | Snapshot Usage |
| 10 | Security Group Usage |
| 11 | Load Balancer Usage |
| 12 | Port Forwarding Usage |
| 13 | Network Offering Usage |
| 14 | VPN users usage |
| 21 | VM Disk usage(I/O Read) |
| 22 | VM Disk usage(I/O Write) |
| 23 | VM Disk usage(Bytes Read) |
| 24 | VM Disk usage(Bytes Write) |
| 25 | VM Snapshot storage usage |
| 27 | VM Snapshot on primary storage usage |
| 28 | VM Backup storage usage |
+-------------+--------------------------------------+
13. C l i c k t o e d i t
The Cloud Specialists
ShapeBlue.com @ShapeBlue
P r o c e s s f l o w – AP I c a l l a n d u s a g e j o b s
API
• Call received
API
• Call processed
DB
• Lifecycle action
written to
cloud.usage_event
Start
• Execute usage job at:
• usage.stats.job.exec.time
• + usage.stats.job.aggregation.range
Copy
• Copy all new events:
• FROM: cloud.usage_event
• TO: cloud_usage.usage_event
• Copy account table for reference
Process
• Summarise each usage type per account:
• Populate helper tables, e.g. usage_storage,
usage_vm_instance, etc
• Aggregate usage types for all accounts
• in cloud_usage.cloud_usage
• based on usage.status.job.aggregation.range
14. C l i c k t o e d i t
The Cloud Specialists
ShapeBlue.com @ShapeBlue
U s ag e even t s
15. C l i c k t o e d i t
The Cloud Specialists
ShapeBlue.com @ShapeBlue
H el p er t ab l es
• Helper tables summarise each resource with:
• start and end datestamp
• usage type (if required)
• Note though – every table has a different schema depending on
the resource usage type
• Example (all in the cloud_usage) DB:
• usage_vm_instance
• usage_storage
• usage_vm_disk
• Etc.
16. C l i c k t o e d i t
The Cloud Specialists
ShapeBlue.com @ShapeBlue
H e l p e r t a b l e s - e xa m p l e s
17. C l i c k t o e d i t
The Cloud Specialists
ShapeBlue.com @ShapeBlue
H e l p e r t a b l e s - e xa m p l e s
18. C l i c k t o e d i t
The Cloud Specialists
ShapeBlue.com @ShapeBlue
H e l p e r t a b l e s - e xa m p l e s
19. C l i c k t o e d i t
The Cloud Specialists
ShapeBlue.com @ShapeBlue
U s ag e ag g r eg at i on
• Once helper tables are processed the
cloud_usage.cloud_usage table is populated.
• The usage service analyses the helper table
entries and divides the time usage for each by
the time period defined in
“usage.stats.job.aggregation.range”.
• E.g.:
• how many hours has a VM been running in each 1440
minute (24 hour) period
• how many hours has each GB of storage been used in
1440 minute period
20. C l i c k t o e d i t
The Cloud Specialists
ShapeBlue.com @ShapeBlue
C l o u d _ u s a g e . c l o u d _ u s a g e
Usage
type
Raw
usage
Resource
ID
Vol/snap
size
Start /
end date
21. C l i c k t o e d i t
The Cloud Specialists
ShapeBlue.com @ShapeBlue
Q u er y t h r ou g h AP I w i t h C l o u d Mo n k e y
22. C l i c k t o e d i t
The Cloud Specialists
ShapeBlue.com @ShapeBlue
R e p o r t i n g e xa m p l e – r u n n i n g VMs
23. C l i c k t o e d i t
The Cloud Specialists
ShapeBlue.com @ShapeBlue
R ep or t i n g exam p l e – n et w or k u t i l i z at i on
24. C l i c k t o e d i t
The Cloud Specialists
ShapeBlue.com @ShapeBlue
R ep or t i n g exam p l e – vol u m es
25. C l i c k t o e d i t
The Cloud Specialists
ShapeBlue.com @ShapeBlue
R e p o r t i n g e x a mp l e – I P a d d r e s s e s , p o r t f o r w a r d i n g a n d VP N
26. C l i c k t o e d i t
The Cloud Specialists
ShapeBlue.com @ShapeBlue
Tr o u b l e s h o o t i n g
• Check usage job status in the database:
• Usage jobs:
• On service startup the usage service will check the DB whether last job was ran – if not this job is
ran once.
• Note if execution time takes longer than “usage.stats.job.aggregation.range” – default 1 day. If so
jobs may overlap and cause problems, hence avoid this situation.
• Logs:
• /var/log/cloudstack/usage/usage.log
• Relativelyverbose – will show processing of usage types, accounts, etc.
27. C l i c k t o e d i t
The Cloud Specialists
ShapeBlue.com @ShapeBlue
Tr o u b l e s h o o t i n g a n d h o u s e k e e p i n g
• DB inconsistencies:
• Back up databases before editing…….. ☺
• If usage job fails it will report in logs what the problem is.
• Watch out for inconsistencies like STOP entry without START, DESTROY without CREATE or double
entries – e.g. 2 x START entries for a VM.
• If required you can add/delete entries in the cloud.usage_event table.
• Housekeeping of cloud_usage table:
• Use the removeRawUsageRecords API call to delete entries older than 5 days.
# cloudmonkey removeRawUsageRecords interval=5
success = true
• Regenerating usage data:
• Use the generateUsageRecords API call to rerun usage in case of failure:
# cloudmonkey generateUsageRecords startdate=2018-09-01 enddate=2018-09-30
success = true
28. C l i c k t o e d i t
The Cloud Specialists
ShapeBlue.com @ShapeBlue
A b r i ef w or d on t h e q u ot a s er vi c e
• In the cloud_usage database you will see a number of “quota_*” tables.
• These are not directly used by the usage service, rather the separate
Quota service.
• Check the following for more information:
• https://cwiki.apache.org/confluence/display/CLOUDSTACK/Quota+Service+-
+FS
• http://docs.cloudstack.apache.org/projects/cloudstack-
administration/en/4.11/plugins/quota.html
29. C l i c k t o e d i t
The Cloud Specialists
ShapeBlue.com @ShapeBlue
Any questions?
30. C l i c k t o e d i t
The Cloud Specialists
ShapeBlue.com @ShapeBlue
Some references
• Blog post:
http://www.shapeblue.com/cloudstack-usage-service-deep-dive
This has more technicaldetail.
• CloudStack usage references:
• http://docs.cloudstack.apache.org/projects/cloudstack-
installation/en/4.11/optional_installation.html
• http://docs.cloudstack.apache.org/projects/cloudstack-
administration/en/4.11/usage.html
31. C l i c k t o e d i t
The Cloud Specialists
ShapeBlue.com @ShapeBlue
Mo r e i n f o r m a t i o n
• Slide decks: http://www.slideshare.net/shapeblue
• Blog: http://shapeblue.com/blog
http://dsonstebo.wordpress.com
• Email: dag.sonstebo@shapeblue.com
• Twitter: @dagsonstebo
• Web: http://shapeblue.com