usage server broken in 4.12.0.0

[konstantinjfk]

ISSUE TYPE

• Bug Report

COMPONENT NAME

Usage server

CLOUDSTACK VERSION

version 4.12, downloaded from official repo

CONFIGURATION

vanilla install

OS / ENVIRONMENT

Centos 7

SUMMARY

Usage server not starting jobs and ignore any db.usage settings

[root@ref-trl-1013-k-M7-apanic-mgmt1 usage]# grep -i usage /etc/cloudstack/usage/db.properties | grep invalid
db.usage.name=invalid_database_name
db.usage.port=invalid_port
db.usage.username=invalid_username
db.usage.password=invalid_password!
[root@ref-trl-1013-k-M7-apanic-mgmt1 usage]# tail -n 5 /var/log/cloudstack/usage/usage.log
2019-03-07 22:40:36,738 INFO [cloud.usage.UsageManagerImpl] (main:null) (logid:) Implementation Version is 4.12.0.0
2019-03-07 22:40:37,569 DEBUG [cloud.usage.UsageManagerImpl] (main:null) (logid:) Usage stats aggregation time zone: GMT
2019-03-07 22:40:37,570 DEBUG [cloud.usage.UsageManagerImpl] (main:null) (logid:) Execution Time: Thu Mar 07 22:28:00 UTC 2019
2019-03-07 22:40:37,570 DEBUG [cloud.usage.UsageManagerImpl] (main:null) (logid:) Current Time: Thu Mar 07 22:40:37 UTC 2019
2019-03-07 22:40:37,575 INFO [cloud.usage.UsageServer] (main:null) (logid:) UsageServer ready..

STEPS TO REPRODUCE

try to install usage server from http://download.cloudstack.org/centos7/4.12/
and force it to proceed with usage records, no luck
My info here in user mail list
http://mail-archives.apache.org/mod_mbox/cloudstack-users/201903.mbox/%3CVI1PR07MB4816C12D23D511AD0904D9D9E44C0%40VI1PR07MB4816.eurprd07.prod.outlook.com%3E

EXPECTED RESULTS

usage server need to be fixed. That is show stopper for the release

ACTUAL RESULTS

[andrijapanicsb]

Confirmed the issue myself as well.
Changes to db.properties (setting invalid user/pass/ip/port) and immediately restarting usage server did NOT raise an exception, but after leaving it over night, and restarting it again in the morning (usage server), it did detect invalid settings, and logged lines "failing back to default settings of cloud/cloud/localhost/3306" - which happened to be in place in my test env) - so again started happily with default settings...

More serious issue:
After setting aggregation period to 60 (execute usage jobs every hour) - usage server never actually run a job even after 8h

[konstantinjfk]

Here is 4.12.0.0 debug log files

Not so much info here

https://pastebin.com/89XycZgq

Is you can see, I updated usage.stats.job.exec.time every time before usage server start to force job start

There is nothing after line

(main:null) (logid:) UsageServer ready...

in both cases.

in the same time the version 4.11.2 with same approach, when I changed usage.stats.job.exec.time to the next minute at the start moment gave me this result:

https://pastebin.com/DiLU2vHn

As you can see, the job started and proceed as it should be

I made the test on same VM, by replacing the repos from 11.2.0 to 12.0.0 and back and reinstalling usage server without any changes to other software

bug is in cloudstack-usage-4.12.0.0-1.el7.centos.x86_64
this version works fine cloudstack-usage-4.11.2.0-1.el7.centos.x86_64

[ustcweizhou]

@konstantinjfk /etc/cloudstack/usage/db.properties should be linked to /etc/cloudstack/management/db.properties

# ls -l /etc/cloudstack/usage/db.properties
lrwxrwxrwx 1 root root 40 Feb 28 09:55 /etc/cloudstack/usage/db.properties -> /etc/cloudstack/management/db.properties

[GabrielBrascher]

The error is happening due to the migration to the systemd unit. I will link a PR fixing it as soon as possible.

[GabrielBrascher]

@konstantinjfk @andrijapanic please review PR #3207.

[GabrielBrascher]

#3210 fixes this issue and it has been merged. Thanks for reporting this issue. RC5 is available for VOTE under the mail [VOTE] Apache CloudStack 4.12.0.0 [RC5], sent to users and dev CloudStack mailing lists.

Git Branch: 4.12.0.0-RC20190314T1011
https://github.com/apache/cloudstack/tree/4.12.0.0-RC20190314T1011
https://gitbox.apache.org/repos/asf?p=cloudstack.git;a=shortlog;h=refs/heads/4.12.0.0-RC20190314T1011

Commit: a137398

Source release for 4.12.0.0-RC20190314T1011:
https://dist.apache.org/repos/dist/dev/cloudstack/4.12.0.0/

PGP release keys (signed using 25908455):
https://dist.apache.org/repos/dist/release/cloudstack/KEYS

The vote will be open for 3 business days (until 19th March).