ubuntu16: fix three issues with ubuntu 16.04 hosts by ustcweizhou · Pull Request #3227 · apache/cloudstack

ustcweizhou · 2019-03-14T10:11:45Z

Description

This fixes three issues with ubuntu 16.04 hosts

fix unable to add host if cloudbrX is not configured
Stop service libvirt-bin.socket while add a host
Diable libvirt default network

Details can be found in each commit.

Types of changes

Breaking change (fix or feature that would cause existing functionality to change)
New feature (non-breaking change which adds functionality)
Bug fix (non-breaking change which fixes an issue)
Enhancement (improves an existing feature and functionality)
Cleanup (Code refactoring and cleanup, that may add test cases)

yadvr · 2019-03-14T11:23:49Z

@blueorangutan package

blueorangutan · 2019-03-14T11:24:03Z

@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

yadvr · 2019-03-14T11:24:33Z

plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/resource/LibvirtComputingResource.java

+            if (network.isActive() == 1) {
+                network.destroy();
+            }
+            if (network.getAutostart()) {


If we destroy the network and set autostart to false, would that cause an issue?

@rhtyd We have destroyed the default network and did not see any issue in our production.
it seems to break packaging/debian/cloudstack-agent.init, however, as /etc/init.d/cloudstack-agent is not in use any more after systemd changes, it should not be a problem.

We still do support initd with centos6 packages.

@rhtyd
I checked the code again, the packaging/debian/cloudstack-agent.init will be packaged in ubuntu/debs and deployed on ubuntu servers as /etc/init.d/cloudstack-agent.
Since we do not support ubuntu 12.04 any more , and systemd is used in ubuntu 14.04 and later, I think this change will not break other functions.

blueorangutan · 2019-03-14T11:57:19Z

Packaging result: ✔centos6 ✔centos7 ✔debian. JID-2632

yadvr · 2019-03-14T12:24:19Z

@blueorangutan test

blueorangutan · 2019-03-14T12:25:04Z

@rhtyd a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

blueorangutan · 2019-03-14T20:21:14Z

Trillian test result (tid-3421)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 26752 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr3227-t3421-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_host_maintenance.py
Smoke tests completed. 68 look OK, 0 have error(s)
Only failed tests results shown below:

Test	Result	Time (s)	Test File

wido · 2019-03-18T17:54:14Z

I have never seen these issues, but I can understand why they happen.

LGTM

yadvr · 2019-05-04T15:17:21Z

@ustcweizhou can you rebase to latest master? We've only deprecated older Ubuntu versions recently, but not in 4.11.

while add a ubuntu16.04 host with native eth0 (cloudbrX is not configured), the operation failed and I got the following error in /var/log/cloudstack/agent/setup.log ``` DEBUG:root:execute:ifconfig eth0 DEBUG:root:[Errno 2] No such file or directory File "/usr/lib/python2.7/dist-packages/cloudutils/serviceConfig.py", line 38, in configration result = self.config() File "/usr/lib/python2.7/dist-packages/cloudutils/serviceConfig.py", line 211, in config super(networkConfigUbuntu, self).cfgNetwork() File "/usr/lib/python2.7/dist-packages/cloudutils/serviceConfig.py", line 108, in cfgNetwork device = self.netcfg.getDefaultNetwork() File "/usr/lib/python2.7/dist-packages/cloudutils/networkConfig.py", line 53, in getDefaultNetwork pdi = networkConfig.getDevInfo(dev) File "/usr/lib/python2.7/dist-packages/cloudutils/networkConfig.py", line 157, in getDevInfo elif networkConfig.isBridge(dev) or networkConfig.isOvsBridge(dev): ``` The issue is caused by commit 9c7cd8c 2017-09-19 16:45 Sigert Goeminne ● CLOUDSTACK-10081: CloudUtils getDevInfo function will now return "bridge" instead o

service libvirt-bin.socket will be started when add a ubuntu 16.04 host DEBUG:root:execute:sudo /usr/sbin/service libvirt-bin start However, libvirt-bin service will be broken by it after restarting Stopping service libvirt-bin.socket will fix the issue. An example is given as below. ``` root@node32:~# /etc/init.d/libvirt-bin restart [ ok ] Restarting libvirt-bin (via systemctl): libvirt-bin.service. root@node32:~# virsh list error: failed to connect to the hypervisor error: no valid connection error: Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory root@node32:~# systemctl stop libvirt-bin.socket root@node32:~# /etc/init.d/libvirt-bin restart [ ok ] Restarting libvirt-bin (via systemctl): libvirt-bin.service. root@node32:~# virsh list Id Name State ---------------------------------------------------- ```

By default, libvirt will create default network virbr0 on kvm hypervisors. If vm uses the same ip range 192.168.122.0/24, there will be some issues. In some cases, if we run tcpdump inside vm, we will see the ip of kvm hypervisor as source ip.

ustcweizhou · 2019-05-04T18:13:45Z

@rhtyd thanks for review. rebased with latest master.

yadvr · 2019-05-04T18:19:19Z

Thanks Wei
@blueorangutan package

blueorangutan · 2019-05-04T18:19:39Z

@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

blueorangutan · 2019-05-04T18:39:54Z

Packaging result: ✔centos6 ✔centos7 ✔debian. JID-2732

yadvr · 2019-05-04T19:09:11Z

@blueorangutan test

blueorangutan · 2019-05-04T19:09:38Z

@rhtyd a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

blueorangutan · 2019-05-05T05:15:58Z

Trillian test result (tid-3549)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 34765 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr3227-t3549-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_internal_lb.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_redundant.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_vpn.py
Intermittent failure detected: /marvin/tests/smoke/test_hostha_kvm.py
Smoke tests completed. 68 look OK, 2 have error(s)
Only failed tests results shown below:

Test	Result	Time (s)	Test File
test_05_rvpc_multi_tiers	`Failure`	408.79	test_vpc_redundant.py
test_05_rvpc_multi_tiers	`Error`	433.55	test_vpc_redundant.py
test_hostha_enable_ha_when_host_in_maintenance	`Error`	304.45	test_hostha_kvm.py

yadvr reviewed Mar 14, 2019

View reviewed changes

yadvr changed the base branch from 4.11 to master May 4, 2019 15:15

ustcweizhou added 3 commits May 4, 2019 18:12

ubuntu16: Diable libvirt default network

fd8fff4

By default, libvirt will create default network virbr0 on kvm hypervisors. If vm uses the same ip range 192.168.122.0/24, there will be some issues. In some cases, if we run tcpdump inside vm, we will see the ip of kvm hypervisor as source ip.

ustcweizhou force-pushed the 4.11.2-ubuntu16-issues branch from 1e90215 to fd8fff4 Compare May 4, 2019 18:12

yadvr merged commit 3729511 into apache:master May 5, 2019

yadvr added this to the 4.13.0.0 milestone May 5, 2019

Conversation

ustcweizhou commented Mar 14, 2019

Description

Types of changes

Uh oh!

yadvr commented Mar 14, 2019

Uh oh!

blueorangutan commented Mar 14, 2019

Uh oh!

yadvr Mar 14, 2019

Choose a reason for hiding this comment

Uh oh!

ustcweizhou Mar 14, 2019

Choose a reason for hiding this comment

Uh oh!

yadvr Apr 8, 2019

Choose a reason for hiding this comment

Uh oh!

ustcweizhou Apr 30, 2019

Choose a reason for hiding this comment

Uh oh!

yadvr May 4, 2019

Choose a reason for hiding this comment

Uh oh!

blueorangutan commented Mar 14, 2019

Uh oh!

yadvr commented Mar 14, 2019

Uh oh!

blueorangutan commented Mar 14, 2019

Uh oh!

blueorangutan commented Mar 14, 2019

Uh oh!

wido commented Mar 18, 2019

Uh oh!

yadvr commented May 4, 2019

Uh oh!

ustcweizhou commented May 4, 2019

Uh oh!

yadvr commented May 4, 2019

Uh oh!

blueorangutan commented May 4, 2019

Uh oh!

blueorangutan commented May 4, 2019

Uh oh!

yadvr commented May 4, 2019

Uh oh!

blueorangutan commented May 4, 2019

Uh oh!

blueorangutan commented May 5, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants