Fixing Openstack VM spawning issue: No suitable host found/vif_type=binding_failed error

Once in a while, on a new setup, when you try to spawn a VM on Openstack and it may fail with the error :  No Suitable Host Found, it points to the fact the the Nova Scheduler failed to filter out a host to spawn the VM on.

[The following trouble shooting is related to my setup. There are a lot of other factors that may lead to this error. So google a bit and check them in addition. This article can help you in figuring out the way to do troubleshooting.]

A quick look at /var/log/nova/scheduler.log and compute.log shows an error:
  • NovaException: Unexpected vif_type=binding_failed
  • Filter RetryFilter returned 0 hosts
This points to the fact that something is wrong with the Neutron setup. So a quick look into the neutron logs show the following:

ERROR neutron.openstack.common.rpc.common [-] AMQP server on ***** is unreachable: [Errno 111] ECONNREFUSED. Trying again in 5 seconds.
ERROR neutron.agent.linux.ovsdb_monitor [-] Error received from ovsdb monitor: ovsdb-client: unix:/var/run/openvswitch/db.sock: receive failed (End of file)
ERROR neutron.agent.linux.ovs_lib [-] Unable to execute ['ovs-ofctl', 'dump-flows', 'br-int', 'table=22']. Exception:
Command: ['sudo', 'neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ovs-ofctl', 'dump-flows', 'br-int', 'table=22']
Exit code: 1
Stdout: ''
Stderr: 'ovs-ofctl: br-int is not a bridge or a socket\n'

INFO neutron.plugins.openvswitch.agent.ovs_neutron_agent [req-d0057d1b-79fd-41ac-8616-7b86b5c900a6 None] Mapping physical network physnet1 to bridge br-eth1
ERROR neutron.plugins.openvswitch.agent.ovs_neutron_agent [req-d0057d1b-79fd-41ac-8616-7b86b5c900a6 None] Bridge br-eth1 for physical network physnet1 does not exist. Agent terminated!

Whoa!! My agent looks dead. A quick status check [ /etc/init.d/neutron-openvswitch-agent status] confirms this.

The problem now seems to be that my OVS agent is trying to bind a port on physnet1 to br-eth1 bridge and is failing to find it. So, to fix the problem, create the bridge [which got missed out in the initial setup]:

ovs-vsctl add-br br-eth1
ovs-vsctl add-port br-eth1 eth1

Restarting the ovs-agent fixes the issue.

Hope this helps someone.


Popular posts from this blog

Solved: Fix for Git clone failure due to GnuTLS recv error (-9)

QuickBite: Tap Vs Veth

Openstack : Fixing Failed to create network. No tenant network is available for allocation issue.