1, 问题:
glance 镜像 raw 格式,启动云主机报错
2015-12-13 11:53:27.597 11654 INFO nova.virt.libvirt.driver [req-db7b8ac1-8218-4ab0-b769-9c4d4379843b e198d9f41e784308858a52b174fc822a a7fc63796b934498ba9b6d1a918cb2c7 - - -] [instance: 18efe519-e86b-43e9-842a-f974a751e29a] Creating p_w_picpath2015-12-13 11:53:27.932 11654 ERROR nova.compute.manager [req-db7b8ac1-8218-4ab0-b769-9c4d4379843b e198d9f41e784308858a52b174fc822a a7fc63796b934498ba9b6d1a918cb2c7 - - -] [instance: 18efe519-e86b-43e9-842a-f974a751e29a] Instance failed to spawn2015-12-13 11:53:27.932 11654 ERROR nova.compute.manager [instance: 18efe519-e86b-43e9-842a-f974a751e29a] Traceback (most recent call last):2015-12-13 11:53:27.932 11654 ERROR nova.compute.manager [instance: 18efe519-e86b-43e9-842a-f974a751e29a] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2155, in _build_resources2015-12-13 11:53:27.932 11654 ERROR nova.compute.manager [instance: 18efe519-e86b-43e9-842a-f974a751e29a] yield resources2015-12-13 11:53:27.932 11654 ERROR nova.compute.manager [instance: 18efe519-e86b-43e9-842a-f974a751e29a] File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2009, in _build_and_run_instance2015-12-13 11:53:27.932 11654 ERROR nova.compute.manager [instance: 18efe519-e86b-43e9-842a-f974a751e29a] block_device_info=block_device_info)2015-12-13 11:53:27.932 11654 ERROR nova.compute.manager [instance: 18efe519-e86b-43e9-842a-f974a751e29a] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 2437, in spawn2015-12-13 11:53:27.932 11654 ERROR nova.compute.manager [instance: 18efe519-e86b-43e9-842a-f974a751e29a] admin_pass=admin_password)2015-12-13 11:53:27.932 11654 ERROR nova.compute.manager [instance: 18efe519-e86b-43e9-842a-f974a751e29a] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 2861, in _create_p_w_picpath2015-12-13 11:53:27.932 11654 ERROR nova.compute.manager [instance: 18efe519-e86b-43e9-842a-f974a751e29a] instance, size, fallback_from_host)2015-12-13 11:53:27.932 11654 ERROR nova.compute.manager [instance: 18efe519-e86b-43e9-842a-f974a751e29a] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 6282, in _try_fetch_p_w_picpath_cache2015-12-13 11:53:27.932 11654 ERROR nova.compute.manager [instance: 18efe519-e86b-43e9-842a-f974a751e29a] size=size)2015-12-13 11:53:27.932 11654 ERROR nova.compute.manager [instance: 18efe519-e86b-43e9-842a-f974a751e29a] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/p_w_picpathbackend.py", line 240, in cache2015-12-13 11:53:27.932 11654 ERROR nova.compute.manager [instance: 18efe519-e86b-43e9-842a-f974a751e29a] *args, **kwargs)2015-12-13 11:53:27.932 11654 ERROR nova.compute.manager [instance: 18efe519-e86b-43e9-842a-f974a751e29a] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/p_w_picpathbackend.py", line 777, in create_p_w_picpath2015-12-13 11:53:27.932 11654 ERROR nova.compute.manager [instance: 18efe519-e86b-43e9-842a-f974a751e29a] prepare_template(target=base, max_size=size, *args, **kwargs)2015-12-13 11:53:27.932 11654 ERROR nova.compute.manager [instance: 18efe519-e86b-43e9-842a-f974a751e29a] File "/usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py", line 254, in inner2015-12-13 11:53:27.932 11654 ERROR nova.compute.manager [instance: 18efe519-e86b-43e9-842a-f974a751e29a] return f(*args, **kwargs)2015-12-13 11:53:27.932 11654 ERROR nova.compute.manager [instance: 18efe519-e86b-43e9-842a-f974a751e29a] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/p_w_picpathbackend.py", line 230, in fetch_func_sync2015-12-13 11:53:27.932 11654 ERROR nova.compute.manager [instance: 18efe519-e86b-43e9-842a-f974a751e29a] fetch_func(target=target, *args, **kwargs)2015-12-13 11:53:27.932 11654 ERROR nova.compute.manager [instance: 18efe519-e86b-43e9-842a-f974a751e29a] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 2853, in clone_fallback_to_fetch2015-12-13 11:53:27.932 11654 ERROR nova.compute.manager [instance: 18efe519-e86b-43e9-842a-f974a751e29a] backend.clone(context, disk_p_w_picpaths['p_w_picpath_id'])2015-12-13 11:53:27.932 11654 ERROR nova.compute.manager [instance: 18efe519-e86b-43e9-842a-f974a751e29a] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/p_w_picpathbackend.py", line 808, in clone2015-12-13 11:53:27.932 11654 ERROR nova.compute.manager [instance: 18efe519-e86b-43e9-842a-f974a751e29a] if self.driver.is_cloneable(location, p_w_picpath_meta):2015-12-13 11:53:27.932 11654 ERROR nova.compute.manager [instance: 18efe519-e86b-43e9-842a-f974a751e29a] File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/storage/rbd_utils.py", line 193, in is_cloneable2015-12-13 11:53:27.932 11654 ERROR nova.compute.manager [instance: 18efe519-e86b-43e9-842a-f974a751e29a] if p_w_picpath_meta.disk_format != 'raw':2015-12-13 11:53:27.932 11654 ERROR nova.compute.manager [instance: 18efe519-e86b-43e9-842a-f974a751e29a] AttributeError: 'dict' object has no attribute 'disk_format'
2,问题分析
1,查找问题定位:/usr/lib/python2.7/site-packages/nova/virt/libvirt/storage/rbd_utils.py
import ipdb;ipdb.set_trace() if p_w_picpath_meta.disk_format != 'raw': reason = ("rbd p_w_picpath clone requires p_w_picpath format to be " "'raw' but p_w_picpath {0} is '{1}'").format( url, p_w_picpath_meta.disk_format) LOG.debug(reason) return False
2, ipdb 启动服务追踪
[root@AthComputeLiberty10325 ~]# ipdb /usr/bin/nova-compute --config-file=/etc/nova/nova.conf
3,发现蛛丝马迹
AttributeError: "'dict' object has no attribute 'disk_format'"> /usr/lib/python2.7/site-packages/nova/virt/libvirt/storage/rbd_utils.py(194)is_cloneable() 193 --> 194 if p_w_picpath_meta.disk_format != 'raw': 195 reason = ("rbd p_w_picpath clone requires p_w_picpath format to be "ipdb> argsself =p_w_picpath_location = {'url': u'rbd://03e180bd-7504-4883-8d0f-5506c63d4a4f/p_w_picpaths/05cfd97d-f0d8-43b4-9796-180f650e79ce/snap', 'metadata': {}}p_w_picpath_meta = {'status': u'active', 'deleted_at': None, 'name': u'Ceph-linux6.6.v2.0', 'deleted': None, 'container_format': u'ovf', 'created_at': datetime.datetime(2015, 12, 12, 8, 47, 40, tzinfo= ), 'disk_format': u'raw', 'updated_at': datetime.datetime(2015, 12, 12, 9, 35, 22, tzinfo= ), 'locations': [{'url': u'rbd://03e180bd-7504-4883-8d0f-5506c63d4a4f/p_w_picpaths/05cfd97d-f0d8-43b4-9796-180f650e79ce/snap', 'metadata': {}}], 'owner': u'a7fc63796b934498ba9b6d1a918cb2c7', 'id': u'05cfd97d-f0d8-43b4-9796-180f650e79ce', 'min_ram': 0, 'checksum': u'0909ad3eebe539efc2054196abffdb41', 'min_disk': 0, 'is_public': None, 'direct_url': u'rbd://03e180bd-7504-4883-8d0f-5506c63d4a4f/p_w_picpaths/05cfd97d-f0d8-43b4-9796-180f650e79ce/snap', 'properties': {}, 'size': 128849018880}ipdb> p_w_picpath_meta.disk_format*** AttributeError: 'dict' object has no attribute 'disk_format'ipdb> p_w_picpath_meta.disk_format()*** AttributeError: 'dict' object has no attribute 'disk_format'ipdb> p_w_picpath_meta.disk_format.values()*** AttributeError: 'dict' object has no attribute 'disk_format'ipdb> dir(p_w_picpath_meta.disk_format)*** AttributeError: 'dict' object has no attribute 'disk_format'ipdb> argsself = p_w_picpath_location = {'url': u'rbd://03e180bd-7504-4883-8d0f-5506c63d4a4f/p_w_picpaths/05cfd97d-f0d8-43b4-9796-180f650e79ce/snap', 'metadata': {}}p_w_picpath_meta = {'status': u'active', 'deleted_at': None, 'name': u'Ceph-linux6.6.v2.0', 'deleted': None, 'container_format': u'ovf', 'created_at': datetime.datetime(2015, 12, 12, 8, 47, 40, tzinfo= ), 'disk_format': u'raw', 'updated_at': datetime.datetime(2015, 12, 12, 9, 35, 22, tzinfo= ), 'locations': [{'url': u'rbd://03e180bd-7504-4883-8d0f-5506c63d4a4f/p_w_picpaths/05cfd97d-f0d8-43b4-9796-180f650e79ce/snap', 'metadata': {}}], 'owner': u'a7fc63796b934498ba9b6d1a918cb2c7', 'id': u'05cfd97d-f0d8-43b4-9796-180f650e79ce', 'min_ram': 0, 'checksum': u'0909ad3eebe539efc2054196abffdb41', 'min_disk': 0, 'is_public': None, 'direct_url': u'rbd://03e180bd-7504-4883-8d0f-5506c63d4a4f/p_w_picpaths/05cfd97d-f0d8-43b4-9796-180f650e79ce/snap', 'properties': {}, 'size': 128849018880}ipdb> p_w_picpath_meta{'status': u'active', 'deleted_at': None, 'name': u'Ceph-linux6.6.v2.0', 'deleted': None, 'container_format': u'ovf', 'created_at': datetime.datetime(2015, 12, 12, 8, 47, 40, tzinfo= ), 'disk_format': u'raw', 'updated_at': datetime.datetime(2015, 12, 12, 9, 35, 22, tzinfo= ), 'locations': [{'url': u'rbd://03e180bd-7504-4883-8d0f-5506c63d4a4f/p_w_picpaths/05cfd97d-f0d8-43b4-9796-180f650e79ce/snap', 'metadata': {}}], 'owner': u'a7fc63796b934498ba9b6d1a918cb2c7', 'id': u'05cfd97d-f0d8-43b4-9796-180f650e79ce', 'min_ram': 0, 'checksum': u'0909ad3eebe539efc2054196abffdb41', 'min_disk': 0, 'is_public': None, 'direct_url': u'rbd://03e180bd-7504-4883-8d0f-5506c63d4a4f/p_w_picpaths/05cfd97d-f0d8-43b4-9796-180f650e79ce/snap', 'properties': {}, 'size': 128849018880}ipdb> type(p_w_picpath_meta) ipdb> p_w_picpath_meta{'status': u'active', 'deleted_at': None, 'name': u'Ceph-linux6.6.v2.0', 'deleted': None, 'container_format': u'ovf', 'created_at': datetime.datetime(2015, 12, 12, 8, 47, 40, tzinfo= ), 'disk_format': u'raw', 'updated_at': datetime.datetime(2015, 12, 12, 9, 35, 22, tzinfo= ), 'locations': [{'url': u'rbd://03e180bd-7504-4883-8d0f-5506c63d4a4f/p_w_picpaths/05cfd97d-f0d8-43b4-9796-180f650e79ce/snap', 'metadata': {}}], 'owner': u'a7fc63796b934498ba9b6d1a918cb2c7', 'id': u'05cfd97d-f0d8-43b4-9796-180f650e79ce', 'min_ram': 0, 'checksum': u'0909ad3eebe539efc2054196abffdb41', 'min_disk': 0, 'is_public': None, 'direct_url': u'rbd://03e180bd-7504-4883-8d0f-5506c63d4a4f/p_w_picpaths/05cfd97d-f0d8-43b4-9796-180f650e79ce/snap', 'properties': {}, 'size': 128849018880}ipdb> p_w_picpath_meta['disk_format']u'raw'
3,解决问题bug:
修改前:
if p_w_picpath_meta.disk_format != 'raw':
修改后:
if p_w_picpath_meta['disk_format'] != 'raw':
179 180 def is_cloneable(self, p_w_picpath_location, p_w_picpath_meta):181 import ipdb;ipdb.set_trace()182 url = p_w_picpath_location['url']183 try:184 fsid, pool, p_w_picpath, snapshot = self.parse_url(url)185 except exception.ImageUnacceptable as e:186 LOG.debug('not cloneable: %s', e)187 return False188 189 if self._get_fsid() != fsid:190 reason = '%s is in a different ceph cluster' % url191 LOG.debug(reason)192 return False193 194 # Add william , p_w_picpath_meta.disk_format != 'raw'195 # so ...196 if p_w_picpath_meta['disk_format'] != 'raw':197 reason = ("rbd p_w_picpath clone requires p_w_picpath format to be "198 "'raw' but p_w_picpath {0} is '{1}'").format(199 url, p_w_picpath_meta.disk_format)200 LOG.debug(reason)201 return False
4, 完成操作
2015-12-13 13:18:36.659 19827 INFO nova.virt.libvirt.firewall [req-7d3b097b-701d-4352-a8cc-87bc6fb3f9f1 e198d9f41e784308858a52b174fc822a a7fc63796b934498ba9b6d1a918cb2c7 - - -] [instance: 36025fe9-5abe-44de-bdb8-71983a3645f2] Called setup_basic_filtering in nwfilter2015-12-13 13:18:36.660 19827 INFO nova.virt.libvirt.firewall [req-7d3b097b-701d-4352-a8cc-87bc6fb3f9f1 e198d9f41e784308858a52b174fc822a a7fc63796b934498ba9b6d1a918cb2c7 - - -] [instance: 36025fe9-5abe-44de-bdb8-71983a3645f2] Ensuring static filters2015-12-13 13:18:37.418 19827 INFO nova.compute.manager [-] [instance: 36025fe9-5abe-44de-bdb8-71983a3645f2] VM Started (Lifecycle Event)2015-12-13 13:18:37.441 19827 INFO nova.virt.libvirt.driver [-] [instance: 36025fe9-5abe-44de-bdb8-71983a3645f2] Instance spawned successfully.2015-12-13 13:18:37.525 19827 INFO nova.compute.manager [req-a92e6077-014c-421c-9fcc-9d21e6f7d420 - - - - -] [instance: 36025fe9-5abe-44de-bdb8-71983a3645f2] During sync_power_state the instance has a pending task (spawning). Skip.