Join Log in
Score:0
Moha avatar Server

Metadata (as in hostname) is not set during instance creation in OpenStack

de flag
Moha

By creating an instance in OpenStack, IP is injected into the instance, but not other metadata as in Hostname and Key Pair. There's no error in the nova and neutron log files.

The problem:

...
checking http://169.254.169.254/2009-04-04/instance-id
failed 1/20: up 5.45. request failed
failed 2/20: up 7.47. request failed
...
failed 19/20: up 746.84. request failed
failed 20/20: up 795.86. request failed
failed to read iid from metadata. tried 20
failed to get instance-id of datasource
Top of dropbear init script
Starting dropbear sshd: failed to get instance-id of datasource
OK
...

The above snippet is part of the Cirros instance creation log. (The full log of the creation process is posted here)

The other error in the logs is this line: acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge. Is this related to the issue?

Some of the config files:

# /etc/neutron/dhcp_agent.ini
[DEFAULT]
interface_driver = openvswitch
dhcp_driver = neutron.agent.linux.dhcp.Dnsmasq
enable_isolated_metadata = true
force_metadata = true


#/etc/neutron/plugins/ml2/ml2_conf.ini
[ml2]
type_drivers = vxlan,vlan,flat
tenant_network_types = vxlan
mechanism_drivers = openvswitch,l2population
extension_drivers = port_security
[ml2_type_flat]
flat_networks = provider
[ml2_type_vxlan]
vni_ranges = 1:1000


#/etc/neutron/plugins/ml2/openvswitch_agent.ini
[agent]
tunnel_types = vxlan
l2_population = True
[dhcp]
[network_log]
[ovs]
local_ip = 10.0.0.1
bridge_mappings = provider:br-provider
[securitygroup]
firewall_driver = iptables_hybrid

Compute node configurations: https://p.teknik.io/Ck1Ae

Here are errors from the cloud-init.log of the instance:

errors in cloud-init.log file

ip netns exec qdhcp-3b5bbc53-7638-4480-9349-407944933413 ip a

77: tap588fefad-6c: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UNKNOWN group default qlen 1000
    link/ether fa:16:3e:df:33:e1 brd ff:ff:ff:ff:ff:ff
    inet 169.254.169.254/32 brd 169.254.169.254 scope global tap588fefad-6c
       valid_lft forever preferred_lft forever
    inet 10.0.1.1/24 brd 10.0.1.255 scope global tap588fefad-6c
       valid_lft forever preferred_lft forever
    inet6 fe80::f816:3eff:fedf:33e1/64 scope link
       valid_lft forever preferred_lft forever

Since an instance creation,

+ I have followed this instruction to implement self-service networking over OVS.

454
1 + 12
us flag
eblock
Have you also checked cloud-init.log? Maybe it shows why those other data have not been injected. You can test if the instance can query the metadata server with `curl http://169.254.169.254/latest` and then add the paths you want to check (like hostname etc.)
0
Reply
Moha avatar
de flag
Moha
@eblock, From inside the instance, `curl` on http://169.254.169.254/latest waits and waits! I updated the post with errors from cloud-init.log: _Calling http://169.254.169.254/openstack failed ... timeout_
0
Reply
us flag
eblock
Apparently the metadata service is not running or not responsive. Can you check neutron-metadata agent log and if the service is up?
0
Reply
Moha avatar
de flag
Moha
I greped for ERROR/fail on the whole /var/log and there's nothing for these days! All services, included neutron-metadata-agent, are running.
0
Reply
us flag
eblock
Are any apparmor/selinux or firewall applications running? I’ve had similar effects with apparmor enabled. Has it ever worked for other instances or is this a fresh installation and the first attempt?
0
Reply
Moha avatar
de flag
Moha
This is the first attempt over OVS; It's been working with LinuxBridge well; Now I'm trying to move it to OVS. Apparmor was running, I stopped it and created an instance, but the problem still exists: `checking http://169.254.169.254/2009-04-04/instance-id failed `
0
Reply
us flag
eblock
The metadata agent should log something if the compute node tries to request metadata information for the instance, can you paste the relevant log output (not just grep for error)? And maybe also check (and paste) neutron-server.log output from the time of instance creation. And does the nova-compute.log reveal anything? Do you see the namespace for the magic ip (169.254.169.254) on the control node? I don't have access to an openstack env right now.
0
Reply
us flag
eblock
So each subnet with dhcp enabled should have a namespace containing the "magic ip", for example: `ip netns exec qdhcp-0db3acf6-d01e-484b-ace3-085422d90448 ip a` shows: `13: tapfe22e4cc-d9: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000 link/ether fa:16:3e:26:27:76 brd ff:ff:ff:ff:ff:ff inet 169.254.169.254/32 brd 169.254.169.254 scope global tapfe22e4cc-d9` Do you see the same in your env for the subnet the instance has a tap device in?
0
Reply
us flag
eblock
Have you tried restarting neutron-metadata-agent? It should log something, at least the daemon restart.
0
Reply
Moha avatar
de flag
Moha
"Empty since the beginning of instance creation"; restarting the service leads to log generation: http://p.teknik.io/Raw/oRcMe; Should I config something other than the Neutron itself and OVS on the computing side (http://p.teknik.io/Ck1Ae)? I have a controller node and a compute node. If I install and activate nova-compute on the controller node, instance creation works well, but still no metadata log; It just does not work if the compute node is separated!
0
Reply
us flag
eblock
Oh damn, I didn't think of it earlier. Are you using a provider network (you are according to `bridge_mappings = provider:br-provider`)? In that case try to start the instance with `config-drive = true`.
0
Reply
us flag
eblock
Either you enable the check mark if you create the instance via Horizon dashboard (Tab "configuration") or if you launch it via command line you add the option `--config-drive true` to the `openstack server create` command. Can you verify if config drive works? If it doesn't we might need to check the cloud.cfg within the vm image.
0
Reply
Score:0
avatar Server
us flag
eblock

One way to inject metadata into an instance is the config drive. If using the command line it can be added as an option to the openstack server create command:

openstack server create --flavor [...] --config-drive true my-vm

You can also force nova to always use a config drive, for that add this line to /etc/nova/nova.conf:

force_config_drive = true

Nova attaches a virtual drive to the instance where it stores the metadata, this drive is mounted by cloud-init and the desired config parameters are applied. It appears that in your case the metadata service does not work properly, I don't see a metadata_agent.ini section in your output, did you configure it? You can find more details in the installation guide.

+ 0
ธงชาติไทย
Elon Musk
I sit in a Tesla and translated this thread with Ai:

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.