Join Log in
Score:0
bowfgc avatar Ubuntu

Automated Server Installs config file top-level keys ”late-commands“ cant use wget or curl

ps flag
bowfgc

i have encountered a problem about Automated server install.

This is my env: Server: centos 7.6 x86_64(its a kvm system) Node1: no system Node2: no system

I want to automated install Ubuntu-20.04.3 Server system.

About Server1 i install tftpd,dhcpd,httpd. some config like this:

# tftpd
# cat /etc/xinetd.d/tftp
service tftp
{
    socket_type     = dgram
    protocol        = udp
    wait            = yes
    user            = root
    server          = /usr/sbin/in.tftpd
    server_args     = -s /var/lib/tftpboot
    disable         = yes
    per_source      = 11
    cps         = 100 2
    flags           = IPv4
}


# dhcpd
# cat  /etc/dhcp/dhcpd.conf
ddns-update-style none;
subnet 10.10.8.0 netmask 255.255.252.0 {
     option routers             10.10.11.254;
     option domain-name-servers 10.10.9.250;
     option subnet-mask         255.255.252.0;
     range dynamic-bootp        10.10.11.10 10.10.11.220;
     default-lease-time         21600;
     max-lease-time             43200;
     next-server                10.10.11.245;
     filename "pxelinux.0";
}

# httpd
# cat /etc/httpd/conf.d/tftp.conf
<Directory /var/lib/tftpboot>
        Options +FollowSymLinks +Indexes
        Require all granted
</Directory>
Alias /tftp /var/lib/tftpboot

Files in /var/lib/tftpboot

# ll  /var/lib/tftpboot/
总用量 1328744
drwxrwxrwx 2 root root         22 12月  8 18:40 grub
-rwxrwxrwx 1 root root   86017541 11月 25 17:40 initrd
-rwxrwxrwx 1 root root         31 11月 25 17:45 meta-data
-rwxrwxrwx 1 root root        336 12月  8 20:03 network.sh
-rwxrwxrwx 1 root root    1435512 4月  16 2020 pxelinux.0
-rwxrwxrwx 1 root root 1261371392 8月  24 17:09 ubuntu-20.04.3-live-server-amd64.iso
-rwxrwxrwx 1 root root       1353 12月  8 20:03 ubuntu_init.sh
-rwxrwxrwx 1 root root       2856 12月  8 20:17 user-data
-rwxrwxrwx 1 root root   11772160 11月 25 17:40 vmlinuz

grub.cfg

# cat grub/grub.cfg
default=autoinstall
timeout=5
timeout_style=menu

set menu_color_normal=white/black
set menu_color_highlight=black/light-gray

menuentry "Focal Live Installer - automated" --id=autoinstall {
    echo "Loading Kernel...1111"
    linux /vmlinuz  root=/dev/ram0 ramdisk_size=1500000 root=/dev/ram0 ramdisk_size=1500000  ip=dhcp url=http://10.10.11.245/tftp/ubuntu-20.04.3-live-server-amd64.iso autoinstall ds=nocloud-net\;s=http://10.10.11.245/tftp/ ---
    echo "Loading Ram Disk...111"
    initrd /initrd
}
menuentry "Focal Live Installer" --id=install {
    echo "Loading Kernel...22"
    linux /vmlinuz   root=/dev/ram0 ramdisk_size=1500000 ramdisk_size=1500000 ip=dhcp url=http://10.10.11.245/tftp/ubuntu-20.04.3-live-server-amd64.iso
    echo "Loading Ram Disk...22"
    initrd /initrd
}

meta-data

# cat meta-data
instance-id: focal-autoinstall

user-data

# cat user-data
#cloud-config
autoinstall:
  apt:
    geoip: true
    preserve_sources_list: false
    primary:
    - arches: [amd64, i386]
      uri: http://mirrors.aliyun.com/ubuntu
    - arches: [default]
      uri: http://ports.ubuntu.com/ubuntu-ports
  user-data:
    timezone: Asia/Shanghai
    disable_root: false
    chpasswd:
      list: |
        root:$6$YMYlEcE71$/bcl2qd2rSYePvOkmgAyQR/h/
  identity: {hostname: 111, password: $6$ereNXNuj9WVUt77B$.dbpuHaLernNl6qq6r14meIk1W5VAyaMV9.,
    realname: 11, username: 11}
  keyboard: {layout: us, toggle: null, variant: ''}
  locale: en_US.UTF-8
  network:
    ethernets:
      eno12399:
        critical: true
        dhcp-identifier: mac
        dhcp4: true
        nameservers:
          addresses: [10.10.9.250]
      eno12409: {dhcp4: true}
      eno12419: {dhcp4: true}
      eno12429: {dhcp4: true}
      eno8303: {dhcp4: true}
      eno8403: {dhcp4: true}
    version: 2
  proxy: http://mirrors.aliyun.com/ubuntu
  ssh:
    allow-pw: true
    authorized-keys: []
    install-server: true
  storage:
    grub:
      reorder_uefi: False
    config:
    - {ptable: gpt, path: /dev/sda, wipe: superblock-recursive, preserve: false, name: '', grub_device: false,
      type: disk, id: disk-sda}
    - {device: disk-sda, size: 536870912, wipe: superblock, flag: boot, number: 1,
      preserve: false, grub_device: true, type: partition, id: partition-2}
    - {fstype: fat32, volume: partition-2, preserve: false, type: format, id: format-2}
    - {device: disk-sda, size: -1, wipe: superblock, flag: '', number: 2,
      preserve: false, grub_device: false, type: partition, id: partition-3}
    - {fstype: xfs, volume: partition-3, preserve: false, type: format, id: format-3}
    - {path: /, device: format-3, type: mount, id: mount-3}
    - {path: /boot/efi, device: format-2, type: mount, id: mount-2}
  updates: security
  packages:
  - linux-generic
  - net-tools
  late-commands:
  - curtin in-target --target=/target -- /usr/bin/wget -P /root/ http://10.10.11.245/tftp/ubuntu_init.sh
  - curtin in-target --target=/target -- /usr/bin/wget -P /root/ http://10.10.11.245/tftp/network.sh
  - curtin in-target --target=/target -- /usr/bin/bash /root/ubuntu_init.sh
  #- curtin in-target --target=/target -- sed -i 's/#PermitRootLogin prohibit-password/PermitRootLogin yes/' /etc/ssh/sshd_config && systemctl restart sshd
  #- curtin in-target --target=/target -- systemctl stop ufw.service && systemctl disable ufw.service
  #- curtin in-target --target=/target -- echo -e "NTP=ntp1.aliyun.com\nFallbackNTP=ntp.ubuntu.com" >> /etc/systemd/timesyncd.conf && systemctl restart systemd-timesyncd
  version: 1

Ubuntu_init.sh

#!/bin/bash
sed -i 's/#PermitRootLogin prohibit-password/PermitRootLogin yes/' /etc/ssh/sshd_config

systemctl restart sshd

systemctl stop ufw.service
systemctl disable ufw.service
# 
echo -e "NTP=ntp1.aliyun.com\nFallbackNTP=ntp.ubuntu.com" >> /etc/systemd/timesyncd.conf
systemctl restart systemd-timesyncd
# 
cat >> /etc/security/limits.conf << EOF
*       soft        nofile  655350
*       hard        nofile  655350
*       soft        nproc   655350
*       hard        nproc   655350
root        soft        nofile  655350
root        hard        nofile  655350
root        soft        nproc   655350
root        hard        nproc   655350
EOF
########################################
cp /etc/sysctl.conf /etc/sysctl.conf.bak
cat > /etc/sysctl.conf << EOF
vm.swappiness = 0
kernel.sysrq = 1

net.ipv4.neigh.default.gc_stale_time = 120

net.ipv4.conf.all.rp_filter = 0
net.ipv4.conf.default.rp_filter = 0
net.ipv4.conf.default.arp_announce = 2
net.ipv4.conf.lo.arp_announce = 2
net.ipv4.conf.all.arp_announce = 2
net.ipv4.tcp_max_tw_buckets = 5000
net.ipv4.tcp_syncookies = 1
net.ipv4.tcp_max_syn_backlog = 1024
net.ipv4.tcp_synack_retries = 2
net.ipv4.tcp_slow_start_after_idle = 0
EOF

/sbin/sysctl -p

My problem

When i run pxe automated install on Dell R750 or other open UEFI service server machines. tftp,dhcpd,httpd are ok. but when the server execute "late-commands" like wget or curl then report an error. other bash commands can execute haven't no error,like echo ,sed,systemctl start service and so on.

when the automated install error then report an error and broken this system install. i use wget can download bash file from my httpd server and exectue.

picture for this: error pic

If you open the error pic slowly, here is the error text describe:

start: subiquity/Late/run/command_0: curtin in-target --target=/target -- /usr/bin/wget -P /root/ http://10.10.11.245/tftp/ubuntu_init.sh
start: subiquity/Meta/status_GET
finish: subiquity/Late/run/command_0: Command '['system-cat', '--level-prefix=false', '--identifier=subiquity_log.3140', 'sh', 'c', 'curtin in-target --target=/target --/usr/bin/wget -P /root/ http://10.10.11.245/tftp/ubuntu_init.sh']' returned non-zero exit status 8.ip a

httpd log

10.10.11.94 - - [13/Dec/2021:15:58:40 +0800] "GET /tftp/ubuntu-20.04.3-live-server-amd64.iso HTTP/1.1" 200 1261371392 "-" "Wget"
10.10.11.94 - - [13/Dec/2021:15:59:01 +0800] "GET /tftp/ubuntu-20.04.3-live-server-amd64.iso HTTP/1.1" 200 1261371392 "-" "Cloud-Init/21.2-3-g899bfaa9-0ubuntu2~20.04.1"
10.10.11.94 - - [13/Dec/2021:15:59:13 +0800] "GET /tftp/ubuntu-20.04.3-live-server-amd64.iso HTTP/1.1" 200 1261371392 "-" "Cloud-Init/21.2-3-g899bfaa9-0ubuntu2~20.04.1"
10.10.11.94 - - [13/Dec/2021:15:59:25 +0800] "GET /tftp/meta-data HTTP/1.1" 200 31 "-" "Cloud-Init/21.2-3-g899bfaa9-0ubuntu2~20.04.1"
10.10.11.94 - - [13/Dec/2021:15:59:25 +0800] "GET /tftp/user-data HTTP/1.1" 200 2772 "-" "Cloud-Init/21.2-3-g899bfaa9-0ubuntu2~20.04.1"
10.10.11.94 - - [13/Dec/2021:15:59:25 +0800] "GET /tftp/vendor-data HTTP/1.1" 404 214 "-" "Cloud-Init/21.2-3-g899bfaa9-0ubuntu2~20.04.1"
10.10.11.94 - - [13/Dec/2021:15:59:26 +0800] "GET /tftp/vendor-data HTTP/1.1" 404 214 "-" "Cloud-Init/21.2-3-g899bfaa9-0ubuntu2~20.04.1"
10.10.11.94 - - [13/Dec/2021:15:59:27 +0800] "GET /tftp/vendor-data HTTP/1.1" 404 214 "-" "Cloud-Init/21.2-3-g899bfaa9-0ubuntu2~20.04.1"
10.10.11.94 - - [13/Dec/2021:15:59:28 +0800] "GET /tftp/vendor-data HTTP/1.1" 404 214 "-" "Cloud-Init/21.2-3-g899bfaa9-0ubuntu2~20.04.1"
10.10.11.94 - - [13/Dec/2021:15:59:29 +0800] "GET /tftp/vendor-data HTTP/1.1" 404 214 "-" "Cloud-Init/21.2-3-g899bfaa9-0ubuntu2~20.04.1"
10.10.11.94 - - [13/Dec/2021:15:59:30 +0800] "GET /tftp/vendor-data HTTP/1.1" 404 214 "-" "Cloud-Init/21.2-3-g899bfaa9-0ubuntu2~20.04.1"
10.10.11.94 - - [13/Dec/2021:15:59:31 +0800] "GET /tftp/vendor-data HTTP/1.1" 404 214 "-" "Cloud-Init/21.2-3-g899bfaa9-0ubuntu2~20.04.1"
10.10.11.94 - - [13/Dec/2021:15:59:32 +0800] "GET /tftp/vendor-data HTTP/1.1" 404 214 "-" "Cloud-Init/21.2-3-g899bfaa9-0ubuntu2~20.04.1"
10.10.11.94 - - [13/Dec/2021:15:59:33 +0800] "GET /tftp/vendor-data HTTP/1.1" 404 214 "-" "Cloud-Init/21.2-3-g899bfaa9-0ubuntu2~20.04.1"
10.10.11.94 - - [13/Dec/2021:15:59:34 +0800] "GET /tftp/vendor-data HTTP/1.1" 404 214 "-" "Cloud-Init/21.2-3-g899bfaa9-0ubuntu2~20.04.1"
10.10.11.94 - - [13/Dec/2021:15:59:35 +0800] "GET /tftp/vendor-data HTTP/1.1" 404 214 "-" "Cloud-Init/21.2-3-g899bfaa9-0ubuntu2~20.04.1"

I execute it manually without error.

If anybody know how to fix this problem or have same as with my problem, please send me message, thanks.

if I did't express clearly, you can point out.

138
0 + 0
pxe
muru avatar
us flag
muru
Please don't post images of text. Copy the text here and use code formatting. It looks like it says "exit status 8", which for [`wget`](https://manpages.ubuntu.com/manpages/jammy/en/man1/wget.1.html) is "8 Server issued an error response."
0
Reply
bowfgc avatar
ps flag
bowfgc
thx, i update text describe. if httpd server not response, why i can use `wget` download manually without error.@muru
0
Reply
muru avatar
us flag
muru
Thanks. Do you have access to the server logs to see what might be happening? What error do you get when you use curl?
0
Reply
bowfgc avatar
ps flag
bowfgc
yes, i check my httpd log, nothing, just record some requests, GET `ubuntu-20.04.3.iso`, `meta-data`,`user-data`, i post the journal entry in the ariticle.
0
Reply
Score:0
Andrew Lowther avatar Ubuntu
jp flag
Andrew Lowther

Your problem is likely caused by the proxy config

  proxy: http://mirrors.aliyun.com/ubuntu

I've found if you specify a proxy then all web traffic will use that proxy. The wget attempt to download your script is being sent via the proxy and failing.

One of the ways the proxy is used is that the environment variables http_proxy and https_proxy will be set to your proxy value in the installer environment. When making web requests wget uses these environment variables to determine the proxy. Here are some simple commandlines you can try in the installer environment shell (or really any Linux system) to verify.

This should work since no proxy is set.

http_proxy= wget http://10.10.11.245/tftp/ubuntu_init.sh

This should error and exit with the return code of 8 like your logs show.

http_proxy=http://mirrors.aliyun.com/ubuntu wget http://10.10.11.245/tftp/ubuntu_init.sh
echo $?

Options

These are a few available ways to work around this

  • don't configure a proxy.
  • early_commands do not appear to be affected by the proxy, so this works to download the script.
  early-commands:
    - /usr/bin/wget -P /run/ http://10.10.11.245/tftp/ubuntu_init.sh
  • unset the relevant environment variable in late-commands. This would have to be done for each command.
  late-commands:
    - http_proxy= curtin in-target --target /target -- /usr/bin/wget -P /root/ http://10.10.11.245/tftp/ubuntu_init.sh

Notes

I tested using the Ubuntu 20.04.3 installer (subiquity 21.08.2)

0 + 0
ธงชาติไทย
Elon Musk
I sit in a Tesla and translated this thread with Ai:

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.