How to log URL to a file only if it’s HTTP status code is 200?

I want to make a script so I can check if I can connect to several URL.

For example I do curl -i -L --silent URL | grep "HTTP/2 200" >> *.txt

I want to combine it with an if statement so I can do if grep "HTTP/" 200" >> *.txt", then: grep "Location" >> *.txt

So if it does grep "HTTP/2 200" then it should grep the "location" (URL) as well. I can't get it to work though.

Also I want to do echo "-------------------------------------" between every URL output so it shows like this in a file:

#URL1

HTTP(1)

URL(1)

#URL2

HTTP(2)

URL(2)

and so on.

thanks for every help.

If you want the HTTP response/status and the URL saved to a file in the format you desire. then curl alone can do that ... It has the option -w or --write-out(Which defines what to display on stdout after a completed and successful operation. The format is a string that may contain plain text mixed with any number of variables ... e.g. -w "-----------\nStatus: %{http_code}\nURL: %{url}\n") and if you redirect the rest of the output to e.g. /dev/null then you end up with a clean result of just what you want and you can then redirect that result and append it to a file called e.g. url.txt like so:

curl -o /dev/null -s -w "-----------\nStatus: %{http_code}\nURL: %{url}\n" https://askubuntu.com >> url.txt

If you have a list of URLs(each on a new line) in a file called e.g. list.txt like so:

$ cat list.txt
https://askubuntu.com/
https://unix.stackexchange.com/
https://notworkingurlblabla.com/

Then, you can check all the URLs in that file, filter-out non 200 status URLs and append the result to url.txt all at once e.g. like so:

xargs -n 1 <list.txt curl -o /dev/null -s -w "%{http_code} %{url}\n" | \
awk '{if ($1 == "200") printf "------------\nStatus: "$1"\nURL: "$2"\n"}' >> url.txt

You can also, obviously, use awk to filter output with a single URL as well like so:

curl -o /dev/null -s -w "%{http_code} %{url}\n" https://askubuntu.com/ | \
awk '{if ($1 == "200") printf "------------\nStatus: HTTP "$1"\nURL: "$2"\n"}' >> url.txt