Join Log in
Score:1
Justin Washek avatar Ubuntu

How to grab only the 2nd-level domains from a list of subdomains

ma flag
Justin Washek

What I need

I have a list of domains like so:

a.example.com
b.foo.com
a.b.bar.com

I only want the output to grab the second-level domains and nothing else, i.e., no 3rd-level or higher. This is what I'm looking for from my example list above:

example.com
foo.com
bar.com

What I tried

I've tried using sed, awk, and cut as follows:

sed

cat domains.txt | sed 's/\.$//g'
cat domains.txt | sed -r 's/^(.*)_/\1\\/; s/.$//g'  # this removes the last character for some reason

awk

awk '{ sub(/\.$/, ""); print $NF }' domains.txt
cat domains.txt | awk -F\. '{print $1,$2}' | tr ' ' '.' # won't work since there are 4th level domains

cut

cat domains.txt | cut -d '.' -f[field] # won't work since there are 4th level domains
71
0 + 0
bac0n avatar
cn flag
bac0n
`rev domains.txt | cut -d . -f-2 | rev`
1
Reply
Justin Washek avatar
ma flag
Justin Washek
That was annoyingly easy... thank you so much for the comment, this worked!
1
Reply
bac0n avatar
cn flag
bac0n
you could use grep too: `grep -Po '[^.]+\.[^.]+$' domains`
0
Reply
bac0n avatar
cn flag
bac0n
`while IFS=. read -a a; do echo ${a[@]:(-2):2}; done < domains`, if the list is short, probably the one I use.
0
Reply
Justin Washek avatar
ma flag
Justin Washek
Awesome, thank you so much! I am always down to learn different ways to do things. You've been extremely helpful!
0
Reply
Will avatar
id flag
Will
@bac0n - ?pop those as an answer - it’s obviously been helpful!
0
Reply
cn flag
glenn jackman
The grep answer should also work with `-E`, nothing pcre-specific about that one.
0
Reply
bac0n avatar
cn flag
bac0n
@glennjackman Not sure if it's just me, but `-E` is in some cases almost x3 slower than `-P` especially redirection.
0
Reply
Score:4
bac0n avatar Ubuntu
cn flag
bac0n

In cases where you need to start your match from the right, you can use an end anchor $ to fixate the pattern to the end of the line.

grep:

grep -Po '[^.]+\.[^.]+$' domains.txt

sed:

sed  's/.*\.\([^.]\+\.[^.]\+\)$/\1/' domains.txt

awk has a pre-defined variable named NF holding the number of fields for the current record. You may combine the NF variable with the field specifier $ to reference the value instead.

awk:

awk -F . -vOFS=. '{print $(NF-1), $NF}' domains.txt

You can also reverse the text for commands like: read or cut that purely reads from left to right.

rev, cut:

rev domains.txt | cut -d . -f1,2 | rev

Bash only example:

while read -r; do \
    printf %s\\n ${REPLY/#${REPLY%.*.*}.}; \
done < domains.txt
0 + 0
ธงชาติไทย
Elon Musk
I sit in a Tesla and translated this thread with Ai:

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.