Trying to remove all text except for IPs, data is in a file I downloaded

Jrecho

1/31/24, 10:19 PM

This is what I normally use to normalize the data, but it is not working. I need to keep only the IPs, one IP per line, and everything else must go. I'm using this in an .sh shell script.

grep -oE '[0-9]+[.][0-9]+[.][0-9]+[.][0-9]+(/[0-9]+[^\# ]+)?' |\
        sort -V | uniq >> ipban/threat-50.txt

# Format: IP # ThreatLevel ThreatLevel Timestamp CountryCode
104.218.166.164 # 100 2023-01-20T01:33:10Z VN
61.240.137.169 # 100 2023-01-27T00:35:06Z CN
42.119.111.155 # 100 2023-01-20T00:25:25Z VN
190.137.180.107 # 100 2023-01-09T03:28:00Z AR

1 + 1

scripts

grep

text-processing

steeldriver

1/31/24, 10:42 PM

Can you be more specific about how it is "not working"? Is it failing to match things that it should, or matching things that it should not? Or something else?

Score:0

Ubuntu

karel

1/31/24, 10:37 PM

Create a file named input.txt which contains the following example data that you downloaded.

# Format: IP # ThreatLevel ThreatLevel Timestamp CountryCode
104.218.166.164 # 100 2023-01-20T01:33:10Z VN
61.240.137.169 # 100 2023-01-27T00:35:06Z CN
42.119.111.155 # 100 2023-01-20T00:25:25Z VN
190.137.180.107 # 100 2023-01-09T03:28:00Z AR

Run the following commands to create a file named threat-50.txt and append the output list of unique IPs to threat-50.txt:

touch threat-50.txt
cat input.txt | grep -oE '[0-9]+[.][0-9]+[.][0-9]+[.][0-9]+(/[0-9]+[^\# ]+)?' | sort -V | uniq >> threat-50.txt

Results of cat threat-50.txt :

42.119.111.155
61.240.137.169
104.218.166.164
190.137.180.107

+ 1

karel

1/31/24, 11:07 PM

I know this is a useless use of cat, but I indifferently used it anyway because it's easier to read that way.

Elon Musk

I sit in a Tesla and translated this thread with Ai:

EN: Trying to remove all text except for IPs, data is in a file I downloaded

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.