Score:2

Ubuntu

Passing multiple file patterns to grep

Fatipati

3/29/24, 8:00 PM

I have a sequence of search patterns stored in a bash array (ptrn), which I want to pass to a grep command. How can I do it?

  ptrn=("FN" "DS")
  for fl in "$@"; do  # loop through number of files
    if [[ -f "$fl" ]]; then
      printf '\n%s%s%s\n\n' "$mgn" "==>  $flnm  <==" "$rst"
      grep --color "$ptrn" "$flnm"
    fi
  done

758

3 + 6

command-line

bash

grep

petep

3/29/24, 8:18 PM

Use ${ptrn[@]} for grepping

Fatipati

3/29/24, 8:36 PM

Does not seem to be enough because I get `grep: DS: No such file or directory`. It is thinking that the pattern is a file.

petep

3/29/24, 8:40 PM

Try "${ptrn[@]}" instead for grep

Fatipati

3/29/24, 8:46 PM

Same problem. Does not recognise as a search pattern.

petep

3/29/24, 8:51 PM

How about grep "${ptrn[0]}\|${ptrn[-1]}" . First and last.

Fatipati

3/29/24, 8:57 PM

Have done this `pa=$(printf '%s\|' "${ptrn[@]}")` followed by `grep --color -E "$pa" "$flnm"`. But nothing gets printed.

Score:3

Ubuntu

Thor

3/29/24, 10:07 PM

How about giving the patterns to grep through a sub-shell, e.g.:

grep -f <(printf "%s\n" "${ptrn[@]}") FILE...

+ 3

Raffa

3/29/24, 10:27 PM

You mean `bash` specific process substitution syntax `<(...)` ... That is smart, processing array elements into newlines and reading by `grep` as a file of patterns ... If, however, you quote the expansion of the array i.e. `<(printf "%s\n" "${ptrn[@]}")` , then each element will be passed as a single token, which might be desirable to preserve exact spaces ... +1

steeldriver

3/29/24, 11:04 PM

Even without process substitution, you could do `printf "%s\n" "${ptrn[@]}" | grep -f - FILE` (of course, assumes you are not grepping stdin itself)

Fatipati

3/30/24, 12:14 AM

I confirm that quoting the expansion of the array is required.

Score:2

Ubuntu

Raffa

3/29/24, 9:05 PM

If your patterns/words stored as array elements are guaranteed to not have spaces or non-escaped shell special characters in them, then you can use bash's parameter expansion to pass the array elements to grep as separate individual patterns e.g. -e FN -e DS ... like so:

ptrn=("FN" "DS")

# Prepend "-e" to each array element
ptrn2="${ptrn[@]/#/-e }"

# Don't quote "$ptrn2" or it will be passed as a single token and wouldn't work.
grep --color $ptrn2 file

Or if they might contain non-escaped shell special characters, you can build the regular expressions around |(or) (splitting on all spaces as well but not failing) and use it with something like:

ptrn=("FN" "DS")

# Translate spaces " " single or multiple including spaces between words in a single array element into "|".
ptrn2="$(echo ${ptrn[@]} | tr " " "|")"

# -E enables extended regular expressions ... needed for this to work.
grep --color -E "$ptrn2" file

Or to preserve exact spaces inside each regular expression pattern passing each array element as an individual token and building an extended regular expression of them using the |(logical or), you can do something like this:

ptrn=("FN" "DS")

# Append "|" to each array element.
ptrn2="$(printf '%s|' "${ptrn[@]}")"

# Delete the last character i.e. the extra "|".
ptrn2="${ptrn2::-1}"

# -E enables extended regular expressions ... needed for this to work.
grep --color -E "$ptrn2" file

+ 9

Fatipati

3/29/24, 9:16 PM

I want the search pattern to be any allowable regex, with exact strings only being a special case. This means that the demand of not to have spaces in the patterns is too restrictive.

Fatipati

3/29/24, 9:27 PM

Would that not be the same as `pa=$(printf '%s\|' "${ptrn[@]}")`?

Fatipati

3/29/24, 9:30 PM

Something specific to bash must be going on because `grep -E "## FN|## DS" ./firefly.rc` works perfectly fine.

Fatipati

3/29/24, 9:38 PM

Got things to work by removing the first `"|"`. Therefore `pa=$(printf '|%s' "${ptrn[@]}") ; pa="${pa:1}"`.

Raffa

3/29/24, 9:52 PM

@Backy Well done :-) ... I added extra info to the answer.

Fatipati

3/30/24, 12:22 AM

There also exists the capability of using the `-e` flag multiple times up to the desired number of search patterns.

Fatipati

3/30/24, 12:30 AM

How could I possibly handle the supply of multiple `-e PATTERN` expressions ?

Raffa

3/30/24, 12:50 AM

@Backy Other than the example in my answer. .. IIRC `-e` can also be passed as an option with `-f` to read patterns from file

Fatipati

3/30/24, 1:21 AM

In a loop I could do `ptrn[$i]="-e ${ptrn[$i]}"`. But I wonder whether blanks in the array element could be misinterpreted, as the `-e` would not be passed separately to the pattern.

Score:2

Ubuntu

muru

3/30/24, 5:12 AM

Two options:

Standards-compliant way: Join the patterns with newline and provide it as single argument:
```
grep -e "$(printf "%s\n" "${ptrn[@]}")" ...
```
(This feature is specified by the POSIX standard: "The pattern_list's value shall consist of one or more patterns separated by <newline> characters ...")
Non-standard, but still safe way: When using a shell with arrays, like, e.g., bash, build an array of arguments to grep:
```
args=()
for p in "${ptrn[@]}"
do
   args+=(-e "$p")
done
grep "${args[@]}" ...
```
This is safe from field-splitting and globbing, and in general is how command lines should be built from variables.

+ 2

Fatipati

3/30/24, 10:07 AM

I would that say that storing in a single string, for instance: `mptrn=$( printf -- ' -e %s' "${ptrn[@]}" ) ; grep "$mptrn" ..` introduces certain problems.

muru

3/30/24, 1:50 PM

Yes, but that's irrelevant to my answer.

Elon Musk

I sit in a Tesla and translated this thread with Ai:

EN: Passing multiple file patterns to grep

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.