Join Log in
Score:1
basin avatar Server

RewriteRule is applied to all files during directory listing

vn flag
basin

With Options Indexes and LogLevel rewrite:trace6 I see that my rewrite rule is applied to all files in the directory being listed. It even goes inside sub-directories and looks for index.html. This is bad for performance. Is this really necessary? How to disable it? In the actual configuration I have a "prg" RewriteMap and this program receives all listed files on its STDIN.

The problem is reproducible with this minimum of configuration:

Environment:

  • OpenSUSE Tumbleweed
  • apache2-2.4.51-1.1.x86_64

Configs:

  • /etc/apache2/conf.d/test.conf (see below the full httpd.conf)

    LoadModule rewrite_module /usr/lib64/apache2/mod_rewrite.so
LogLevel debug rewrite:trace6
<Directory "/srv/www/htdocs/test">
    Options Indexes FollowSymLinks
    RewriteEngine on
    RewriteRule badrule neverhappen
</Directory>

Commands:

mkdir /srv/www/htdocs/test
touch /srv/www/htdocs/test/zzzzzzzzzzzzzzzzzzzzz{a..z}
systemctl restart apache2
tail -F /var/log/apache2/error_log &
curl -s -o /dev/null http://localhost/test/

Output:

[rid#556dab9a2060/initial] [perdir /srv/www/htdocs/test/] strip per-dir prefix: /srv/www/htdocs/test/ ->
[rid#556dab9a2060/initial] [perdir /srv/www/htdocs/test/] applying pattern 'badrule' to uri ''
[rid#556dab9a2060/initial] [perdir /srv/www/htdocs/test/] pass through /srv/www/htdocs/test/
[rid#556dab9a8090/subreq] [perdir /srv/www/htdocs/test/] strip per-dir prefix: /srv/www/htdocs/test/index.html -> index.html
[rid#556dab9a8090/subreq] [perdir /srv/www/htdocs/test/] applying pattern 'badrule' to uri 'index.html'
[rid#556dab9a8090/subreq] [perdir /srv/www/htdocs/test/] pass through /srv/www/htdocs/test/index.html
[rid#556dab9a8090/subreq] [perdir /srv/www/htdocs/test/] strip per-dir prefix: /srv/www/htdocs/test/index.html.var -> index.html.var
[rid#556dab9a8090/subreq] [perdir /srv/www/htdocs/test/] applying pattern 'badrule' to uri 'index.html.var'
[rid#556dab9a8090/subreq] [perdir /srv/www/htdocs/test/] pass through /srv/www/htdocs/test/index.html.var
[rid#556dab9a8090/subreq] [perdir /srv/www/htdocs/test/] strip per-dir prefix: /srv/www/htdocs/test/HEADER.html -> HEADER.html
[rid#556dab9a8090/subreq] [perdir /srv/www/htdocs/test/] applying pattern 'badrule' to uri 'HEADER.html'
[rid#556dab9a8090/subreq] [perdir /srv/www/htdocs/test/] pass through /srv/www/htdocs/test/HEADER.html
[rid#556dab9a8090/subreq] [perdir /srv/www/htdocs/test/] strip per-dir prefix: /srv/www/htdocs/test/zzzzzzzzzzzzzzzzzzzzza -> zzzzzzzzzzzzzzzzzzzzza
[rid#556dab9a8090/subreq] [perdir /srv/www/htdocs/test/] applying pattern 'badrule' to uri 'zzzzzzzzzzzzzzzzzzzzza'
[rid#556dab9a8090/subreq] [perdir /srv/www/htdocs/test/] pass through /srv/www/htdocs/test/zzzzzzzzzzzzzzzzzzzzza
[rid#556dab9a8090/subreq] [perdir /srv/www/htdocs/test/] strip per-dir prefix: /srv/www/htdocs/test/zzzzzzzzzzzzzzzzzzzzzb -> zzzzzzzzzzzzzzzzzzzzzb
[rid#556dab9a8090/subreq] [perdir /srv/www/htdocs/test/] applying pattern 'badrule' to uri 'zzzzzzzzzzzzzzzzzzzzzb'
[rid#556dab9a8090/subreq] [perdir /srv/www/htdocs/test/] pass through /srv/www/htdocs/test/zzzzzzzzzzzzzzzzzzzzzb
[rid#556dab9a8090/subreq] [perdir /srv/www/htdocs/test/] strip per-dir prefix: /srv/www/htdocs/test/zzzzzzzzzzzzzzzzzzzzzc -> zzzzzzzzzzzzzzzzzzzzzc
[rid#556dab9a8090/subreq] [perdir /srv/www/htdocs/test/] applying pattern 'badrule' to uri 'zzzzzzzzzzzzzzzzzzzzzc'
[rid#556dab9a8090/subreq] [perdir /srv/www/htdocs/test/] pass through /srv/www/htdocs/test/zzzzzzzzzzzzzzzzzzzzzc
...

Full httpd.conf:

# sudo -u nobody /sbin/httpd -d /etc/httpd-test -f /etc/httpd-test/httpd.conf -DFOREGROUND
#
# The following is not needed on SUSE
LoadModule mpm_event_module modules/mod_mpm_event.so
#LoadModule mpm_prefork_module modules/mod_mpm_prefork.so
#LoadModule mpm_worker_module modules/mod_mpm_worker.so

# The following is not needed on SUSE
LoadModule unixd_module modules/mod_unixd.so

LoadModule authz_core_module modules/mod_authz_core.so
LoadModule autoindex_module  modules/mod_autoindex.so
LoadModule rewrite_module    modules/mod_rewrite.so

ErrorLog "|/bin/cat"
LogLevel debug rewrite:trace6
PidFile /dev/shm/httpd-test.pid

ServerName localhost

Listen 8080

DocumentRoot "/srv/www/htdocs"

<Directory "/srv/www/htdocs/test">
    Require all granted
    Options Indexes FollowSymLinks
    RewriteEngine on
    RewriteRule badrule neverhappen
</Directory>
70
0 + 0
Score:0
avatar Server
kz flag
MrWhite

With a directory listing generated by mod_autoindex an internal subrequest is triggered for all listed items in the directory. This is indicated in the log with subreq (as opposed to initial).

You can prevent the RewriteRule directive being processed for subrequests by using the NS (nosubreq) flag. For example:

RewriteRule badrule neverhappen [NS]

Alternatively, you can move the RewriteRule directive out of the <Directory> container and have it directly under the <VirtualHost> instead. Note that the directive might need to be changed slightly since it's now working in a different context (virtualhost as opposed to directory) and processed earlier. This naturally prevents the directive being processed for each item in the directory listing (without having to include the NS flag). However, it can still be called for other subrequests, such as calls for HeaderName and ReadmeName files associated with the directory listing (if these directives are set, regardless of whether the files exist or not), so you may still choose to include the NS flag on these directives if this is an issue.

It even goes inside sub-directories and looks for index.html

This is caused by mod_dir testing for DirectoryIndex documents. If this is not required then it can be disabled:

DirectoryIndex disabled

Reference:

0 + 0
ธงชาติไทย
Elon Musk
I sit in a Tesla and translated this thread with Ai:

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.