Blocking Port 8080 from direct IP access but allow access via domain (Virtualhost Config)

I want to access web server from http://subdomain2.domain.com from URL but not from direct IP (http://1.1.1.1:8080). For instance, below is my virtualhost config file (site2.conf) in /etc/apache2/sites-available/ but section <VirtualHost *:8080> ... </VirtualHost> doesn't seem to have any affect.

<VirtualHost *:80>
    ServerName subdomain2.domain.com
    ServerAlias www.subdomain2.domain.com
    ServerAdmin [email protected]
    DocumentRoot /var/www/site2
    Keepalive On

    ErrorLog ${APACHE_LOG_DIR}/error.log
    CustomLog ${APACHE_LOG_DIR}/access.log combined

    RewriteEngine on
    RewriteCond %{HTTP:UPGRADE} ^WebSocket$ [NC]
    RewriteCond %{HTTP:CONNECTION} ^Upgrade$ [NC]
    RewriteRule .* ws://localhost:8080%{REQUEST_URI} [P]

    ProxyPreserveHost On
    ProxyPass "/stream" ws://localhost:8080/ retry=0 timeout=5
    ProxyPass "/" http://localhost:8080/ retry=0 timeout=5
    ProxyPassReverse / http://localhost:8080/
</VirtualHost>

<VirtualHost *:8080>
        ServerName 1.1.1.1:8080
        ServerAlias  2001:0db8:85a3:0000:0000:8a2e:0370:7334:8080
        DocumentRoot /var/www/site2
        <Location />
            Require all denied
        </Location>
        ErrorLog ${APACHE_LOG_DIR}/error.log
        CustomLog ${APACHE_LOG_DIR}/access.log combined
</VirtualHost>

For http://subdomain1.domain.com, I have already implemented this successfully in (000-default.conf) in /etc/apache2/sites-available/ and its working fine after adding section <VirtualHost *:80> ... </VirtualHost>. Below is the code for reference:

<VirtualHost *:80>
        ServerName subdomain1.domain.com
        ServerAlias www.subdomain1.domain.com
        ServerAdmin [email protected]
        DocumentRoot /var/www

        ErrorLog ${APACHE_LOG_DIR}/error.log
        CustomLog ${APACHE_LOG_DIR}/access.log combined
</VirtualHost>

<VirtualHost *:80>
        ServerName 1.1.1.1
        ServerAlias  2001:0db8:85a3:0000:0000:8a2e:0370:7334
        DocumentRoot /var/www
        <Location />
            Require all denied
        </Location>

        ErrorLog ${APACHE_LOG_DIR}/error.log
        CustomLog ${APACHE_LOG_DIR}/access.log combined
</VirtualHost>

Since you are reverse proxying to localhost:8080, there is probably some application server other than Apache listening on port 8080. You cannot control that other software by adding VirtualHost configurations to Apache. Instead, change the settings of that software and make it only listen on local loopback (localhost) instead of every interface.

As it turns out the other server is Gotify, we can consult its configuration documentation.

gotify/server looks in the following paths for config files

./config.yml /etc/gotify/config.yml

server:
  keepaliveperiodseconds: 0 # ...
  listenaddr: "" # the address to bind on, leave empty to bind on all addresses
  port: 80 # the port for the http server

In this config.yml, change the parameter server.listenaddr to 127.0.0.1 as it defaults to listening on all addresses.