Why is the WordPress router non-responsive for some static files but not others?

Context

I have a website of approximately 50k files (mostly static, a handful of PHP files) and plan to gradually migrate all of them into WordPress. There are about 500 files/folders in the root folder and the total of all is about 10GB.

Due to the amount of time the migration into WordPress will take, I need my URL to be able to serve the new the WordPress content and the old static content as transparently as possible. So, example.com/ and example.com/news might go to WordPress while example.com/about.html and example.com/products (which corresponds to example.com/products/index.html) might go to the static assets.

At first I was thinking I might have to do a series of redirects (ugly for the user) or a reverse proxy (complicated to maintain), but then I realized that the default behavior of the WordPress router seems like what I want. From what I read, the behavior of the WordPress router is basically:

1. If the path corresponds to a file on the file system, serve that file.
2. Else if the path corresponds to a WordPress page, serve that page.
3. Else, serve 404 message.

I have WordPress in the root of my site (ex: example.com/wp-content).

Attempts

First, I tried tossing example.com/test/index.html on the WordPress server. It behaved as described above and my pages in WordPress worked fine. Awesome!

Next, I tossed a "real" file online, example.com/test.php. It loaded fine, but threw an error because it refers to a bunch of other files (php includes, js, css, images). So, I tossed our assets folder online. This folder contains all of the libraries and includes that the all of the static sites use so it's about 2k files and 50MB. Now, when the page loaded, the css, php includes and javascript worked perfectly, but none of the images from the server loaded. (It did load cleardot.gif from google.com just fine.). The browser console shows that the requests for these images timed out. For reference, one of the images is 250K so they aren't huge and, in a given folder, there's probably anywhere from a dozen to a few hundred files. All the while, loading a WordPress page would continue to work as expected.

From there, I figured maybe there was something particular about those image files, so I decided to cross that bridge later and I uploaded approximately 10k / 2GB more files to test. I started trying other URLs... example.com/test/test2.html, example.com/test3/index.htm, ... and these time out too. However, the main WordPress site still loads fine and that earlier example.com/test/index.php still loads fine minus the images. This behavior was the same in a new browser or when I skipped the browser cache.

For the file that "worked" (minus the images), I tried deleting it and got a 404 and then re-added it and it showed up okay. So, it doesn't appear to be some cache thing or to relate to when I uploaded the file.

I looked on the server with SSH and the files that are served have the exact same permissions, owner and group settings as the files that aren't served.

So, all that considered, I'm thinking maybe there is some limitation in the way the the Wordpress router engages with the file system but I can't figure out what it is or how to get around it. Unfortunately, Googling a lot of these issues like timeouts, WordPress routing, handling static assets on WordPress, etc. is drowned out by posts about other topics.