However, when I connect a USB-A to USB-C converter and then plug the USB-C <-> Displayport cable in, nothing shows up on the monitor.
Do you mean an adapter with a male USB-A and female USB-C like that linked to below?
https://www.amazon.com/dp/B07KCL8WZK
Those adapters violate the USB specs and so their function is not defined, and further they have the potential to cause damage because their function is not defined.
There are USB-A to DisplayPort adapters, and they work by having a GPU in them as USB-A does not support video output any other way, at least by no other means defined by the people that created and maintain the USB spec. A cheap converter is not going to have a GPU, and those that do tend to use DisplayLink GPUs, some examples can be found on the DisplayLink website. https://www.displaylink.com/products/find?cat=3&vid_dp=1
Not all computers with USB-C support video out on USB-C, and supporting one display by USB-C does not mean it will support two displays by USB-C.
How can I connect 2x4k@60Hz monitors now?
That depends on the capabilities of the computer. Using a USB-A DisplayLink adapter is one way to add another display but since USB-A was not designed to support video this has limitations. USB-A will handle only 5 or 10 Gbps, and 2x4k@60Hz will require 12.54 Gbps without compression as shown on the chart linked below. https://en.wikipedia.org/wiki/DisplayPort#Refresh_frequency_limits_for_standard_video
DisplayLink adapters might be able to output 4k@60Hz with some kind of compression but that means the computer needs to use the CPU to create this compressed video stream, and that is asking a lot of the CPU as a CPU is less than optimal for this. There is a reason we use a GPU for graphics and a CPU for more general computing, graphics processing requires vastly different kinds of math to be done. A CPU can be put into a GPU role, and it's the DisplayLink drivers that tell the CPU how to do this. DisplayLink chips are not doing a lot of processing but instead are mostly just for adding the hardware interface from the CPU to the display. For lightweight video needs a DisplayLink adapter often works well, but are not likely to get 60Hz refresh because of bandwidth limits.
If your computer supports Thunderbolt then another option is a more capable Thunderbolt GPU, as opposed to a USB GPU. Thunderbolt was made for adding a GPU and other high bandwidth devices to a computer, this means not relying on the CPU to do graphics processing and not trying to squeeze more than 12 Gbps of data through a 5 Gbps USB pipe.
Thunderbolt external GPUs can be quite expensive and still require a lot of the computer. It may make more sense on a bang/buck calculation to buy a new computer than try to get an old computer to support 2x4K displays.
