Firstly try really hard not to trust LTT when it comes to Production workloads - they routinely make work for themselves building unsupportable, unstable "serverlike" systems - because it gets views, as do the subsequent videos where they 'fix' the problems they cause. Truth is that professional system design, building and administration is kind of dull because all we care about is uptime and serving our customers - it's dull, doesn't get clicks but keeps hundreds of thousands of professionals in work - and the world turning.
Secondly it's perfectly possible to build GPU virtualisation that's reliable and secure - I do it myself - but I use VMware's ESXi as my hypervisor and NVidia's GRiD software and supported hardware to achieve this. None of this is free but it's been working well for half-a-decade or more for me.
Obviously you'll have your own budget in mind, which may preclude some or all of the tools I use to do this but to answer your question - yes it's very do'able - but your tool set will vary this. I'm sure there's someone else on here who's maybe tried to do this with Hyper-V.
Certainly full PCIe device passthrough works with most hypervisors given enough configuration, and should be as secure as any other virtualised resource within your VMs.
Anyway I hope you understand my answer, get even better help off others, but please - friends don't let friends trust LTT designs - not for Production anyway, seriously what they do with their disk arrays sends shudders down the spines of storage-pros :)