Applying lossless compression (like LZMA) to a database with encrypted or/and signed data is mostly neutral from a security standpoint: it's neither good, nor necessarily bad (but see second point below).
- It provides no serious protection when the database is transferred across an hostile network. Argument: an attack working with the database not compressed can most likely be adapted to be carried with the database compressed, by decompressing, performing the attack, and re-compressing the database if it was modified by the attack.
- If compression introduces a vulnerability that did not exist without compression, and under the assumption that the hostile network has full access to the data (rather than e.g. limited to the data length), said vulnerability can only be due to an error in the decompression code. These can lead to denial of service, or even code injection. That has happened many tines to many decompression code, see e.g. Security and Crash Bugs there, because decompression code is complex. Depending on your taste, that can be mitigated by one or more of:
- Using a language or/and execution environment less likely to lead to exploitable bugs than bare C/C++ code is.
- Reviewing the decompression code, perhaps even proving it correct.
- Testing, perhaps by fuzzing, coverage analysis…
- Authentication (including authenticated encryption) of the compressed data.
From a functional standpoint, the main drawback of compressing a database is that the database in unusable while compressed, and decompressing/re-compressing is costly in CPU and to some degree space and SSD/disk wear. But this is justifiable during transit, to reduce the amount of data transferred.
A simple approach to insuring database security while in transit in an hostile network is to send it PGP/GPG encrypted. This has built-in compression, and can give both confidentiality and integrity, if keys can be safely stored. As pointed in the question, what's stored encrypted in the database won't compressed (unless it's re-encoded to e.g. Base64). I don't know an alternative with post-quantum crypto. But then Cryptographically Relevant Quantum Computers remain highly hypothetical.