Score:-2

Server

How is the underlying disk space (and associated technologies) for persistent storage usually configured in Docker

BitShift

4/6/23, 7:02 PM

I want to attach persistent storage to my docker containers. I have tried NFS using TrueNAS and seem to have no luck there.

Surely, I am approaching this in the wrong way. What are the common conventions for attaching persistent storage OUTSIDE of the container so that if the container fails or is remade, the data is re-attached or at the very least remains existent.

EDIT: Apologies for the confusion. I am not talking about bind mounts or volumes, I am well aware of those technologies. I am referring to the underlying filesystems i.e. NFS etc.. are there standard toolings or management systems e.g. Longhorn for block storage on kubernetes. is it better to use a file, object, or block storage solution etc...

EDIT 2: I have ready everything in the Docker docs. I know about volumes and drivers. What I don't know is how to use them properly with respective options as they are not documented properly, or how these things are commonly installed into a solution. Yes I can just "use the driver" but how exactly? Is NFS a common theme? What about block storage? Object storage? How is this architected on a system level.

EDIT: I'm adding this edit because some weasal closed it for not being specific enough. As if asking about this problem isn't relevant. I'm the only one with had an actual answer. Not even the common users of docker have a clue as to how this all works under the hood. You don't think we should have a post about that question? Oh no of course not.. just have ppl repeat the same shit over and over again.

0 + 0

docker

docker-compose

Moshe Katz

4/6/23, 11:54 PM

Have you read about [bind mounts](https://docs.docker.com/storage/bind-mounts/) and [volumes](https://docs.docker.com/storage/volumes/)? One or both of those is what you need.

BitShift

4/7/23, 12:48 AM

That's not what I'm referring to... let me clarify the question

Gerald Schneider

4/7/23, 8:22 AM

Docker only knows volumes. It doesn't care how you store them.

Gerald Schneider

4/7/23, 8:26 AM

You should read the section about volumes more carefully, especially the part about volume drivers.

BitShift

4/8/23, 6:44 AM

@GeraldSchneider I've read everything there is to read about storage from the docker docs, what remains unclear is how these drivers are useful. They are sparse in documenting use cases, options, and general architectural design. The last of which I have come here to clarify.

Score:2

Server

Massimo

4/8/23, 6:58 AM

What are the common conventions for attaching persistent storage OUTSIDE of the container

There aren't.

The whole Docker paradigm is based on the assumption that you can create and destroy containers as you wish. They are not persistent, and they are not supposed to be.

If your containers expect something to be there, You're Doing It Wrong™.

0 + 0

BitShift

4/8/23, 7:01 AM

Well sure, it needs to be as stateless as possible and lack of dependency is my aim. Things like Wikijs content, Organizr/Heimdall dashboard config files, Ansible cookbooks that I can save outside the container. This way, in the event I need to rebuild the container, I can simply reload my saved content. Surely that sits within the ethos and philosophy of containers no?

Massimo

4/8/23, 5:48 PM

Yes, of course. But for all those cases the solution is to store your persistent data *outside* the containers, in some place they can access. *Where*, exactly, is up to your implementation.

BitShift

4/8/23, 8:31 PM

Yes. That is the purpose of the question. I'm trying to learn more about the underlying technologies used to facilitate that and I've since learnt about different storage technologies. I'll do some kind of write up here when I'm finished. For now it looks as though NFS for non-performant persistent storage outside the container is preferable. Though for the way I want to configure these containers, I think that block-level with iSCSI will be better.

Score:0

Server

BitShift

4/9/23, 5:12 AM

The convention in terms of underlying file system technology appears to be using block storage solutions such as Cinder, BGFS etc... (which can use Ceph, iSCSI, and other block file types) or object storage like Minio

These kinds of solutions help to manage the underlying disk space easily and also take into account an implicit need of distributed file systems.

It does depend on use cases and NFS is perfectly suitable for use cases where performance is not a concern, like writing files. In this way, a wikijs container would be a perfect candidate for NFS so that writing the user-generated content can remain separate from the container.

Where block storage solutions also shine for containers is providing ease of deployment in rapid create-destroy cycles. As is the typical use case.

Then again, if you are running lots and lots of containers that have indefinite life spans (like I am with Nextcloud, Heimdall, pihole, etc...) then block storage solutions may also provide you with a more flexible arrangement. I like the idea of using block storage in this scenario as the disk and file system are handled by the application/container, rather than it depending on external factors. That may or may not be a real world concern, but I could imagine there being potential issues with NFS because of those reasons or more specifically: less compromise with a block storage solution.

Finally, and I need to confirm this, tracking the volumes associated with each container becomes a task in and of itself. These block storage solutions may provide a neat and manageable way to track that across create-destroy cycles for backup and restore.

0 + 0

AlexD

4/19/23, 9:37 AM

Minio is not a block storage

BitShift

4/19/23, 10:35 PM

Yes @AlexD it's object storage. Next time please just edit the entry.

Elon Musk

I sit in a Tesla and translated this thread with Ai:

EN: How is the underlying disk space (and associated technologies) for persistent storage usually configured in Docker

TH: พื้นที่ดิสก์พื้นฐาน (และเทคโนโลยีที่เกี่ยวข้อง) สำหรับการจัดเก็บข้อมูลแบบถาวรมีการกำหนดค่าอย่างไรใน Docker

RO: Cum este de obicei configurat spațiul pe disc (și tehnologiile asociate) pentru stocarea persistentă în Docker

RU: Как базовое дисковое пространство (и связанные с ним технологии) для постоянного хранилища обычно настраивается в Docker

VI: Không gian đĩa bên dưới (và các công nghệ liên quan) để lưu trữ liên tục thường được định cấu hình như thế nào trong Docker

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.