Score:0

Why does my container not contain DB data after I commit it?

fr flag

I'm trying to create a PostgreSQL container that already contains development data. I can see the data I have created, but when I commit, push, and pull the image, the data is not there.

Below is my Dockerfile:

FROM postgis/postgis:11-2.5
MAINTAINER Me

# Contents of 20-init_db.sql:
# CREATE EXTENSION IF NOT EXISTS btree_gist;
# CREATE EXTENSION IF NOT EXISTS hstore;
COPY ./20-init_db.sql /docker-entrypoint-initdb.d/20-init_db.sql
COPY ./make_data.sh /make_data.sh

I create the container using the script below, and data is successfully populated.

#!/bin/bash

aws ecr get-login-password --region us-east-1 | \
    docker login --username AWS --password-stdin 23456789JQKA.dkr.ecr.us-east-1.amazonaws.com

aws ecr create-repository --repository-name dev_postgres --region us-east-1

# Make sure the build environment is clean
docker stop dev_postgres_container
docker rm dev_postgres_container
docker rmi dev_postgres_img

# Prepare the container
docker build -f postgres.Dockerfile -t dev_postgres_img .
docker run -d --name dev_postgres_container \
    -e POSTGRES_DB=the_database \
    -e POSTGRES_USER=the_user \
    -e POSTGRES_PASSWORD=the_password \
    -e PGDATA=/var/lib/postgresql/data/pgdata
    dev_postgres_img
docker exec -it dev_postgres_container bash /make_data.sh

###
### This returns the correct number of users, so there is data in the container at this point.
###
docker exec -it dev_postgres_container /usr/bin/psql -U the_user the_database -c "select count(*) from auth_user;"

# Commit the container state and save it to Amazon ECR:
docker commit $(docker ps -q --filter "name=dev_postgres_container") 23456789JQKA.dkr.ecr.us-east-1.amazonaws.com/dev_postgres:11-2.5-with-data

# Also tag it with "latest" and the timestamp
docker tag 23456789JQKA.dkr.ecr.us-east-1.amazonaws.com/dev_postgres:11-2.5-with-data 23456789JQKA.dkr.ecr.us-east-1.amazonaws.com/dev_postgres:latest-with-data
docker tag 23456789JQKA.dkr.ecr.us-east-1.amazonaws.com/dev_postgres:11-2.5-with-data 23456789JQKA.dkr.ecr.us-east-1.amazonaws.com/dev_postgres:$( date -u +%Y%m%d_%H%MZ )-with-data

# Push
docker push 23456789JQKA.dkr.ecr.us-east-1.amazonaws.com/dev_postgres --all-tags

However, when I pull the container in an isolated environment, there's no data:

brew install virtualbox
brew install multipass
sudo multipass set local.driver=virtualbox
multipass launch --name ubuntu2
multipass mount ~/.aws ubuntu2:/home/ubuntu/.aws
multipass mount ./db ubuntu2:/home/ubuntu/db
multipass exec ubuntu2 -- bash /home/ubuntu/db/test_db_container.sh

### Contents of test_db_container.sh are below: ###

curl -fsSL https://get.docker.com -o get-docker.sh
sudo sh get-docker.sh

sudo apt-get install -y awscli

# Pull Docker container:
aws ecr get-login-password --region us-east-1 | \
    sudo docker login --username AWS --password-stdin 23456789JQKA.dkr.ecr.us-east-1.amazonaws.com
sudo docker pull 23456789JQKA.dkr.ecr.us-east-1.amazonaws.com/dev_postgres:latest-with-data

sudo docker run -d --name the_db 23456789JQKA.dkr.ecr.us-east-1.amazonaws.com/dev_postgres:latest-with-data

###
### The auth_user table does not exist.
###
sudo docker exec -it the_db /usr/bin/psql -U the_user the_database -c "select count(*) from auth_user;"

sudo docker history 23456789JQKA.dkr.ecr.us-east-1.amazonaws.com/dev_postgres:latest-with-data

## Result:
IMAGE          CREATED        CREATED BY                                      SIZE      COMMENT
854dbef239c2   23 hours ago   postgres                                        70B
4b1bd66a8d33   23 hours ago   COPY ./make_data.sh /make_data.sh               3.03kB    buildkit.dockerfile.v0
<missing>      23 hours ago   COPY ./20-init_db.sql /docker-entrypoint-ini…   188B      buildkit.dockerfile.v0
<missing>      23 hours ago   MAINTAINER Me                                   0B        buildkit.dockerfile.v0
[...]

Side note: I've considered creating a data volume container, but this method seemed simpler and better documented.

Other side note: I can't add the data in the Dockerfile because Postgres needs to be running before data can be inserted.

Why isn't docker commit saving the state of my container? Why does the image not contain data?

mreferre avatar
nl flag
It's likely because the data volume is a .... volume. See [here](https://stackoverflow.com/questions/27377876/docker-postgres-with-initial-data-is-not-persisted-over-commits)
Travis avatar
fr flag
Yes, that was the problem. I didn't realize the PostgreSQL container set `/var/lib/postgresql/data` as a volume. The solution is [here](https://stackoverflow.com/a/29956964/1477364), although the more I read, the more it looks like a data-only container is the best practice for this. (Post your comment as an answer, and I'll accept it.)
mreferre avatar
nl flag
Cool. Happy to hear that. Done.
Score:1
nl flag

As per the comments under the main thread the solution was similar to the one suggested in this existing answer. It boils down to the data directory being a volume and hence living outside of the context of the container being modified/committed.

mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.