Score:1

Converting specific folder of S3 into browsable directory list without making it public

ua flag

I have a bucket that I'd like to access using a browser similar to http://data.openspending.org/ and I'd like only a subfolder to be visible. So if Bucket1 has multiple folders, I only wanna show and let users download contents of zipFiles folder(Bucket1/zipFiles/*)

Taking instructions from https://github.com/rufuspollock/s3-bucket-listing I was able to get it done with the following 5 changes

  1. Disable public access blocking in bucket permissions

  2. Add bucket policy

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AllowPublicRead",
            "Effect": "Allow",
            "Principal": {
                "AWS": "*"
            },
            "Action": "s3:GetObject",
            "Resource": [
                "arn:aws:s3:::Bucket1/index.html",
                "arn:aws:s3:::Bucket1/zipFiles/*"
            ]
        },
        {
            "Sid": "AllowPublicList",
            "Effect": "Allow",
            "Principal": "*",
            "Action": "s3:ListBucket",
            "Resource": "arn:aws:s3:::Bucket1",
            "Condition": {
                "StringLike": {
                    "s3:prefix": "zipFiles/*"
                }
            }
        }
    ]
}
  1. Add CORS in bucket permissions

  2. Add index.html in the root of the bucket

<!DOCTYPE html>
<html>
<head>
  <title>S3 Bucket Listing Generator</title>
</head>
<body>
  <div id="navigation"></div>
  <div id="listing"></div>

<script type="text/javascript" src="https://ajax.googleapis.com/ajax/libs/jquery/3.1.1/jquery.min.js"></script>
<script type="text/javascript">
  var S3BL_IGNORE_PATH = false;
  // var BUCKET_NAME = 'Bucket1';
  var BUCKET_URL = 'https://Bucket1.s3-eu-west-1.amazonaws.com';
  // var S3B_ROOT_DIR = 'SUBDIR_L1/SUBDIR_L2/';
  // var S3B_SORT = 'DEFAULT';
  // var EXCLUDE_FILE = 'index.html';  // change to array to exclude multiple files, regexp also supported e.g. /^(.*\/)?index.html$/ to exclude all index.html
  // var AUTO_TITLE = true;
  // var S3_REGION = 's3-eu-west-1';
</script>
<script type="text/javascript" src="https://rufuspollock.github.io/s3-bucket-listing/list.js"></script>

</body>
</html>
  1. Enable static website hosting

It works to the point of restricting folder access to zipFiles. My issue is I don't want to make this website public. I want to keep it visible to organization users only. I tried two things. First is limiting by IP addresses. The other is limiting by vpc. Both do not work because IP address does not compare with private IP but the external IP. and vpc limiting has dependencies I havent fully figured out

Is there a way to expose S3 folder contents as browsable directory structure(not xml format) without making it public? The org's IP range is peered with aws account so employees can access ec2 with private IPs. Is same thing possible for S3? Or if making it public is the only way, how can I limit the access to org users only?

John Rotenstein avatar
in flag
It's not easy. You _might_ be able to use a **VPC Endpoint for S3** to access S3 from the VPC, with a Bucket Policy that limits access to requests that are coming from the VPC. Or, add a Bucket Policy that permits access only from the Public IP address that all your corporate traffic appears to be coming from when it hits the Internet. I was thinking that an S3 Access Point might work, but I doubt it works with anonymous web browsing (since buckets need to be accessed via an ARN instead of a URL).
Magnus avatar
my flag
You might also try mounting the S3 bucket to disk using s3fs on a separate server and have e.g. apache serve the particular subfolder(s) you want to make browseable
mangohost

Post an answer

Most people don’t grasp that asking a lot of questions unlocks learning and improves interpersonal bonding. In Alison’s studies, for example, though people could accurately recall how many questions had been asked in their conversations, they didn’t intuit the link between questions and liking. Across four studies, in which participants were engaged in conversations themselves or read transcripts of others’ conversations, people tended not to realize that question asking would influence—or had influenced—the level of amity between the conversationalists.