Server Fault Stack Exchange
Server Fault Stack Exchange

Questions tagged [amazon-s3]

This is for questions on Amazon's cloud storage service.

1120 questions
387
votes
27 answers

How can I get the size of an Amazon S3 bucket?

I'd like to graph the size (in bytes, and # of items) of an Amazon S3 bucket and am looking for an efficient way to get the data. The s3cmd tools provide a way to get the total file size using s3cmd du s3://bucket_name, but I'm worried about its…
Garret Heaton
  • 4,325
  • 3
  • 18
  • 10
156
votes
8 answers

Amazon Cloudfront with S3. Access Denied

We're trying to distribute out S3 buckets via Cloudfront but for some reason the only response is an AccessDenied XML document like the following: AccessDenied Access Denied …
Jordan Adams
  • 1,661
  • 2
  • 11
  • 4
108
votes
3 answers

How long before an s3 bucket can be created with same name after deletion?

Title is quite explanatory, but I have just deleted an s3 bucket as it was in the wrong region and am wanting to recreate it in the correct region with same name as the just deleted one. Is there any documentation of this or user experience?
ljs.dev
  • 1,254
  • 2
  • 9
  • 15
78
votes
8 answers

Copy directory structure intact to AWS S3 bucket

I want to use the AWS S3 cli to copy a full directory structure to an S3 bucket. So far, everything I've tried copies the files to the bucket, but the directory structure is collapsed. (to say it another way, each file is copied into the root…
agentv
  • 970
  • 1
  • 6
  • 11
68
votes
2 answers

Why does AWS recommend against public S3 buckets?

"We highly recommend that you never grant any kind of public access to your S3 bucket." I have set a very granular public policy (s3:GetObject) for one bucket that I use to host a website. Route53 explicitly supports aliasing a bucket for this…
Andrew Johnson
  • 791
  • 1
  • 5
  • 6
62
votes
8 answers

What is causing Access Denied when using the aws cli to download from Amazon S3?

I'm really flailing around in AWS trying to figure out what I'm missing here. I'd like to make it so that an IAM user can download files from an S3 bucket - without just making the files totally public - but I'm getting access denied. If anyone can…
Josh Gagnon
  • 755
  • 1
  • 5
  • 6
53
votes
6 answers

How to move files between two S3 buckets with minimum cost?

I have millions of files in a Amazon S3 bucket and I'd like to move these files to other buckets and folders with minimum cost or no cost if possible. All buckets are in the same zone. How could I do it?
Daniel Cukier
  • 823
  • 1
  • 10
  • 18
51
votes
8 answers

Chrome S3 Cloudfront: No 'Access-Control-Allow-Origin' header on initial XHR request

I have a webpage (https://smartystreets.com/contact) that uses jQuery to load some SVG files from S3 through the CloudFront CDN. In Chrome I will open an Incognito window as well as the console. Then I will load the page. As the page loads, I will…
SunSparc
  • 998
  • 3
  • 11
  • 23
51
votes
3 answers

Rsync to AWS S3 bucket

For a server I am hosting a website on I want to backup the data and settings to an S3 bucket. I found out that you can't directly use rsync to backup to an S3 bucket. Is there another way to achieve the following rsync command to backup the data to…
Bart Bergmans
  • 633
  • 1
  • 6
  • 9
50
votes
6 answers

How do I back up an AWS S3 Bucket without versioning the source bucket

Is there any way to recover from accidental deletions of an Amazon S3 Bucket? We've got critical info in our buckets and I need to mitigate the risk of accidental or malicious deletions of the bucket itself. I know I can sync the entire bucket…
Nikhil Gupte
47
votes
7 answers

How can one efficiently use S3 to back up files incrementally?

I understand how rsync works on a high-level, but there are 2 sides. With S3 there is no daemon to speak of — well there is, but it's basically just HTTP. There look to be a few approaches. s3rsync (but this just bolts on rsync to s3). …
Jaimie Sirovich
  • 571
  • 1
  • 4
  • 5
43
votes
10 answers

Most efficient way to batch delete S3 Files

I'd like to be able to batch delete thousands or tens of thousands of files at a time on S3. Each file would be anywhere from 1MB to 50MB. Naturally, I don't want the user (or my server) to be waiting while the files are in the process of being…
SudoKill
  • 533
  • 1
  • 4
  • 5
42
votes
7 answers

Using RSYNC with Amazon S3

I am interested in using Amazon S3 to backup our ~ 100gb server images (created via Acronis backup tools) Obviously, this uploading to S3 every night would be expensive, in terms of bandwidth and cost. I'm considering using rsync with S3 and came…
alex
  • 1,720
  • 15
  • 43
  • 63
40
votes
2 answers

AWS S3 charging for 4 TB of storage when only using less than 1 GB

I'm having trouble understanding my large S3 bill, and figured I'd ask here before dropping $30 on AWS monthly support. Basically, I have an Amazon EC2 instance that makes an API to different cryptocurrency exchanges and saves the responses to the…
Garglesoap
  • 512
  • 1
  • 4
  • 8
36
votes
1 answer

using CloudFormation with an existing S3 bucket

Using CloudFormation, I want to set some of the properties in AWS::S3::Bucket on an existing bucket. In other words, I don't want to create the bucket, I just want to enforce some of the settings. Here's an example of my CloudFormation JSON: …
tedder42
  • 853
  • 1
  • 9
  • 20
1
2 3
…
74 75