Moving large >1TB AMIs to S3
  • NE1Scott
    0
    Does MSP have any solution to the problem that you cannot copy AMI files to S3 larger than 1TB using create-store-image-task ?
    I have about 7 AMis now ranging from 1.1TB to 5.6 TB that I want to move to S3 glacier deep archive but I can't get them to copy using create-store-image-task due to their 1 TB limitation on the source AMI.

    The CLI command I use for any AMI under 1 TB
    -----------------------------------------------------------------
    aws ec2 create-store-image-task --image-id ami-0921bd9XXXXb86c63 --bucket "ami-backup-CompanyName/VmName"

    Does cloudberry explorer or some other product have access to AMIs to be able to move them ?
    NE1Scott Options
  • David Gugick
    118
    ↪NE1Scott
    The maximum object size for Amazon S3 is 5 terabytes, so you're not going to be able to move one of those AMIs regardless. I'm surprised to see an AMI that large. I checked the Amazon S3 documentation and didn't see any limitations listed in the API for that particular command. Having said that I don't think we're going to be able to assist given that one of those AMIs is larger than the maximum object size that Amazon supports anyway. For those that are under five terabytes just make sure you're using the current version of the API, which I think is version 2, as version 1 may have some limitations I'm not aware of.
    David Gugick Options
  • NE1Scott
    0
    I have only tried the "aws ec2 create-store-image-task tool and it has a limitation of 1TB.

    A limitation of 5TB would be a welcome relief but was hoping I would find a tool that could utilize it in moving AMIs to S3.
    NE1Scott Options
  • David Gugick
    118
    ↪NE1Scott
    If that's a limitation by Amazon, then I do not have an answer for you. Maybe you'd be better off exporting the AMI and keeping copies locally instead of in S3. I would also make sure to check your restore costs from Amazon Glacier Deep Archive to make sure you are prepared for what it may cost to restore that amount of data should you figure out how to leverage that storage class for your AMIs.
    David Gugick Options
  • NE1Scott
    0
    Company mandate requires everything to be at AWS as there are no ioffices or datacenters anymore.

    I will look into those costs though, hopefully we will never need them.
    NE1Scott Options
bold
italic
underline
strike
code
quote
ulist
image
url
mention
reveal
youtube
tweet
Add a Comment