IT 4100 : File Systems and Storage Technologies

Cloud - S3

Cloud Storage

• Why?
  • Administration Costs
  • Hardware Costs
  • Availability
  • Redundancy
  • Automation

Cloud Storage

• What is it?
  • Online space where you can store data
• Simple Storage Service (S3)
  • Object storage
• Supports various storage tiers

Object storage

• Good overview
• virtually unlimited storage
  • On aws a single object is limited to 5TB

Object vs File vs Block storage

• Object - data is designated as an object (s3)
• File - data stored as single piece of information in a folder (hierarchy) (nfs)
• Block - takes a file apart into singular blocks of data and then stores these blocks as separate pieces of data. Each piece of data has a different address, so they don’t need to be stored in a file structure. (iscsi)

Buckets

• Basic container to hold data
• Cannot be nested
• Specified with name, storage class, and geographic location.
• Have a universal namespace
  • Must be unique across all AWS
• Stored redundantly for 11 9’s of availability.

Buckets

• Can access over http or https
• Can manage permissions (data access) through various policies and control lists.
• Data is not public by default
• Data can be encrypted in-transit and at rest
• Can transition to various storage tiers

S3 Storage classes

• S3 Standard: high durability, availability, and performance (mobile, web, gaming, analytics)
• S3 Intelligent Tiering: automatically can move data to the most cost-effective tier. Can monitor your access patters and automatically move to better tier. (unpredictable data access patterns)
• S3 standard IA: Less frequently accessed data, but needs rapid access when required (backups)
• S3 Once Zone IA: Same as previous but doesn’t need to be as resilient (secondary backups, easily replicatable data)
• S3 Glacier: data archiving, data access takes minutes to hours
• S3 Glacier Deep archive: archiving, data isn’t accessed very often

Buckets

Demo time!