AWS S3

What is Amazon S3?

Simple Storage Service is a scalable and secure cloud storage service provided by Amazon Web Services
(AWS). It allows you to store and retrieve any amount of data from anywhere on the web.

which has the capable of

scalable

/highly available >this is globally accessable. we can basically aces through hhttp protocal.

we can access it from any where in the world.

uniq name across all aws account-

/secure and cost effective./performance.

aws region> why> s3 buckets scoped in region but content we can access globally. to solve the latency
problem.

by default public access is blocked. anywhere in the world can access this bcket.

default encryption>

we can store anything like> app logs, data bases, s3 doest have any restrictions to store anything.

it solve a very common problem deal with storage. like logs, data, images, docs, anything.

s3 99.11 9s > it indicates amount of reliability of aws s3.

aws says s3 is 99.119s reliable. if u wan to understand, we k=can put any object in s3 , we can expect
99.119s error budget. 100croses and 100 years only 1 object may get delete remain all object wl be safe.

What are S3 buckets?

S3 buckets are containers for storing objects (files) in Amazon S3. Each bucket has a unique name
globally across all of AWS. You can think of an S3 bucket as a top-level folder that holds your data.
Why use S3 buckets?

S3 buckets provide a reliable and highly scalable storage solution for various use cases. They are
commonly used for backup and restore, data archiving, content storage for websites, and as a data
source for big data analytics.

Key benefits of S3 buckets

S3 buckets offer several advantages, including:

Durability and availability: S3 provides high durability and availability for your data.

1 bl of data through out 100 years only 1 object may loss. that asurance ha sgiven by aws.

Scalability: You can store and retrieve any amount of data without worrying about capacity constraints.

we can keep upload , and aws s3 will not restrict us. and only restrictionis object should not exceed 5tb.

Security: S3 offers multiple security features such as encryption, access control, and audit logging.

any sencitive info , s3 says lot of options, with encryption, and s3 has acls, bucket policies, we canr
restrict access.

unauthorized access attempts.

Performance: S3 is designed to deliver high performance for data retrieval and storage operations.
multipart uploads.

Cost-effective: S3 offers cost-effective storage options and pricing models based on your usage patterns.

Creating and Configuring S3 Buckets

Creating an S3 bucket
To create an S3 bucket, you can use the AWS Management Console, AWS CLI (Command Line Interface),
or AWS SDKs (Software Development Kits). You need to specify a globally unique bucket name and select
the region where you want to create the bucket.

Choosing a bucket name and region

The bucket name must be unique across all existing bucket names in Amazon S3. It should follow DNS
naming conventions, be 3-63 characters long, and contain only lowercase letters, numbers, periods, and
hyphens. The region selection affects data latency and compliance with specific regulations.

Bucket properties and configurations

Versioning: Versioning allows you to keep multiple versions of an object in the bucket. It helps protect
against accidental deletions or overwrites.
Bucket-level permissions and policies

Bucket-level permissions and policies define who can access and perform actions on the bucket. You can
grant permissions using IAM (Identity and Access Management) policies, which allow fine-grained
control over user access to the bucket and its objects.

Uploading and Managing Objects in S3 Buckets

Uploading objects to S3 buckets

You can upload objects to an S3 bucket using various methods, including the AWS Management Console,
AWS CLI, SDKs, and direct HTTP uploads. Each object is assigned a unique key (name) within the bucket
to retrieve it later.

Object metadata and properties

Object metadata contains additional information abouteach object in an S3 bucket. It includes attributes
like content type, cache control, encryption settings, and custom metadata. These properties help in
managing and organizing objects within the bucket.

File formats and object encryption

S3 supports various file formats, including text files, images, videos, and more. You can encrypt objects
stored in S3 using server-side encryption (SSE). SSE options include SSE-S3 (Amazon-managed keys), SSE-
KMS (AWS Key Management Service), and SSE-C (customer-provided keys).

Lifecycle management

Lifecycle management allows you to define rules for transitioning objects between different storage
classes or deleting them automatically based on predefined criteria. For example, you can move
infrequently accessed data to a lower-cost storage class after a specified time or delete objects after a
certain retention period.

Multipart uploads

Multipart uploads provide a mechanism for uploading large objects in parts, which improves
performance and resiliency. You can upload each part in parallel and then combine them to create the
complete object. Multipart uploads also enable resumable uploads in case of failures.

normally if we upload a file,

ex: 100 mb file upload to cloud storage, data local to s3 small chunks upload to s3.

100 mb file in 10 parts, after 5 parts any disruption happen, if upload failed.

5 uploaded to s3 and if again start 5 parts itwont upload first 5 parts.

we can store upto 5 tb file size, inside s3, while upload any object it cant more than 5 gb.

here multi part upload will come

spilts files into small parts.

required for files larger than 5 gb.

parts can be uploaded in parllel.

it wl improve upload efficiency.

how multiparts upload.

start upload and get upload id.

system has to upload that we hv remain files parts of this.

demo:

how to split file: select file 1. 7.zip software 2. slit the file in to 100 mb or 50 mb like that.

then start upload.

create bucket:

how to make an object oublic ?

how to make s3 bucket as a static web

dynamic website:

Managing large datasets with S3 Batch Operations

S3 Batch Operations is a feature that allows you to perform bulk operations on large numbers of objects
in an S3 bucket. It provides an efficient way to automate tasks such as copying objects, tagging, and
restoring archived data.

Advanced S3 Bucket Features

S3 Storage Classes

S3 offers multiple storage classes, each designed for different use cases and performance requirements:

how can we choose: depends on progect need.

cheapest s3 glacier,

version: Versioning in Amazon S3 is a means of keeping multiple variations of an object in the same
bucket. Versioning-enabled buckets can help us to recover objects from accidental deletion or overwrite

create bucket

uploD an object multiple times and observer the overwrite.

enable versioning and upload a file multiple files

observer how versions are areated with VID

reupload same file with same filename

observe how s3 stores all the version of deleted object

tags: identify the resources, project and which project that belongs to, like logical grouping,
maintainance and fetch the data.

encryption: S3 automatically applies server-side encryption (SSE-S3) for each new object, unless you
specify a different encryption option.
s3 bucket access loging: S3 bucket access logging captures information on all requests made to a bucket,
such as PUT, GET, and DELETE actions. Bucket access logging is a recommended security best practice
that can help teams identifying unauthorized access to your data.

event notifications:

event brodge:

object locking: i want to lock the object then , we can lock the object. probably no over ride and view, we
can go with it.

Object Lock works only on Versioned buckets. You must enable Versioning on this bucket before you can
enable Object Lock.

static web site:

object public url >click on object properties> object url> default private

it will add s3-ap-south-amazon.com base don bucket name.

check object url able to access.

we need to change it to public. goto object actions> make public>

goto permissions and change , select object make public and change it to public.
disable block p-access. save changes.

website hosting:

html5 free html templates>

download and make enable public access.

default it will showing as broken website> reason remain supported files not enabled as public

select all in public access.

---permissions:

S3 Replication

S3 replication enables automatic and asynchronous replication of objects between S3 buckets in

S3 event notifications allow you to configure actions when specific events occur in an S3 bucket. For
example, you can trigger AWS Lambda functions, send messages to Amazon Simple Queue Service (SQS),
or invoke other services using Amazon SNS when an object is created or deleted.

S3 Batch Operations

S3 Batch Operations allow you to perform large-scale batch operations on objects, such as copying,
tagging, or deleting, across multiple buckets. It simplifies managing large datasets and automates tasks
that would otherwise be time-consuming.

Security and Compliance in S3 Buckets

S3 bucket security considerations

Ensure that S3 bucket policies, access control, and encryption settings are appropriately configured.
Regularly monitor and audit access logs for unauthorized activities.

Data encryption at rest and in transit

Encrypt data at rest using server-side encryption options provided by S3. Additionally, enable encryption
in transit by using SSL/TLS for data transfers.

Access logging and monitoring

Enable access logging to capture detailed records of requests made to your S3 bucket. Monitor access
logs and configure alerts to detect any suspicious activities or unauthorized access attempts.
S3 Bucket Management and Administration

S3 bucket policies

Create and manage bucket policies to control access to your S3 buckets. Bucket policies are written in
JSON and define permissions for various actions and resources.

S3 access control and IAM roles

Use IAM roles and policies to manage access to S3 buckets. IAM roles provide temporary credentials and
fine-grained access control to AWS resources.

S3 APIs and SDKs

Interact with S3 programmatically using AWS SDKs or APIs. These provide libraries and methods for
performing various operations on S3 buckets and objects.

Monitoring and logging with CloudWatch

Utilize Amazon CloudWatch to monitor S3 metrics, set up alarms for specific events, and collect and
analyze logs for troubleshooting and performance optimization.

S3 management tools

AWS provides multiple management tools, such as the AWS Management Console, AWS CLI, and third-
party tools, to manage S3 buckets efficiently and perform operations like uploads, downloads, and
bucket configurations.
Troubleshooting and Error Handling

Common S3 error messages and their resolutions

Understand common S3 error messages like access denied, bucket not found, and exceeded bucket
quota. Troubleshoot and resolve these errors by checking permissions, bucket configurations, and
network connectivity.

Debugging S3 bucket access issues

Investigate and resolve issues related to access permissions, IAM roles, and bucket policies. Use tools like
AWS CloudTrail and S3 access logs to identify and troubleshoot access problems.

Data consistency and durability considerations

Ensure data consistency and durability by understanding S3's data replication and storage mechanisms.
Verify that data is correctly uploaded, retrieve objects using proper methods, and address any data
integrity issues.

Recovering deleted objects

If an object is accidentally deleted, you can often recover it using versioning or S3 event notifications.
Additionally, consider enabling Cross-Region Replication (CRR) for disaster recovery scenarios.