Module 3
Lecture 5
Google Cloud
Storage
Google Cloud Storage
Google Cloud Storage is a cloud-based object storage platform provided by
Google Cloud. It enables developers to store and access data in the cloud,
making it easy to build and scale applications that require data storage.
• It is a managed service for storing unstructured data.
It is scalable, durable, and secure and can be accessed from anywhere i.e. region
independent. It is more suited for storing static data like videos, photos, etc.
Google Cloud Storage
Object storage offers data encryption, data replication, and lifecycle
management services which make it more reliable.
It is easy to integrate the object storage with multiple GCP services like Google
Cloud Functions, BigQuery, and AI Platform, enabling you to build powerful
applications.
Why Object Storage?
Object storage, also known as object-based storage, is a computer data
storage architecture designed to handle large amounts of unstructured data.
Each object includes the data, metadata, and a unique identifier that applications can use
for easy access and retrieval.
It can be stored on-premises or in cloud. It is less costly to store large data volumes than
other options
Much of today’s data is unstructured: email, media and audio files, web pages, sensor
data, and other types of digital content that do not fit easily into traditional databases.
How does object storage work?
To access data, object storage systems will use the unique identifier and the
metadata to find the object you need, such as an image or audio file.
Locate and access objects using RESTful APIs, HTTP, and HTTPS to query
object metadata. Since objects are stored in a global storage pool, it’s fast and
easy to locate the exact data you need
Storage pools can be spread across multiple object storage devices and
geographical locations, allowing for unlimited scale
Google Cloud Storage
Objects are stored in containers called buckets, and each bucket is associated
with a project.
A bucket can hold multiple objects yet, a single object will belong to
only one bucket.
This model for storage is widely popular in cloud-native systems due to its
low cost combined with the serverless approach and simplicity.
The drawback of object storage is that there is no other way to access
the data besides the REST API
Special Features of Cloud Storage
Object Storage: Google Cloud Storage is an object storage platform, which
means that data is stored in containers called buckets and accessed using unique
object identifiers.
Highly Scalable: It is highly scalable and can handle large amounts of data.
Durability and Availability: Data is replicated across multiple regions,
providing redundancy and ensuring that data is always available.
Flexible Access Control: It allow developers to set permissions for buckets and
objects, and control who can access the data.
Special Features
Object Lifecycle Management: It allows users to define and assign conditions
to a bucket that could trigger a data deletion or move to a less costly storage
class.
Object Versioning: It allows users to keep track of the objects they have stored
and have access to them if they are deleted or overwritten. It does increase
storage costs, but Object Lifecycle Management can be used to remove older
versions of the objects.
Featur
es
Multiple redundancy options: Cloud Storage has an ever-growing list of
storage bucket locations where users can store their data with multiple automatic
redundancy options.
Storage Classes
In Google Cloud Storage, you are required to select one storage class for your
buckets.
A storage class is a piece of metadata that is used by every object. The storage
class set for an object affects the object's availability and pricing model.
The bucket's default storage class is Standard storage.
Storage Classes - types
Storage Classes - types
Standard Storage: It is appropriate for “hot” data that is accessed frequently,
including websites, streaming videos, and mobile apps.
Nearline Storage: It is a low-cost option suited for data that can be stored for at
least 30 days, including data backup and long-tail multimedia content.
Coldline Storage: Very low cost suited for data that can be stored for at least 90
days, including disaster recovery.
Archive Storage: It offers the lowest cost and suited for data that can be stored
for at least 365 days, including regulatory archives.
Steps to create object and upload in
bucket
Step 1: Navigate to Google Cloud Console and log in with your
registered Email ID.
Step 2: In the console, click the Navigation menu and click on
Storage in the Storage Service Section.
Step 3: Click on Create Bucket option.
Step 4: Enter your bucket information and click Continue to complete
each step.
Step 5: Your bucket will be successfully created.
Step 7: Select the file from your system and click on Upload. You can see
that the file will be uploaded to the bucket.
Features of Storage Classes
The following features apply to all storage classes:
• Unlimited storage with unlimited access.
• No minimum object size.
• Worldwide accessibility and worldwide storage locations.
• Low latency with no offline data retrieval.
• High durability (99.999999999% annual durability).
• Redundancy across regions when data is stored in a multi-region or dual-region.
• A uniform experience with Cloud Storage features, security, tools, and APIs.
Google Persistent Disk
Reliable, high-performance block storage for virtual
machine instances.
Persistent Disk is a durable and high performance block storage service for
Google Cloud Platform.
Persistent Disk provides SSD and HDD storage that can be attached to instances
running in either Compute Engine or Google Kubernetes Engine.
Google Persistent Disk
By default, each Compute Engine virtual machine (VM) instance has a single boot
Persistent Disk volume that contains the operating system.
When your apps require additional storage space, one possible solution is to attach
additional Persistent Disk volumes to your VM.
Persistent Disk volumes are durable network storage devices that your VMs can access
like physical disks in a desktop or a server.
The data on each persistent disk is distributed across several physical disks. Compute
Engine manages the physical disks and the data distribution for you to ensure redundancy
and optimal performance.
Google Persistent Disk
• Google Cloud Persistent Disks provide virtual instances with the storage capabilities those
instances need to operate.
• The level of flexibility and configuration options available make Persistent Disk a very
accessible cloud storage service for customers new to cloud computing.
Google Persistent Disk types
The user can simply start by choosing a geographical region and one of three types of
persistent disks that better suits the workload’s requirements:
1. Standard disks for workloads that primarily use sequential I/Os.
2. SSD for enhanced performance and single-digit millisecond latencies.
3. Local SSDs for even lower latencies but without redundancy capabilities and availability
bound to the instance lifetime.
Steps to create a Persistent Disk
• Go to the VM instances page.
• Click the name of the VM where you want to add a disk.
• Edit -> Additional disks, click Add new disk.
• Specify a namefor the disk, configure the disk's properties,
and select Blank as the Source type.
• Under the heading Device name, select the option Use a custom
device name. The
name you enter is used to generate a symlink which makes
disk
for the disk, identification easier.
• Click Done to complete the disk's configuration.
• Click Save to apply your changes to the VM and add
the new disk.
Google Persistent Disk - Features
High-performance block storage for any workload
Its performance scales with the size of the disk and with the number of
vCPUs on your VM instance.
Durability and availability that keep your business running
Automatically store your data redundantly to ensure the highest level of
data integrity
Automatic security and encryption
Each Persistent Disk remains encrypted with system-defined keys
Data protection for business continuity
Protectyour data with cross-zone synchronous replication,
Google Persistent Disk - Benefits
• Easy to deploy and scale
• Optimal price and performance
• Flexibility that comes with no downtime
Google Cloud
Filestore
Google Cloud Filestore
High-performance, fully managed file storage.
Cloud Filestore is a scalable and highly available shared file service fully-
managed by Google.
Cloud Filestore provides persistent storage ideal for shared workloads.
It is best suited for enterprise applications requiring persistent, durable, shared
storage which is accessed by NFS or requires a POSIX compliant file system.
Google Cloud Filestore
Filestore instances are fully managed NFS file servers on Google Cloud for use
with applications running on Compute Engine virtual machine (VM) instances,
Google Kubernetes Engine clusters, external datastores such as Google Cloud
VMware Engine, or your on-premises machines.
Google Cloud Filestore - Features
It offers low latency for file operations.
It is a fully managed, NoOps service that is integrated with the rest of
the Google Cloud portfolio.
Users can scale file storage elastically to suit the evolving needs of
their business.
Supports storage of unstructured objects, transactions, and SQL-like queries.
Steps to create a Google Cloud
Filestore instance
1. create a Google Cloud account
2. select or create a Google Cloud project.
3. Create a Compute Engine VM instance to
be the client
4. Create a Filestore instance
5. Mount the Filestore file share on the client
6. Create a file on the file share
7. Clean up
Cloud Storage for Firebase
Cloud Storage for Firebase is built on fast and secure Google Cloud
infrastructure for app developers who need to store and serve user-
generated content, such as photos or videos.
Cloud Storage for Firebase is a powerful, simple, and cost-effective object storage service
built for Google scale.
The Firebase SDKs for Cloud Storage add Google security to file uploads and downloads for
your Firebase apps, regardless of network quality.
Cloud Storage for Firebase
Client SDKs are used to store images, audio, video, or other user-
generated content.
On the server, Firebase Admin SDK are used to manage buckets and
create download URLs, and use Google Cloud Storage APIs to access your
files.
Cloud Storage for Firebase
Cloud Firestore is a flexible, scalable database for mobile, web, and
server development from Firebase and Google Cloud.
The Firebase Realtime Database stores JSON application data, like game state or chat
messages, and synchronizes changes instantly across all connected devices.
Firebase Remote Config stores developer-specified key-value pairs to change
the behavior and appearance of your app without requiring users to download an update.
Firebase Hosting hosts the HTML, CSS, and JavaScript for your website as well as other
developer-provided assets like graphics, fonts, and icons.
How does it work?
Developers use the Firebase SDKs for Cloud Storage
to upload anddownload files directly from clients.
If the network connection is poor, the client is able to retry the
operation right where it left off, saving your users time and bandwidth.
Cloud Storage for Firebase stores your files in a Google Cloud Storage
bucket, making them accessible through both Firebase and Google
Cloud.
This allows you the flexibility to upload and download files from
mobile clients via the Firebase SDKs for Cloud Storage.
How does it work?
In addition, you can do server-side processing such as image
filtering or video transcoding using the Google Cloud Storage APIs.
Cloud Storage scales automatically, meaning that there's no need to
migrate to any other provider.
The Firebase SDKs for Cloud Storage integrate seamlessly with Firebase
Authentication to identify users, and provides a declarative security
language that lets you set access controls on individual files or groups
of files, so you can make files as public or private as you want.
Key Capabilities
Robust operations: Firebase SDKs for Cloud Storage perform
uploads and downloads regardless of network quality. Uploads and
downloads are robust, meaning they restart where they stopped, saving
your users time and bandwidth.
Strong security: Firebase SDKs for Cloud Storage integrate
with Firebase
Authentication to provide simple and intuitive authentication for
developers. You can use our declarative security model to allow access
based on filename, size, content type, and other metadata.
High scalability: Cloud Storage is built for exabyte scale when
Referen
ces
• https://cloud.google.com/appengine/docs/an-overview-of-app-engine
• https://cloud.google.com/filestore/docs/create-instance-console
• https://firebase.google.com/docs/storage
