Artificial intelligence Practical Manual.
1)Describe Artificial Intelligence workloads and
considerations.
What is AI?
Simply put, AI is the creation of software that imitates human behaviors and
capabilities. Key workloads include:
• Machine learning - This is often the foundation for an AI system, and is the
way we "teach" a computer model to make prediction and draw conclusions
from data.
• Anomaly detection - The capability to automatically detect errors or unusual
activity in a system.
• Computer vision - The capability of software to interpret the world visually
through cameras, video, and images.
• Natural language processing - The capability for a computer to interpret
written or spoken language, and respond in kind.
• Knowledge mining - The capability to extract information from large volumes
of often unstructured data to create a searchable knowledge store.
Machine learning in Microsoft Azure
Microsoft Azure provides the Azure Machine Learning service - a cloud-based
platform for creating, managing, and publishing machine learning models. Azure
Machine Learning provides the following features and capabilities:
Feature Capability
Automated machine This feature enables non-experts to quickly create an effective machine learning
learning
model from data.
Azure Machine Learning A graphical interface enabling no-code development of machine learning solutions.
designer
Data and compute Cloud-based data storage and compute resources that professional data scientists
management
can use to run data experiment code at scale.
Iranagouda.Patil
Artificial intelligence Practical Manual.
Feature Capability
Pipelines Data scientists, software engineers, and IT operations professionals can define pipelines t
training, deployment, and management tasks.
Understand anomaly detection
Imagine you're creating a software system to monitor credit card transactions and
detect unusual usage patterns that might indicate fraud. Or an application that tracks
activity in an automated production line and identifies failures. Or a racing car
telemetry system that uses sensors to proactively warn engineers about potential
mechanical failures before they happen.
These kinds of scenario can be addressed by using anomaly detection - a machine
learning based technique that analyzes data over time and identifies unusual
changes.
Computer Vision models and capabilities
Most computer vision solutions are based on machine learning models that can be
applied to visual input from cameras, videos, or images. The following table describes
common computer vision tasks.
Task Description
Image classification Image classification involves training a machine learning model to classify images based on
their contents. For example, in a traffic monitoring solution you might use an image
classification model to classify
images based on the type of vehicle they contain, such as taxis, buses, cyclists, and so on.
Object detection Object detection machine learning models are trained to classify individual objects
within an image, and identify their location with a bounding box. For example,
a traffic monitoring solution might use object detection to identify the location
of different classes of vehicle.
Semantic segmentation Semantic segmentation is an advanced machine learning technique in which individual
pixels in the image are classified according to the object to which they belong.
For example, a traffic monitoring solution might overlay traffic images with "mask"
layers to highlight different vehicles using specific colors.
Iranagouda.Patil
Artificial intelligence Practical Manual.
Task Description
Image analysis You can create solutions that combine machine learning models with advanced
image analysis techniques to extract information from images, including "tags" that
could help catalog the image or even descriptive captions that summarize the scene
shown in the image.
Face detection, analysis,
and recognition Face detection is a specialized form of object detection that locates human faces in an imag
with classification and facial geometry analysis techniques to recognize individuals based on
Optical character Optical character recognition is a technique used to detect and read text in images. You can
recognition (OCR) photographs (for example, road signs or store fronts) or to extract information from scanne
letters, invoices, or forms.
Computer vision services in Microsoft Azure
Microsoft Azure provides the following cognitive services to help you create
computer vision solutions:
Service Capabilities
Computer Vision You can use this service to analyze images and video, and extract descriptions, tags,
objects, and text.
Custom Vision Use this service to train custom image classification and object detection models using
your own images.
Face The Face service enables you to build face detection and facial recognition solutions.
Form Recognizer Use this service to extract information from scanned forms and invoices.
Understand natural language processing
Natural language processing (NLP) is the area of AI that deals with creating software
that understands written and spoken language.
NLP enables you to create software that can:
• Analyze and interpret text in documents, email messages, and other sources.
• Interpret spoken language, and synthesize speech responses.
• Automatically translate spoken or written phrases between languages.
Iranagouda.Patil
Artificial intelligence Practical Manual.
• Interpret commands and determine appropriate actions.
For example, Starship Commander, is a virtual reality (VR) game from Human Interact,
that takes place in a science fiction world. The game uses natural language
processing to enable players to control the narrative and interact with in-game
characters and starship systems.
Natural language processing in Microsoft Azure
In Microsoft Azure, you can use the following cognitive services to build natural
language processing solutions:
Service Capabilities
Language Use this service to access features for understanding and analyzing text,
training language models that can understand spoken or text-based commands, and building
intelligent applications.
Translator Use this service to translate text between more than 60 languages.
Speech Use this service to recognize and synthesize speech, and to translate spoken languages.
Azure Bot This service provides a platform for conversational AI, the capability of a software "agent" to
participate in a conversation. Developers can use the Bot Framework to create a bot and manage it
with Azure Bot Service - integrating back-end services like Language, and connecting to c
hannels for web chat, email, Microsoft Teams, and others.
2) Describe fundamental principles of machine
learning on Azure.
Machine Learning is the foundation for most artificial intelligence solutions. Creating
an intelligent solution often begins with the use of machine learning to train
predictive models using historic data that you have collected.
Azure Machine Learning is a cloud service that you can use to train and manage
machine learning models.
Iranagouda.Patil
Artificial intelligence Practical Manual.
Types of machine learning
There are two general approaches to machine learning, supervised and unsupervised
machine learning. In both approaches, you train a model to make predictions.
The supervised machine learning approach requires you to start with a
dataset with known label values. Two types of supervised machine learning tasks
include regression and classification.
• Regression: used to predict a continuous value; like a price, a sales total, or
some other measure.
• Classification: used to determine a binary class label; like whether a patient has
diabetes or not.
The unsupervised machine learning approach starts with a dataset without known
label values. One type of unsupervised machine learning task is clustering.
• Clustering: used to determine labels by grouping similar information into label
groups; like grouping measurements from birds into species.
What is Azure Machine Learning studio?
Completed100 XP
• 5 minutes
Training and deploying an effective machine learning model involves a lot of work,
much of it time-consuming and resource-intensive. Azure Machine Learning is a
cloud-based service that helps simplify some of the tasks it takes to prepare data,
train a model, and deploy a predictive service.
Most importantly, Azure Machine Learning helps data scientists increase their
efficiency by automating many of the time-consuming tasks associated with training
models; and it enables them to use cloud-based compute resources that scale
effectively to handle large volumes of data while incurring costs only when actually
used.
Azure Machine Learning workspace
To use Azure Machine Learning, you first create a workspace resource in your Azure
subscription. You can then use this workspace to manage data, compute resources,
code, models, and other artifacts related to your machine learning workloads.
Iranagouda.Patil
Artificial intelligence Practical Manual.
After you have created an Azure Machine Learning workspace, you can develop
solutions with the Azure machine learning service either with developer tools or the
Azure Machine Learning studio web portal.
Azure Machine Learning studio
Azure Machine Learning studio is a web portal for machine learning solutions in
Azure. It includes a wide range of features and capabilities that help data scientists
prepare data, train models, publish predictive services, and monitor their usage. To
begin using the web portal, you need to assign the workspace you created in the
Azure portal to Azure Machine Learning studio.
3) Describe features of computer vision workloads
on Azure.
Azure resources for Computer Vision
To use the Computer Vision service, you need to create a resource for it in your
Azure subscription. You can use either of the following resource types:
• Computer Vision: A specific resource for the Computer Vision service. Use this
resource type if you don't intend to use any other cognitive services, or if you
want to track utilization and costs for your Computer Vision resource
separately.
• Cognitive Services: A general cognitive services resource that includes
Computer Vision along with many other cognitive services; such as Text
Analytics, Translator Text, and others. Use this resource type if you plan to use
multiple cognitive services and want to simplify administration and
development.
Whichever type of resource you choose to create, it will provide two pieces of
information that you will need to use it:
• A key that is used to authenticate client applications.
• An endpoint that provides the HTTP address at which your resource can be
accessed.
Iranagouda.Patil
Artificial intelligence Practical Manual.
Analyzing images with the Computer Vision service
After you've created a suitable resource in your subscription, you can submit images
to the Computer Vision service to perform a wide range of analytical tasks.
Describing an image
Computer Vision has the ability to analyze an image, evaluate the objects that are
detected, and generate a human-readable phrase or sentence that can describe what
was detected in the image. Depending on the image contents, the service may return
multiple results, or phrases. Each returned phrase will have an associated confidence
score, indicating how confident the algorithm is in the supplied description. The
highest confidence phrases will be listed first.
To help you understand this concept, consider the following image of the Empire
State building in New York. The returned phrases are listed below the image in the
order of confidence.
Tagging visual features
The image descriptions generated by Computer Vision are based on a set of
thousands of recognizable objects, which can be used to suggest tags for the image.
These tags can be associated with the image as metadata that summarizes attributes
of the image; and can be particularly useful if you want to index an image along with
a set of key terms that might be used to search for images with specific attributes or
contents.
For example, the tags returned for the Empire State building image include:
• skyscraper
• tower
• building
Detecting objects
The object detection capability is similar to tagging, in that the service can identify
common objects; but rather than tagging, or providing tags for the recognized
objects only, this service can also return what is known as bounding box coordinates.
Not only will you get the type of object, but you will also receive a set of coordinates
that indicate the top, left, width, and height of the object detected, which you can
use to identify the location of the object in the image, like this:
Iranagouda.Patil
Artificial intelligence Practical Manual.
Detecting brands
This feature provides the ability to identify commercial brands. The service has an
existing database of thousands of globally recognized logos from commercial brands
of products.
When you call the service and pass it an image, it performs a detection task and
determine if any of the identified objects in the image are recognized brands. The
service compares the brands against its database of popular brands spanning
clothing, consumer electronics, and many more categories. If a known brand is
detected, the service returns a response that contains the brand name, a confidence
score (from 0 to 1 indicating how positive the identification is), and a bounding box
(coordinates) for where in the image the detected brand was found.
For example, in the following image, a laptop has a Microsoft logo on its lid, which is
identified and located by the Computer Vision service.
Detecting faces
The Computer Vision service can detect and analyze human faces in an image,
including the ability to determine age and a bounding box rectangle for the location
of the face(s). The facial analysis capabilities of the Computer Vision service are a
subset of those provided by the dedicated Face Service. If you need basic face
detection and analysis, combined with general image analysis capabilities, you can
use the Computer Vision service; but for more comprehensive facial analysis and
facial recognition functionality, use the Face service.
The following example shows an image of a person with their face detected and
approximate age estimated.
Iranagouda.Patil
Artificial intelligence Practical Manual.
Categorizing an image
Computer Vision can categorize images based on their contents. The service uses a
parent/child hierarchy with a "current" limited set of categories. When analyzing an
image, detected objects are compared to the existing categories to determine the
best way to provide the categorization. As an example, one of the parent categories
is people_. This image of a person on a roof is assigned a category of people_.
A slightly different categorization is returned for the following image, which is
assigned to the category people_group because there are multiple people in the
image:
Detecting domain-specific content
When categorizing an image, the Computer Vision service supports two specialized
domain models:
• Celebrities - The service includes a model that has been trained to identify thousands
of well-known celebrities from the worlds of sports, entertainment, and business.
• Landmarks - The service can identify famous landmarks, such as the Taj Mahal and the
Statue of Liberty.
For example, when analyzing the following image for landmarks, the Computer
Vision service identifies the Eiffel Tower, with a confidence of 99.41%.
Iranagouda.Patil
Artificial intelligence Practical Manual.
Optical character recognition
The Computer Vision service can use optical character recognition (OCR) capabilities
to detect printed and handwritten text in images. This capability is explored in
the Read text with the Computer Vision service module on Microsoft Learn.
Additional capabilities
In addition to these capabilities, the Computer Vision service can:
• Detect image types - for example, identifying clip art images or line drawings.
• Detect image color schemes - specifically, identifying the dominant foreground,
background, and overall colors in an image.
• Generate thumbnails - creating small versions of images.
• Moderate content - detecting images that contain adult content or depict violent, gory
scenes.
4) Describe features of Natural Language Processing
(NLP) workloads on Azure.
Text Analytics Techniques
Text analytics is a process where an artificial intelligence (AI) algorithm, running on a
computer, evaluates these same attributes in text, to determine specific insights. A
person will typically rely on their own experiences and knowledge to achieve the
insights. A computer must be provided with similar knowledge to be able to perform
the task. There are some commonly used techniques that can be used to build
software to analyze text, including:
• Statistical analysis of terms used in the text. For example, removing common
"stop words" (words like "the" or "a", which reveal little semantic information
about the text), and performing frequency analysis of the remaining words
(counting how often each word appears) can provide clues about the main
subject of the text.
• Extending frequency analysis to multi-term phrases, commonly known as N-
grams (a two-word phrase is a bi-gram, a three-word phrase is a tri-gram, and
so on).
Iranagouda.Patil
Artificial intelligence Practical Manual.
• Applying stemming or lemmatization algorithms to normalize words before
counting them - for example, so that words like "power", "powered", and
"powerful" are interpreted as being the same word.
• Applying linguistic structure rules to analyze sentences - for example, breaking
down sentences into tree-like structures such as a noun phrase, which itself
contains nouns, verbs, adjectives, and so on.
• Encoding words or terms as numeric features that can be used to train a
machine learning model. For example, to classify a text document based on the
terms it contains. This technique is often used to perform sentiment analysis, in
which a document is classified as positive or negative.
• Creating vectorized models that capture semantic relationships between words
by assigning them to locations in n-dimensional space. This modeling
technique might, for example, assign values to the words "flower" and "plant"
that locate them close to one another, while "skateboard" might be given a
value that positions it much further away.
While these techniques can be used to great effect, programming them can be
complex. In Microsoft Azure, the Language cognitive service can help simplify
application development by using pre-trained models that can:
• Determine the language of a document or text (for example, French or English).
• Perform sentiment analysis on text to determine a positive or negative
sentiment.
• Extract key phrases from text that might indicate its main talking points.
• Identify and categorize entities in the text. Entities can be people, places,
organizations, or even everyday items such as dates, times, quantities, and so
on.
In this module, you'll explore some of these capabilities and gain an understanding
of how you might apply them to applications such as:
• A social media feed analyzer to detect sentiment around a political campaign or
a product in market.
• A document search application that extracts key phrases to help summarize the
main subject matter of documents in a catalog.
• A tool to extract brand information or company names from documents or
other text for identification purposes.
These examples are just a small sample of the many areas that the Language service
can help with text analytics.
Azure resources for the Language service
Iranagouda.Patil
Artificial intelligence Practical Manual.
To use the Language service in an application, you must provision an appropriate
resource in your Azure subscription. You can choose to provision either of the
following types of resource:
• A Language resource - choose this resource type if you only plan to use natural
language processing services, or if you want to manage access and billing for the
resource separately from other services.
• A Cognitive Services resource - choose this resource type if you plan to use the
Language service in combination with other cognitive services, and you want to
manage access and billing for these services together.
Language detection
Use the language detection capability of the Language service to identify the
language in which text is written. You can submit multiple documents at a time for
analysis. For each document submitted to it, the service will detect:
• The language name (for example "English").
• The ISO 6391 language code (for example, "en").
• A score indicating a level of confidence in the language detection.
For example, consider a scenario where you own and operate a restaurant where
customers can complete surveys and provide feedback on the food, the service, staff,
and so on. Suppose you have received the following reviews from customers:
Review 1: "A fantastic place for lunch. The soup was delicious."
Review 2: "Comida maravillosa y gran servicio."
Review 3: "The croque monsieur avec frites was terrific. Bon appetit!"
You can use the text analytics capabilities in the Language service to detect the
language for each of these reviews; and it might respond with the following results:
Document Language Name ISO 6391 Code
Review 1 English en
Review 2 Spanish es
Review 3 English en
Notice that the language detected for review 3 is English, despite the text containing
a mix of English and French. The language detection service will focus on
the predominant language in the text. The service uses an algorithm to determine
Iranagouda.Patil
Artificial intelligence Practical Manual.
the predominant language, such as length of phrases or total amount of text for the
language compared to other languages in the text. The predominant language will
be the value returned, along with the language code. The confidence score may be
less than 1 as a result of the mixed language text.
Ambiguous or mixed language content
There may be text that is ambiguous in nature, or that has mixed language content.
These situations can present a challenge to the service. An ambiguous content
example would be a case where the document contains limited text, or only
punctuation. For example, using the service to analyze the text ":-)", results in a value
of unknown for the language name and the language identifier, and a score
of NaN (which is used to indicate not a number).
Sentiment analysis
The text analytics capabilities in the Language service can evaluate text and return
sentiment scores and labels for each sentence. This capability is useful for detecting
positive and negative sentiment in social media, customer reviews, discussion forums
and more.
Using the pre-built machine learning classification model, the service evaluates the
text and returns a sentiment score in the range of 0 to 1, with values closer to 1
being a positive sentiment. Scores that are close to the middle of the range (0.5) are
considered neutral or indeterminate.
For example, the following two restaurant reviews could be analyzed for sentiment:
"We had dinner at this restaurant last night and the first thing I noticed was how
courteous the staff was. We were greeted in a friendly manner and taken to our table
right away. The table was clean, the chairs were comfortable, and the food was
amazing."
Indeterminate sentiment
A score of 0.5 might indicate that the sentiment of the text is indeterminate, and
could result from text that does not have sufficient context to discern a sentiment or
insufficient phrasing. For example, a list of words in a sentence that has no structure,
could result in an indeterminate score. Another example where a score may be 0.5 is
in the case where the wrong language code was used. A language code (such as "en"
for English, or "fr" for French) is used to inform the service which language the text is
Iranagouda.Patil
Artificial intelligence Practical Manual.
in. If you pass text in French but tell the service the language code is en for English,
the service will return a score of precisely 0.5.
Key phrase extraction
Key phrase extraction is the concept of evaluating the text of a document, or
documents, and then identifying the main talking points of the document(s).
Consider the restaurant scenario discussed previously. Depending on the volume of
surveys that you have collected, it can take a long time to read through the reviews.
Instead, you can use the key phrase extraction capabilities of the Language service to
summarize the main points.
You might receive a review such as:
"We had dinner here for a birthday celebration and had a fantastic experience. We
were greeted by a friendly hostess and taken to our table right away. The ambiance
was relaxed, the food was amazing, and service was terrific. If you like great food and
attentive service, you should try this place."
Key phrase extraction can provide some context to this review by extracting the
following phrases:
• attentive service
• great food
• birthday celebration
• fantastic experience
• table
• friendly hostess
• dinner
• ambiance
• place
Not only can you use sentiment analysis to determine that this review is positive, you
can use the key phrases to identify important elements of the review.
Entity recognition
You can provide the Language service with unstructured text and it will return a list
of entities in the text that it recognizes. The service can also provide links to more
information about that entity on the web. An entity is essentially an item of a
particular type or a category; and in some cases, subtype, such as those as shown in
the following table.
Iranagouda.Patil
Artificial intelligence Practical Manual.
Type SubType Example
Person "Bill Gates", "John"
Location "Paris", "New York"
Organization "Microsoft"
Quantity Number "6" or "six"
Quantity Percentage "25%" or "fifty percent"
Quantity Ordinal "1st" or "first"
Quantity Age "90 day old" or "30 years old"
Quantity Currency "10.99"
Quantity Dimension "10 miles", "40 cm"
Quantity Temperature "45 degrees"
DateTime "6:30PM February 4, 2012"
DateTime Date "May 2nd, 2017" or "05/02/2017"
DateTime Time "8am" or "8:00"
DateTime DateRange "May 2nd to May 5th"
DateTime TimeRange "6pm to 7pm"
DateTime Duration "1 minute and 45 seconds"
DateTime Set "every Tuesday"
URL "https://www.bing.com"
Email "[email protected]"
US-based Phone Number "(312) 555-0176"
IP Address "10.0.1.125"
The service also supports entity linking to help disambiguate entities by linking to a
specific reference. For recognized entities, the service returns a URL for a
relevant Wikipedia article.
For example, suppose you use the Language service to detect entities in the
following restaurant review extract:
"I ate at the restaurant in Seattle last week."
Iranagouda.Patil
Artificial intelligence Practical Manual.
Entity Type SubType Wikipedia URL
Seattle Location https://en.wikipedia.org/wiki/Seattle
last week DateTime DateRange
Iranagouda.Patil
