Understand computer vision

https://learn.microsoft.com/en-us/training/modules/get-started-ai-fundamentals/3-understand-
computer-vision

Completed100 XP

 5 minutes

Computer Vision is an area of AI that deals with visual processing. Let's explore some of the
possibilities that computer vision brings.

The Seeing AI app is a great example of the power of computer vision. Designed for the blind and
low vision community, the Seeing AI app harnesses the power of AI to open up the visual world and
describe nearby people, text and objects.

View the following video to learn more about Seeing AI.

To find out more, check out the Seeing AI web page.

Computer Vision models and capabilities

Most computer vision solutions are based on machine learning models that can be applied to visual
input from cameras, videos, or images. The following table describes common computer vision tasks.

Expand table

Task Description

Image classification

Image classification involves training a machine learning model to classify images based on their c
traffic monitoring solution you might use an image classification model to classify images based o
contain, such as taxis, buses, cyclists, and so on.
Task Description

Object detection

Object detection machine learning models are trained to classify individual objects within an imag
location with a bounding box. For example, a traffic monitoring solution might use object detectio
of different classes of vehicle.

Semantic
segmentation

Semantic segmentation is an advanced machine learning technique in which individual pixels in th

Image analysis

You can create solutions that combine machine learning models with advanced image analysis tec
information from images, including "tags" that could help catalog the image or even descriptive c
the scene shown in the image.

Face detection,
analysis, and
recognition

Face detection is a specialized form of object detection that locates human faces in an image. This
classification and facial geometry analysis techniques to recognize individuals based on their facia
Task Description

Optical character
recognition (OCR)

Optical character recognition is a technique used to detect and read text in images. You can use O
photographs (for example, road signs or store fronts) or to extract information from scanned docu
invoices, or forms.

Computer vision services in Microsoft Azure

You can use Microsoft's Azure AI Vision to develop computer vision solutions. The service features
are available for use and testing in the Azure Vision Studio and other programming languages. Some
features of Azure AI Vision include:

 Image Analysis: capabilities for analyzing images and video, and extracting descriptions, tags,
objects, and text.

 Face: capabilities that enable you to build face detection and facial recognition solutions.

 Optical Character Recognition (OCR): capabilities for extracting printed or handwritten text
from images, enabling access to a digital version of the scanned text.

Next unit: Understand natural language processing

Previous

Understand natural language processing

Completed100 XP

 4 minutes

Natural language processing (NLP) is the area of AI that deals with creating software that
understands written and spoken language.

NLP enables you to create software that can:

 Analyze and interpret text in documents, email messages, and other sources.

 Interpret spoken language, and synthesize speech responses.

 Interpret commands and determine appropriate actions.

For example, Starship Commander is a virtual reality (VR) game from Human Interact that takes place
in a science fiction world. The game uses natural language processing to enable players to control the
narrative and interact with in-game characters and starship systems.

Natural language processing in Microsoft Azure

https://learn.microsoft.com/en-us/training/modules/get-started-ai-fundamentals/4-understand-
natural-language-process

You can use Microsoft's Azure AI Language to build natural language processing solutions. Some
features of Azure AI Language include understanding and analyzing text, training conversational
language models that can understand spoken or text-based commands, and building intelligent
applications.

Microsoft's Azure AI Speech is another service that can be used to build natural language processing
solutions. Azure AI Speech features include speech recognition and synthesis, real-time translations,
conversation transcriptions, and more.

Microsoft's Azure AI Translator uses a Neural Machine Translation (NMT) model for translation,
which analyzes the semantic context of the text and renders a more accurate and complete
translation as a result.

You can explore Azure AI Language features in the Azure Language Studio and Azure AI Speech
features in the Azure Speech Studio. The service features are available for use and testing in the
studios and other programming languages.

Understand document intelligence and knowledge mining

https://learn.microsoft.com/en-us/training/modules/get-started-ai-fundamentals/5-document-
intelligence-knowledge-mining

Completed100 XP

 3 minutes

Document Intelligence is the area of AI that deals with managing, processing, and using high
volumes of a variety of data found in forms and documents. Document intelligence enables you to
create software that can automate processing for contracts, health documents, financial forms and
more

Document intelligence in Microsoft Azure

You can use Microsoft's Azure AI Document Intelligence to build solutions that manage and
accelerate data collection from scanned documents. Features of Azure AI Document Intelligence help
automate document processing in applications and workflows, enhance data-driven strategies, and
enrich document search capabilities. You can use prebuilt models to add intelligent document
processing for invoices, receipts, health insurance cards, tax forms, and more. You can also use Azure
AI Document Intelligence to create custom models with your own labeled datasets. The service
features are available for use and testing in the Document Intelligence Studio and other
programming languages.
Knowledge Mining

Knowledge mining is the term used to describe solutions that involve extracting information from
large volumes of often unstructured data to create a searchable knowledge store.

Knowledge mining in Microsoft Azure

One Microsoft knowledge mining solution is Azure AI Search, a private, enterprise, search solution
that has tools for building indexes. The indexes can then be used for internal only use, or to enable
searchable content on public facing internet assets.

Azure AI Search can utilize the built-in AI capabilities of Azure AI services such as image processing,
document intelligence, and natural language processing to extract data. The product's AI capabilities
makes it possible to index previously unsearchable documents and to extract and surface insights
from large amounts of data quickly.

Understand generative AI

Completed100 XP

 2 minutes

Generative AI describes a category of capabilities within AI that create original content. People
typically interact with generative AI that has been built into chat applications. Generative AI
applications take in natural language input, and return appropriate responses in a variety of formats
including natural language, image, code, and audio.
Generative AI in Microsoft Azure

https://learn.microsoft.com/en-us/training/modules/get-started-ai-fundamentals/6-understand-
generative-ai

In Microsoft Azure, you can use the Azure OpenAI service to build generative AI solutions. Azure
OpenAI Service is Microsoft's cloud solution for deploying, customizing, and hosting generative AI
models. It brings together the best of OpenAI's cutting edge models and APIs with the security and
scalability of the Azure cloud platform.

Azure OpenAI supports many foundation model choices that can serve different needs. The service
features are available for use and testing with Azure AI Foundry, Microsoft's platform for designing
enterprise-grade AI solutions. You can use the Azure AI Foundry portal to manage, develop, and
customize generative AI models.
Challenges and risks with AI

https://learn.microsoft.com/en-us/training/modules/get-started-ai-fundamentals/7-challenges-with-
ai

Completed

100 XP

3 minutes

Artificial Intelligence is a powerful tool that can be used to greatly benefit the world. However, like
any tool, it must be used responsibly.

The following table shows some of the potential challenges and risks facing an AI application
developer.

Challenge or Risk Example

Bias can affect results A loan-approval model discriminates by gender due to bias in the data with
which it was trained

Errors may cause harm An autonomous vehicle experiences a system failure and causes a collision
Data could be exposed A medical diagnostic bot is trained using sensitive patient data, which is
stored insecurely

Solutions may not work for everyone A home automation assistant provides no audio output for
visually impaired users

Users must trust a complex system An AI-based financial tool makes investment
recommendations - what are they based on?

Who's liable for AI-driven decisions? An innocent person is convicted of a crime based on
evidence from facial recognition – who's responsible?

Understand Responsible AI

Completed100 XP

 10 minutes

At Microsoft, AI software development is guided by a set of six principles, designed to ensure that AI
applications provide amazing solutions to difficult problems without any unintended negative
consequences.

Fairness

https://learn.microsoft.com/en-us/training/modules/get-started-ai-fundamentals/8a-understand-
responsible-ai

AI systems should treat all people fairly. For example, suppose you create a machine learning model
to support a loan approval application for a bank. The model should predict whether the loan should
be approved or denied without bias. This bias could be based on gender, ethnicity, or other factors
that result in an unfair advantage or disadvantage to specific groups of applicants.

Azure Machine Learning includes the capability to interpret models and quantify the extent to which
each feature of the data influences the model's prediction. This capability helps data scientists and
developers identify and mitigate bias in the model.

Another example is Microsoft's implementation of Responsible AI with the Face service, which retires
facial recognition capabilities that can be used to try to infer emotional states and identity attributes.
These capabilities, if misused, can subject people to stereotyping, discrimination or unfair denial of
services.

For more details about considerations for fairness, watch the following video.

Reliability and safety

AI systems should perform reliably and safely. For example, consider an AI-based software system for
an autonomous vehicle; or a machine learning model that diagnoses patient symptoms and
recommends prescriptions. Unreliability in these kinds of systems can result in substantial risk to
human life.

AI-based software application development must be subjected to rigorous testing and deployment
management processes to ensure that they work as expected before release.

For more information about considerations for reliability and safety, watch the following video.
Privacy and security

AI systems should be secure and respect privacy. The machine learning models on which AI systems
are based rely on large volumes of data, which may contain personal details that must be kept
private. Even after the models are trained and the system is in production, privacy and security need
to be considered. As the system uses new data to make predictions or take action, both the data and
decisions made from the data may be subject to privacy or security concerns.

For more details about considerations for privacy and security, watch the following video.

Inclusiveness

AI systems should empower everyone and engage people. AI should bring benefits to all parts of
society, regardless of physical ability, gender, sexual orientation, ethnicity, or other factors.

For more details about considerations for inclusiveness, watch the following video.

Transparency

AI systems should be understandable. Users should be made fully aware of the purpose of the
system, how it works, and what limitations may be expected.

For more details about considerations for transparency, watch the following video.

Accountability

People should be accountable for AI systems. Designers and developers of AI-based solutions should
work within a framework of governance and organizational principles that ensure the solution meets
ethical and legal standards that are clearly defined.

For more details about considerations for accountability, watch the following video.

The principles of responsible AI can help you understand some of the challenges facing developers as
they try to create ethical AI solutions.

Further resources

For more resources to help you put the responsible AI principles into practice,
see https://www.microsoft.com/ai/responsible-ai-resources.

To see these policies in action you can read about Microsoft’s framework for building AI systems
responsibly.

Knowledge check

200 XP

 Module assessment

 2 minutes

1.

You want to create a model to predict sales of ice cream based on historic data that includes daily
ice cream sales totals and weather measurements. Which Azure service should you use?
Azure Machine Learning

Azure AI Bot Service

Azure AI Language

2.

You work for a wildlife sanctuary and are considering using AI to identify bird species from images.
Which AI service should you use to prototype your idea?

Azure AI Vision

Azure AI Search

Azure OpenAI

3.

A predictive app provides audio output for visually impaired users. Which principle of Responsible
AI is reflected here?

Transparency

Inclusiveness

Fairness

Submit answers

Summary

Completed100 XP

 1 minute

Artificial Intelligence enables the creation of powerful solutions to many kinds of problems. AI
systems can exhibit human characteristics to analyze the world around them, make predictions or
inferences, and act on them in ways that we could only imagine a short time ago.

With this power, comes responsibility. As developers of AI solutions, we must apply principles that
ensure that everyone benefits from AI without disadvantaging any individual or section of society.

Module incomplete: