Google Cloud: TensorFlow on Google Cloud

TensorFlow toolkit hierarchy

A tensor is an N-dimensional array of data

GradientTape records operations for automatic differentiation

TensorFlow can compute the derivative of a function with respect to any parameter.

Quiz

  1. Which of the following is true when we compute a loss gradient?
    • It uses tape and the gradients associated with each recorded operation to compute the gradients.
    • The computed gradient of a recorded computation will be used in reverse mode differentiation.
    • TensorFlow records all operations executed inside the context of a tf.GradientTape onto a tape.
  2. Which of the following statements is true of TensorFlow?
    • TensorFlow is a scalable and multi platform programming interface for implementing and running machine learning algorithms, including convenience wrappers for deep learning.
  3. Which API is used to build performant, complex input pipelines from simple, re-usable pieces that will feed your model’s training or evaluation loops.
    • tf.data.Dataset
  4. How does TensorFlow represent numeric computations?
    • Using a Directed Acyclic Graph (or DAG)
  5. What operations can be performed on tensors?
    • They can be both reshaped and sliced
  6. Which are useful components when building custom Neural Network models?
    • tf.losses
    • tf.metrics
    • tf.optimizers

Design and Build an input Data Pipeline

A tf.data.Dataset allows you to

Datasets can be created from different file formats

# Using TextLineDataset
dataset = tf.data.TextLineDataset(csv_file_path)
dataset = dataset.map(parse_row)
dataset = dataset.shuffle(1000).repeat(15).batch(128)

# Read a set of shared CSV files using TextLineDataset
dataset = tf.data.Dataset.list_files(path).flat_map(tf.data.TextLineDataset).map(parse_row)
dataset = dataset.shuffle(1000).repeat(15).batch(128)

Embeddings

A good starting point for number of embedding dimensions

EX) 50,000 movies => 25 dimensions => search space 15 to 35

fc.crossed_column enables a model to learn separate for combination of features

fc_crossed_ploc = fc.crossed_column([fc_bucketized_plat, fc_bucketized_plon], hash_bucket_size=NBUCKETS * NBUCKETS)
# crossed_column is backed by a hashed_column, so you must set the size of the hash bucket.

Data preprocessing

  1. Text features preprocessing => tf.keras.layers.TextVectorization
  2. Numerical features preprocessing => tf.keras.layers.Normalization
  3. Numerical preprocessing => tf.keras.layers.Discretization
  4. Categorical features preprocessing
    • CategoryEncoding
    • Hashing
    • StringLookup
    • integerLookup
  5. The adapt() method
    • TextVectorization => StringLookup and IntegerLookup => Normalization =>Discretization
Categorical features preprocessing

Option 1: Make them part of the model

This is for good for GPU training.

Option 2: Apply it to your tf.data.Dataset

Benefits of doing preprocessing inside the model at inference time

This is especially powerful if you’re exporting your model to another run time, such as TensorFlow.js. You won’t have to re-implement your preprocessing pipeline in JavaScript.

Makes your model portable and helps reduce the training-serving skew.

Quiz

  1. Which of the following is true about embedding?
    • An embedding is a weighted sum of the feature crossed values.
    • Embedding is a handy adapter that allows a network to incorporate sparse or categorical data.
    • The number of embeddings is the hyperparameter to your machine learning model.
  2. What are distinct ways to create a dataset?
    • A data source constructs a Dataset from data stored in memory or in one or more files and a data transformation constructs a dataset from one or more tf.data.Dataset objects.
  3. Which of the following is not a part of Categorical features preprocessing?
    • tf.keras.layers.Discretization
  4. What is the use of tf.keras.layers.TextVectorization?
    • It turns raw strings into an encoded representation that can be read by an Embedding layer or Dense layer.
  5. Which of the following is a part of Keras preprocessing layers?
    • Numerical features preprocessing
    • Image data augmentation
    • Image preprocessing
  6. When should you avoid using the Keras function adapt()?
    • When working with lookup layers with very large vocabularies
  7. Which is true regarding feature columns?
    • Feature columns describe how the model should use raw input data from your features dictionary.
  8. Which of the following layers is not non-trainable?
    • Normalization

Leave a Reply


Create your own website, free on WordPress.com.Create your own website.
Start free