For complete BTECH CSE, AIML, DS subjects tutorials visit : ns lectures youtube channel
REINFORCEMENT LEARNING
Unitwise Important Questions
Unit-1
1. Explain the Basics of Probability and their significance in Reinforcement Learning.
How do probabilistic models relate to decision-making?
2. Describe the fundamental concepts of Linear Algebra and their relevance in
Reinforcement Learning. Provide examples of linear algebra operations used in RL.
3. Define a Stochastic Multi-Armed Bandit. How does it relate to the exploration-
exploitation trade-off in decision-making?
4. Explain the concept of Regret in the context of Multi-Armed Bandits. Why is
minimizing regret important, and how is it measured?
5. Discuss strategies for achieving Sublinear Regret in Multi-Armed Bandit problems.
What are the key techniques used to optimize decision-making in this context?
6. Describe the Upper Confidence Bound (UCB) algorithm for Multi-Armed Bandits.
How does UCB balance exploration and exploitation, and what are its advantages?
7. Explain the KL-UCB algorithm and its role in Multi-Armed Bandit problems. How
does it differ from traditional UCB, and in what scenarios is it preferred?
8. Discuss the concept of Thompson Sampling as a Bayesian approach to Multi-
Armed Bandit problems. How does it incorporate uncertainty into decision-making?
UNIT-2
1. Explain the fundamentals of a Markov Decision Problem (MDP) in reinforcement
learning. What are the key components, and how do they relate to decision-
making?
2. Define policy and value function in the context of MDPs. How are these concepts
used to represent and solve reinforcement learning problems?
3. Describe different types of reward models in reinforcement learning, including
infinite discounted reward, total reward, finite horizon reward, and average reward.
Provide examples of scenarios where each type is applicable.
4. Differentiate between episodic and continuing tasks in reinforcement learning. How
does the task type affect the formulation and solution of an RL problem?
5. Explain Bellman's optimality operator and its role in dynamic programming
approaches to reinforcement learning. How does it facilitate the computation of
optimal policies and values?
6. Describe the concept of Value Iteration as a dynamic programming method for
solving MDPs. What are the key steps involved in the Value Iteration algorithm?
7. Explain the concept of Policy Iteration as another dynamic programming approach
to solving MDPs. How does it alternate between policy evaluation and policy
improvement?
UNIT-3
Prepared by Chennuri Nagendra Sai (Asst.prof)
For complete BTECH CSE, AIML, DS subjects tutorials visit : ns lectures youtube channel
1. Explain the essence of the Reinforcement Learning problem. Describe its key
components, including agents, environments, and rewards. Discuss the
fundamental challenges faced in reinforcement learning.
2. Differentiate between prediction and control problems in Reinforcement Learning.
Provide real-world examples for each type of problem and discuss the key
distinctions.
3. Elaborate on the concept of model-based Reinforcement Learning algorithms. How
do these algorithms employ models of the environment to make informed
decisions? Provide examples of situations where model-based methods are
beneficial.
4. Describe the Monte Carlo method for solving prediction problems in Reinforcement
Learning. How does it estimate value functions based on sampled episodes?
Explain the key characteristics of Monte Carlo methods.
5. Discuss the online implementation of Monte Carlo policy evaluation. How does this
approach update value estimates as new data becomes available? Provide insights
into the advantages and limitations of online Monte Carlo methods.
UNIT-4
1. Explain the concept of Bootstrapping in Reinforcement Learning. How does it differ
from traditional Monte Carlo methods, and what are its advantages?
2. Describe the TD(0) algorithm in detail. How does it update value estimates, and
what is its significance in reinforcement learning?
3. Discuss the convergence properties of Monte Carlo and batch TD(0) algorithms.
What conditions ensure the convergence of these methods, and under what
circumstances do they differ in their convergence behavior?
4. Explain the concept of Model-Free Control in Reinforcement Learning. Discuss the
key algorithms used for model-free control, including Q-learning, Sarsa, and
Expected Sarsa. How do these algorithms learn optimal policies without explicit
models of the environment?
UNIT-5
1. Explain the concept of n-step returns in Reinforcement Learning. How do they
balance the trade-off between bootstrapping and sampling? Provide examples to
illustrate their use.
2. Describe the TD(λ) algorithm in detail. How does it extend the TD(0) algorithm, and
what role does the eligibility trace play in TD(λ)?
3. Discuss the need for generalization in Reinforcement Learning practice. Why is
generalization important, and how does it address issues related to scalability and
transferability?
4. Explain Linear Function Approximation and the geometric view of it in the context of
Reinforcement Learning. How does linear function approximation enable the
handling of high-dimensional state spaces?
Prepared by Chennuri Nagendra Sai (Asst.prof)
For complete BTECH CSE, AIML, DS subjects tutorials visit : ns lectures youtube channel
5. Describe Linear TD(λ) and its application in reinforcement learning. What are the
advantages and limitations of using linear function approximation with eligibility
traces?
6. Explain the concept of Tile Coding as a method for discretizing continuous state
spaces. How does it work, and what are its benefits in function approximation?
7. Discuss Control with Function Approximation. How can you apply function
approximation techniques to solve control problems in reinforcement learning?
8. Describe Policy Search methods in Reinforcement Learning. What are the key
ideas behind policy search, and how do they differ from value-based methods?
9. Explain Policy Gradient methods and their significance in Reinforcement Learning.
How do they optimize parameterized policies directly?
10. Discuss the concept of Experience Replay and its role in improving the stability and
efficiency of reinforcement learning algorithms. Why is experience replay
particularly valuable in deep reinforcement learning?
11. Describe Fitted Q Iteration as an approach to approximate Q-values using function
approximation. How does it work, and what are its advantages?
12. Provide case studies or examples illustrating the practical application of the
discussed topics in real-world reinforcement learning scenarios.
Prepared by Chennuri Nagendra Sai (Asst.prof)
