Q-Learning

More documents

Recommendations

Info

Properties ● ● ● ● Convergence: Our approximation will converge to the true Q function – But we must visit every state-action pair infinitely many times! Table size can be very large for complex environments like a game We do not estimate unseen values How to we fix these problems?
Neural Network Approximation ● ● Instead of the table, use a neural network – Inputs are the state and action – Output is a number between 0 and 1 that represents the utility Encoding the states and actions *properly* will be challenging – Helpful idea: multiple neural networks, one for each action
Page 1 and 2: Q-Learning ● ● ● Reinforcemen
Page 3 and 4: Our Problem ● ● ● ● ● We
Page 5 and 6: Formal Definition Q s , a=r s ,amax
Page 7 and 8: Example Problem a12 a23 s1 s2 s3 a2
Page 9 and 10: The Algorithm s1, a12 0 s1, a14 0 s
Page 11 and 12: Next Move s1, a12 0 s1, a14 0 s2, a
Page 13 and 14: Next Move s1, a12 0 s1, a14 0 s2, a
Page 15 and 16: New Game s1, a12 0 s1, a14 0 s2, a2
Page 17: Final State (after many iterations)
Page 21 and 22: Exploration Strategy ● ● ● Wa

Q-Learning

Create successful ePaper yourself

Delete template?

Save as template?