Introduction -- RMAB in Opportunistic Scheduling -- Optimality of Myopic Policy with Imperfect Sensing -- Whittle Index Policy with Imperfect Sensing -- Heuristic Policy with Imperfect Sensing -- Optimality of Myopic Policy with Imperfect Observation -- Whittle Index Policy for Multi-State Channel Scheduling -- Conclusion.

This book provides foundations for the understanding and design of computation-efficient algorithms and protocols for those interactions with environment, i.e., wireless communication systems. The book provides a systematic treatment of the theoretical foundation and algorithmic tools necessarily in the design of computation-efficient algorithms and protocols in stochastic scheduling. The problems addressed in the book are of both fundamental and practical importance. Target readers of the book are researchers and advanced-level engineering students interested in acquiring in-depth knowledge on the topic and on stochastic scheduling and their applications, both from theoretical and engineering perspective. Introduces Restless Multi-Armed Bandit (RMAB) and presents its relevant tools involved in machine learning and how to adapt them for application; Elaborates on research bringing the conventional decision theory and stochastic optimal technology into wireless communication applications involving machine learning; Delivers a comprehensive treatment on problems ranging from theoretical modeling and analysis, to practical algorithm design and optimization.