Most manufacturing systems are large, complex, and subject to uncertainty. Obtaining exact feedback policies to run these systems is nearly impossible, both theoretically and computationally. It is a common practice, therefore, to manage such systems in a hierarchical fashion. This book articulates a theory that shows that hierarchical decision making in the context of a goal-seeking manufacturing system can lead to near optimization of its objective. The approach in this book considers manufacturing systems in which events occur at different time scales. For example, changes in demand may occur far more slowly than breakdowns and repairs of production machines. This suggests that capital expansion decisions that respond to demand are relatively longer term than those decisions regarding production. Thus, long-term decisions such as those dealing with capital expansion can be based on the average existing production rapacity, and can be expected to be nearly optimal even though the short-term capacity fluctuations are ignored. Having the long-term decisions in hand, one can then solve the simpler problem of obtaining production rates. Increasingly complex and realistic models of manufacturing systems with failure-prone machines facing uncertain demands are formulated as stochastic optimal control problems. Partial characterization of their solutions is provided when possible along with their hierarchical decomposition based on event frequencies. In the latter case, multilevel decisions are constructed in the manner described above and these decisions are shown to be asymptotically optimal as the average time between successive short-term events becomes much smaller than that between successive long-term events. Much attention is given to establish that the order of deviation of the cost of the hierarchical solution from the optimal cost is small. The striking novelty of the approach is that this is done without solving for the optimal solution, which as stated earlier is an insurmountable task. This approach presents a paradigm in convex production planning, whose roots go back to the classical work of Arrow, Karlin and Scarf (1958). It also represents a new research direction in control theory. Finally, the material covered in the book cuts across the disciplines of Operations Management, Operations Research, System and Control theory, Industrial Engineering, Probability and Statistics, and Applied Mathematics. It is anticipated that the book would encourage development of new models and techniques in these disciplines.