By and large, this book grew out of research conducted in my group as well as classes and lectures given at the University of California, Irvine (UCI) and elsewhere over the years. It can be used as a textbook for an undergraduate or graduate course in machine learn- ing, or as an introduction to the topic for scientists from other fields. Basic prerequisites for understanding the material include college-level algebra, calculus, and probability. Familiarity with information theory, statistics, coding theory, and computational com- plexity at an elementary level are also helpful. I have striven to focus primarily on fundamental principles and provide a treatment that is both self-contained and rigorous, sometimes referring to the literature for well-known technical results, or to the exercises, which are an integral part of the book.