When we watch other people perform actions, this involves many interacting processes comprising cognitive, motor, and visual system interactions. These processes change based on the context of our observations, particularly if the actions are novel and our intention is to learn those actions so we can later reproduce them, or respond to them in an effective way. Over the past 20 years or so I have been involved in research directed at understanding how we learn from watching others, what information guides this learning, and how our learning experiences, whether observational or physical, impact our subsequent observations of others, particularly when we are engaged in action prediction. In this review I take a historical look at action observation research, particularly in reference to motor skill learning, and situate my research, and those of collaborators and students, among the common theoretical and methodological frameworks of the time.