The theory for time-resolved, pump-probe, photoemission spectroscopy and other pump-probe experiments is developed. The formal development is completely general, incorporating all of the nonequilibrium effects of the pump pulse and the finite time width of the probe pulse, and including possibilities for taking into account band structure and matrix element effects, surface states, and the interaction of the photoexcited electrons with the system leading to corrections to the sudden approximation. We also illustrate the effects of windowing that arise from the finite width of the probe pulse in a simple model system by assuming the quasiequilibrium approximation.