Video indexing based on motion is an emerging research area. While most previous work focused on video indexing using motion vectors, a detailed quantitative characterization of the spatial and temporal change of motion vectors in a video has not received much attention. We characterize motion in terms of motion activity and propose novel methods for motion activity description. In the particular context of human motion activity analysis, we propose new algorithms for motion activity capture and recognition. Two new motion activity descriptors are introduced for low level video indexing. The first one, the motion intensity descriptor, represents the degree of change in motion in a scene. The second descriptor, the motion intensity histogram, represents the temporal statistics of motion intensity. The motion activity information is extracted in compressed domain based on MPEG macroblock type information. We present a system for capturing panoramic video of human motion activity and a novel method for virtual camera control. The proposed method integrates region of interest (ROI) detection, tracking, and virtual camera control, and works on both uncompressed and compressed video streams. Finally, we present a unified approach for human motion activity recognition. The panoramic camera capturing system is used for video capture. The virtual camera control parameters are used for the recognition of activities such as ix walking, and the motion parameters of each frame are used for the recognition of other activities like turning around, sitting down and getting up. For motion parameter based recognition, the likelihood of the motion parameters is represented using a multivariate Gaussian model and their temporal change is characterized using a continuous density hidden Markov model (HMM). Detailed experimental results are provided to demonstrate the efficiency and effectiveness of the proposed descriptors and motion based activity recognition. In summary, the research presented in this dissertation advances the current state of the art in video indexing by proposing new methods for characterizing motion activity at the low level, using motion intensity and motion intensity histogram, and at the semantic level for annotating some of the common human motion activities.