This paper presents a novel descriptor for activity classification. The intuition behind the descriptor is "learning" statistics of optical flow histograms (as opposed to learning "raw" histograms). Towards this end, an activity descriptor capturing histogram statistics is constructed. Further, a technique to make the feature descriptor scale-invariant and parts-based is proposed. The approach is validated on a dataset collected from a camera network. The data presents a challenging real world scenario (variable frame rate recording, significant depth disparity, and severe clutter), where biking, skateboarding, and walking are activities to be classified. Experimental results point to the promise of the proposed descriptor in comparison to state of the art.