In this work, a new approach to fully automatic color image segmentation, called JSEG, is presented. First, col-ors in the image are quantized to several representing classes that can be used to differentiate regions in the image. Then, image pixel colors are replaced by their cor-responding color class labels, thus forming a class-map of the image. A criterion for "good" segmentation using this class-map is proposed. Applying the criterion to local win-dows in the class-map results in the "J-image", in which high and low values correspond to possible region bound-aries and region centers, respectively. A region growing method is then used to segment the image based on the multi-scale J-images. Experiments show that JSEG pro-vides good segmentation results on a variety of images.