Title: Non-parametric Classification of Pixels Under Varying Outdoor Illumination Authors: Shashi D. Buluswar and Bruce A. Draper Affiliation: Computer Vision Research Laboratory, Dept. of Computer Science, University of Massachusetts, Box 34610, Amherst, MA 01003-4610 Abstract: Using color for visual recognition outdoors has proven to be a difficult problem, chiefly due to varying illumination. Attempts to classify pixels or image patches in outdoor scenes often fail, partly because of the paucity of data, but partly because color shifts due to changes in illumination are not well modeled as random noise. Approaches which attempt to recover the "true color" of the incident light (i.e. color constancy approaches) appear to work in constrained environments, but are not yet applicable to outdoor scenes. We present a technique that uses training images of an object under daylight to learn the shift in color of an object. Our method uses multivariate decision trees for piecewise linear approximation of the region corresponding to the object's appearance in color space. We then classify pixels in outdoor scenes depending on whether they fall within this region, and group clusters of target pixels into regions of interest (ROIs) for a model-based RSTA system. The techniques presented are demonstrated on a challenging task: recognizing camouflaged vehicles in outdoor military scenes. Keywords: Color, Pixel classification, Decision Trees, Chromatic Variation, Focus of Attention