The probability distribution on the set of naturally occurring images is sparse with most of the probability mass on a small subset of all possible images, hence not all images are equally likely to be seen in nature. This can indirectly be observed by studying the marginal statistics of filter responses on natural images. Intensity differences, or equivalently responses of linear filters, of natural images have a spiky distribution with heavy tails, which puts a large proportion of the probability mass on small intensity differences, but at the same time giving a reasonable probability on large differences. This is due to the fact that images consist mostly of smooth regions separated by discontinuous boundaries. We propose to model natural images as stochastic Lévy processes with kernel distributed intensity differences. We will argue that the scale invariant kernels of the recently proposed scale space theory provides a promising model of the intensity difference distribution (or in general linear filter responses) in conjunction with the Lévy process model of natural images.