Tracking objects across a network of intelligent vision sensors requires an architecture to distribute intelligent processing algorithms locally to the intelligent vision sensor and an algorithm for the communication of the acquired information to nearby sensors for collaboration and hand-offs of tracked objects. Additionally, the selection of which intelligent algorithms need to be performed at each intelligent sensor, and the management of constrained resources of the network, including network capacity (transmission rates), processing capacity (local processing power of sensor node) and in some cases, battery life of the sensor node must also occur. In the case of object tracking, as the number of tracked objects in the network increase, the resources consumed increases, as more processing power is required to create object descriptors and more networking resources are required to transmit information between sensors to collaboratively track the object. The local processing of intelligent vision algorithms at the vision node transforms high data-rate raw video data into low data rate features to be communicated across the network, thus relieving the networking capacity constraint. We focus on, what we view as the key resource, the sensor nodes' processing capacity, in creating a cluster-based distributed object tracking architecture, which includes resource management for processing capacities of the intelligent sensor nodes.