We present a detection based tracking algorithm for tracking melanosomes (organelles containing melanin) in time lapse image sequences imaged using bright field microscopy. Due to heavy imaging noise detecting all the melanosomes accurately in every frame is difficult. Therefore, two sets of imperfect detections are used in a unified probabilistic approach to simultaneously perform melanosome detection and tracking. We propose a novel iterative algorithm which jointly estimates the optimal set of detections and track results in every iteration from the previous tracks and detections. Our algorithm obtains significantly better tracking results than the state of the art tracking-by-detection algorithm.