Piecewise planar city 3D modeling from street view panoramic sequences

City environments often lack textured areas, contain repetitive structures, strong lighting changes and there- fore are very difficult for standard 3D modeling pipelines. We present a novel unified framework for creating 3D city models which overcomes these difficulties by exploiting im- age segmentation cues as well as presence of dominant scene orientations and piecewise planar structures. Given panoramic street view sequences, we first demonstrate how to robustly estimate camera poses without a need for bundle adjustment and propose a multi-view stereo method which operates directly on panoramas, while enforcing the piece- wise planarity constraints in the sweeping stage. At last, w e propose a new depth fusion method which exploits the con- straints of urban environments and combines advantages of volumetric and viewpoint based fusion methods. Our tech- nique avoids expensive voxelization of space, operates di- rectly on 3D reconstructed points through effective kd-tre e representation, and obtains a final surface by tessellationof backprojections of those points into the reference image.