The electromagnetic models of retured signals are used to describe the returned signal of the scattering points on the target and are a function of frequency and/or aspect angle. The Attributed Scattering Center(ASC), is a widely used and precise model which is based on the geometry and type of each scattering center and is inferred using the Geometrical Theory of Differaction(GTD). The Automatic Target Recognition(ATR) is among the hot topics in recent decade’s research. Many methods have been proposed to enhance the recognition performance. In recent years, methods based on neural networks and machine learning were frequently used and have achived improvements in ATR results. In this paper, we model a neural network with two data streams. In the first data stream the images are directly fed into the CNN. In the second stream, however at first, the ASC model parameters are extracted and the distributed and localized scattering centers are classified. Then the data is extraplated both in cross range and down range which creates higher resolution images. These images are then fed into another CNN which at the end the two streams are concatenated and the classification is made using both streams. The final network is trained based on real-world data and the results are discussed.