Deep networks are commonly being used for segmentation of various  applications. One example is thigh and calf muscle images which are being used for estimating fat infiltration into muscular dystrophies. The infiltration of adipose tissue into the diseased muscle region varies in its severity across, and within, patients. In order to efficiently quantify the infiltration of fat, accurate segmentation of muscle and fat is needed. Several algorithmic solutions have been proposed for automatic segmentation. While these methods may work well in mild cases, they struggle in moderate and severe cases due to the high variability in the intensity of infiltration , and the tissue’s heterogeneous nature. This project we address these challenges with a a deep network producing robust muscle and inter-muscular adipose tissue segmentation.