Current efforts to utilize full waveform inversion (FWI) as a tool beyond acoustic imaging applications, for example for reservoir analysis, face inherent limitations on resolution and also on the potential trade-off between elastic model parameters. Adding rock physics constraints does help to mitigate these issues. However, current approaches to add such constraints are based on averaged type rock physics regularization terms. Since the true earth model consists of different facies, averaging over those facies naturally leads to smoothed models. To overcome this, we propose a novel way to utilize facies based constraints in elastic FWI. A so-called confidence map is calculated and updated at each iteration of the inversion using both the inverted models and the prior information. The numerical example shows that the proposed method can reduce the cross-talks and also can improve the resolution of inverted elastic properties.