For instance, MVNLayer reads data from its top blob during the backward pass, under the assumption that this data is exactly the same as the output it created. If it's been modified by a later layer that does in-place computation, the gradient will be computed incorrectly.
In general, caffe should not rely on the user to know under what circumstances a layer can safely be done in-place.