Many organizations "attack their own system"

Many organizations "attack their own system" as a way to harden their security. What is your take regarding such an approach, and what do you think should be considered during such a practice? Please aim to share past experiences without revealing proprietary information.