A truly data-driven organization is in a perpetual state of A/B testing and optimization. And in such an organization, the opportunity cost of A/B testing is trivial.
The hidden costs of A/B testing, however, are of greater consequence. The hidden cost of A/B testing are born from an intellectual over-reliance on iterative, incremental improvements; this over-reliance blinds the organization to fundamental, existential shortcomings in the product. (via The Hidden Costs of A/B Testing)