Do not deprive one another, except perhaps by agreement for a limited time, that you may devote yourselves to prayer; but then come together again, so that Satan may not tempt you because of your lack of self-control.
1 Corinthians 7:5
NKJV - 5 Do not deprive one another except with consent for a time, that you may give yourselves to fasting and prayer; and come together again so that Satan does not tempt you because of your lack of self-control.
For follow-up discussion and general commentary on the topic. Comments are sorted chronologically.
I think this is valid as a rationale for not depriving your spouse. It is not a license for the deprived spouse to wander outside the marriage.