In my experience, yes, you should interact less with TBM family and friends, but with an important caveat. Simply try not to talk about Mormonism in any way unless the context demands it. And when you do talk about Mormonism, talk like you were a non-member. Congratulate your TBM friends when they baptize a child or a son gets home from a mission. Congratulate them on temple sealings or a new calling.
This tactic is a practical application of the central problem of introducing apostasy to someone. If there was one issue that could consistently and reliably lead someone out of the LDS church, the church would have collapsed long ago. Obviously the long list of silliness and tragedy surrounding Mormons has not affected the critical mass of orthodox believers. If any one person is going to become an apostate then it will initiate from having an emotional reaction to an intellectual issue. And their is usually no way to know what that issue will be.
Even more insidious is that even if you somehow magically "knew" what that hot-button issue of apostasy was going to be, it will backfire on you if you introduce it as a known unbeliever. The Sunk Cost Fallacy combined with your status as a philosophical outsider will very often cause a believer to retrench even in the face of evidence.
Believers rarely want to talk about their actual beliefs, even when deep down they are bothered by them. The institutional LDS church quietly knows this, which is why the recent Gospel Doctrine lesson on D&C 132 spent 99% of the time talking about treating your spouse righteously and glorifying the temple sealing. The lesson treats the discussion of 19th century plural marriage like it was as appealing as a condom filled with fire ants. Better that no one talks about it at all rather than putting on that issue.
When you are with believers, talk about their job, their hobbies, and their children. Talk about politics, world events, and science. Talk about upcoming movies and the last season of their favorite show. Discuss problems to be faced and solved. Talk about loved ones that have passed on and babies yet to be born. Let them bring up the issues so that the burden of proof rests squarely on the believer. Let them see that well-adjusted, happy unbeliever you actually are striving to become.