Just a data point, take it for what it's worth:
I'm a 20+ year, VFR-by-choice pilot. My wife (also a VFR-by-choice pilot) has flown for 17 years.
We have flown all over the United States, coast to coast, Canada to Mexico, all VFR. We have attended Oshkosh 33 times in a row, flying in since 1999. We've never missed due to weather, although we've been delayed a few times. (Remember SloshKosh?)
Although we do use our plane for business, both of us fly for the sheer joy of it. Back in 2001, I went for the instrument rating. I logged 54 approaches, a bunch of hours, and was about to be signed off for the checkride when we bought our first hotel. That was the end of focused, concentrated free time as I knew it, and I never finished the rating.
I found that the training made me a more precise pilot -- but so did aerobatics training. I also found that I did not enjoy instrument flying, at all. Flying without seeing anything, staring intently at the instrument panel, surrendering my routing choice to others, all of it was a bore and a chore, and seemed diametrically opposed to why I wanted to fly in the first place.
A few years later, when it became possible to resume my training, I hesitated. Living in Iowa (at the time), Mary and I logged the number of days when an instrument rating would help us. They were amazingly few, given that we could not fly in icing or convective conditions in the piston powered bug-smashers we flew. So, I decided to remain VFR only, by choice, and have never regretted it.
Now that I live on an island in the gulf, there are more times when the rating would help me get out, but they are still very few, and frankly I don't fly anywhere that I absolutely have to be. So, if it's VFR, we go. If it's not, chances are pretty good that it would be a miserable flight, and I don't want to ever fly in miserable conditions.
An aside: I have found that technology has made flying in marginal VFR conditions much more enjoyable and safe. We have had synthetic vision on our EFIS for over a year, now, and it has the marvelous ability of making hazy conditions clear as a bell. The days of not being able to see where the airport is are behind us, because there is a magenta "balloon" floating over it in the virtual world.
And, of course, with ADS-B traffic and weather, it is now possible to fly around weather that, back in the olden days, we used to be unaware of. Before XM and ADS-B weather, VFR cross-country flights were truly sketchy affairs.
Another data point: Remaining IFR current is not enough. You must remain proficient. I know a LOT of pilots, and I can count the number of instrument proficient private instrument pilots on one hand. It is extraordinarily difficult and expensive to maintain instrument proficiency as a non-professional private pilot.
So, there you have it. You have to ask yourself why you are flying, and what kind of flying you enjoy. I made the choice to stay VFR, and have had a ball with it for over two decades. YMMV.