You identified the problem, now correct it. It sounds simple because it is.
And you're not alone. I've owned my own plane for awhile now, and there was a point where my pre-flights were getting a bit lax. I mean, its my plane, and I know I put it away in the locked hangar in flyable...