Does this happen whan you're riding, in gear, idling with the sidestand down, or when?
I've been kinda watching this post, and I'm not seeing enough data to begin a guess.
So far, I'd have to go with the thought of NOT bumping the switch when you're not intending to start or reverse the bike... But that doesn't fix up you issue.
Under what conditions does this happen -- You mention that usually you'd hear the starter spin -- tht would indicate that you're doing this in neutral, or with the sidestand up / clutch lever in... ???