If you swing hard, you better follow through with it to slow the blade down. Im sure if you swing hard enough and stop it suddenly, it might cause harm. I know what you are talking about when you say, "white patches" but I don't think you have to worry about that unless you actually bend the blade by hitting it against a hard edge to create a crease.
Try to treat your lightsaber like a valuable sharp sword. Don't just hack away at anything, choose your opponents wisely.
oh, and FFX sabers should not be used for crutches, canes or walking sticks. hhahhaha