It isn't that it is more direct, it is that affect is usually used as a verb, and effect is usually used as a noun.
You might not be an idiot. I figure there are three possibilities. It could be that you have extreme difficulty translating your thoughts into written words, and therefore only appear to be an idiot. Or you could have terrible trouble extracting the useful information from social interaction you see, and therefore it would be impossible for you to be aware of how much of an idiot you often sound like.
Or, you could be an idiot.
Personally I believe that you are one or both of the first two, because if you were just an idiot, you wouldn't so consistently misunderstand people, and say stupid, inappropriate, or irrelevant things.