Hi Kristin, I know for me I tend to always think the worse of peoples reactions to me. I know that some empaths feel this way quite a bit. We have self doubt. The more I have become grounded as an empath the less I feel like this. Mind you I still have my issues. It is also very normal for us to pull away from be social too. I really tend to isolate myself from people. It could be the same self doubt.
I hope this helps at least a little to know you are not alone with this.
Dan : )