You are correct! I am surprised to read this post of yours because I do think you come across as quite confident and assertive. For me, part of it is your Avatar. I look at it and see a strong leader with head upturned, appearing to lead some sort of ritual. That may not be what it's meant to be, but that is what I pick up from it, and since I have no idea what you look like, my mind fills in the rest.
I used to be just where you are now. With people whose vibration was pretty high, loving kindness worked, and problems were eventually ironed out. But some people vibrate so much lower they are not living in their upper chakras at all.
I am no longer a pushover, but I do NOT recommend that you change the way I did, by letting yourself be mistreated by horrible in-laws, neighbors and so-called friends. I simply reached a point where I was DONE and unleashed all that held in wrath. Where being endlessly kind had gotten me only more bullying, giving them their own medicine in spades scared them so much that they leave me totally alone now. It is much more peaceful. I no longer take any crap from anyone, and I don't give that person ammo to use against me. I have become much less open and wait until I feel I know which type of person I am dealing with before I trust.
I was taught never to lower myself to the level of that sort of person, and never to get angry, but I find they literally live in a different world, where compassion is regarded as weakness, and if you don't play by rules they understand, at least until you get them out of your life, you just get more and worse of the same. We can see this working in a much larger way upon the world stage right now. Some people think they can influence people who live in a society with 11th century values by treating them as if they have 21st century values. They regard this as weakness. It has only resulted in more pain for everyone.
I am working hard on not caring what that type of person thinks, says or does, so that I can say " thank you for the kind words", no matter what is said to me, while walking away as fast as my legs can go. Controlling my reaction afterwards, once I get away and am in private is a lot harder. My goal is to get to where I can laugh and move on right away, without giving it another thought. I have a long way to go and often get so angry that I'm amazed I haven't had a stroke.
The point is not to let them see it, because one thing I pick up from almost all of these people is how much they enjoy seeing that they have made you angry and hurt you. That's how miserable they are; their joy comes from making someone else even more miserable than they are. I have seen them grin with delight. They are the ones who are pathological, not us.
These are the same people honking and giving you the finger for going the speed limit. I think a smile and a wave will break their anger better than anything else. They do not know what to do with it, and do not expect it.
I don't think anything I've said is really going to help with your driving fears, but it may help a bit with the pleasing everyone else part. I felt the same guilt over my car accidents, especially the one where I was declared dead. It backed up a major roadway at rush hour and people were as much as 4 hrs. late getting home that night. I felt guilty for years even though it was not ruled to be my fault. I don't think I would feel that now. I have detached myself from what others feel and think to a much greater degree, simply by affirming it each morning when I put up my shield.