How do I get my family to understand that I don't want them there?

Why don't you tell the person that you respect the things she has to say but you prefer her not to be involved in the situation. Tell her she needs to stop being so nosy and let you decide what you and your grandma and brother should do about it.