Is it possible for ghosts to follow a person from one house to another?

Yes...It does happen. Especially at the ages you and your brother are. Nobody seems to know why, but teenagers or adolescents seem to have the highest amount of paranormal experiences. You sound as if you are a very logical person and that you have tried to explain what has happened rationally. What ever this is, does not appear to be harmful. For some reason, they just have an affinity for your family. Do not be afraid. They do gain energy from fear among other things. At any rate, if you want to get rid of this, you need to be firm and take back your home. Tell them that they do not belong there (firmly) and they must leave. The only concern is they may get aggravated, and step it up a bit. Such as showing themselves more frequently. The thing that has worked for me is to tell them firmly that they can stay if they want, but they must not scare you and must leave you alone. When you do this, you can not be afraid. They need to know that you are not fearful and can get rid of them if you want to. Feel free to e-mail me if you like. I might be able to talk you through this. Best of luck!