Why do people say this about me and my horse?

She definitely sounds like she had been abused. Ignore other people. It is not there place to judge. The horse trusts you because you have shown tremendous patience and gentleness towards the horse. You are a great owner, and your horse is lucky!