How can I get my uncle to stop brainwashing my gram?

My gram offered to have my boyfriend and I stay with her and rent her basement until we save up money to get our own place. My gram had met my boyfriend on many occasions and liked him. She knew what she was letting into her house (a cat and dog). Both of us suffer from severe depression already but I believed we could get it under control by staying there.
My uncle, who does not know him, decided to make a big deal about a stranger living with her, but he's not a stranger. Everyone met my boyfriend but him and his wife.
On the fifth day we were there, they visited my gram. But they also thought they had permission to go downstairs and look at our stuff and then complain about how messy it was cause it wasn't completely organized. I told them they had no place to say anything just because she's your mother and that we are currently and have been sorting it out.
I know my gram has been talking to them possibly daily cause I hear her say our names. I personally had asked her if we were doing anything wrong and she seemed to have shut down and said she couldn't think of anything. A phonecall later, she had a list of things for us to change. I didn't think anything of it other than why couldn't she have told me whenI asked. I believe some of it was her and some of it was stuff she was told to say.
My boyfriend especially felt restricted and only expected the worst. I tried to help him think differently but I ended up giving him no choice but to get help. I called his parents who he had been ignoring for months and told them the situation and we called him in for suicide threats. There were police and a counselor there to talk to him. He didnt want to go with his parents so he chose to go to the hospital because I didnt want him with me in his mindset.
After we finally did that my mom and I went back to my grams to pack some clothes because I had decided to stay with my parents for a few days and my my boyfriend was going to stay with his parents for a couple nights (a lie to avoid something that happened anyway) my mom and I explained that he has a chemical imbalance and was severely depressed and was going to the doctor to get his medication altered.
But the next day she called my mom and said she doesn't want him to come back because she was scared of it. She didn't say that to us before... he is not a threat to anyone, but he was to himself but is getting help, and is going to be much better and actually be able to work soon to pay her.
My gram is ignoring our pleas to believe us that we are not going to hurt her or anyone ever. But shes stuck with her brainwashing son who is trying so hard to ruin our lives. Everything him and his wife is saying is out of nowhere, but my gram is old and they guess they can take advantage of her. Which they've been doing for years since my grampa died.
I've spoken to a lot of counselors for advice and they agree that I can call upon the Adult Protection Agency and charge them with Elder Abuse.
It's crazy that my gram is on their side, even though my parents are the ones that help with everything.

But I would like as much advice as possible to make sure I have all the facts down before calling APA tomorrow.