Cleveland beat Cincinnati, where Jackson is now a special assistant to the head coach, on Sunday, 35-20. The Browns victory came just a few weeks after Jackson was fired.
Mayfield was vocal about Jackson moving onto a division rival so quickly, even calling him fake. But on Monday, he said people were making a big deal out of it.
"People took it as me personally attacking Hue. That's not it. It's the fact that I get to have my own opinion on how it transpired and he gets to do what he wants," Baker said.
"I'm an athlete. I'm not a cookie-cutter quarterback. Never have been, never will be. I speak my mind."
The rookie refused to go into detail about the issues with Jackson but acknowledged there were problems inside the building.
When asked if he was reverting back to his behavior at the University of Oklahoma, Mayfield was direct, as always.
"Some Oklahoma mistakes, as in like sideline gestures and getting arrested? No, those are the only mistakes I made at Oklahoma. People get maturity confused with me being 100 percent comfortable in my own skin."
Mayfield said he's used to blocking out criticism because the most important thing is his ability to do his job.
So far this season, the Heisman winner has thrown more than 2,000 yards with 17 touchdowns over nine games.
Cleveland plays the Texans in Houston on Sunday.