Former England batsman Michael Carberry said that racism cost him a place at one of his four county clubs.
Carberry, who represented Surrey, Kent, Hampshire and Leicestershire, as well as playing six Tests for England, told the Cricket Badger podcast: "I think other players who laugh it off want acceptance, they don't want to get dropped, or put a left hook on that guy.
"Not hit him, but have a harsh word with him and say, 'Listen mate, don't ever say that to me again', because that guy may have a massive powerful influence in the team.
The 39-year-old added: "I've almost come close to making a coach spit 32 (teeth) out on the ground for stuff that he said to me.
"'I couldn't see you in the dark' and 'What are the brothers having tonight? Bit of fried chicken and rice and peas tonight?’
"I had to drag him out on the balcony and say: 'Listen, let me ask you something mate. How much time have you spent in Black company? And he literally wet his pants. He literally hung his head like a little child.
"Bear in mind, I'm putting my career (on the line), and it probably ended up being the final nail in my coffin in that club.
"I won't name the club. But these are the things you have to weigh up when you hear things like this in your company."
Carberry's comments came as cricket weighs up how best to respond to the worldwide Black Lives Matter protests that have taken place following the police killing of George Floyd in the United States.
West Indies captain Jason Holder, speaking the day after his squad arrived in England ahead of next month's Test series, was unsure how his team-mates would react to the campaign.
"Who knows, this could be something serious we could build on and we could get some real positive energy through the group," he said.
"Only after we sit down and discuss and get a common sense of where everybody's mind is at, will we then formulate our plans."