Michael Vaughan's "life and livelihood" are at stake in the Yorkshire racism hearing, the former England captain's lawyer claimed on Tuesday.
Former Yorkshire batsman Vaughan has lost roles as a television pundit since the charges were made and revealed at the hearing that his health and wellbeing had also suffered.
Christopher Stoner, Vaughan's lawyer, told the Cricket Discipline Commission in London: "This is obviously a matter of significant importance to my client. The shape of his life and livelihood are at stake.
"We simply say the evidence does not make up the charge."
Vaughan is alleged to have told Rafiq, Adil Rashid, Rana Naved-ul-Hasan and Ajmal Shahzad: "There's too many of you lot, we need to have a word about that."
Stoner accused the ECB of backtracking due to a discrepancy concerning the second half of the comment, as Rafiq also recalled it was: "We need to do something about it."
ECB lawyer Jane Mulcahy admitted: "It's a bit messy, I see that." But she reiterated that the keywords "you lot" is what the disciplinary panel should focus on.
Vaughan, who attended the hearing on Friday, maintains he has no recollection of making a comment. His lawyer cited what he called "the frailties of human recollection" of an incident that took place almost 14 years ago.
"Mr Rafiq says he has a clear recollection but not of the exact words," Stoner said. He said it made him feel sick and angry, but he didn't speak about it for 11 years.
"We say it's inherently improbable he would not have spoken about it to someone."
Vaughan's 32-page closing written submission stated: "As this hearing closes, now some 5,006 days after the words were alleged to have been spoken, the burden of proving that the words were said, which is so plainly on the ECB, has simply not been discharged."
The commission also heard evidence relating to charges against former Yorkshire head coach Andrew Gale and the county's ex-bowling coach Richard Pyrah.
Gale and Pyrah, who did not attend the hearing, deny using racist and/or discriminatory language. The hearing is scheduled to continue in private on Wednesday and Thursday.