NEW YORK — A federal judge put the NFL on the defensive over its four-game suspension of New England Patriots quarterback Tom Brady on Wednesday, demanding to know what evidence directly links Brady to deflating footballs and belittling the drama of the controversy.

"What is the direct evidence that implicates Mr. Brady?" Judge Richard M. Berman repeatedly asked NFL lawyer Daniel L. Nash at the first hearing in the civil case in Manhattan federal court as Brady and Commissioner Roger Goodell looked on.

Nash responded there was "considerable evidence Mr. Brady clearly knew about this," including records of text messages and phone calls between the quarterback and one of two Patriots employees implicated in the scandal known as "Deflategate." But he also said there was no "smoking gun" showing Brady had direct knowledge that the balls were underinflated for the first half of the Patriots' 45-7 win over the Indianapolis Colts in the AFC championship game Jan. 18.

Brady and Goodell didn't speak during the hearing, except to introduce themselves to Berman. Brady, his head lowered, looked dour throughout a hearing that lasted about 1 hour, 20 minutes. Immediately afterward, Brady smiled slightly as he signed sketches for two court artists. Berman then met individually with each side for more settlement discussions in private. The talks continued for several hours into the afternoon.

Two weeks ago, the NFL asked Berman to declare that its punishment of Brady was properly carried out. The players' union countersued, asking him to nullify the suspension. The judge has signaled from the start that he wants the parties to reach a swift settlement.

On Wednesday, Berman called it "ironic or not" that Brady's statistics were better in the second half of the AFC championship game, after the balls were re-inflated.

"You might say (Brady) got no better advantage from the under-inflation," the judge said.

At one point, the judge also seemed to try to defuse the controversy, saying: "This Deflategate. I'm not sure where the 'gate' comes from."

When the union got its chance to argue, the judge asked attorney Jeffrey L. Kessler why one of the Patriots employees would deflate balls without Brady's knowledge. Kessler said the union does not believe the balls were deflated but, if they were, the employee did it on his own because he "thought it would be good for his quarterback."

The judge also questioned why Brady destroyed his cellphone in the midst of the inquiry — a move that the league argues was further proof of his deception. Kessler claimed that the quarterback got rid of the phone on the advice of his agent to protect his privacy but had otherwise cooperated with the inquiry.

However, in hindsight, "You're right, it could have been done a different way," the lawyer said of the phone.

Both sides are scheduled to return to court next week.

As Goodell arrived at the courthouse, he was greeted by a smattering of boos as he walked inside. Four minutes later, Brady arrived flanked by four security guards. Both men went through a security sweep like everyone else going to court.

Dozens of fans and journalists waited for two of the NFL's most famous faces at the front entrance of the courthouse, including some fans wearing deflated football hats they were hoping to sell.

Goodell suspended Brady after concluding he "knew about, approved of, consented to, and provided inducements and rewards" to support a scheme in which a Patriots employee deflated balls on game day. Brady insists he knew nothing about it.

In a July 28 decision upholding the suspension, Goodell heavily criticized Brady for having an aide destroy a cellphone containing nearly 10,000 text messages from a four-month stretch including the AFC championship game. He accused him of obstructing the NFL probe about a controversy that represented "conduct detrimental to the integrity of, and public confidence in, the game of professional football."

In court documents, the union's lawyers said the suspension was unfair and violated the labor contract and complained that it would cause irreparable harm to Brady by forcing him to miss games.

They called a June appeal hearing before Goodell "a kangaroo court proceeding, bereft of fundamentally fair procedures."