Judge Andrew Napolitano said on "Varney & Co." today that the Confederate flag should be removed from South Carolina state capitol grounds, not because it represents hatred, but because the government has no business expressing an opinion on anything.

Judge Napolitano explained that the Supreme Court handed down a "bizarre" ruling two weeks ago, which said that states have freedom of speech.

He said that this is dangerous, as individual humans should have that natural right, but not the government.

"The government is an artificial entity based on consensus and force. It does not have the natural rights that human beings do. It's dangerous for the government to have the freedom of speech," Judge Napolitano said. "Who cares what the government thinks?"

He said that putting aside his legal, philosophical analysis of the issue, he is opposed to the Confederate flag because it represents, at least in part, "the most horrible racial oppression known to man."

"It should not be flown," Judge Napolitano concluded. "Who cares what the government's opinions are?"

Watch more from "Varney & Co." above.

Huckabee to Ed Henry: 'I Didn't Punt' Questions About Confederate Flag

Judge Napolitano: 'Hatred Is Protected Under the Constitution'

Toby Keith on Charleston Shooting: Stricter Gun Laws Are Not the Answer