Billionaire philanthropist George Soros hopes the U.S. goes to pot, and he is using his money to drive it there.

With a cadre of like-minded, wealthy donors, Mr. Soros is dominating the pro-legalization side of the marijuana debate by funding grass-roots initiatives that begin in New York City and end up affecting local politics elsewhere.

Through a network of nonprofit groups, Mr. Soros has spent at least $80 million on the legalization effort since 1994, when he diverted a portion of his foundation’s funds to organizations exploring alternative drug policies, according to tax filings.

His spending has been supplemented by Peter B. Lewis, the late chairman of Progressive Insurance Co. and an unabashed pot smoker who channeled more than $40 million to influence local debates, according to the National Organization for the Reform of Marijuana Laws. The two billionaires’ funding has been unmatched by anyone on the other side of the debate.

Mr. Soros makes his donations through the Drug Policy Alliance, a nonprofit he funds with roughly $4 million in annual contributions from his Foundation to Promote an Open Society.

Mr. Soros also donates annually to the American Civil Liberties Union, which in turn funds marijuana legalization efforts, and he has given periodically to the Marijuana Policy Project, which funds state ballot measures.

Lewis, who died in November, donated to legalization efforts in his name and through the ACLU and the Marijuana Policy Project, on which he served as the chairman of the board. Lewis‘ estate declined to comment for this article.

“The pro-legalization movement hasn’t come from a groundswell of the people. A great deal of its funding and fraud has been perpetrated by George Soros and then promoted by celebrities,” said John Walters, director of the White House Office of National Drug Control Policy under George W. Bush. “The truth is under attack, and it’s an absolutely dangerous direction for this country to be going in.”

Mr. Soros‘ Open Society Foundations have annual assets of more than $3.5 billion, a pool from which he can dole out grants to pet projects, according to 2011 tax returns, the most recent on file for his charitable organizations.

David and Charles Koch, the billionaire brothers who often are cited for their conservative influence, had $308 million tied up in their foundation and institute in 2011.

Mr. Soros did not respond to a request to be interviewed.

‘A question of when’

In his book “Soros on Soros: Staying Ahead of the Curve,” he said the U.S. policy of criminalizing drug use rather than treating it as a medical problem is so ill-conceived that “the remedy is often worse than the disease.”

Although Mr. Soros didn’t outline an alternative in his book, he wrote that he could imagine legalizing some of the less-harmful drugs and directing the money saved from the criminal justice system to treatment.

“Like many parents and grandparents, I am worried about young people getting into trouble with marijuana and other drugs. The best solution, however, is honest and effective drug education,” Mr. Soros said in a 2010 op-ed in The Wall Street Journal. “Legalizing marijuana may make it easier for adults to buy marijuana, but it can hardly make it any more accessible to young people. I’d much rather invest in effective education than ineffective arrest and incarceration.”