MELBOURNE

 Violence broke out between anti-Islam groups and campaigners against racism in the Australian state of Victoria on Sunday, with some protesters brawling on the steps of a Melbourne town hall.

The Age reported that scuffles began when members of far-right group United Patriots Front (UPF) attempted to storm the building but their efforts were thwarted by hundreds of counter-protesters.

UPF is a splinter group of Reclaim Australia -- a right-wing faction that cites the wearing of the burqa, "Islamic extremism" and "fear that sharia law is on the rise" as issues of concern.

While police -- some on horseback -- attempted to keep the two groups apart, a small number broke through the lines, leading to physical confrontation.

In a Facebook video, Shermon Burgess -- the organizer of the right wing protest -- said the UPF had met Sunday to "protest against the left wing extremists that hate our Anzacs, hate the Australian flag and hate our way of life."

Marchers -- many carrying Australian flags, while some wore t-shirts sporting the Nazi swastika -- told The Age that they had gathered in response to "the left-wing violence" they said they had faced during clashes in Melbourne and Hobart on April 4.

The April incidents also saw anti-Islam and anti-racism groups clash, with many people being treated for minor injuries as Reclaim Australia held rallies to oppose "sharia law, halal tax and Islamization."

Earlier Sunday, the far right groups had launched into speeches railing "left-wing traitors", "halal certification" and what they called "the Islamic terror threat".

Meanwhile, rival protesters chanted "no Nazis, never again."

The far right said they had targeted the town hall in the suburb of Richmond to "bring down" a local left wing councilor.

However, that wasn't to be -- heavily outnumbered they were soon forced to retreat behind police lines.

Around 1-1/2 hours after gathering, the right wing groups began to disperse while jubilant counter-protesters chanted "f... off Nazi scum" and "you'll always lose in Melbourne," reported the Age.

A member of the rival anti-racism group later told Australia's 9NEWS that they had turned up in force to protest the UPF's claims that Muslims weren't welcome in Australia.

"Muslims are more than welcome," the protester said.

Later Saturday, Campaign Against Racism and Fascism activist Jerome Small called the event "a fantastic demonstration of ordinary people in Melbourne coming out to say these Nazis are not going to spread their filth around the streets."

"They didn't achieve their aim today, we're not going to let them achieve their aims in the future," he told the Age.

Police have said that a man is expected to be charged after he was found carrying a knife, but no serious injuries had occurred.