In he months leading up to the Iraq War in March 2003, New York Times journalist Judith Miller wrote a series of front-page reports revealing details of Saddam Hussein’s arsenal of weapons of mass destruction and the regime’s determination to develop nuclear weapons. Miller’s reports were based on anonymous sources, sources, she said, in the intelligence community and in congress.

In 2007, Bill Keller, the then executive editor of the New York Times, apologised for the paper’s coverage of the lead-up to the war and said that the issue of the use of anonymous sources needed to be urgently addressed. Keller said his paper had misled its readers. He said — and I agree — that there can be no greater sin in journalism.

It turned out that Miller’s source for her stories was not in the intelligence community nor in congress, but was one Lewis ‘Scooter’ Libby, a senior official in the office of Vice President Dick Cheney. Libby was later charged, convicted and sentenced to 30 months in prison for lying to investigators over his role in the Valerie Plame affair, which involved the leaking of the fact that Plame worked for the CIA.

The rules of engagement between journalists and politicians in many ways determine what we, the community, get to know about what’s going on. As the Miller affair illustrated, those rules, especially when it comes to the treatment of anonymous sources, background briefings and off-the-record quotes that can only be used without attribution, are problematic at best.

In Australia, journalism has been damaged by the way federal politics has been reported over the past 18 months. More specifically, it has been damaged by the reporting of the leadership contest between Kevin Rudd and Julia Gillard. Those of us who are not political insiders have been told untruths if not outright lies. Journalists have failed to tell us what is really going on in federal politics. This explains why people have been baffled and angry ever since Kevin Rudd was deposed in June 2010. Rudd’s political execution seemed to happen suddenly and without explanation. Gillard and her supporters then essentially lied about why Rudd had been dumped.

In 2010, journalists in the main reported these untruths — this spin — without comment and without challenging the veracity of what Gillard and other senior ministers were saying. Gillard has now admitted that she did not tell the truth in 2010. Why wasn’t she called on this back then?

The rules of engagement between journalists and politicians, in my view, encourage dishonesty from politicians and timidity from journalists. And the community is left in the dark about what those rules are and even why they exist.

Last week, Julia Gillard, having been accused by Kevin Rudd of disloyalty when she was Deputy Prime Minister, said she was wanted to free journalists from any obligation they felt they had about confidentiality and report any instances when she had briefed them in any way that was critical of Prime Minister Rudd. When Rudd asked whether he too would free journalists from confidentiality undertakings and allow them to report his briefings — if there were any — against Gillard over the past 18 months, he simply said journalists should stick to their ethics.

Most people would have found this reply confounding. What could he have meant? Only those who were party to the rules of engagement would have understood what the hell he was talking about.

Some journalists covering the Rudd-Gillard leadership struggle have, in essence, been lying to us. Not just in the past couple of weeks, but for months. Are there journalists who were briefed by Rudd and his supporters, from 2010 all the way to the weeks leading up to the leadership challenge, about Rudd’s long-term strategy to win back the prime ministership? And did Rudd or his supporters leak against Gillard during the 2010 election campaign and were they therefore responsible for the fact that the government failed to win a majority of seats?

It seems to me that on the evidence publicly available and the evidence of gossip among journalists, that the answer to these questions is: yes.

If so, there are journalists who have retailed lies. They have done so on the basis that any briefings they were given by Rudd and his supporters, any leaks they were party to, were received anonymously or even as ‘background’. That’s what Rudd meant when he said journalists had to behave ethically.

This is ‘he said, she said, they said’ journalism — what one senior journalist has called transactional journalism. It is meant to be ‘straight’ but is in reality timid and ultimately dishonest. The rules of engagement in Canberra no longer serve the public interest. They encourage and support dishonesty from politicians and timidity and, yes, dishonesty from reporters and commentators.

The rules of engagement protect ‘insiders’ and keep the rest of us, we poor punters with no access to ‘secrets’, more or less in the dark about what’s really going on. As a result, there is now a great divide between the political elite and the rest of us know- nothings, who sense that we are being fed bullshit but have no way of proving it.

At the heart of the problem is the issue of anonymous sources and the ethics of guaranteeing them protection no matter what happens. But if the consequence of this is that journalists are forced to retail lies, how can this be ethical?

In August 2007, three journalists decided that they would reveal the fact that in 2005 Peter Costello had, at a dinner with the three journalists, outlined for them — off the record — his plan for challenging John Howard for the leadership. They said they did this because Costello was not telling the truth about where he stood on challenging Howard. This was two years after the dinner and on the eve of the 2007 federal election.

The fact that they waited so long, the fact that the timing seemed so sensitive, meant there was little discussion about the ethical issues raised by their decision to report the Costello dinner revelations. It’s a conversation we need to have now. The rules of engagement need to change. Sometimes, anonymity for sources makes sense. So does off-the-record briefing and off-the-record quotes. But when politicians make on-the-record statements which contradict what they have told journalists on the basis of anonymity, I believe journalists are no longer bound to protect sources.

This needs to be made clear publicly by journalists. The community needs to understand the rules of engagement. Otherwise, journalists will continue to have their work treated with suspicion if not contempt by the community they are meant to serve.

Michael Gawenda is a senior fellow at the Centre for Advanced Journalism at Melbourne University and a former editor in chief of the Age.