I’ve quit jobs in the regulated broadcast ‘impartial’ media twice in the last five years. I left Newsnight to go and work for Channel 4. I left Channel 4 to go freelance with one 900 word column a week in the Guardian.

Both times it was a shock and both times I woke up the next day and said to my partner: “I thought I was going to be terrified but I now feel like a secret policeman has been switched off in my brain.”

If you are working in TV that secret policeman should exist. But I felt it was getting in the way of the journalism I wanted to do.

No matter how brilliant you are and how independently minded you are; no matter how many times you are prepared to say “fuck you” to the entire management team to maintain your independence, creativity, inquisitiveness and risk-taking; and suffer the consequences; for some people it’s still not worth it. I want to explain why.

For me, impartiality is not an imperative of journalism. It’s a subset of truth. Truth is the imperative and truth has other subsets like accuracy and fairness.

I don’t want to impose impartiality on the Daily Mail but I would like to see them being more accurate and fair because these are subsets of truth.

We’re talking about political impartiality because that’s what it’s come to mean. I’d say that BBC and ITV Sport are quite impartial between Man Utd and Man City. That’s quite easy to do.

But history has intervened into this concept and made it hard when it comes to journalism.

When there was a two-party system, the basic guidelines of impartiality kind of made sense for broadcasters. In both the Keynesian post-war era and then the neoliberal, post-Thatcher era, you’ve basically had political establishments that think that the world is fine as it is. There’s a bit of a scratchy period in the miners’ strike and with documentaries like Death on the Rock etc. But you had a consensus.

As we know from our American colleagues who, until this year, still had that consensus, that makes really good journalism really possible within an impartiality framework. Because what you get is people from within the system – like those who people who gave the Watergate information to theWashington Post – who believe that the media has a legitimate, almost constitutional, role.

If you look at the great era of journalism in the 1960s and 1970s when John Pilger and Ken Loach were making TV documentaries, and then if you look again at the late 1990s and early 2000s, when the BBC was able to go head to head with the government on Andrew Gilligan’s ‘sexed-up dossier’ report, those were good days.

They are days that we may not see again because not only has the two-party system broken apart and not only is the UK under threat from devolutionary pressures but what has emerged is a kind of almost systemic absence of consensus that the present situation is OK.

Whether you look at Ukip on 3% today or 13% a few years ago, Ukip exists because there is a deep dissatistfaction with the thing that the two parties were trying to manage. Likewise, Corbyn exists for the same ultimate reason: the radical left has taken over the Labour Party and I would say to anybody who hasn’t noticed that it’s a big shock.

What does then mean for how impartiality rules have to work? I do want them to work, even if I don’t want to work under them.

Let’s look at some elephants in the room.

We’re sitting in a College that has a big Media Studies Department where some people learn journalism. If you go anywhere near a university degree or an A Level or even a GCSE, you will hear the word ‘ideology’.

You will hear very learned and peer-reviewed debate about where ideas come from and why they represent the world as it is that’s perfect and immutable.

But impartiality rules have no theory of ideology. In fact, broadcasters do not have a theory of ideology. You will not find any of these ideas (which are not Marxist but mainstream academic concepts) in the rules of broadcasters.

And so the first elephant in the room is: how are you supposed to be impartial when you have no theory about the ideas of society and how they are transmitted?

On top of this: broadcasters have no theory of the state. Now theories of the state of course do partly emanate from Durkheim, Marx and Weber. In general they identify the state as serving a class interest, directly or indirectly. Why is that important?

What shocked me as I moved into broadcasting in the early 2000s is how close senior broadcast people are to the state: to the Met, MI5, MI6, GCHQ and government ministries.

I am talking not just about people who cover security and I am not talking about agents of the security service – even though ex-spy David Shayler claimed there is a spy in every newsroom.

The problem is the wider mindset: that state agencies tell you the truth in general. If you believe this, then when they repeatedly  tell you lies – as during the Iraq war– you wake up the next day very pissed off. But you still don’t have a theory of why that ‘s happened.

This leads to the third elephant in the room: how close so many senior journalists have become to the state as a source of information.

It is not journalists’ fault but the fault of successive governments and the political system. For the past 30 years and we’ve had long governments – a long Tory government, a long Labour government and then a long Tory government again.

In this situation, governments become very used to manipulating broadcast media. I don’t think broadcast media – with all their impartiality regulations – have a standard operating procedure for understanding what is going on.

The result is: on some days all the top stories on broadcasters’ websites are government announcements.

And you then fly to Viktor Orban’s Hungary or Putin’s Russia and you get off the plane and you look at their media and you laugh at them. But they quite rightly pull up the BBC and say ‘show me a story that is not based on a government, or a quasi-government, announcement?’.

We end up doing stenography for the state. I don’t think that is mainly our desire but the result of the state’s ability to manipulate the news cycle.

So what do we do? The Scottish Referendum was the first time I realised how important that impartiality is for the the BBC: because the BBC sets standards of behaviour for everybody else. I thought – and wrote – that the BBC strayed into Iraq-era propaganda in the Scottish referendum. It amplified and became a vehicle for the government’s narrative.

What I want all broadcasters to do is to recognise that when people complain about this stuff, they’re not doing it because they hate you; it’s because they’re seeing, faster than you are seeing, a change in reality which you are failing institutionally to deal with; and whose risks you are refusing to understand.

The final elephant that we don’t want to talk about is the sociological and demographic make-up of decision-makers in newsrooms.

Public school, Oxbridge, passed a psychometric test? I did a survey on Newsnight where that was absolutely the case: the Coalition cabinet was being slagged off as posh and the straw poll revealed our team was posher.

And as long as you have a workforce that is public school, Oxbridge and psychometrically tested, it means most ordinary journalists can’t get into it and therefore there’s very little challenge to the status quo.

So we need big reforms across the broadcast media. However, the organisation that needs to set the standards of behaviour is the BBC. It needs fewer rules but a better understanding of the world it lives in, in order to understand what new checks and balances are necessary.

For this, it needs a theory of ideology, a theory of the state, a critical response to state propaganda and a different recruitment policy aimed at promoting class diversity and diversity of life experience.

The BBC’s bosses should ask, every day: how do those in power try to use us to create the impression that the existing power structure and distribution of wealth is natural and immutable, and what stories can we commission or pursue that resist this?

All media organisations need to realise that the collapse of consent in the centre obliges them to reflect the politics and world view of the system’s critics as normal and legitimate, not abnormal. We need to abandon the implicit view that the world is OK as it is, that the state tells you the truth, and that stenography for the government is in any way a form of journalism.

This is an edited extract of Paul Mason’s contribution to TV News: Fair, balanced, impartial? at Goldsmiths, University of London, May 5, 2017.