The Rise and Fall of Labour's New Left
Andy Beckett's The Searchers charts the Labour New Left from its emergence in the late 1960s until it took the leadership of the party under Jeremy Corbyn. Its analysis of the lethal and frequently destructive political skill of the Labour right offers painful lessons for today's activists.
Radical history is often an act of recovery. When not the straightforward process of documenting struggles and movements otherwise forgotten or ignored by the conventional historical record, it is the far trickier one of challenging the caricatures of superficial and incurious popular narratives. As Edward Thompson described it in The Making of the English Working Class, to write radical history is to engage in a process of rescuing radical actors from ‘the enormous condescension of posterity.’
In The Searchers, a group-biography of the five lead figures of Labour’s ‘New Left’ for the last six decades – Tony Benn, Ken Livingstone, Jeremy Corbyn, Diane Abbott and John McDonnell – Andy Beckett attempts to do both. As Beckett says, ‘in the vast literature written about the party between the mid-1980s and mid-2010s … Corbyn, Abbott and McDonnell rarely ever featured.’ When they did, they were misrepresented as either an absurd irrelevance, a collection of backward-looking purists uninterested in power, or dangerous extremists and bullying demagogues. What Beckett demonstrates, however, is that Benn and his acolytes were serious, imaginative, and committed activists, who wanted to change Britain forever, and in some areas arguably succeeded.
While from widely different backgrounds, and diverging considerably in age, Beckett’s five ‘searchers’ shared in the common radicalising experiences of the late 1960s. Benn, already a high-flying, technocratic minister in the first Harold Wilson government, was radicalised by the student revolts of 1968, even visiting an occupation in Bristol incognito. Livingstone, from his parent’s home in South London, watched the same events, glued to the television screen. For Abbott, 1968 was when she had her first major exposure to racism, in the aftermath of Enoch Powell’s ‘Rivers of Blood’ speech, and took in the inspiring energies of the black power movement. Jeremy Corbyn encountered similar as he followed the activism of Walter Rodney while teaching in Jamaica. A little while later, he read How Europe Underdeveloped Africa and visited a Chile on the brink of electing Salvador Allende. Around the same time, John McDonnell began reading the first English translations of Antonio Gramsci’s Prison Notebooks.
Benn interpreted these new radical energies, found both in the student revolts and the various ‘single issue’ social movements that soon filled the extra-parliamentary politics of the 1970s, as a demand for greater autonomy and self-determination. Tied to this was the decline of deference, and a discontent with the unresponsive bureaucracies of the welfare state. Yet far from being an individualistic threat to the left, Benn believed these new energies could form the basis of a new ‘grassroots socialism.’
The idea of a society organised around participatory democracy, in the workplace, the national and local state, and in political parties, would be the motivating political idea for the rest of Benn and his followers’ political lives. For Benn, this took the form of advocating workers control and an Alternative Economic Strategy against both the wishes of Harold Wilson and the supposedly impartial civil service bureaucracy. Nearly ten years later, for Livingstone and McDonnell at the Greater London Council, it would manifest through the generous allocation of grants to activist and community groups, and the enthusiastic promotion of identity politics and diversity monitoring. For Abbott, it came through activism with the black sections campaign, its culmination being her entry into parliament in 1987.
Each believed that the future advance of the left would require a more expansive progressive coalition than previously achieved, drawing upon the new social movements around feminism, anti-racism and queer politics, as well a more assertive rank-and-file trade union militancy, one that struggled for industrial democracy as much as the traditional union demands over wages and conditions. What Beckett identifies here as the common preoccupation of these figure of the Labour New Left is what historians have dubbed ‘popular individualism’. This growing desire for autonomy and self-determination that emerged in the post-war years was later devastatingly harnessed for right-wing ends by Margaret Thatcher. Yet alternatives, like those proposed by Benn and his followers, were available. The self-actualising politics of identity, in alliance with the workers control movement and Stuart Holland’s Alternative Economic Strategy, was the left-wing off-ramp on the road to Thatcherism.
But Benn’s plans were ultimately obstructed in Whitehall and vetoed by Wilson, despite the wishes of party conference. It was little surprise then that the struggle for workers control developed into a simultaneous struggle for members control within the Labour Party, pioneered by the organising genius of the Campaign for Labour Party Democracy. The members and trade unionists a say in the election of the party leadership, and, for a time, a chance to select or deselect their MP every term, rather than once every few decades. Here too the struggle for grassroots participation in the political process, and the direct accountability of parliamentary elites to ordinary activists, was a part of this general decline of deference and the search for greater self-determination.
****
It was on the back of this that Benn reached the height of his influence. In 1981, during a hard-fought campaign, he came within a handful of votes of usurping deputy leader Denis Healey. Interestingly, Beckett shows how victory for the left could have been clinched had Corbyn more effectively lobbied the old public sector union, NUPE; there is something poetic in the idea that he would make up for such a failure by finally leading the left to victory thirty years later.
From this initial, agonising defeat came only more. The 1983 general election, the end of the miners’ strike, the collapse of the rate-capping rebellion in local government, and the eventual abolition of the GLC: this is typically where most histories of Bennism end. Instead, Beckett diligently traces later overlooked initiatives like the Chesterfield conferences, which gathered thousands of socialists to debate a future strategy for the left, and Livingstone’s Socialist Economic Bulletin. All offered signs of continued creativity, but could do little to offset the process of decline.
Into the New Labour years of the 1990s and 2000s, when the parliamentary left most resembled the ‘Sealed Tomb’ Peter Mandelson infamously said he wished to trap them in, the Socialist Campaign Group became a site of mutual support and free political discussion in the Parliamentary Labour Party. Yet as the opportunities to wield influence in parliament receded, the left’s emphasis on politics beyond Westminster grew. For Livingstone that came in the more conventional form of the Mayor of London, making him for a time the most powerful socialist in Britain. For the others, the emphasis moved to the new protest movements, against globalisation, the Iraq War, tuition fees and austerity. Corbyn and others would, Beckett writes, make diligent appearances ‘at rallies, on picket lines, at occupations and on vigils, however tiny or seemingly futile’. It was these radical energies that Corbyn eventually ‘cashed in’ in the summer of 2015, mobilising a broad coalition of the young and old, a mixture of diehard Labour activists and those politicised by the new protest movements, to finally seize the party leadership.
[book-strip index="1"]
Beckett’s aim is to shift the standard reference points of modern British history away from the high politics of Westminster and the administrations of Thatcher and Blair. In so doing, it integrates these five ‘searchers’ and their causes, long considered fringe or irrelevant, into the wider story of Britain in the latter half of the 20th century, and the first two decades of the 21st. The struggle for gay, women’s and black liberation, for workers control and for a democratised Labour party, all formed part of the rise of popular individualism. The ability of Livingstone and Corbyn to soar through the ranks of ‘empty shell’ constituency Labour parties, hollowed out by demographic change and membership decline, forms a broader story around the immense socio-economic changes that came with the gentrification of inner London. Corbyn’s leadership campaign, and the 2017 general election, tell a wider story about the rise of alternative platforms of communication and political mobilisation through social media, in some ways the culmination of Benn’s early utopian hopes for a new citizenry in a high-tech computer age.
The most significant legacy Beckett highlights, however, is in the politics of identity. Abbott’s entry into parliament as Britain’s first black female MP leaves a lasting, pioneering legacy. Meanwhile, Livingstone’s GLC opened the doors of County Hall, often literally, to minority and community groups, and through its ambitious cultural programme altered the city’s sense of itself. London was no longer to be a ‘declining former imperial capital’ but a cosmopolitan city enthusiastically defined by multiculturalism and ‘new ways of living’. This self-perception has stayed with London ever since, while the GLC’s wider emphasis on diversity, once considered extremist, now forms part of ‘standard memos from human resources.’ For Beckett this legacy in Britain’s social politics is decisively Gramscian, the result of incremental, almost invisible changes in Britain’s everyday life.
In helping to recast this frame of reference, Beckett’s book has already done a great service. The quality of Beckett’s writing makes an already important book a joy to read, elegantly piecing together the complex, interlocking trajectories of the five protagonists, as well as a series of simultaneous political developments with remarkable clarity. It’s a skill that allows him to explain the rise, fall, rise, and fall again of the left without subjecting the general reader, or the left-wing trainspotter, to the tedious details of the Wembley conference, the Collins report, or the Falkirk scandal. Meanwhile, his vivid descriptions bring to life a world of radicalism that a reader who has only known neoliberalism (with austerity characteristics) may sometimes struggle to visualise. Beckett’s description of places and events – Benn’s cluttered home in Holland Park, the queues outside the Chesterfield Conference, the grand, cavernous rooms of the GLC, the bustling Red Rose Centre in Islington, meetings of the Labour Briefing collective – make tangible the spaces where a living, breathing left built sites of resistance to Thatcherite and New Labour hegemony.
Beckett also adds depth to the book through prolonged studies of the five’s respective personalities. Benn was far more ambitious than perhaps he ever admitted to himself, but prone to dark moods and self-doubt. Livingstone was an ingenious Machiavellian politician, at times ideologically ambiguous and addicted to controversy. His fundamental hostility to the ‘puritanism’ of mainstream society encouraged him to adopt a provocative form of social libertarianism, declaring that ‘everyone is bisexual’ and eschewing conventional family structures in his own personal life. McDonnell was disciplined but also curt, confrontational, and occasionally aggressive. Corbyn was ego-free, humble, hard-working, and conflict avoidant, sometimes to the point of evasion. Abbott was thoroughly self-reliant, which at times appeared remote, while also enduring the double-bind experienced by many ambitious people of colour: of racism, on the one hand, and accusations of careerism, or ‘selling-out’, on the other. All possessed a love of politics that could verge on the obsessive. All were stubborn to a fault, but also possessed of an optimistic and relentless energy. It was this defiant optimism that allowed them to ride out defeats and disappointments others found ideologically shattering.
****
What also made Benn’s acolytes unique was that despite being radicalised by the revolts of 1968 and their antecedents, when these radical energies gradually dissipated, they joined Labour rather than the litany of Far-Left parties that were then enjoying a brief renaissance. In this was a pragmatic recognition that while the latter would relegate them to the margins, the Labour Party remained the only vehicle that could translate their ideas and aspirations into tangible political change. While the emphasis, and frankness, varied, all were far from uninterested in power.
Avoiding Far-Left sects also gave each a non-sectarian outlook on the left, and an ability to construct wide-ranging alliances and to draw upon ideas from a variety of traditions. While only mentioned briefly in Beckett’s book, it is perhaps the result of this non-sectarian attitude that ensured that Corbyn’s leadership office included former member of the Trotskyite entryist group Socialist Action alongside former diehard Communists organised around the newspaper Straight Left.
This non-sectarianism was also typified by an eclectic approach to political inspiration. Part of this came from a surprising lack of dogmatism, despite media depictions. Benn did not read The Communist Manifesto till he was in middle age, and perhaps drew inspiration just as heavily from the romantic heroes of the English Revolution as Marx and Engels. Livingstone reportedly preferred science fiction, and gained a remarkable ideological flexibility, taking inspiration from Thatcher’s favourite economist Milton Friedman in drawing up his flagship ‘congestion charge’. His record as Mayor of London bore some remarkable similarities to New Labour’s ‘mix of business promotion and social redistribution’, yet also sought to emulate the state-capitalist model that had facilitated the dramatic rise in China’s economic power.
While Beckett focuses on the forward thinking, modernising tendencies of the Bennite left, from the miners’ strike onwards Benn became increasingly ‘preoccupied by the present and the past… trying to save as many as possible of the socialist transitions that went through them’. Yet this was not just a personal tick on Benn’s part, or a reflection of his marginality. In the 1990s and 2000s – in a trend surprisingly unmentioned in the book – the Labour left increasingly portrayed itself, in contrast to the iconoclastic New Labour, as the defenders of Labour’s traditional values and policies. This sometimes involved a more positive attitude to the post-war social democracy than had previously been the case, and the advocacy of policies more akin to that tradition than any sort of radical departure. While perhaps less surprising than the modernising perspective Beckett uncovers in the left’s earlier activity, such a position had its merits. The privatisations and deregulations of Thatcher and New Labour had eroded the base on which any radical alternative economic strategy was to be built. The left therefore had to rebuild before more ambitious plans could be pursued.
[book-strip index="2"]
This again demonstrates a degree of pragmatism and ideological flexibility on the part of the left, but also its political savvy. As the ideological departures of the New Labour years intensified, particularly in regard to welfare universalism and the defence of public services, while the later Miliband leadership accepted the main parameters of austerity, the Labour left was convincingly presented as the last figures standing in defence of ‘Labour values’. This was demonstrated most famously in 2015 when Corbyn was the only leadership candidate to not vote for the Cameron government’s welfare cap. Unfortunately, this is an area that Beckett does not explore.
****
As the book reaches the Corbyn leadership, and continues to the present, Beckett becomes a more visible observer in the story of the Labour left’s progress; sympathetic, clearly, but aloof. The result is a more of a journalistic narrative of events than the previous sections, covering well-trodden territory for any observer or participant of Labour’s last decade. This is perhaps understandable. The passage of time has yet to reach the point where a more fully formed history of the period can be written. Beckett’s account, however, does reveal some key themes for further exploration.
Much of Corbyn's appeal rested in his extra-parliamentary energy. After all, it was the election campaign in 2017, however haphazard (and partly sabotaged), that rejuvenated his public standing and saw Labour soar to 40% of the vote. Whether it was the lack of a major campaign in 2018 and most of 2019, or the sheer political exhaustion of the public after five years of nationwide elections and referenda, but ‘Peak Corbyn’ soon dissipated. In multiple ways the Corbyn project was undone by Brexit. The question of whether to back a second referendum divided even the leadership’s core supporters, leaving it with a hopelessly muddled and alienating compromise position by the 2019 election. Yet Beckett also makes the insightful point that the endless Commons intrigue sapped Corbynism’s energy, making Corbyn himself a parliamentary figure rather than an insurgent.
From the very beginning Corbyn also faced an orchestrated campaign of delegitimisation by factional opponents in the party, and by the media. Press hostility verged on the hysterical. Policies that amounted to the cautious re-establishment of post-war social democracy was portrayed as extremist. Committed, hardworking, mild mannered activists like Jon Lansman and Andrew Fisher were portrayed as bullies and thugs. The engagement of hundreds of thousands of young people in political activism for the first time was seen as a sinister development. The left’s main organisation, Momentum, described as a cult. Most damagingly of all, the accusations of widespread antisemitism (which Beckett navigates with a commendable level of tact and objectivity) allowed Labour to be portrayed, even in comparison to Boris Johnson’s Conservatives, as uniquely nasty, and potentially institutionally racist.
Lurking throughout Beckett’s book is also the threat of violence. Alongside vicious hostility from political rivals and the media, all five at various points received death threats, violent intimidation and in a few instances direct physical attack. Abbott remains the ‘most abused’ MP in parliament, including from Tory donors. While at the GLC, McDonnell found broken glass left in his children’s sandpit. When Benn’s wife Caroline was once interrupted with news during a school governor meeting in the 1970s, her first thought was that her husband had been assassinated. In the early 2000s the outside of Corbyn’s office was once covered in death threats from pro-Pinochet activists. In 2017, a man was jailed for deliberately driving a van into a crowd near the Finsbury Park Mosque – his intended target was Corbyn. Perhaps most chilling of all was the potential threat from the military and security services. In the 1970s Airey Neave reportedly plotted a coup against a possible Benn Government; in 2019 British soldiers were videoed using pictures of Corbyn as target practice. The memory of what had befallen the Allende government in Chile was a frequent point of reference for Beckett’s five, and at times even a source of dark humour.
****
In recent weeks, Beckett’s analysis of the Labour right, in many ways far more immediate rivals to the left than the various Conservative governments, has increased in relevance. Here radical history can function not only as a tool of recovery, to better inform our understanding of the past, but also shape radical politics for the present and future.
What Beckett captures brilliantly is the lethal and frequently destructive political skill of the Labour right. During the high point of Bennism in the early 1980s, John Golding waged an internal war, going so far as to welcome a landslide election defeat in 1983 in order to discredit the left. According to Beckett, ‘for the sectarians of the Labour right, saving the party required them first to destroy much of it.’ During the 2017 election Golding’s spiritual successors, dug in among the party’s bureaucracy, re-directed funds to the campaigns of factional allies, or deliberately did as little work as possible.
Yet this kind of destructive sectarianism has long been associated with the party’s left in the public imagination. This is particularly seen with parliamentary selection. As Beckett writes: ‘journalists and Labour centrists warn that a “Stalinist” cleansing of the party by the left is coming. It never really does.’ Despite the long struggle to introduce mandatory reselection in the 1980s, in the end the mechanism saw but a handful of Labour MPs deselected; a number, such as Ernie Roberts and Norman Atkinson, being from the party’s left. For a very brief period it helped alter the social complexion of the PLP, but rarely its ideological composition. Under Jeremy Corbyn, the mechanism was never re-introduced, and no right-wing Labour MP, no matter how disloyal, unpopular or intransigent, was ever deselected. Instead, purges of the Labour’s parliamentary party have been most effectively carried out by the right. As Beckett acknowledges in the epilogue, Starmer’s leadership has enforced ‘a level of ideological control not even achieved by New Labour at its most intolerant’. Corbyn had the Labour whip removed three years ago, while over the course of this parliament various left-wing MPs first elected in 2019 have been swiftly deselected. The blatant display of heavy handed and factional party management in the last few weeks, the deselection of Diane Abbott (now called off) and Lloyd Russell-Moyle, the suspension of Faiza Shaheen and the imposition of right-wing NEC members, including those directly involved in the suspension process of other candidates, brings all this into stark relief.
It appears that Starmer’s key cadres, Labour Together and Labour to Win (formerly Labour First), are waging an unprecedentedly vicious war against the Labour left. They are seeking to not only relegate its parliamentary strength to the margins, but to publicly humiliate those they have defenestrated. All of this has gone directly against the wishes of ordinary party members. While the blatancy of these purges could be interpreted as hubristic, it could also be more calculated. Through such public displays of leadership authority, it is probably hoped that more left-wingers will leave the party in disgust, depleting its strength, while those that do stay will become more timid than ever.
Either way, there’s a level of destructive cruelty on display that defies even the most hardnosed pragmatic approach to electoral politics. In the case of Faiza Shaheen it’s clear that factional allies are preferred over electability. The Labour right has always relished destroying the left in internal faction fights and has been more than happy to adopt scorched earth tactics when not in control of the leadership. But now it appears that factional score-setting even takes precedence over election victory. In a twisted irony, a right-wing leadership is having its election campaign partly derailed by its own fixers. It has dominated news headlines, overshadowed major policy announcements and appears to have gone down badly among voters, who tend to disapprove of politicians seen to have thrown a colleague under the bus.
It will not cost Labour the election, but it could set the tone for the future. For a party already failing to enjoy positive support beyond anti-Tory exhaustion, to display such a nasty, arbitrary side will not earn them much goodwill. For Starmer personally, a politician who has never belonged to any single faction, filling the parliamentary party with the most committed right-wing fixers appears deeply risky. Not only do fixers fail as thoughtful or imaginative politicians, capable of intellectual reflection or strategic renewal, but also as trustworthy allies. It’s foreseeable that a Starmer government could fall apart not due to the litany of economic and social challenges it faces, but simply due to the internal wranglings of a faction addicted to score setting and muscle flexing.
****
Lewis Minkin, the late scholar of the Labour Party’s institutions, documented in painstaking detail the ways in which New Labour’s culture of ‘party management’ often came back to haunt it. The attempts to prevent Ken Livingstone winning Labour’s nomination for Mayor of London, or Rhodri Morgan from becoming leader of Welsh Labour, all backfired and lost New Labour much of its early goodwill, and coming to be labelled as ‘control freaks’. While gradually losing millions of voters, and hundreds of thousands of party members, it also facilitated a gradual swing back to the left inside the party. New Labour’s great unintended consequences were to lay the foundations for the revival of their fiercest opponents. It is possible to imagine the same happening to a Starmer government that prides control above all else.
The failure to reintroduce mandatory reselection, to provide a powerful counterweight for party members against unrepresentative parliamentary elites, and if necessary to transform the composition of the parliamentary party, must now be seen as the left’s monumental error. In the early years of his leadership, Corbyn tried to construct an inclusive shadow cabinet, and in the years of ‘Peak Corbyn’ when such a reform was possible, the focus remained on replacing a Conservative government that seemed capable of collapsing at any moment. The creation of such a counterweight was put on the backburner. It is that error that allowed the right to mount such a comprehensive overthrow of Labour’s most radical leadership. A leadership, however flawed, frustrating and disappointing, that was the product of decades of diligent activism, inspired by the hope of a more egalitarian and democratic Britain.
If the swing back to the left should ever happen again, on the back of a disappointing Starmer government, the left might not make the same error twice.
Alfie Steer is a doctoral student at the University of Oxford, researching the history of the Labour left from the late 1980s to 2015.
