The pioneers of modern robotics, Norbert Wiener and John von Neumann, were both involved in the Manhattan Project which produced the atomic bomb. Wiener is recognized as having originated cybernetics, while Neumann is acknowledged to be the principal progenitor of the mathematical theory of games. They were fully aware that the sciences they were developing opened up possibilities that stretched far beyond the struggle against Nazism. Writing in 1954, Wiener mused on the power that humans were acquiring with this new knowledge:
[Humans are] playing a game against the arch enemy, disorganization. Is this devil Manichaean or Augustinian? Is it a contrary force opposed to order or is it the very absence of order itself? The difference between these two sorts of demons will make itself apparent in the tactics to be used against them. The Manichaean devil is an opponent, like any other opponent, who is determined on victory and will use any trick of craftiness or dissimulation to obtain this victory. In particular, he will keep his policy of confusion secret, and if we show any signs of beginning to discover his policy, he will change it in order to keep us in the dark. On the other hand, the Augustinian devil, which is not a power in itself, but the measure of our own weakness, may require our full resources to uncover, but when we have uncovered it, we have in a certain sense exorcised it…
For Wiener science was a game played against nature. Whether nature was a malign demiurge or a mere absence of order was left open. Even in the latter case nature exhibits a kind of intelligence, and there is no reason to rule out the possibility that machines will do so too. If nature in the form of the human species could bring forth intelligent machines, the process of evolution would continue among the machines.
In 1964, Wiener envisioned such a process:
Man makes man in his own image. This seems to be the echo or the prototype of the act of creation, by which God is supposed to have made man … What is the image of a machine? Can this image, as embodied in one machine, bring a machine of a general sort, not yet committed to a particular specific identity, to reproduce the original machine, either absolutely or under some change that may be construed as a variation?
Could a game be played between humans and machines, the effect of which would be to leave machines beyond the comprehension of their human inventors? Might the process whereby new types of machines developed come to be as much of a mystery as the act of creation in religion? Wiener thought the answer to these questions was ‘Yes’, and just such a prospect was also envisioned by Neumann:
It is not unlikely that if you had to build an automaton now you would plan the automaton, not directly, but on some general principles which concern it, plus a machine which could put these into effect, and will construct the ultimate automaton and do it in [such] a way that you yourself don’t know any more what the automaton will be.
Towards the end of his life Neumann became preoccupied with the relations of computers with the human mind. An unfinished manuscript published posthumously as The Computer and the Human Brain (1958) explored similarities and differences between the two. In a foreword to the third edition of the book, Ray Kurzweil writes that Neumann ‘define[s] the essential equivalence of the human brain and a computer’. He declares, ‘Artificial intelligence … will ultimately soar past unenhanced human thinking.’ Kurzweil has no fears regarding this prospect: ‘the purpose of this endeavour is not to displace us but to expand the reach of what is already a human-machine civilization.’ It is not obvious why Kurzweil is so sure that human purpose will prevail.
The pioneers of robotics were more sceptical. Wiener and Neumann envisaged situations arising when thinking machines could cease to be either controllable or comprehensible by their makers. Implicitly, they recognized that machines would develop by natural selection – a process without purpose or direction. Eventually humans could find themselves displaced by thinking machines they had originally created. The upshot of progress in human knowledge and invention might well be human redundancy.
Kurzweil and other scientific futurists celebrate the increase of knowledge as enhancing human power. By controlling natural processes, they believe, humans can gain mastery of the planet and even the universe. It does not occur to them to inquire who or what will exercise this mastery. Dreaming of a more fully self-aware species, they are attempting to create another version of humankind – one that reflects the flattering image they cherish of themselves as rational beings.
The icons of the prevailing faith in science came into the world as a result of the imperatives of war. Emerging towards the end of the Second World War and developing in the Cold War that followed, the new technologies of robotics and artificial intelligence were tools of human conflict. During the Second World War Wiener suggested that funds be made available for research on computers as part of a project on automatic gun control – an early example of what would prove to be a continuing interaction between war and the rise of computer-controlled machines. Later, Neumann’s work in game theory was used to deal with the strategic dilemmas that resulted when the Soviet Union acquired nuclear weapons.
It was not long before the new sciences escaped from what Philip Mirowski, in his study of their role in economics, has described as their ‘military incubator’. Theories of computation, information and dynamic systems, which had been confined to engineering and the physical sciences, were applied to the human world. It came to be believed that society could be understood using the same methods that are used to understand machines, and from there it was a small step to think that society is in fact a kind of machine. Long bewitched by the idea of a mathematical model of human behaviour, economists were captivated by the prospect.
As Mirowski writes of the spread of cybernetic thinking into economics in the decades after the Second World War: ‘If there was one tenet of that era’s particular faith in science, it was that logical rigour and the mathematical idiom of expression would produce transparent agreement over the meaning and significance of various models and their implications.’ What cybernetics offered economics was not just the power of prediction and control – though that was certainly part of the appeal of the new science – but the possibility of understanding human behaviour in non-human terms. If the economy could be modelled as a machine, the values and meanings that human beings brought to the market could be discounted. Whether they knew it or not, human actors were incidental to the operation of a system that was more rational than they could ever be. The economy was becoming a computer in which human judgement was superfluous.
Curiously, though perhaps not unpredictably, this vision of the market attracted some who had been enthusiasts for central economic planning. As one of them wrote: ‘When I think of it, it’s not such a great distance from communist cadre to software engineer. I may have joined the party to further social justice, but a deeper attraction could have been to a process, a system, a program. I’m inclined to think I’ve always believed in the machine.’ For former communists as for those who had never questioned the free market, the idea that the economy was a highly sophisticated machine was irresistible. Human labour would continue to be necessary. But with their mercurial passions and irrational longings, human beings were obstacles to the machine’s efficient functioning.
A few decades later it is no longer clear that the machine needs large inputs of human labour. Many have observed how the internet has decimated some industries and fundamentally altered others. As banking, the allocation of capital in markets, medical diagnostics and many managerial functions are automated, whole swaths of professional occupations seem close to being wiped out. It is not just the superior computational powers of computers that are eliminating these jobs. The developing capacity for pattern-recognition is displacing human judgement.
Unskilled labour is being automated, while many functions that have been assumed to require human contact will no longer do so. Robot nurses and teachers, sex workers and soldiers are ceasing to be merely the stuff of speculative fiction. If these replacements for human labour are not yet feasible, it is likely that they soon will be. Self-driving cars and telephones that interact with human voices are the front line of a rapidly advancing trend. Occupations that seemed safe because they required a level of skill or education are no longer secure.
There is no reason to expect technological innovation to stop or slow. As we are forever being reminded, the advance of knowledge is now an exponential process. Some believe computers will soon pass the Turing test – named after the great mathematician who played a vital role breaking German codes at Bletchley Park during the Second World War – and display intelligent behaviour indistinguishable from that of humans. Kurzweil may well be right in his forecast that within a decade or so computers will be joking and flirting with their users.
Economists may object that in the past technological innovation has not reduced employment permanently – as old occupations have died out, others have been born. But robotic technologies are unparalleled in their scope and reach. If an earlier burst of technological advance left behind a lumpenproletarian underclass, the current wave looks set to create a lumpenbourgeoisie. Denied any prospect of a lifelong career, lacking pensions or savings, the former middle classes can expect a life of precarious insecurity for the foreseeable future. A few may recreate the trappings of Edwardian privilege, but for most a bourgeois life of any kind will soon be as remote as feudalism.
The inherent tendency of this wave of technological innovation seems to be to render the human majority superfluous in the process of production. In a more remote future envisioned by techno-enthusiasts, human redundancy could be more complete. There is no way even a small elite will be able to keep up with the development of artificial intelligence. In the longer run the only rational course of action will be to reconstruct the humans that remain so that they more closely resemble machines. A technologically enhanced species will join in in the ongoing evolutionary advance. As for the remnants that are left behind, human obsolescence is a part of progress.
GRAY, John, The Soul of the Marionette: A Short Enquiry into Human Freedom. London: Penguin, 2015.
Portal E.M. Cioran Brasil 🇧🇷 