Machine Learning and Artificial Intelligence in Neurosurgery: Status, Prospects, and Challenges

INTRODUCTION AND OVERVIEW: THE ROAD TO ARTIFICIAL INTELLIGENCE “Create a model that is as sophisticated as the problem requires – but not more so.” Craig MacDonald1 The purpose of this article is to introduce artificial intelligence (AI), machine learning (ML), and related technologies to neurosurgeons, to review their current status, and to comment on the trajectory for their incorporation into neurosurgery. On the order of 25 studies explicitly utilizing AI technologies have been published in the neurosurgical literature to date. This is only the beginning.2-6 The conceptual origins of these technologies can be traced to ancient legendary androids, humanoid automatons, and mechanical imitations of animals, some of which were bestowed with a form of intelligence and even will. The history leading to AI is summarized in Table 1. Mechanical calculators have been deployed throughout recorded history. Charles Babbage, a Cambridge mathematician and inventor, conceived and designed the first computers between 1833 and 1871 to address mathematical calculation errors, but they were never successfully built. In fact, his efforts were later described as a “false dawn.”7 A number of other computing machines were created prior to World War II and proved essential to the war effort.8 One problem that had to be resolved before computers could be commercialized was command and data storage. Another was cost. In 1950, Engineering Research Associates introduced magnetic drum storage, and in 1952 both UNIVAC and IBM introduced magnetic tape.9 Both advances were seminally important. In 1950, A.M. Turing published a sentinel paper exploring whether computers might be construed as thinking. He proposed a test to determine whether they could.10,11 A wide-reaching debate ensued around whether he was right and whether, in any case, machine intelligence (as it then was called – and the term the British retained in preference to AI) was anything worth pursuing. The introduction of the term “artificial intelligence” is ascribed to John McCarthy of Dartmouth, Marvin Minsky of Harvard, Nathaniel Rochester from IBM, and Claude Shannon from Bell Telephone Laboratories who submitted a proposal for a “2 month, 10 man” workshop in 1955. When the meeting took place at Dartmouth in the summer of 1956, Allen Newell, Cliff Shaw, and Herbert Simon presented Logic Theorist, a program funded by the Research and Development (RAND) Corporation designed to mimic human problem-solving skills. This served as proof of concept for AI, and is generally acknowledged to have launched the field. Research programs emerged at Massachusetts Institute of Technology, Stanford, Carnegie Mellon University, and other institutions. Many were supported by the Defense Advanced Research Projects Agency in the US Department of Defense.12 As the field evolved, so did debates about whether AI should pursue the simulation of human reasoning, or whether the simulation of human performance sufficed. Those interested in simulating human reasoning were called neats and those focused on performance were known as scruffies.13,14 By the late 1960s, researchers began applying AI-based tools to the natural and physical sciences. This work crossed multiple scientific disciplines. One early collaboration involved the geneticist and Nobel Laureate Joshua Lederberg, the computer scientist and Turing Award Laureate Edward Feigenbaum, the chemist and National Medalist of Science and Technology Carl Djerassi (inventor of oral birth control), and the mathematician, computer scientist, and philosopher Bruce Buchanan. The project was DENDRAL, commissioned by National Aeronautics and Space Administration to encode the knowledge of expert chemists in order to infer the structures of organic compounds from mass-spectral data.15 DENDRAL was significant not only for the huge volume of data processed, but also for its emphasis on knowledge engineering (KE), a complementary field that became central to AI. KE included knowledge acquisition, validation, and representation (mapping and encoding into a knowledge base); TABLE 1. - Milestones Along the Path From Robots to AI in Medicine Pre-1946 Automatic machines and calculating device but not AI. Wondrous ancient automata described 1920s The word “robot” replaces the word “automaton” 1928 Eric, a battery-powered, aluminum-skinned robot with 11 electromagnets and a motor that could move its hands and head and be controlled remotely or by voice presented at the Model Engineer's Society in London 1930s Industrial robots introduced in the United States 1939 Elektro, a 7-foot tall, walking, talking, voice-controlled, humanoid robot weighing 120 kg presented at the World's Fair. It could smoke, speak 700 words and move its head and arms 1949 Manchester Mark 1, first stored program computer, installed. Named “The Electronic Brain” 1950 Alan Turing writes “Can Machines Think?” 1955 Logic Theorist – first AI program presented and funded by the RAND Corporation 1956 Dartmouth Summer Research Project on Artificial Intelligence 1963 DARPA funds AI at Massachusetts Institute of Technology 1965 Edward Feigenbaum introduces expert systems at Stanford (The Heuristic Programming Project) 1968 The famed science fiction writer, Arthur C. Clarke, predicts that by 2001, machines will be smarter than humans 1970s Automated, computer-assisted EKG readings 1973 Image analysis of digitized retinal angiography 1973 Expert system assistance for renal disease 1978 Mirsky and others predict no more than 3 to 8 years before human intelligence is surpassed by computers 1978 CASNET introduced for expert system computer-assisted diagnosis of glaucoma 1981 The PC is introduced with the PC DOS operating system 1980s Early investigation of machine vision adaptations to medical image analysis 1983 Two expert medical systems, the “Internist-I” and “Cadeuceus” introduced 1988 Computer-assisted resection of subcortical lesions 1988 Automated computer-assisted detection of peripheral lung lesions 1990 Human Genome Project begins 1997 An IBM computer defeats Gary Kasparove in chess 1997 Dragon Software introduces first public speech recognition system 1998 Image Checker computer-assisted diagnostic system for mammography introduced 2000 Proliferation of cheap storage and increasing computer power 2000 Introduction of DL for medical applications 2004 Early reports of computer-assisted diagnosis of retinal disease 2007 IBM Watson introduced 2010 Passage of the Patient Protection and Affordable Care Act. EMRss proliferate 2010 Computer-assisted diagnosis in endoscopy 2011 Digital assistant introduced commercially 2012 Computer-assisted segmentation of sectional brain images 2012 Computer-assisted brain tumor grading 2017 Chatbots introduced for patient intake 2018 AI trials for gastroenterology diagnosis begin 2018 FDA approves Viz.AI, AI-assisted clinical decision support system for stroke triage 2020 Stacked neural networks applied to EKG interpretation EKG, electrocardiogram; EMRs, electronic medical record. inferencing (inferring answers for the user from the knowledge stored); and explaining what information was needed or how conclusions were reached.16 Public and corporate interest in intelligent computing systems expanded during the 1980s as access to computers proliferated. AI was expected to revolutionize society. Health care was a principal focus of these expectations.17-23 Experts bickered about whether medical informatics ought to be considered as a field of computer science, engineering, or biomedicine. The intellectual identity of and control over the field mattered more than might otherwise be imagined. Edward Shortliffe, editor of the Journal of Biomedical Informatics, an early and influential contributor to AI, curtailed the debate by proposing that AI be classified as a form of biomedical informatics. This was the view that prevailed and allowed for effective crossdisciplinary collaboration.14,24,25 AI suffered from inadequate computer storage and inadequate computer power. Faith in the ability of AI to deliver solutions diminished. AI was widely perceived to have failed. A period characterized as “AI winter” descended. The field of AI became notably self-reflective, perhaps because of the sociological and ethical implications of reproducing human intelligence.14,26-28 AI was revived by the advent of the Human Genome Project in 1990. The Genome Project was accompanied by a mandate for tools to handle an unprecedented eruption of data.29 The demands of an enormous data flow reinvigorated interest in data analytics and catalyzed the development of the AI-related technologies.30-33 In 1997, scientists at the National Aeronautics and Space Administration described the problem of datasets too large to be stored in a computer's main memory. This restriction severely limited the extent of data processing that was possible.34 The scientists introduced the term “big data” to describe datasets of this magnitude. In 2001, Douglas B. Laney of the Gartner Group enlarged on this concept, characterizing big data as “high-volume, high-velocity and/or high-variety information assets that demand… innovative forms of… processing…” Volume, velocity, and variety came to be known as the 3 “Vs,” alluding to the amount of low density, unstructured data (volume); the rate at which data must be processed (velocity); and the diversity of data encountered (variety).35 Three more “Vs” were added later: value, variability, and veracity.36,37 The challenge of confronting big data is balanced by the promise of unearthing novel insights that are not otherwise accessible. DEFINITIONS AND TECHNOLOGIES The terms data mining (DM), algorithm, AI, ML, deep learning (DL), neural networks, and expert systems are defined and described in Table 2.38-42 The terms may sometimes overlap or be defined with slight variation. TABLE 2. - Definitions of Essential Terms Term Abbreviation Definition Artificial intelligence AI Describes the abilities of computer systems able to perform tasks that otherwise normally require human intelligence. Algorithm n/a Sets of rules or processes to be followed in making calculations in other problem-solving operations. Data mining DM A field of computer science focused on the properties of datasets. It can extract rules for algorithms, for example, from data. A prerequisite for other forms of data processing. Machine learning ML A subset of AI that allows systems to create algorithms capable of modifying themselves by reading structured data without human intervention after being trained, and of improving from experience without being programmed explicitly. Almost always requires structured data. If outputs are wrong, the algorithms need to be retrained by humans. Deep learning DL A subset of ML which uses neural networks and multiple layers of algorithms to reiterate a task and learn progressively in order to gradually improve outcomes. Depends on adequate but not necessarily structured data. Mimics human learning more closely. May still produced flawed outputs if the quality of data is insufficient. Neural network NN Also known as artificial neural network, and is used to describe a series of algorithms that aim to recognize underlying relationships in a set of data through processes that mimic the way the human brain operates. The algorithms adapt appropriately to changing inputs. Sometimes construed as the next evolutionary stage of ML. Expert systems n/a Refers to software programmed using AI techniques to offer advice or make decisions in such areas as medical diagnosis where the judgment of human experts is emulated. ML constitutes a subfield of AI. DL is sometimes classified as a subfield of ML (and therefore of AI) and sometimes of neural networks. Neural networks are the basis for DL algorithms. The “deep” in “deep learning” refers to the depth of the node layers in a neural network. A node is the equivalent of a neuron with multiple dendrites providing inputs for processing (translation) and an axon transmitting the output. A DL algorithm by definition must have at least 3 nodes.43,44 Effective approaches may draw from AI, ML, DL, or a combination, depending on the specific application. We adopt the term “Intelligent Computing Systems” (ICS) to refer to AI, ML, DL, neural networks, and expert systems as a group, but refer to each technology by name when discussing them individually. While the earliest clinical applications of ICS revolved around expert systems for decision support, their ambit has broadened as technologies have evolved.17-23 The number of publications invoking medical ICS burgeoned from 596 in 2010 to 12 422 in 2019.23 About 51% of all papers in the field involve or invoke DM (not ICS, strictly speaking, but closely related) and ML.28,45-50 DM AND KNOWLEDGE EXTRACTION DM is defined formally as a field of computer science focused on the properties of datasets and enabling the examination of large datasets to elicit correlations and patterns that may not be evident otherwise. It has the same practical meaning as “knowledge extraction,” a term used to refer to the automated or semiautomated extraction of useful information from structured and unstructured sources.51 Knowledge extraction was trademarked “Database Mining” by the Hecht-Nielsen Neurocomputer Corporation in San Diego in the 1980s, but similar terms had appeared earlier.52-54 It also became known by an alternative designation, “Knowledge Discovery in Data (KDD),” introduced in 1989 and was framed as follows: “The basic problem addressed by the KDD process is one of mapping low-level data (which are typically too voluminous to understand and digest easily) into other forms that might be more compact (for example, a short report), more abstract (for example, a descriptive approximation or model of the process that generated the data), or more useful (for example, a predictive model for estimating the value of future cases). At the core of the process is the application of specific data-mining methods for pattern discovery and extraction.”55,56 The term KDD was initially preferred by academics, but it has been largely eclipsed by the later term, “data mining” (see Table 3).57 TABLE 3. - DM and ML120 Differences DM ML 1. Scope DM explores the properties of datasets. Large datasets are analyzed to elicit or confirm correlations and patterns of significance that may be useful inherently, or applied to the prediction of outcomes or actions. The output of DM is an input for ML. ML is a branch of AI that automatically improves the accuracy of the algorithms on which it depends to analyze inputs without being explicitly programmed to do so. ML depends on DM methods. 2. Methodology DM is a method of eliciting useful information from complex datasets. ML automatically uses training datasets to improve its complex data processing algorithms. 3. Uses DM is a research instrument primarily directed at eliciting information. ML is a tool primarily directed at predicting outcomes. 5. Method DM is typically deployed to analyze data in batches. ML algorithms typically run continuously. Changes in input data patterns can be incorporated without reprogramming or human interference. 6. Nature DM requires human direction to choose and apply techniques to extract information. ML is designed to proceed automatically. 7. Learning capability DM is a manual technique insofar as it requires analysis to be initiated by a human being. ML uses the same techniques as DM to automatically learn and to adapt to changes. 8. Implementation DM involves building models to which particular correlation and pattern evaluation techniques are applied. ML uses AI, neural networks, and automated algorithms to accomplish its objectives. 9. Data needs Relative to ML, DM can produce results on lesser volumes of data. ML requires large amounts of data presented in standardized format. ALGORITHMS AND AI AI depends on the deployment of algorithmic software routines designed for data analysis and developed on the basis of large sets of structured and curated data. During development, algorithms are “taught” on training datasets and then tested against testing datasets. Algorithm outputs are then assessed, after which the algorithm can be rejected, modified, or adopted. Poor training sets and misleading inputs can lead to misleading results. Algorithms can be standalone or embedded in devices, and or algorithms do not to with and to be by the algorithms may or as data are processed and may be by the that the algorithm is capable of and is an but a challenge from the because the algorithm first presented for may be from the into Algorithms for medical must be tested prior to using called and The process of and is in Table and do not necessarily to clinical TABLE - Software and of test whether software early in development whether the software the needs and in the and software and the testing and user and testing on Software and and not focus on Software and and software essential Essential on and of testing a system with no prior knowledge of its and testing the of the applications of AI in or data as inputs. The AI algorithms are on models that process the data to produce a a or a an algorithm might be designed to the that a of on a is with a whether the is or a form of that in and became in the of clinical In are the basis for what is to as clinical In the performance of AI on a diagnostic task is tested against a judgment the This test of its clinical its clinical its clinical value, its are an essential of predictive ML ML is a branch of AI which and algorithms. The term was first used by Arthur in to describe algorithms the ability to learn without being explicitly computers to ML is the basis for advances image and While ML may to be a of all algorithms, it is not always for devices, as ML not or DL DL is a subset of ML. It has promise in from to and voice Many that DL is the next in ML and that it will have an in in the applications of DL have been early and largely AI and other forms of ML, DL is a may be The software and algorithms in ICS may be by and An of the intellectual for software in the US is in Table The for software the are in the TABLE 5. - of Protection Term years from the of the earliest US or application to which is an to or an or an in the to An must be or to be and not and to than or useful of or any and useful is May not be a discovery (as to an An must be and not and other in be medical devices, may be by both and used with or in medical devices, may for and of the have or work of be into a and must be must an and not an and computer and The around software is and may in the from the and the can can or but must be in other than the but about the it is to will be any or any combination, used or to be used to and the of one or from those of and to the of the and on and for not is in the US for and The is not or by any ICS Many studies have the quality and value of ICS in the and interpretation of large datasets and in clinical decision AI has been successfully deployed for applications from image interpretation to the of and glaucoma diagnosis in decision support in and and clinical The ability to improve outcomes and is widely The National of Medicine to and detection as the The of AI in and is also The introduction of ICS into medical has been than A review published in 2020 that only devices, devices, and 10 medical from the US and Administration explicitly neurosurgical were At least a of the problem is with the and of the FDA The FDA is with software that as a medical that may be deployed on or other computing or that has a in the or control of a ICS for will be to by the software those for research may be or even not at medical device software are not to review to introduce in the software are in the early testing may have the form of a but with or called are still not to be even after they are launched This from that by the medical device where and are more and are by a using computer is a of some computer It can be by human before being into in machine which is only by the can be to the program and is generally and with that the user from to or in is public and designed to to to improve Software are about the with which their may be or if the is published and widely that the is to that in the process may be one of the in the of medical to clinical designed to be or embedded into a medical algorithms must generally be to for This is a process for and may be perceived as a to intellectual the with algorithms and ML, the FDA has This for to algorithms on the basis of learning and still to and the for by the FDA have been by device A of using ML and AI through a 8 the and through a process have the in order to set a and ICS – TO THE As of only neurosurgical studies using ML or This number may not be of the research because papers to the or neurosurgeons, the neurosurgical The for this number may to in and neurosurgical as as preferred research in the clinical image and and clinical trials have of applications for and are commercially Almost all can be from that It is to have automated and and elicit a AI and DM applications the history against medical expert systems support diagnostic processing solutions and systems offer accuracy and in and technologies in machine and image the resection of and later machine vision and expert systems offer analysis in or at least at systems and and medical ML and control against to the medical intelligent and AI technologies support and clinical Table some principal and clinical applications and applications TABLE 6. - and Uses of ICS in applications Automated analysis trials Computer-assisted review Image applications system Human Electronic medical analysis systems Chatbots for Patient care decision support control systems tasks ICS ICS both and scientific The to and The of and are an because the of algorithms used in ICS is not necessarily in It is to determine a DL model a specific As a are that may clinical have been around the need for when ICS is The ethical implications of ICS are only to be are or framed as a form of with solutions than as While this is the of the current it is to increasing as ICS The of ICS in has to be are areas in which ICS particular the data analytics big decision support, diagnostic image and clinical trials and It is to and describe the algorithms that are ICS are to of and in the same way as other medical the of AI in on and or into algorithms. with a of the that may will require as AI more widely This did not any or The have no or interest in any of the or described in this from in the