LLM-based “AMIE: A Research AI System for Diagnosis, Medical Reasoning and Conversation” released as a result of a collaboration between Google Research and DeepMind. I think it’s almost a nature-level paper, but it was released first through arXiv. Some say it’s virtually a Turing test in healthcare…
The main contents include developing Artificial Medical Intelligence Explorer (AMIE), an LLM-based AI system optimized for clinical history collection and diagnostic dialogue, developing a self-play based simulation diagnostic dialogue environment to scale AMIE across different specialties and scenarios, developing a pilot evaluation baseline to evaluate medical history collection, diagnostic reasoning, communication skills, and empathy capabilities of diagnostic interactive medical AI covering both clinical and patient-centered indicators, and conducting comparisons through blind remote OSCE studies using 149 case scenarios from clinical providers in Canada, the United Kingdom, and India…
제목: Towards Conversational Diagnostic AI
Summary:
At the heart of healthcare is doctor-patient conversation, and skilled medical history listening is the foundation of accurate diagnosis, effective management, and continuous trust. Artificial intelligence (AI) systems that can have diagnostic conversations can improve accessibility, consistency, and quality of care. However, approaching the clinician’s expertise is a very challenging task. Here, we introduce Artificial Medical Intelligence Explorer (AMIE), a large language model (LLM)-based AI system optimized for diagnostic conversation. AMIE employs a novel self-play-based simulation environment with automated feedback mechanisms to scale learning according to different disease conditions, specialties, and situations. We designed a framework for evaluating clinically meaningful performance axes such as medical history listening, diagnostic accuracy, management reasoning, communication skills, and empathy. We compared the performance of AMIE with that of primary care physicians (PCPs) through a randomized, double-blind cross-study of text-based counseling with patient actors validated by an objective structured clinical trial (OSCE) method. The study included 149 case scenarios provided by clinical providers in Canada, the United Kingdom, and India, 20 PCPs for comparison with AMIE, and evaluations by professional physicians and patient actors. AMIE showed higher diagnostic accuracy and superior performance on 28 of 32 axes evaluated by professional physicians and 24 of 26 axes evaluated by patient actors. This study has some limitations and should be interpreted with appropriate care. Clinicians were limited to unfamiliar synchronous text chats, which allow for large-scale LLM-patient interactions but are not representative of typical clinical practice. Although further studies are needed to apply AMIE to real-world settings, the findings will be a milestone toward interactive diagnostic AI.
That’s one of the most interesting news about CES. This sentence seems especially meaningful. “Volkswagen will be the first mass manufacturer to provide Chat GPT as standard features for many production vehicles starting in the second quarter of 2024.”
Volkswagen will showcase its first vehicle with an artificial intelligence-based chatbot ChatGPT integrated into the IDA voice assistant at CES 2024, the world’s largest electronics fair, taking place Jan. 9-12. Going forward, customers will have seamless access to the ever-growing AI database on all Volkswagen models with an IDA voice assistant and will be able to survey content read while driving. Technology partner Cerence Chat Pro from Cerence Inc.’s Cerence Chat Pro is the foundation of a new feature that offers unique intelligent car-grade ChatGPT integration. Volkswagen will be the first mass manufacturer to offer Chat GPT as a standard feature in many production vehicles starting in the second quarter of 2024.
https://www.volkswagen-newsroom.com/en/press-releases/world-premiere-at-ces-volkswagen-integrates-chatgpt-into-its-vehicles-18048
Blog: https://blog.research.google/2024/01/amie-research-ai-system-for-diagnostic_12.html
arXiv: https://arxiv.org/abs/2401.05654
Browse: https://browse.arxiv.org/pdf/2401.05654.pdf
PDF: https://arxiv.org/pdf/2401.05654.pdf
arXiv-vanity: https://www.arxiv-vanity.com/papers/2401.05654
Paper page: https://huggingface.co/papers/2401.05654
HTML : https://browse.arxiv.org/html/2401.05654v1
Papers with code: https://paperswithcode.com/paper/towards-conversational-diagnostic-ai Prospects of thousands of authors on future AI. Results of a survey of 2778 people who contributed AI-related papers in academic journals. It was also conducted in 2016 and 2022, this time in cooperation with NeurIPS and ICML, with four times more participants than in 2022. Overall, the forecast for achieving AI milestones is 13 years earlier than in 2022.
In 2028, users of AI systems said they would not know the true reason for the choice of AI systems, and by 2028, AI systems would autonomously build payment processing sites from scratch, create songs indistinguishable from new songs from popular musicians, autonomously download and fine-tune large language models, and expect to reach 10% in 2027 and 50% in 2047 to emerge machines that outperform humans in all possible tasks….even concerns about human extinction ultimately. So, more than 70% of users said AI safety should be prioritized higher than it is today.
Jung-Woo Ha, the director of the center, will have more work to do. Let’s work together to improve AI reliability and safety.
제목: Thousands of AI Authors on the Future of AI
Summary:
The largest-ever survey of 2,778 researchers who published papers in top-notch artificial intelligence (AI) journals predicted that by 2028, AI systems would have a 50% or more chance of achieving multiple milestones, autonomously building payment processing sites from scratch, creating songs indistinguishable from new songs from popular musicians, autonomously downloading and fine-tuning large language models. If scientific advances do not stop, the chances of human-beating machines emerging in all possible tasks are expected to reach 10% by 2027 and 50% by 2047. The latter estimate is 13 years ahead of results from a similar survey conducted just a year ago [Grace et al., 2022]. However, the chances of all jobs becoming fully automated are predicted to reach 10% by 2037 and 50% by 2116 at the latest (2164 in the 2022 survey).
Most respondents expressed considerable uncertainty about the long-term value of AI development: 68.3% thought that the good outcomes of superhuman AI outweigh the bad outcomes, but 48% of these net optimists suggested a greater than 5% chance of extremely bad outcomes, such as human extinction, and 59% of net pessimists suggested a greater than 5% chance of extremely good outcomes. 38% to 51% of respondents said that the development of AI had a greater than 10% chance of bad outcomes, such as human extinction. More than half of respondents said that six AI-related scenarios, including misinformation, authoritarian control, and inequality, required “significant” or “extreme” concerns. There was disagreement as to which one was better for the future of humanity, the fast or slow development of AI. However, there was broad agreement that more priority should be given to research to minimize the potential risks of AI systems.
arXiv: https://arxiv.org/abs/2401.02843
Browse: https://browse.arxiv.org/pdf/2401.02843.pdf
PDF: https://arxiv.org/pdf/2401.02843.pdf
arXiv-vanity: https://www.arxiv-vanity.com/papers/2401.02843
Paper page: https://huggingface.co/papers/2401.02843