Answers are grounded in this paper and cite their sources.
Developers of proprietary large language models (LLMs), like OpenAI and Anthropic, often roll out updates, some announced and others silent, impacting transparency for users. We interviewed 21 LLM users, including end-users, developers, and academics, mainly using ChatGPT and GPT4. Many reported feelings of loss of control and uncertainty, and diminished trust due to silent updates and deprecations. Participants overall felt that proprietary LLMs were similar to traditional software with regards to updates, only differing in the impossibility in full transparency and explainability. Consequently, participants felt LLM companies departed from established software update norms without justification, in some cases appealing to the novelty of AI in order to excuse bad practices. As a result, users are inclined to shift towards fixed, open-source models, reserving proprietary LLMs for prototyping. We suggest strategies that LLM providers can adopt to better uphold transparency and trust during model updates and deprecations.
No takes yet. Share an insight, caveat, or question.
Ma et al. (2024) studied this question.
Synapse has enriched 8 closely related papers on similar clinical questions. Consider them for comparative context: