Authors' response to Hughes et al. (2024)

Response to the Editor We thank the Editor for the opportunity to respond to the comments raised in the letter to the editor by Hughes et al. (2024). We also thank Hughes et al. for their interest in our paper (Pu et al., 2024), but we do not believe there are “inaccuracies and contradictions” in our paper. Below we will respond to the comments raised by Hughes et al. Considering the complexity and multidimensional nature of pain, we agree with the Hughes et al. (2024) regarding the need for a comprehensive approach to better capture the pain experiences of people with dementia. The results from our study support the feasibility of incorporating automated facial analysis into pain assessment for people with dementia. We disagree with Hughes et al.'s comment that “the authors focus their discussion and conclusions on potential limitations of the AI algorithms used in the App without providing any evidence for their claims.” In the Discussion and Conclusion sections, we first focussed on the feasibility of the PainChek app. We supported the potential of artificial intelligence-enabled real-time facial analysis for pain assessment in people with dementia. This was followed by a broader discussion on the relationship between facial expressions and pain in people with dementia, referencing previous research findings (Atee et al., 2022; Lautenbacher et al., 2018; van Dalen-Kok et al., 2018). We then discussed factors influencing the development of algorithms for automated pain assessment, such as gender, cultural background, and ethnicity, with supporting evidence from an editorial paper (Dildine & Atlas, 2019). Limitations of the PainChek app were raised in the Discussion regarding the small number of facial Action Units (AUs) (see response to Remark 3) and the use of either the presence or absence of AUs (see response to Remark 4). These limitations were carefully considered in light of the presented supporting evidence (as outlined below). Science and clinical applications can only progress if we remain critical and identify limitations so they can be addressed. Hughes et al. raised the question “how many AUs should be used for pain assessment.” We acknowledge that PainChek includes nine core AUs associated with pain. However, none of these facial expressions (or their combinations) are pathognomonic for pain and they also occur in other perceptions and emotions (Dildine et al., 2023). Moreover, Kunz et al. (2019) suggested that an absence of certain facial expressions does not mean pain does not exist, it is therefore more effective to employ a personalized approach, considering that these pain-related AUs are manifested in various combinations and clusters to express pain rather than seeking a standardized or universal pain expression. Similarly, Chen et al. (2022) mentioned pain identification is insufficient based only on the occurrence of individual AUs. Furthermore, some people with dementia may present unusual and unique pain behaviors such as grinding of teeth (Pu et al., 2023). Therefore, additional facial expressions should be considered to help distinguish pain from other emotions with consideration of individualized features. We agree that binary responses (presence or absence) could facilitate simplicity of use in clinical settings where nurses might prefer the binary scoring system over the complex ordinal scoring system. However, other commonly used tools have adopted the ordinal scoring format to assess pain in people with dementia. Examples include the Abbey Pain Scale (scoring 0–3 for each item) (Abbey et al., 2004), which is commonly used in clinical practice, and the Pain Assessment in Advanced Dementia (PAINAD, scoring 0–2 for each item) (Warden et al., 2003), which represents one of the best pain assessment tools with the strongest psychometric evidence for pain assessment in dementia (Ellis-Smith et al., 2016). Moreover, unlike the binary responses, the ordinal scoring system of each item would produce sensitivity to change/responsiveness when assessing the impact of interventions on pain in people with dementia (Husebo et al., 2016; Lukas et al., 2019). This has been highlighted as a limitation of the previous observational pain assessment tools (Husebo et al., 2016). Therefore, both the recently developed Pain Assessment in Impaired Cognition Scale (PAIC-15) (Corbett et al., 2014) and the Pain Intensity Measure for Persons with Dementia (PIMP) (Ersek et al., 2019) used a four-point Likert scale ranging from 0 to 3 to rate each item for pain intensity. Ideally, automated apps that use facial expressions to assess pain should have good clinimetric properties, including responsiveness, so it can be used to assess the effect of pain management in people with dementia. We acknowledge the multidimensionality of the PainChek app and its potential use to assess pain in people with dementia by focusing on facial expressions and behavioral indicators, such as body movements and vocalizations. However, we also highlighted that pain is an individualized experience, and each person may have different ways to express their pain via facial expressions and behaviors, including the absence of facial expressions. For example, Kunz et al. (2021) found four types of patterns of facial expressions in painful situations. Therefore, we proposed the potential of an individualized automated estimation of pain intensity using machine-learning techniques, which has been reported in some research (Martinez et al., 2017; Xu & Sa, 2021). We disagree with Hugh et al.'s assertion that the comments in ‘The limitations of the PainChek app’ and ‘Implications for Practice’ sections are unfounded and misleading. While we acknowledged the feasibility of using PainChek app to assess pain in people with dementia, we also raised our concerns in terms of the potential limitations of using the PainChek app in clinical practice with supporting research evidence as indicated in our responses to Remarks 3 and 4. Given that Hughes et al, as co-inventors, shareholders and/or paid consultants for PainChek Ltd, are not more open to acknowledge limitations and utilize them to further improve applications, we present the results of this study to facilitate their and the fields advancement in assessing pain in people with dementia. Hughes et al. questioned the study's conclusions. The results of our study indicated the potential of using automated real-time facial analysis as part of the pain assessment in people with dementia. However, it remains unclear which specific facial expressions accurately signal pain in this population, as current AI-based algorithms are predominantly developed from data obtained from limited, young, and healthy cohorts (Dildine & Atlas, 2019). These algorithms may not apply to older individuals with dementia who demonstrate emotional blunting (Lautenbacher et al., 2022; Taati et al., 2019). Moreover, there is a paucity of research examining pain-related facial expressions in clinical settings for individuals with dementia. Existing facial analysis methods primarily rely on experimentally induced pain, conducted in controlled laboratory environments with consistent front views and adequate lighting (Prkachin & Hammal, 2021). These factors cannot be disregarded as they present practical challenges to the widespread implementation of automated facial recognition for pain assessment in clinical practice, particularly in settings such as nursing homes, where residents might be uncooperative or display severe behavioral symptoms, or in areas with inadequate lighting. These concerns were highlighted in the Background and the Implications for Further Research sections. We disagree that a contradictory statement was provided regarding “the potential of automated real-time facial assessment.” Our research findings support the potential of automated real-time facial assessment for people with dementia; however, given the discussions on the number of facial expressions to be considered as indicative of pain in people with dementia (Prkachin & Hammal, 2021), the methods of calculating the pain intensity with binary or ordinal scoring format (Husebo et al., 2016), the population samples (e.g., people with dementia or not, people with different cultural background, gender, etc.) (Dildine et al., 2023), and the environment (controlled settings or real clinical settings) (Prajod et al., 2022; Pu et al., 2023), we concluded the necessity for further research aimed at improving the existing AI-based algorithms by taking into account these factors. Moreover, the development of the AI-based algorithms should be reported according to guidelines, such as the Reporting Guidelines for Artificial Intelligence in Medical Research (Ibrahim et al., 2021) and the early-stage clinical evaluation of decision support systems driven by artificial intelligence: DECIDE-AI (Vasey et al., 2022). Only through such efforts can the technology be effectively and widely implemented to support healthcare professionals in nursing homes. Hughes et al. raised the need for a rationale for the higher (adjusted) observational pain score and the pain conditions (with or without pain) of participants. The study aims to identify independent associations between nine pain-related facial expressions measured by the PainChek app using AI and the observational pain scores collected by assessors (i.e., 33 indicators of pain behaviors of vocalizations, movement, behaviors, activity, and body language). To achieve this, the adjusted observational pain scores were calculated as the total score of 33 items of pain behaviors excluding the face domain. In the data analysis, we did not define ‘a higher adjusted observational score’ as the adjusted observational pain score was treated as a continuous outcome (not a binary outcome) in the linear mixed model (LMM). Moreover, the results from LMM were similar between the adjusted pain score based on the 33 items and the unadjusted based on the total score of 42 items. The adjusted scores included all participants with pain. In this study, all participants were assumed to be experiencing chronic pain with a diagnosis of at least one pain-related condition, for example, osteoarthritis, fracture, injury, and with prescriptions of regular pain medication. We believe we appropriately used the PainChek app for the target population. PainChek was initially developed for residents who are not able to self-report their pain, such as people with moderate to severe dementia (Atee et al., 2018). In our study, the participants had an average Mini-Mental State Examination (MMSE) score of 12.1 (±8.7), indicating moderate to severe dementia (Perneczky et al., 2006). Hughes et al. asked how we ascertained the chronicity of pain. In this study, participants were assumed to be experiencing chronic pain if they were receiving regular pain medications or this was evaluated through registered nurses' proxy reports of pain. Furthermore, eligible participants included in this study had at least one diagnosed pain-related condition, for example, osteoarthritis, fracture, injury. In summary, instead of “focusing on the limitations of the PainChek app,” our results support the feasibility and the potential of AI applications incorporating automated facial analysis as part of the assessment indicators, alongside behavioral indicators, to assess pain in people with dementia. Although automated pain assessment is promising to support healthcare professionals as an additional tool for pain assessment, due to the complexity of expressions of pain in people with dementia (including the absence of facial expressions), there is a need for improvement (e.g., more personalized assessment) for better implementation of automated facial analysis of pain assessment for people with dementia in clinical settings. W. Moyle and L. Pu prepared the first manuscript draft. All authors contributed to the ideas, writing and approval of this Letter to the Editor. All authors have agreed on the final version and meet at least one of the following criteria (recommended by the ICMJE*): (1) substantial contributions to conception and design, acquisition of data, or analysis and interpretation of data; (2) drafting the article or revising it critically for important intellectual content. New funder details included. Funder name: Griffith University, DOI: 10.13039/501100001791. No funding was received for this letter. The authors declare that they have no conflict of interest in relation to this letter. Not applicable.