Can LLMs help with XML?

LLMs have become popular for a huge range of tasks. They train over vast collections of text, yet largely discard markup of all kinds. Historically, many AI and NLP tools just stripped out not only explicit markup but even punctuation, case distinctions, or inflection. Recent work is a bit less drastic, but still commonly discards even simple notions like paragraphing, headings, block quotes, and notes. Likewise, few systems create output with any useful markup beyond paragraph breaks (if that). The conventions of written language leverage all these things, and it seems odd that LLMs should waste the opportunity of using them. “Cleaning” to avoid inappropriate text is also ubiquitous and also leads to unintended consequences. Exclusion granularity is coarse – a work may be discarded because it (perhaps critically) includes a rude quote. Just like markup “cleaning,” text selection may get minimal and non-expert attention. Censorship is often done by word-list, or worse, string -list, and even the best-known lists seem almost random. Underlying all these issues is the substitution of deceptively simple proxies for more complex and subtle goals. This is understandable especially at scale; but it has extensive and varied consequences. Such cleaning may seem to work fine, but we have little idea of precisely where, how often, and in how biased a way, it goes wrong. Figuring that out post facto is hard, in part because there is little transparency about any of these decisions and processes. How can markup (broadly construed) and LLMs work better together? Why and how can LLMs make use of structure, both going in and coming out? Can LLMs help with structure-related questions and tasks? How can users construct effective prompts in this area? And finally, how does the semantic orientation of markup relate to LLM's probabilistic rather than semantic treatment of language? This paper begins to explore such questions.