Non-permutation phenomena in trivariate families over F2ᵐ and resolution of a conjecture

Permutation polynomials over finite fields with simple multivariate expressions are of interest in cryptography and coding theory. Li and Kaleyski (IEEE Trans. Inf. Theory, 2024) generalized two sporadic quadratic APN permutations into infinite families of trivariate APN functions. Motivated by their constructions, we study the permutation property of a related trivariate family. We reduce its non-degenerate eight-parameter form to a four-parameter model and derive polynomial conditions that imply non-permutation. For two parameter subfamilies, we obtain exact root criteria when the odd extension degree is sufficiently large relative to the Frobenius exponent. We also give a complete computational classification over F₈³, with exactly $ 20 $ parameter choices yielding permutations for each of the two nontrivial Frobenius automorphisms of F₈. As an application of elimination and algebraic geometry, we prove that the trivariate function Cᵤ introduced by Beierle et al. (Finite Fields Appl., 2022) is not a permutation of F2ᵐ³ for every odd m≥23 and every u∈ F2ᵐ^*. This settles the remaining permutation part of their conjecture for all odd extension degrees at least $ 23 $.