Privacy in stochastic control: A Markov Decision Process perspective

Cyber physical systems, which rely on the joint functioning of information and physical systems, are vulnerable to information leakage through the actions of the controller. In particular, if an external observer has access to observations in the system exposed through cyber communication links, then critical information can be inferred about the internal states of the system and consequently compromise the privacy in system operation. In this work, a mathematical framework based on a Markov Process model is proposed to investigate the design of controller actions when a privacy requirement is imposed as part of the system objective. Quantifying privacy using information theoretic equivocation, the tradeoff between achievable privacy and system utility is studied analytically. Specifically, for a sub-class of Markov Decision Processes (MDP), where the system output is independent of present and future states, the optimization is expressed as a solution to a Bellman equation with convex reward functions. Further, when the state evolution is a deterministic function of the states, actions and inputs, the Bellman equation is reduced to a series of recurrence relations. For the general MDP with privacy constraints, the optimization is expressed as a Partially Observable Markov Decision Process with belief dependent rewards (ρ-POMDP). Computable inner and outer bounds are provided on the achievable privacy utility tradeoff using greedy policies and rate distortion optimizations respectively.