HiQ: A Hierarchical Q-Learning Algorithm to Solve the Reader Collision Problem

HiQ is a hierarchical, online learning algorithm that finds dynamic solutions to the reader collision problem in RFID systems. When the transmissions from one reader interfere with the operation of another reader, a reader collision occurs. The objective of the reader collision problem is to minimize the reader collisions experienced by RFID readers while using the minimum number of frequencies and using the minimum total time for all readers to communicate successfully. HiQ attempts to minimize reader collisions by learning the collision patterns of the readers and by effectively assigning frequencies over time to ensure neighboring readers do not experience collisions from one another. HiQ is arranged hierarchically with distributed, local control. The algorithm is based on a type of reinforcement learning called Q-learning, which is used to determine frequency and time assignments. Through repeated interaction with the system, Q-learning attempts to discover an optimum frequency assignment over time. We show that HiQ finds optimal or near optimal solutions to the reader collision problem.