Software and Hardware FPGA-Based Digital Watermarking and Steganography Approaches: Toward New Methodology for Evaluation and Benchmarking Using Multi-Criteria Decision-Making Techniques

Evaluating and benchmarking software and hardware field programmable gate array (FPGA)-based digital watermarking are considered challenging tasks because of multiple and conflicting evaluation criteria. A few evaluation and benchmarking techniques/frameworks have been implemented to digital watermarking or steganography; however, these approaches still present certain limitations. In particular, fixing some attributes on account of other attributes and well-known benchmarking approaches are limited to robust watermarking techniques. Thus, this study aims toward a new methodology for evaluation and benchmarking using multi-criteria analysis for software and hardware “FPGA”-based digital watermarking or steganography. To achieve this objective, two iterations are conducted. The first iteration consists of two stages: discussing software and hardware “FPGA”-based digital watermarking or steganography to create a dataset with various samples for benchmarking and discussing the evaluation method and then discussing the test for software and hardware “FPGA”-based digital watermarking or steganography according to multi-criteria evaluation (i.e., complexity, payload and quality) to create a decision matrix. The second iteration applies different decision-making techniques (i.e., SAW, MEW, HAW, TOPSIS, WSM and WPM)) to benchmark the results of the first iteration (i.e., software or hardware FPGA-based digital watermarking or steganography approaches). Then, the discussed mean, standard deviation and paired sample [Formula: see text]-test results are used to measure the correlations among different techniques based on the ranking results. The discussion findings are described as follows: (1) the integration of developer and evaluator preferences into the evaluation and benchmarking for software and hardware FPGA-based digital watermarking or steganography, (2) the process of assigning weights and (3) visualizing large-scale data sample in either software or hardware FPGA-based digital watermarking or steganography algorithms.