Pre-Emphasis for 1.2 Tb/s DP-64QAM Transmission Simulated in OptiSystem

We investigate analog and digital pre-emphasis for ultra-high-bit-rate coherent dual-polarization 64-QAM (DP-64QAM) transmission using OptiSystem. Two representative single-wavelength configurations are studied: 64 Gbaud (600 Gb/s payload, 768 Gb/s line rate) and 100 Gbaud (1000 Gb/s payload, 1.2 Tb/s line rate). The transmitter employs raised-cosine pulse shaping (roll-off 0.1) and a 9-bit DAC, while the receiver uses a 9-bit ADC; bandwidth-limiting Bessel/Gaussian filters emulate practical transmitter (Tx) and receiver (Rx) front-end constraints. Analog pre-emphasis (APE) is realized by uploading a measured analog filter response immediately after the DAC to compensate high-frequency roll-off. Digital pre-emphasis (DPE) is implemented before the DAC as a finite-impulse-response (FIR) pre-distortion stage, with taps obtained from the measured frequency response via spectrum mirroring, inverse FFT, Hamming-window smoothing, and normalization. We compare four cases: (i) ideal reference without bandwidth limits; (ii) bandwidth-limited without pre-emphasis; (iii) APE; and (iv) DPE. Bit-error-rate–versus–optical signal-to-noise ratio (OSNR) results show that both APE and DPE substantially mitigate bandwidth-induced penalties and approach the theoretical bound, reducing the OSNR gap to 5.8 dB at 64 Gbaud and 6.6 dB at 100 Gbaud, with operation near the forward error correction (FEC) threshold (BER=10−2). While DPE offers full programmability, it increases peak-to-average power ratio (PAPR) and may require additional gain headroom. Overall, APE provides an effective rapid-prototyping step prior to DPE deployment, confirming the feasibility of 768 Gb/s and 1.2 Tb/s DP-64QAM links with commercially realistic components, including a 150 GSa/s DAC operating at 1.5 samples/symbol for 100 Gbaud.