Vortex solitons in atomic-molecular Bose-Einstein condensates with a square-optical-lattice potential

We propose a theoretical framework, based on the two-component Gross-Pitaevskii equation (GPE), for the investigation of vortex solitons (VSs) in hybrid atomic-molecular Bose-Einstein condensates under the action of the stimulated Raman-induced photoassociation and square-optical-lattice potential. Stationary solutions of the coupled GPE system are obtained by means of the imaginary-time integration, while the temporal dynamics are simulated using the fourth-order Runge-Kutta algorithm. The analysis reveals stable rhombus-shaped VS shapes, with topological charges m = 1 and 2 of the atomic component. The stability domains and spatial structure of these VSs are governed by three key parameters: the parametric-coupling strength ( χ ), atomic-molecular interaction strength ( g 12 ), and the optical-lattice potential depth ( V 0 ). By varying χ and g 12 , we demonstrate a structural transition where four-core rhombus-shaped VSs evolve into eight-core square-shaped modes, highlighting the nontrivial nonlinear dynamics of the system. This work establishes a connection between interactions of cold atoms and topologically structured matter waves in hybrid quantum systems.