Nonlinear Rayleigh quotient approach to a fractional p -Laplacian problem with concave–convex sign-changing nonlinearities

It is established existence and multiplicity of solutions for a fractional p-Laplacian problem in the whole space RN. More specifically, we consider the following nonlocal elliptic problem: {(−Δ)psu+V(x)|u|p−2u=λh(x)|u|q−2u+g(x)|u|r−2u in RN,u∈Ws,p(RN), where λ∈[0,λ∗), λ∗>0, N>ps and s∈(0,1) is fixed. Furthermore, we assume that 1<q<p<r<ps∗:=Np/(N−ps). The potential V is a continuous function which is bounded from below by a positive constant. The main objective of this work is to study the quasilinear elliptic problems assuming that the nonlinearity has a sign-changing function. Hence, by employing the nonlinear Rayleigh quotient, we establish the existence of at least two nontrivial solutions to our main problem for each λ∈(0,λ∗). We also consider the asymptotic behavior of solutions as λ→0. Furthermore, we assume some suitable restrictions on h and g, which ensure the applicability of the nonlinear Rayleigh quotient.