The Brezis–Nirenberg problem for fractional elliptic operators

Let L = div ( A ( x ) ∇ ) be a uniformly elliptic operator in divergence form in a bounded open subset Ω of R n . We study the effect of the operator L on the existence and nonexistence of positive solutions of the nonlocal Brezis–Nirenberg problem( − L ) s u=u n + 2 s n − 2 s+ λ uin Ω ,u = 0on ∂ Ωwhere( − L ) s denotes the fractional power of − L with zero Dirichlet boundary values on ∂ Ω , 0 < s < 1 , n > 2 s and λ is a real parameter. By assuming A ( x ) ≥ A ( x 0 )for all x ∈ Ω ¯and A ( x ) ≤ A ( x 0 ) + | x − x 0 | σ I nnear some pointx 0 ∈ Ω ¯ , we prove existence theorems for any λ ∈ ( 0 , λ 1 , s( − L ) ) , whereλ 1 , s( − L )denotes the first Dirichlet eigenvalue of( − L ) s . Our existence result holds true for σ > 2 s and n ≥ 4 s in the interior case ( x 0 ∈ Ω ) and for σ > 2 s ( n − 2 s ) n − 4 sand n > 4 s in the boundary case ( x 0 ∈ ∂ Ω ). Nonexistence for star‐shaped domains is obtained for any λ ≤ 0 .