Please answer my question with appropriate solution.my answer didn't match by applying conservation of angular momentum. Answer is 2nd.

Let m1 and m2 be masses of satellites P1 and P2.

If M is mass of the earth, then

GMm1/R^2=m1v1^2/R OR

GM/R=v1^2 OR

v1=(GM/R)^1/2……..(1)

Similarly,for P2 satellite,

v2=(GM/4R)^1/2………..(2)

Then, linear velocity of P2 relative to P1 is v2-v1.

When both satellites will be nearest to each other,

angular velocity of the line joining them will be maximum. This maximum angular velocity will be

w( max)=(v2-v1)/3R= (GM)^1/2[ (1/4R)^1/2-(1/R)^1/2]/3R and for maximum distance of 5R between them

w(min.)=(GM)^1/2 [ (1/4R)^1/2-(1/R)^1/2]/5R. Then,

w(max)/w(min)=1/3R/(1/5R)=5/3