It is said that the separation between two charges forming an electric dipole should be small. Small compared to what?

hii nikhil !

Good question...:-

If you say that d is very small.....it doesn't mean that you are comparing d with any other parameter...it simply means that d is somewhat greater than zero...and has some value which is measurable....

in other word...smallest possible distance which cannot be zero and can be measurable...like 10^-2 mm or  10^-6 m ....

These are some distance which can be considered as small....but infact you are not comparing these with some quantity ....

Similarly if it says tha d is very large......it simply means...d is the largest possible no. close to infinty and measurable ....(can be measured with help of some scale)

I hope this is clear to you now 

Regards

Yagya