Sahih Muslim #2413 a
The Book of the Merits of the Companions 44:70
Sa'd reported:
This verse was revealed in relation to six persons and I and Ibn Mas'ud were amongst them. The polytheists said to him (the Holy Prophet): Do not keep such persons near you. It was upon this that (this verse was revealed):" Drive not away those who call upon their Lord morning and evening desiring only His pleasure" (vi. 52).