Binky question for a 3 mo. old?

Would you try and take the pacifier away from a 3 mo. old who has been acustomed to falling asleep with it, if he wakes several times a night, and goes back to sleep only after I replace it in his mouth? He can sleep about a 5 hour stretch without waking, but after that..its every couple of hours, and he's not hungry. I've tried lying him down for his nap w/out it, but with a blanky instead...at first he snuggles with it, but a minute later is crying histericaly, and I can't see him like that...not this young.