Pondicherry is a city where you don't need specific days for your stay. It depends on your schedule, i know people who stayed there for 2 nights & also for 2 weeks. As there are no specific monuments in the city, you go Pondicherry to wandered through French streets admiring "La culture Française". You can stay upto 3-4 days, there's lot to do in the city like meditation, early walks in quartier française, promenades on beach, going to Auroville and most important eating like french ; Ratatouille, Coq au vin, Crêpe, Crème brûlée, Croissants, Baguette, tasting some french wines etc etc...the list goes on. Bon Voyage
2 days would be sufficient for Pondy. Aurobindo ashram, Paradise beach, French colony are the main attractions. You can spend night along the rock beach and see the sunrise in the morning which is an experience.