From Delhi there is this direct bus service that takes about 15 hours and costs 1500 in Indian rupees (last year figures) to reach Nepal through the border near Lucknow. Alternatively, one could take a bus to kathgodam and enter Nepal from the border at Sanauli.
Last year i went nepal on a very budgeted trip, you can do the same, we went raxual by train crossed the border and reached birgunj by using any local public conveyance, they are many and easily available. Then from birgunj you will get many bus options for kathmandu.
all the best.
If you're talking about the cheapest way then what can be the cheapest way than bus service?.But wait! train up to boarder and then local bus service is also an important option to try out.Since Nepal and india are some of those few countries in the world with Open boarder you don't need to worry for any visa procedure.But still if you don't wannet to be trouble by SSB or police force at both the sides, you can carry your citizenship or other legal document like driving license issued by the Government.I guess the train to Raxaul from Delhi would be cheapest.But its not a good conditioned.You can book the direct ticket to Kathmandu from Delhi.Still if you need to reach very shortly you can try out air service which is also not so expensive.