Yes thats true diwali and baisakhi are two festivals when visit to golden temple is just extraordinary.
Well 7 days are not required for amritsar depending upon the location you are travelling from.
I have done my trip from delhi which concluded in 3 days that comprises of 11 hours travel by train one side and staying for a day in amritsar.