Milford Sound (actually a Fjord) on the southwest coast is justly famous and definitely worth a visit on a complete tour of New Zealand. There are cruises that take you out to the Tasman Sea and by many waterfalls.