There are banks all over even in small places, but why not use your debit card ? There are loads of ATM machines everywhere and it would save you having to carry a lot of cash. If you get an account with Nationwide you don't even have to pay a fee.
When I went to NZ I took a couple of hundred pounds which I exchanged on arrival then used my debit and credit cards. I certainly wouldn't want to be carrying large amounts of cash with me.

Hi

I was in New Zealand in May you will find Places to exchance your money in airports and most towns have an ATM at banks and some of the big service stations for fuel have them also just be aware you will have to have a $35.00 NZ departure tax to leave the country it is not in your ticket so have the cash at the end of your trip.
I hope you will like your trip there is so much to see in both islands we love NZ so much and will be back some day.

Thank you for your replies, very helpful I was aware of the departure tax but didn't know how much it was, so thank you for that. Is the $35NZ per party or per person?
We are so looking forward to this holiday

Hi again

It is per person cost I do hope you enjoy your holiday also if you are in the north island go and are up around Matakohe it is north of Auckland go to the Kauri Museum it is very good and take a cruise to the hole in the rock at Paihia we went on a cruise called the cream run it goes around all the islands and goes through the hole in the rock it was a very good day trip there is so much to see over there i could go on and on

Thanks again

The departure tax is NZ $25 per person. We were able to pay for ours at the airport hotel or there are desks at the airport. Please also be aware of strict bio-security on the way in - don't take any food, etc, as it may be confiscated and if you have hiking boots, etc, they need to be declared at customs and well cleaned.

We spent 3 weeks in New Zealand in October and loved the place. Thought you might like to see my pictures:

http://www.worldisround.com/articles/317654/index.html

Thanks for all your suggestions and tips
Maria, your photos are great thanks for sharing them