If you are looking for recreational riding you'd do well to visit the
NZ Pony Club website which caters for riders of all ages.
They'll also be able to help you with pricing or the actual pony/horse, grazing and vet fees. Horses are generally grazed year round and depending on which part of
NZ you are interested in grazing may be available locally.