We are growing peaches in the Far North... a bit further north than the Waikato. Picked our first Golden Queen, April White and white heritage peaches this autumn.
We did get some leaf curl on the Golden Queen, but not bad. April White is not affected by it, neither is Black Boy or my white seedling which has been grown from seed.
Don't take a "you can't grow that here" as a definite no go... experiment and see for yourself... you just might have the perfect spot for growing peaches and other stone fruit.
That is why I have a low chill cherry and low chill apricots in the orchard as well... experiments. I am keen to get hold of the Hokianga Harbour apricot that is acclimatised and does well up there.
As to the chill factor... it is not how many frosts you get that count... it is how many hours you get between 7 and 5 degrees C that really matter.
Frosts damage fruit trees coming out of dormancy and need to be watched for especially when the trees are budding and starting to flower.
We had one frost last winter which damaged the bananas but made no significant difference to the peaches.They flowered well and I removed most of the little peaches as the trees were planted one year earlier and still establishing themselves. This winter we will need to put in frost protection for the little avocados... just in case.
This coming summer all the fruit trees in the orchard will all be under timed drip irrigation through the dry period. Adequate water and some rock dust fertilizer makes a huge difference to how well they do.