Bring a pot of water to a boil. Fill a large bowl with ice and water; set aside. Score bottom of each peach with an X. Place in boiling water until skin begins to pull away, about 30 seconds. Transfer to cold water; let sit until cool, about 30 seconds.

Preheat oven to 350°. Pull skin off peaches starting at the X. Cut peaches in half, remove pits, and slice into 8 wedges each, placing in a large bowl as you go. Put butter in a 3-quart baking dish; transfer dish to oven until butter is melted and dish is hot, about 5 minutes. Add berries, cornstarch, cinnamon, and 1/4 cup sugar to peaches; toss to combine.

In a medium bowl, whisk together flour, remaining 3/4 cup sugar, baking powder, nutmeg, and salt; whisk in milk. Pour batter into dish. Top with fruit. Bake until golden brown, about 45 minutes. Serve hot with vanilla ice cream.