In a large saucepan, boil potatoes, in enough water to cover, for 30 min. Or until barely tender. Drain and cool under cold running water. Peel and cut lengthwise into ½-inch slices. Place in large, shallow baking dish. In a medium-size saucepan, combine sugar, water, brown sugar, and nutmeg. Mix well. Add lemon and orange. Bring to a boil. Reduce heat and simmer, uncovered, for 20 min. Heat oven to 350°. Pour syrup over potatoes, then cover with foil and bake 1 hour.