Preheat the oven to 400 degrees. Bake the potatoes until a fork inserted in center meets no resistance, about 40 minutes. Remove the skins and mash until smooth. Combine the the sweet potatoes, sugar, milk, egg yolks, butter, vanilla, cinnamon, ginger, nutmeg, and salt in a large bowl, and beat until smooth. In another bowl, whip egg whites until soft peaks form. Fold into the sweet potato mixture. Pour the batter into pastry shell and bake 10 minutes. Reduce the oven temperature to 350 degrees and bake until set, about 45 minutes. Cool at least 15 minutes before serving.