Preheat your oven to 350°F (175°C).
Prepare the sweet potatoes by washing, peeling, and chopping them into chunks. Boil or steam the sweet potato chunks until they are tender, then drain and mash them.
In a large mixing bowl, combine the mashed sweet potatoes, granulated sugar, eggs, milk, melted butter, vanilla extract, ground cinnamon, ground nutmeg, and a pinch of salt. Mix until well combined and smooth.
Transfer the sweet potato mixture into a greased 9x13-inch baking dish, spreading it out evenly.
In a separate bowl, prepare the topping by mixing together the brown sugar, flour, softened butter, and chopped pecans (if using) until the mixture forms a crumbly texture.
Sprinkle the topping evenly over the sweet potato filling in the baking dish.
Bake in the preheated oven for 45 minutes, or until the top is golden brown and the filling is set.
Remove the sweet potato casserole from the oven and let it cool for a few minutes before serving.