Preheat the oven to 350°F (175°C) and line a muffin tin with paper liners.
In a large bowl, whisk together the flour, granulated sugar, baking powder, salt, and ground cinnamon.
In another bowl, combine the melted butter, egg, milk, and vanilla extract. Mix until well combined.
Pour the wet ingredients into the dry ingredients and stir until just combined. Do not overmix.
In a small bowl, mix together the brown sugar, ground cinnamon, and softened butter until crumbly.
Spoon half of the muffin batter into the prepared muffin cups, filling each about halfway. Add a teaspoon of the cinnamon filling mixture on top of the batter in each cup, then cover with the remaining muffin batter.
Bake for 18-20 minutes or until a toothpick inserted into the center comes out clean.
While the muffins are baking, prepare the glaze by whisking together the powdered sugar, milk, and vanilla extract in a small bowl until smooth.
Once the muffins are done, let them cool in the pan for 5 minutes before transferring them to a wire rack. Drizzle the glaze over the warm muffins.