In a large bowl, beat the butter and granulated sugar on medium-high speed for about 3 minutes until smooth. Scrape down the bowl, then add eggs, 2 teaspoons vanilla, and almond extract, beating until combined.

On low speed, slowly add the flour and baking powder to the wet ingredients and beat until just incorporated. Divide the dough into two equal portions.

Roll each dough portion between two sheets of parchment paper to 1/4-inch thickness. Chill the rolled dough on a baking sheet for at least 1 hour or up to 1 day.

Preheat the oven to 350°F (175°C) and line baking sheets with parchment paper. Remove dough from the fridge, cut into shapes with cookie cutters, and transfer to prepared baking sheets.

If the dough is warm, freeze for 15 minutes before baking to prevent spreading. Bake for 10 minutes, or until the cookies are set and lightly browned. Cool on a wire rack completely.

For the icing, whisk together the sifted powdered sugar, 1/2 teaspoon vanilla extract, and 1 to 2 tablespoons of milk until smooth. Decorate cooled cookies as desired.