In a large bowl, use an electric mixer to beat the butter and granulated sugar on medium-high speed until well-combined and smooth, about 3 minutes. Add the eggs, vanilla extract, and almond extract and beat until just combined, about 30 seconds. On low speed, slowly add in the all-purpose flour and baking powder and beat until incorporated.

Divide the dough into 2 equal portions. Place one dough portion between two sheets of parchment paper and roll out to 1/4-inch thickness. Repeat with the remaining portion of dough. Place the two sheets of dough on a baking sheet and chill for at least 1 hour or up to 1 day. You can also freeze for 30 minutes.

Preheat the oven to 350°F. Line large baking sheets with parchment paper or silicone baking mats.

Remove the dough from the fridge and cut into shapes with a cookie cutter, rerolling any scraps and cutting more shapes. Transfer the shapes to the prepared baking sheets.

Freeze for 15 minutes, or until firm.

Bake for 10 minutes, or until the cookies are set and beginning to brown at the edges. Be careful not to over-bake. Let the cookies cool for 5 minutes before removing to a wire rack to cool completely. Baked un-iced sugar cookies can be frozen in an airtight container for up to 1 month.

Make the icing by whisking together the powdered sugar, vanilla extract, and water or milk in a medium bowl until smooth and of desired consistency, about 2-3 minutes. Decorate the cooled cookies as desired.