Whisk dry ingredients: In a medium bowl, whisk flour, baking powder, and salt.
Set aside.
Cream butter and sugar: In a large bowl, beat softened butter and sugar with a hand or stand mixer on medium until light and fluffy, about 2–3 minutes.
Add egg and extracts: Mix in the egg, vanilla, and almond extract until fully combined, scraping the bowl as needed.
Bring the dough together: Add the dry ingredients in two additions. Mix on low until the dough looks mostly combined. If it seems crumbly, add milk or cream, 1 teaspoon at a time, just until it comes together and feels soft but not sticky.
Divide and roll: Split dough in half.
Roll each portion between two sheets of parchment to about 1/4 inch thick (6 mm) for sturdy cookies or 3/16 inch (5 mm) for a slightly softer bite.
Chill the sheets: Transfer the parchment-wrapped sheets to a baking sheet and chill for 20–30 minutes. This firms the dough for clean cut-outs and minimal spread.
Cut shapes: Preheat oven to 350°F (175°C). Cut shapes with cookie cutters, then place on a parchment-lined baking sheet, about 1 inch apart.
Re-roll scraps as needed.
Bake: Bake for 8–11 minutes, depending on thickness and size. Edges should look set and just barely golden on the bottom, not brown on top. Flat is good for decorating.
Cool completely: Let cookies rest on the baking sheet for 2 minutes, then transfer to a wire rack to cool fully before icing.
Decorate with royal icing: Make sure cookies are completely cool and dust-free.
Use a stiff icing to outline and a thinner flood icing to fill. Let dry uncovered until the surface is fully set.