Let's review.
Earlier you should have looked up and saw the scaffolding overhead.
You clicked and know there's an active spot.
The you get the ladder that you can use here - the one that is in the stone building NOT the one that is in the unfinished dining room inside the house. That ladder is short and can only be used in the boiler room.
Once you know you need a ladder at the scaffolding and it is the right time to go up there - then you can place the ladder from the stone building on the scaffold. If not time yet, then you have to wait. The reason for this is that there are a lot of inventory items that can be picked up and used; so the game limits what can be picked up only for that chapter.
The game also tells you where you might need an item as a clue for later on.
