Lamayuru is a small village in Ladakh, midway between Kargil and Leh. To see the monastery and explore the village you explicitly have to turn left and go down the hill.
.
Going straight would lead you to ladakh. That small signboard is easy to miss!