Cream together butter and sugar until fluffy. Grate peel from lemon and add to creamed mixture. Squeeze juice from lemon and set aside. Add eggs, flour, baking powder, salt and milk to creamed mixture and blend well. Stir in nuts.
Turn batter into greased 9x5-inch loaf pan and bake at 350 degrees one hour. Combine lemon juice and confectioners' sugar and mix well. Pour over hot loaf and allow syrup to sink into bread. Turn loaf out onto a rack and cool.