Lamb Holm on Wikipedia.
 

The Italian Chapel and its interior. The chapel was built and decorated at the northern end of the island by Italian POW's during WWII. HY 4884 0066. Both © Bill Henderson. This photo, © Martin Briscoe, shows something of the Nissen hut used to construct the chapel. Another view, and interior, both © Peter Morgan (2021). Link.

Some O.S. maps show a Chapel (Site of) near the southern tip of the island (Kirk Point). Maps I have access to don't mark it with the usual cross symbol, so it's difficult to give a precise grid reference, but there are some Cairns marked at HY 4828 0001 which could be associated with the vanished chapel. Had it survived, it would probably have been visible in this view, from Streetview in 2021. It's mentioned here (at the bottom of the Parish History section) as being attested to in 1595.