Your "thought process" tells you that "there must be a better way".
Surely you don't just expect to pick ore out of clusters on the walls!
You'll need some kind of tool to mine the various gems you find.