Most caves are formed when water creeps through cracks into limestone and eats away at it with natural acids, forming an open space inside the ground.