HEy Jeff,
I would suggest a rock pick or some other type of prying tool. Normally knappable stone is also associated with very thick, compacted clay that holds on to every piece of stone. You'll need a stout tool to remove any sizable pieces.
Sledge hammers tend to be too hard, and when you hit a piece of stone with it, it tends to shatter the stone. You'll probably need some large hammerstones (basically hard, round river cobbles) that you can use to break open the stone.