I grew up fishing. We had a local hardware store that sold "Cane Poles" as you mentioned. They are cheap and have no moving parts.
For starters, I would say go to Walmart and get a Zebco 202 spin-cast. You can get them bundled with the rod or separately. They are cheap and last a long time. Watch for sales since Spring is coming soon.
I would suggest learning to tie a proper fishing knot. There are instructions online, better if you can get a fisherman to show you how. Once you learn that you can tie your own hooks, lures, or whatever. If you break a line, you just tie another one on.
You will also need to buy a fishing license. You can get those at Wally World too when you get your rod and reel.
You can use a lot of things for bait: worms, crickets, grasshoppers, minnows, etc. My grandfather did well using colored mini marshmallows. I have also used peanut butter or Velveeta cheese. The type of hook varies by bait. Marshmallows, peanut butter, and cheese use a treble hook. Live baits like worms, insects, frogs, and crickets use a single barb hook.