Cant help with the trans related activities but am all ears for others input. My only encounter was driving through Castro and it just seemed like any other neighbourhood but with a few rainbow flags.
What you do for only a few hours though will probably be affected by your starting point and how many those few hours there are. SF can be real easy to get around on public transport, but it can also be a right pain, so if there is a bus route up to golden gate from where you are its not far away, but if you need to make a couple of changes it will seem like its in another state.
You could ride the cable car, hop off and take a look at Lombard st, maybe carry on walking down and back up to Coit tower. If your on market you could hop on the F-Train to the embarcadero and on to fisherman's wharf to look for sea lions, maybe take a look around the Boudin bakery/museum. If your in the financial district, china town is next door, head for bush and grant for the china town "gate". Maybe head out to take a look at the painted ladies, no not our sisters in castro, the picture postcard row of pastel painted houses, though its a bit of a hilly trek from the main bus route and just someone's home at the end of the day lol.
If you have your own transport then you can get a bit more out of the city and head over the golden gate bridge, stop off at the view points and take a walk out over the bridge, Drive up the road to the west just on the north side that has nice views back to the city behind the golden gate bridge.
In April presumably it will still be quiet, The cable cars are so much nicer at night when you can sit comfortably, rather than jammed in (you have to experience that too though). maybe only a few of you getting an impromptu private tour. We have been know to just ride the circuit on the same car at 10pm in January just to take in the sights and sounds of the city at night. Also good if your nervous about being out in the city at night as you can see it from the protection of the cable car and I get the impression the drivers and brake man wont take any s*&t from anyone causing problems on board. On that note the F-Train at night can be a little more raucous with drunks etc.
Whatever you chose to do, you cant do it all in that time, but have fun.