Note: I’ve moved my blog! You can find this post here.
Some airplanes (such as the Cessna 172s I fly in) have a steerable nosewheel. That is, they have a mechanical link between the rudder pedals and the nosewheel allowing you to turn the nosewheel when you press the rudder pedals. It’s primarily this, and not the rudder control surfaces, that allow you to turn when you press the rudder pedals while taxiing.
I feel silly for not knowing this!