Supposedly the iPhone tracks and stores a "year's worth" of location data even if you opt out of location-based services. It's not clear how/when this location log is shipped to Apple.
Next thing they will be saving and scanning our text messages....
(yes that last was a joke as I know that they can't possibly have the time, engery, bandwidth or manpower to do that
)