You're conflating magnetic poles, geomagnetic poles and geographic poles. Two are wandering, one is fixed. The magnetic poles are wandering rapidly towards Siberia and the Indian Ocean. When North reaches the "40 degree mark" they will probabably reverse. Together with a collapse of the magnetic field, this could be a fatal scenario. It hasn't happened for 12,000 years (Gothenburg excursion), so modern humans don't know the consequences, other than increased radiation at surface level, and extinction of some species.