Toddlers Have No Sense of Time
Any parent who has lost their cool over a toddler not understanding that you "need to leave N-O-W" knows that they just don't understand the concept of time. What you probably didn't know is that they don't have the brain structure for it. According to Dr. Tovah Klein, "Because toddlers lack this sense of time and their brains are still developing, they have not internalized the skills that adults (well, most, anyway) use to help them manage time, control their impulses, express their needs, thoughtfully plan their actions, and cope with stress."
Toddlers exist in the present. So when you say "we're leaving in five minutes," they just know that they're not leaving now — which is why their answer to how much longer they want to do something is usually five minutes — it's a placeholder. Repetitious routines and use of sequential language (first, next, then, etc.) helps teach younger tots the concept of time, but they may not fully grasp it until the later elementary school years.
Source: Flickr user hlkljgk