The answer to your question is actually a bit more complicated than you might think. A good question and set of answers that deal with many of the related details are at: What is the Proper Way to Destroy a Map Instance?.
I'm not sure from your question, but it sounds like you may have created a page that loads the Google Maps API more than one time (or could, depending on user choices) and you should avoid that entirely. Google admits there are memory leak bugs associated with reloading the map and strongly recommends against multiple map reloads. Google essentially does not support multiple map load use cases.
Check out some of the information available at the question link included above; it contains some good discussion and information.
EDIT in Response to @Engineer's Comment:
Check out the Google Maps API Office Hours May 9 2012 Video where Chris Broadfoot and Luke Mahe from Google discuss: that they don't support use cases that involve reloading the map, that the API is intended to be loaded only once, and their acknowledgement that there is a memory leak bug. Set the playback to ~12:50 to view the section about destroying the map, problems with reloading the map, and suggestions they offer to avoid problems. Primarily, if you must hide and then show a map, they recommend reusing a single map instance.