javascript - How to Pan a div using JQuery

Question

I have been trying for a while now to get a simple pan of a div to work, I however can not seem to get it to 100%. It partially works with bugs.

$("#view_point").mousedown(function(e) {
    start_x = e.pageX;
    start_y = e.pageY;
    e.preventDefault()
    //On Click set start x and y vars
}).mousemove(function(e) {
    temp_x = e.pageX;
    temp_y = e.pageY;
    e.preventDefault();
}).mouseup(function(e) {
    temp_x = Math.abs(temp_x - start_x);
    temp_x = Math.abs(temp_y - start_y);

    console.log(temp_x + " - " + temp_y);

    //Animate the map
    $("#tiles").animate({
        marginTop: '-' + temp_x,
        marginLeft: '-' + temp_y
    }, 50);
});​

How do I go about making a pan script that will pan inside a div that has its overflow property set to hidden.