1

We are actually migrating db from oracle to postgres. We are using postgreSQL 11. So the packages are converted to procedures/functions accordingly.

For global variables implementation we used the local configuration setting as below

SET pkg.variable = value;
select current_setting('pkg.variable');

However, these values are becoming empty if it goes into exception block and we needed the value to be available in the exception block to proceed with some other logic on failure.

So we took an approach of using global variables using extension plperl/plpython as per below,

CREATE OR REPLACE FUNCTION set_var(name text, val text)
    RETURNS void
    LANGUAGE 'plperl'
AS $BODY$
    undef $_SHARED{$_[0]};
    $_SHARED{$_[0]} = $_[1];
$BODY$;

CREATE OR REPLACE FUNCTION get_var(name text)
    RETURNS text
    LANGUAGE 'plperl'
AS $BODY$
    return $_SHARED{$_[0]};
$BODY$;

plpython as below

CREATE OR REPLACE FUNCTION set_var(var text, val text)
 RETURNS void
 LANGUAGE plpythonu
AS $function$
    if var not in GD:
        GD[var] = val 
    else:
        del GD[var]
        GD[var] = val
$function$ ;

    CREATE OR REPLACE FUNCTION get_var(var text)
 RETURNS text
 LANGUAGE plpythonu
AS $function$
    if var in GD:
        return GD[var]
    else:
        return None
$function$

We set the value and retrieve the value like,

PERFORM set_var('pkg.variable',value) -- to set the variable
select get_var('pkg.variable') --to get the value.

The procedure that we are calling has a for loop inside and inreturn calls many other procedures which will set a total of approx 270 of such global variables altogether for each loop. The value of the global variables will be updating to the new value for the same 270 variables that we already created in the first loop. This has to happen for around 300,000 rows in the for loop.

However with this implementation, the memory consumption is going very high and at one point it reaches an out of memory state and stopping at around 157000th row.

I do not know much of perl or python but based on things i understood from online i have added undef() and del for perl and python respectively which would release the memory. However it is not happening so.

We also took the address of the variable set to see if it is updating on the same address or taking new memory address on each invocation. It is updating the value on the same address when the variable got created.

But the memory keeps accumulating overtime until it uses all of the available RAM and throw error as out of memory.

If anyone can help me understanding on what i have done wrong or what i can change to make it work fine it would be of great help. Thanks in advance.

UPDATE: The issue was because of using Procedures. We changed the code to Functions and the memory consumption issue is not there. I am still yet to findout why there is such a massive difference in using procedures vs functions.

4

0 回答 0