0

我正在运行一个用 Go 编写的服务器，并且 RSS 非常高，甚至没有释放内存（回到操作系统）。

我使用 pprof 进行检查，但似乎没有内存泄漏。

我也试过：

GODEBUG=madvdontneed=1 ./memorytest

请告诉我如何使用 madvdontneed。

OS: CentOS 7 (Linux)
Arch: amd64
Go version: 1.14.2

代码：

package main

import (
    "fmt"
    "os"
    "runtime"
    "time"
)

var garr []string

var chn chan int

func main() {
    chn = make(chan int, 1)

    go Alloc()

    <-chn
}

func Alloc() {
    for {
        arr := make([]string, 100000000)
        //copy(garr,arr)
        garr = arr
        print_heap_info()
        time.Sleep(5 * time.Second)
    }
}

func print_heap_info() {
    var m runtime.MemStats
    runtime.ReadMemStats(&m)
    fmt.Printf("env:%v, heapsys:%d,heapalloc:%d,heapidel:%d,heapreleased:%d,heapinuse:%d\n",
        os.Getenv("GODEBUG"), m.HeapSys, m.HeapAlloc, m.HeapIdle, m.HeapReleased, m.HeapInuse)
}

输出：</p> 

env:madvdontneed=1, heapsys:1677262848,heapalloc:1600074304,heapidel:76955648,heapreleased:76914688,heapinuse:1600307200
env:madvdontneed=1, heapsys:3287875584,heapalloc:3200081392,heapidel:87556096,heapreleased:87506944,heapinuse:3200319488
env:madvdontneed=1, heapsys:4898619392,heapalloc:4800086512,heapidel:98295808,heapreleased:98115584,heapinuse:4800323584
env:madvdontneed=1, heapsys:6509101056,heapalloc:6400090640,heapidel:108773376,heapreleased:108724224,heapinuse:6400327680
env:madvdontneed=1, heapsys:6509232128,heapalloc:4800086176,heapidel:1708908544,heapreleased:108724224,heapinuse:4800323584
env:madvdontneed=1, heapsys:6509002752,heapalloc:6400090304,heapidel:108675072,heapreleased:108560384,heapinuse:6400327680
env:madvdontneed=1, heapsys:6509199360,heapalloc:4800086712,heapidel:1708875776,heapreleased:108560384,heapinuse:4800323584
env:madvdontneed=1, heapsys:6509068288,heapalloc:6400090744,heapidel:108740608,heapreleased:108560384,heapinuse:6400327680
env:madvdontneed=1, heapsys:6509199360,heapalloc:4800086712,heapidel:1708875776,heapreleased:108560384,heapinuse:4800323584
env:madvdontneed=1, heapsys:6509068288,heapalloc:6400090840,heapidel:108740608,heapreleased:108462080,heapinuse:6400327680

在此处输入图像描述

4

1 回答 1

2

优化内存分配以降低峰值。

参见：debug.FreeOSMemory()

FreeOSMemory 强制进行垃圾收集，然后尝试将尽可能多的内存返回给操作系统。（即使不调用它，运行时也会在后台任务中逐渐将内存返回给操作系统。）

您的格式化输出是：

sys: 1599 MB alloc: 1525 MB idel:   73 MB released:   73 MB inuse: 1526 MB 
sys: 3135 MB alloc: 3051 MB idel:   83 MB released:   83 MB inuse: 3052 MB 
sys: 4671 MB alloc: 4577 MB idel:   93 MB released:   93 MB inuse: 4577 MB 
sys: 6207 MB alloc: 6103 MB idel:  103 MB released:  103 MB inuse: 6103 MB 
sys: 6207 MB alloc: 4577 MB idel: 1629 MB released:  103 MB inuse: 4577 MB 
sys: 6207 MB alloc: 6103 MB idel:  103 MB released:  103 MB inuse: 6103 MB 
sys: 6207 MB alloc: 4577 MB idel: 1629 MB released:  103 MB inuse: 4577 MB 
sys: 6207 MB alloc: 6103 MB idel:  103 MB released:  103 MB inuse: 6103 MB 
sys: 6207 MB alloc: 4577 MB idel: 1629 MB released:  103 MB inuse: 4577 MB 
sys: 6207 MB alloc: 6103 MB idel:  103 MB released:  103 MB inuse: 6103 MB

这是（最后一行）：
sys：从操作系统获得的堆内存字节：6207 MB
alloc：已分配堆对象的字节：6103 MB
idel：空闲（未使用）的字节跨度：103 MB
已释放：物理内存字节返回到操作系统：103 MB
使用中：使用中的字节跨度：6103 MB

这不是内存泄漏。具有总 8GB RAM 的系统的输出（系统内存监视器）： 在此处输入图像描述

命令：

GODEBUG=madvdontneed=1  go run .

输出：

env: madvdontneed=1, sys: 1087 MB, alloc: 1024 MB, idel:   63 MB, released:   63 MB, inuse: 1024 MB
env: madvdontneed=1, sys: 2111 MB, alloc: 2048 MB, idel:   63 MB, released:   63 MB, inuse: 2048 MB
env: madvdontneed=1, sys: 3135 MB, alloc: 3072 MB, idel:   63 MB, released:   63 MB, inuse: 3072 MB
env: madvdontneed=1, sys: 4159 MB, alloc: 4096 MB, idel:   63 MB, released:   63 MB, inuse: 4096 MB
env: madvdontneed=1, sys: 4159 MB, alloc: 3072 MB, idel: 1087 MB, released:   63 MB, inuse: 3072 MB
env: madvdontneed=1, sys: 4159 MB, alloc: 4096 MB, idel:   63 MB, released:   63 MB, inuse: 4096 MB
env: madvdontneed=1, sys: 4159 MB, alloc: 3072 MB, idel: 1087 MB, released:   63 MB, inuse: 3072 MB
env: madvdontneed=1, sys: 4159 MB, alloc: 4096 MB, idel:   63 MB, released:   63 MB, inuse: 4096 MB
env: madvdontneed=1, sys: 4159 MB, alloc: 3072 MB, idel: 1087 MB, released:   63 MB, inuse: 3072 MB
env: madvdontneed=1, sys: 4159 MB, alloc: 4096 MB, idel:   63 MB, released:   63 MB, inuse: 4096 MB

代码：

package main

import (
    "fmt"
    "os"
    "runtime"
    "time"
)

func main() {
    for i := 0; i < 10; i++ {
        a = make([]byte, 1024*meg)
        var m runtime.MemStats
        runtime.ReadMemStats(&m)
        fmt.Printf("env: %v, sys: %4d MB, alloc: %4d MB, idel: %4d MB, released: %4d MB, inuse: %4d MB\n",
            os.Getenv("GODEBUG"), m.HeapSys/meg, m.HeapAlloc/meg, m.HeapIdle/meg, m.HeapReleased/meg, m.HeapInuse/meg)
        time.Sleep(1 * time.Second)
    }
}

var a []byte

const meg = 1024 * 1024

顶部： 在此处输入图像描述

命令的输出vmstat 1 -S MB，运行两个终端：

procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu-----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
 0  0   1161   6369     25    595    0    0   702   452  276  567  6  3 91  1  0
 3  0   1160   6300     28    632    0    0 43432     0 6288 14013 13  5 82  1  0
 1  0   1160   6243     28    634    0    0   528     0 6342 13008 11  3 85  0  0
 1  0   1160   6177     28    634    0    0    56    56  979 2090  4  1 95  0  0
 2  0   1160   6110     28    634    0    0     0     0 1061 2664  5  2 94  0  0
 0  0   1160   6044     28    634    0    0    12    36  994 2233  4  1 94  0  0
 0  0   1160   3991     28    634    0    0    32   188 1074 1787  5  6 89  0  0
 2  0   1160   2120     28    634    0    0     0   136 1016 1634  4  5 90  0  0
 1  0   1160    973     28    633    0    0     0     4 1077 1660  4  5 92  0  0
 2  0   1160    143     25    390    0    0  1780   356 1849 2341  4  9 87  0  0
 0  9   1323    102      0    108    0  165 51964 169652 20277 21017  1 20 53 25  0
 1  5   1510     99      0    129    2  189 54376 193996 57829 52152  1 16 56 27  0
 1  5   1794     99      0    129    1  286 10068 293856 81160 59511  0 16 77  6  0
 4  5   2047    101      0     98    5  257 21236 263292 69923 65485  0 23 54 23  0
 1  2   1479   2867      0    217   43   23 233508 24380 24023 48536  4 27 47 22  0
 2  0   1452   2824      0    232   27    0 43960     0 8168 21085  4  5 90  1  0
 0  1   1443   2814      0    233    9    0 10956     0 3341 8468  5  2 93  0  0
 3  0   1425   2796      0    231   18    0 19688     0 5780 15490  4  3 92  1  0
 1  1   1420   2672     10    337    3    0 121628  1920 3292 7934  5  7 81  7  0
 0  0   1394   2646     10    338   25    0 27360     0 7975 21555  3  5 92  0  0
 0  1   1359   6856     10    339    1    0  2416     0 1035 2108  3  2 95  0  0
 0  0   1353   6847     10    348    4    0 13660     0 1696 3471  4  1 95  0  0

GODEBUG=madvdontneed=1 go run .第一个命令的输出（自动终止）：

env: madvdontneed=1, sys: 1087 MB, alloc: 1024 MB, idel:   63 MB, released:   63 MB, inuse: 1024 MB
env: madvdontneed=1, sys: 2111 MB, alloc: 2048 MB, idel:   63 MB, released:   63 MB, inuse: 2048 MB
env: madvdontneed=1, sys: 3135 MB, alloc: 3072 MB, idel:   63 MB, released:   63 MB, inuse: 3072 MB
env: madvdontneed=1, sys: 4159 MB, alloc: 4096 MB, idel:   63 MB, released:   63 MB, inuse: 4096 MB
env: madvdontneed=1, sys: 4159 MB, alloc: 3072 MB, idel: 1087 MB, released:   63 MB, inuse: 3072 MB
env: madvdontneed=1, sys: 4159 MB, alloc: 4096 MB, idel:   63 MB, released:   63 MB, inuse: 4096 MB
env: madvdontneed=1, sys: 4159 MB, alloc: 3072 MB, idel: 1087 MB, released:   63 MB, inuse: 3072 MB
signal: killed

第二个GODEBUG=madvdontneed=1 go run .命令的输出：

env: madvdontneed=1, sys: 1087 MB, alloc: 1024 MB, idel:   63 MB, released:   63 MB, inuse: 1024 MB
env: madvdontneed=1, sys: 2111 MB, alloc: 2048 MB, idel:   63 MB, released:   63 MB, inuse: 2048 MB
env: madvdontneed=1, sys: 3135 MB, alloc: 3072 MB, idel:   63 MB, released:   63 MB, inuse: 3072 MB
env: madvdontneed=1, sys: 4159 MB, alloc: 4096 MB, idel:   63 MB, released:   63 MB, inuse: 4096 MB
env: madvdontneed=1, sys: 4159 MB, alloc: 3072 MB, idel: 1087 MB, released:   63 MB, inuse: 3072 MB
env: madvdontneed=1, sys: 4159 MB, alloc: 4096 MB, idel:   63 MB, released:   63 MB, inuse: 4096 MB
env: madvdontneed=1, sys: 4159 MB, alloc: 3072 MB, idel: 1087 MB, released:   63 MB, inuse: 3072 MB
env: madvdontneed=1, sys: 4159 MB, alloc: 4096 MB, idel:   63 MB, released:   62 MB, inuse: 4096 MB
env: madvdontneed=1, sys: 4159 MB, alloc: 3072 MB, idel: 1087 MB, released:   62 MB, inuse: 3072 MB
env: madvdontneed=1, sys: 4159 MB, alloc: 4096 MB, idel:   63 MB, released:   62 MB, inuse: 4096 MB

代码：

package main

import (
    "fmt"
    "os"
    "runtime"
    "time"
)

func main() {
    for i := 0; i < 10; i++ {
        a = make([]byte, 1024*meg)
        var m runtime.MemStats
        runtime.ReadMemStats(&m)
        fmt.Printf("env: %v, sys: %4d MB, alloc: %4d MB, idel: %4d MB, released: %4d MB, inuse: %4d MB\n\n",
            os.Getenv("GODEBUG"), m.HeapSys/meg, m.HeapAlloc/meg, m.HeapIdle/meg, m.HeapReleased/meg, m.HeapInuse/meg)
        time.Sleep(1 * time.Second)
    }
}

var a []byte

const meg = 1024 * 1024

命令：

GODEBUG=gctrace=1 go run .

输出：

gc 1 @0.008s 2%: 0.071+0.67+0.034 ms clock, 0.57+0.88/0.76/0.041+0.27 ms cpu, 4->4->0 MB, 5 MB goal, 8 P
gc 2 @0.014s 3%: 0.069+1.3+0.027 ms clock, 0.55+0.48/0.80/0.73+0.21 ms cpu, 4->4->0 MB, 5 MB goal, 8 P
gc 3 @0.029s 2%: 0.056+0.62+0.040 ms clock, 0.45+0.39/0.75/0.67+0.32 ms cpu, 4->4->0 MB, 5 MB goal, 8 P
gc 4 @0.045s 3%: 0.31+0.97+0.12 ms clock, 2.5+0.91/1.2/0.92+1.0 ms cpu, 4->4->0 MB, 5 MB goal, 8 P
gc 5 @0.060s 2%: 0.056+0.60+0.019 ms clock, 0.45+0.44/0.70/1.1+0.15 ms cpu, 4->4->0 MB, 5 MB goal, 8 P
gc 6 @0.071s 2%: 0.025+1.0+0.018 ms clock, 0.20+0.47/1.1/3.5+0.15 ms cpu, 4->4->0 MB, 5 MB goal, 8 P
gc 7 @0.084s 2%: 0.12+0.88+0.042 ms clock, 0.97+0.77/1.2/0.88+0.33 ms cpu, 4->4->1 MB, 5 MB goal, 8 P
gc 8 @0.093s 2%: 0.039+0.83+0.028 ms clock, 0.31+0.38/0.66/1.2+0.22 ms cpu, 4->4->0 MB, 5 MB goal, 8 P

gc 1 @0.007s 3%: 0.013+1.7+0.005 ms clock, 0.11+0.86/2.0/1.2+0.042 ms cpu, 4->5->4 MB, 5 MB goal, 8 P

gc 1 @0.002s 5%: 0.024+2.0+0.042 ms clock, 0.19+0.25/1.5/1.3+0.34 ms cpu, 4->6->5 MB, 5 MB goal, 8 P
gc 2 @0.017s 3%: 0.014+4.6+0.044 ms clock, 0.11+0.13/3.3/1.9+0.35 ms cpu, 9->10->7 MB, 10 MB goal, 8 P
gc 3 @0.058s 2%: 0.030+6.7+0.036 ms clock, 0.24+0.12/5.3/1.8+0.29 ms cpu, 13->15->10 MB, 15 MB goal, 8 P
gc 4 @0.100s 2%: 0.034+5.6+0.015 ms clock, 0.27+0/7.2/0.65+0.12 ms cpu, 18->18->12 MB, 21 MB goal, 8 P
gc env: gctrace=1, sys: 1087 MB, alloc: 1024 MB, idel:   63 MB, released:   63 MB, inuse: 1024 MB

1 @0.024s 0%: 0.028+0.41+0.016 ms clock, 0.22+0.11/0.14/0.093+0.13 ms cpu, 1024->1024->1024 MB, 1025 MB goal, 8 P
env: gctrace=1, sys: 2111 MB, alloc: 2048 MB, idel:   63 MB, released:   63 MB, inuse: 2048 MB

gc 2 @1.049s 0%: 0.021+0.44+0.005 ms clock, 0.16+0.12/0.15/0.12+0.045 ms cpu, 2048->2048->2048 MB, 2049 MB goal, 8 P
env: gctrace=1, sys: 3135 MB, alloc: 3072 MB, idel:   63 MB, released:   63 MB, inuse: 3072 MB

env: gctrace=1, sys: 4159 MB, alloc: 4096 MB, idel:   63 MB, released:   63 MB, inuse: 4096 MB

gc 3 @3.096s 0%: 0.023+0.56+0.017 ms clock, 0.18+0.13/0.20/0.17+0.13 ms cpu, 4096->4096->2048 MB, 4097 MB goal, 8 P
env: gctrace=1, sys: 4159 MB, alloc: 3072 MB, idel: 1087 MB, released:   63 MB, inuse: 3072 MB

env: gctrace=1, sys: 4159 MB, alloc: 4096 MB, idel:   63 MB, released:   63 MB, inuse: 4096 MB

gc 4 @5.619s 0%: 0.023+0.31+0.018 ms clock, 0.18+0.18/0.18/0.22+0.15 ms cpu, 4096->4096->2048 MB, 4097 MB goal, 8 P
env: gctrace=1, sys: 4159 MB, alloc: 3072 MB, idel: 1087 MB, released:   63 MB, inuse: 3072 MB

env: gctrace=1, sys: 4159 MB, alloc: 4096 MB, idel:   63 MB, released:   63 MB, inuse: 4096 MB

gc 5 @8.141s 0%: 0.025+0.26+0.004 ms clock, 0.20+0.16/0.15/0.15+0.033 ms cpu, 4096->4096->2048 MB, 4097 MB goal, 8 P
env: gctrace=1, sys: 4159 MB, alloc: 3072 MB, idel: 1087 MB, released:   63 MB, inuse: 3072 MB

gc 6 @10.413s 0%: 0.028+0.51+0.013 ms clock, 0.22+0.23/0.28/0.29+0.11 ms cpu, 4096->4096->2048 MB, 4097 MB goal, 8 P
env: gctrace=1, sys: 4159 MB, alloc: 4096 MB, idel:   63 MB, released:   63 MB, inuse: 4096 MB

此行的格式可能会发生变化。目前，它是：

    gc # @#s #%: #+#+# ms clock, #+#/#/#+# ms cpu, #->#-># MB, # MB goal, # P
where the fields are as follows:
    gc #        the GC number, incremented at each GC
    @#s         time in seconds since program start
    #%          percentage of time spent in GC since program start
    #+...+#     wall-clock/CPU times for the phases of the GC
    #->#-># MB  heap size at GC start, at GC end, and live heap
    # MB goal   goal heap size
    # P         number of processors used

注：
go版本go1.15.5 linux/amd64

另请参阅：
Go 1.13 RSS 不断增加，疑似清理问题
https://github.com/golang/go/issues/36398
https://github.com/golang/go/issues/39295
https://go.googlesource .com/proposal/+/master/design/14951-soft-heap-limit.md
https://docs.google.com/document/d/1wmjrocXIWTr1JxU-3EQBI6BK6KgtiFArkG47XK73xIQ/edit#
https://blog.cloudflare.com/去不要收集我的垃圾/

于 2020-11-23T07:42:25.210 回答