6

我在处理软引用和弱引用时遇到了问题。代码有一个标志，可以在软引用和弱引用之间切换所有逻辑。虽然使用弱引用似乎一切正常，但使用软引用我一直得到 OutOfMemoryError。MacOSX 上的 JDK7 和 JDK6 以及 Debian 上的 IcedTea6 都会发生这种情况。但是，带有 G1 收集器的 JDK7 是我发现使用软引用的设置，我尝试过的所有其他操作（串行/并行 GC、-client/-server 等）都未能引发异常。

代码有点大，但我试图在保留问题的同时尽可能缩小范围。
我在上面留下了大量评论，更详细地描述了如何重现该问题。

/*
 *
 * Leakling.java
 *
 *
 * Issue:
 *
 *
 *   This code throws OutOfMemoryError when using soft references, whereas weak references
 *   work ok. Moreover, with JDK7 G1 garbage collector soft references work as well. Other
 *   collectors seem to fail. Code was tested with MacOSX 10.8.2 JDKs 1.7.0_10-b18 and
 *   1.6.0_37-b06-434-11M3909, with Debian 6.0 IcedTea6 1.8.13.
 *   Various command line options including -Xmx, -client/-server, -XX:+UseParallelOldGC,
 *   -XX:+UseSerialGC were tested.
 *
 *
 * Examples:
 *
 *
 * 1. Default options, weak references, this works, counters go up and down,
 *    but everything keeps going just as expected:
 *
 *   java -Xmx50m Leakling \
 *       --loop-delay=10 --min-chunk-size=1000 --max-chunk-size=100000 --use-soft-references=false
 *
 *
 * 2. Default options, soft references, this eventually throws the exception:
 *
 *   java -Xmx50m Leakling \
 *       --loop-delay=10 --min-chunk-size=1000 --max-chunk-size=100000 --use-soft-references=true
 *
 *
 * 3. G1 collector (IcedTea6 doesn't support it), weak references, this works, but it did anyway:
 *
 *   java -XX:+UseG1GC -Xmx50m Leakling \
 *       --loop-delay=10 --min-chunk-size=1000 --max-chunk-size=100000 --use-soft-references=false
 *
 *
 * 4. G1 collector, soft references, this works with JDK7.
 *    JDK6 seems to just stop after having hit memory limit (with no message whatsoever).
 *
 *   java -XX:+UseG1GC -Xmx50m Leakling \
 *       --loop-delay=10 --min-chunk-size=1000 --max-chunk-size=100000 --use-soft-references=true
 *
 *
 * jarek, 02.01.2013
 *
 *
 */

import java.lang.ref.*;
import java.util.*;

public class Leakling {
  private static final String TAG = "memory-chunk-";


  class Chunk {
    final String name;
    final int size;
    final private byte[] mem;

    Chunk(String name, int minSize, int maxSize, Random randomizer) {
      int currSize = minSize;
      if (maxSize > minSize) {
        currSize += randomizer.nextInt(maxSize - minSize + 1);
      }
      this.size = currSize;
      this.mem = new byte[currSize];
      this.name = name;
      log(this + " has been created (" + currSize + " bytes)");
    }

    @Override
    public void finalize() throws Throwable {
      log(this + " is finalizing");
    }

    @Override
    public String toString() {
      return name + " of " + getReadableMemorySize(size);
    }
  }


  class WeakChunk extends WeakReference<Chunk> {
    final String name;

    public WeakChunk(Chunk chunk, ReferenceQueue<Chunk> queue) {
      super(chunk, queue);
      this.name = chunk.name;
    }

    @Override
    public String toString() {
      return "weak reference of " + name + " is " + ((get() == null) ? "null" : "alive");
    }
  }


  class SoftChunk extends SoftReference<Chunk> {
    final String name;

    public SoftChunk(Chunk chunk, ReferenceQueue<Chunk> queue) {
      super(chunk, queue);
      this.name = chunk.name;
    }

    @Override
    public String toString() {
      return "soft reference of " + name + " is " + ((get() == null) ? "null" : "alive");
    }
  }

  // Logging as implemented here gives extra timing info (secs.milis starting from the initialization).
  private final long start = System.currentTimeMillis();
  private final Formatter formatter = new Formatter(System.err);
  private final String formatString = "%1$d.%2$03d %3$s\n";

  // I found this be better synchronized...
  synchronized void log(Object o) {
    long curr = System.currentTimeMillis();
    long diff = curr - start;
    formatter.format(formatString, (int) (diff / 1000), (int) (diff % 1000), o.toString());
  }

  private final ArrayList<Chunk> allChunks = new ArrayList<Chunk>();
  private final ReferenceQueue<Chunk> softReferences = new ReferenceQueue<Chunk>();
  private final ReferenceQueue<Chunk> weakReferences = new ReferenceQueue<Chunk>();
  private final HashSet<Reference<Chunk>> allReferences = new HashSet<Reference<Chunk>>();
  private final Random randomizer = new Random();

  private int loopDelay = 200;
  private int minChunkSize = 100;
  private int maxChunkSize = 1000;
  private int chunkCounter = 0;
  private boolean useSoftReferences = false;
  private long minMemory = 10 * 1024 * 1024;  // Default range is 10-30MB
  private long maxMemory = 3 * minMemory;
  private long usedMemory = 0;

  private String getReadableMemorySize(long size) {
    if (size >= 1024 * 1024) {
      return (float) (Math.round((((float) size) / 1024f / 1024f) * 10f)) / 10f + "MB";
    }
    if (size >= 1024) {
      return (float) (Math.round((((float) size) / 1024f) * 10f)) / 10f + "kB";
    } else if (size > 0) {
      return size + "B";
    } else {
      return "0";
    }
  }

  private void allocMem() {
    Chunk chunk = new Chunk(TAG + chunkCounter++, minChunkSize, maxChunkSize, randomizer);
    allChunks.add(chunk);
    Reference ref = useSoftReferences
            ? (new SoftChunk(chunk, softReferences)) : (new WeakChunk(chunk, weakReferences));
    allReferences.add(ref);
    log(ref);
    usedMemory += chunk.size;
  }

  private void freeMem() {
    if (allChunks.size() < 1) {
      return;
    }
    int i = randomizer.nextInt(allChunks.size());
    Chunk chunk = allChunks.get(i);
    log("freeing " + chunk);
    usedMemory -= chunk.size;
    allChunks.remove(i);
  }

  private int statMem() throws Exception {
    for (Reference ref; (ref = softReferences.poll()) != null;) {
      log(ref);
      allReferences.remove(ref);
    }
    for (Reference ref; (ref = weakReferences.poll()) != null;) {
      log(ref);
      allReferences.remove(ref);
    }
    int weakRefs = 0;
    int softRefs = 0;
    for (Iterator<Reference<Chunk>> i = allReferences.iterator(); i.hasNext();) {
      Reference<Chunk> ref = i.next();
      if (ref.get() == null) {
        continue;
      }
      if (ref instanceof WeakChunk) {
        weakRefs++;
      }
      if (ref instanceof SoftChunk) {
        softRefs++;
      }
    }
    log(allChunks.size() + " chunks, "
            + softRefs + " soft refs, "
            + weakRefs + " weak refs, "
            + getReadableMemorySize(usedMemory) + " used, "
            + getReadableMemorySize(Runtime.getRuntime().freeMemory()) + " free, "
            + getReadableMemorySize(Runtime.getRuntime().totalMemory()) + " total, "
            + getReadableMemorySize(Runtime.getRuntime().maxMemory()) + " max");
    if (loopDelay > 1) {
      Thread.sleep(loopDelay);
    }
    return (int)((100 * usedMemory) / maxMemory); // Return % of maxMemory being used.
  }

  public Leakling(String[] args) throws Exception {
    for (String arg : args) {
      if (arg.startsWith("--min-memory=")) {
        minMemory = Long.parseLong(arg.substring("--min-memory=".length()));
      } else if (arg.startsWith("--max-memory=")) {
        maxMemory = Long.parseLong(arg.substring("--max-memory=".length()));
      } else if (arg.startsWith("--min-chunk-size=")) {
        minChunkSize = Integer.parseInt(arg.substring("--min-chunk-size=".length()));
      } else if (arg.startsWith("--max-chunk-size=")) {
        maxChunkSize = Integer.parseInt(arg.substring("--max-chunk-size=".length()));
      } else if (arg.startsWith("--loop-delay=")) {
        loopDelay = Integer.parseInt(arg.substring("--loop-delay=".length()));
      } else if (arg.startsWith("--use-soft-references=")) {
        useSoftReferences = Boolean.parseBoolean(arg.substring("--use-soft-references=".length()));
      } else {
        throw new Exception("Unknown command line option...");
      }
    }
  }

  public void run() throws Exception {
    log("Mem test started...");

    while(true) {
      log("going up...");
      do {// First loop allocates memory up to the given limit in a pseudo-random fashion.
          // Randomized rate of allocations/frees is about 4:1 as per the 10>=8 condition.
        if (randomizer.nextInt(10) >= 8) {
          freeMem();
        } else {
          allocMem();
        }
      } while (statMem() < 90); // Repeat until 90% of the given mem limit is hit...

      log("going down...");
      do {// Now do the reverse. Frees are four times more likely than allocations are.
        if (randomizer.nextInt(10) < 8) {
          freeMem();
        } else {
          allocMem();
        }
      } while (usedMemory > minMemory);
    }
  }

  public static void main(String[] args) throws Exception {
    (new Leakling(args)).run();
  }
}
4

2 回答 2

6

首先，不要将终结器与参考混合。两者都会影响从内存中删除对象的速度，并且使用终结器可以执行的所有操作都可以使用适当的引用类型更好地完成。

其次，正如我所提到的，可能存在与使用引用相关的 gc 延迟。至少对于“常见的” gc 算法，弱/软引用的对象可能会在完全回收之前进行额外的 gc 运行。弱引用和软引用之间的本质区别在于，弱引用被积极地 gc'ed 并且软引用通常被“尽可能长地”持有。这很可能是你得到的。

当你使用弱引用的对象运行时，东西会随着你去清理，让你避免 OOME。

当您使用软引用的对象运行时，所有软引用的对象都会被保留，直到您接近 limit。然后，当内存变得紧张时，gc 会尝试开始释放软引用的对象，但它需要的时间太长（因为它可能需要多次 gc 传递才能完全回收内存）并且你最终会得到一个 OOME。

我对 G1 gc 只了解肤浅的知识，所以我不知道为什么它在那个场景中“有效”。

总之，软引用有点好，但由于回收延迟，它们并不总是像您希望的那样工作。此外，这是一篇很棒的文章，其中包含一些额外的有用细节。

于 2013-01-02T19:58:21.607 回答
5

摆脱终结者。

对象终结发生在一个单独的线程中，直到终结完成后才能真正回收内存。在您的终结器中，您正在进行系统调用（输出），这将在该线程中引入等待。这很容易，当您处于内存限制时，任何终结器都会等待导致 OOM。

至于软引用与弱引用的区别：我们的引用将在次要收集期间被回收，而软引用不会（我没有检查过；可能是控制软引用生命周期的标志只会允许它存在在多个次要集合上）。您的终结器线程很可能能够跟上丢弃的弱引用对象。

于 2013-01-02T20:00:59.587 回答