0

Will MemoryMappedFile.CreateViewStream(0, len) allocate a managed block of memory of size len, or will it allocate smaller buffer that acts as a sliding window over the unmanaged data?

I wonder because I aim to replace an intermediate buffer for deserialization that is a MemoryStream today, which is giving me trouble for large datasets, both because of the size of the buffer and because of LOH fragmentation.

If the viewstream's internal buffer becomes the same size then making this switch wouldn't make sense.

Edit:

In a quick test I found these numbers when comparing the MemoryStream to the MemoryMapped file. Readings from GC.GetTotalMemory(true)/1024 and Process.GetCurrentProcess.VirtualMemorySize64/1024

Allocate an 1GB memory stream: 

                     Managed           Virtual 
Initial:               81 kB        190 896 kB
After alloc:    1 024 084 kB      1 244 852 kB

As expected, a gig of both managed and virtual memory. Now, for the MemoryMappedFile:

                     Managed           Virtual 
Initial:               81 kB        189 616 kB    
MMF allocated:         84 kB        189 684 kB
1GB viewstream allocd: 84 kB      1 213 368 kB
Viewstream disposed:   84 kB        190 964 kB

So using a not very scientific test, my assumption is that the ViewStream uses only unmanaged data. Correct?

4

2 回答 2

2

像这样的 MMF 并不能解决您的问题。由于虚拟内存空间中没有足够大的孔以适应分配，因此程序会在 OOM 上发生炸弹。如您所知，您仍在使用 MMF 消耗 VM 地址空间。

使用小的滑动视图是一种解决方法，但这与写入文件没有任何不同。这就是 MMF 在重新映射视图时所做的事情，它需要将脏页刷新到磁盘。简单地流式传输到 FileStream 是正确的解决方法。那仍然使用 RAM，文件系统缓存有助于加快写入速度。如果您有 1 GB 的可用 RAM，而且现在不难获得，那么写入 FileStream 只是内存到内存的复制。非常快，5 GB/秒及以上。该文件在后台以一种懒惰的方式写入。

在 Windows 中，过于努力地将数据保存在内存中是徒劳的。内存中的私有数据由分页文件支持，当 Windows 需要 RAM 用于其他进程时，它们将被写入该文件。并在您再次访问时回读。这很慢，你使用的内存越多，它变得越糟糕。像任何按需分页的虚拟内存操作系统一样，磁盘和内存之间的区别很小。

于 2013-06-21T16:21:04.113 回答
0

给出http://msdn.microsoft.com/en-us/library/system.io.memorymappedfiles.memorymappedfile.aspx上的示例上的示例，在我看来你得到了一个滑动窗口，至少这是我在阅读时所解释的例子。

为方便起见，这里举个例子：

    using System;
using System.IO;
using System.IO.MemoryMappedFiles;
using System.Runtime.InteropServices;

class Program
{
    static void Main(string[] args)
    {
        long offset = 0x10000000; // 256 megabytes 
        long length = 0x20000000; // 512 megabytes 

        // Create the memory-mapped file. 
        using (var mmf = MemoryMappedFile.CreateFromFile(@"c:\ExtremelyLargeImage.data", FileMode.Open,"ImgA"))
        {
            // Create a random access view, from the 256th megabyte (the offset) 
            // to the 768th megabyte (the offset plus length). 
            using (var accessor = mmf.CreateViewAccessor(offset, length))
            {
                int colorSize = Marshal.SizeOf(typeof(MyColor));
                MyColor color;

                // Make changes to the view. 
                for (long i = 0; i < length; i += colorSize)
                {
                    accessor.Read(i, out color);
                    color.Brighten(10);
                    accessor.Write(i, ref color);
                }
            }
        }
    }
}

public struct MyColor
{
    public short Red;
    public short Green;
    public short Blue;
    public short Alpha;

    // Make the view brigher. 
    public void Brighten(short value)
    {
        Red = (short)Math.Min(short.MaxValue, (int)Red + value);
        Green = (short)Math.Min(short.MaxValue, (int)Green + value);
        Blue = (short)Math.Min(short.MaxValue, (int)Blue + value);
        Alpha = (short)Math.Min(short.MaxValue, (int)Alpha + value);
    }
}
于 2013-06-21T10:57:17.053 回答