Memory mapped data files

how to use memory mapped data files?

why? Example of reverse string in a file: MMF can do this with One File & no Buffer!

steps of using MMF:

Step 1: Creating or Opening a File Kernel Object 盡可能保持read only,create file fail will return -1

Step 2: Creating a File-Mapping Kernel Object

  • SEC_Reserve & commit不適用於實體file
  • mapping size > min page size
  • 如果大小 < 4G, max size high 放 0, 但high & low兩個不能同時為0
int WINAPI _tWinMain(HINSTANCE, HINSTANCE, PTSTR, int) {

   // Before executing the line below, C:\ does not have
   // a file called "MMFTest.Dat"
   HANDLE hFile = CreateFile(TEXT("C:\\MMFTest.Dat"),
      GENERIC_READ | GENERIC_WRITE,
      FILE_SHARE_READ | FILE_SHARE_WRITE, NULL, CREATE_ALWAYS,
      FILE_ATTRIBUTE_NORMAL, NULL);

   // Before executing the line below, the MMFTest.Dat
   // file does exist but has a file size of 0 bytes.
   HANDLE hFileMap = CreateFileMapping(hFile, NULL, PAGE_READWRITE,
      0, 100, NULL);

   // After executing the line above, the MMFTest.Dat
   // file has a size of 100 bytes.
   // 系統會自動加到這個大小, 就算我甚麼都沒做, close handle後還
   // 會是100 bytes, 因為是map到section的大小

   // Cleanup
   CloseHandle(hFileMap);
   CloseHandle(hFile);

   // When the process terminates, MMFTest.Dat remains
   // on the disk with a size of 100 bytes.
   return(0);
}

createFile will return NULL if fail but CreateFileMapping return -1 if fail.

Step 3: Mapping the File's Data into the Process' Address Space 要注意mapViewOfFile跟CreateFileMapping的file權限要要有一致性

offset要是multiple of the system allocation granularity. number of byte to map可以寫0讓系統自己來

HANDLE hFile = CreateFile(pszFileName, GENERIC_READ | GENERIC_WRITE, 0, NULL,
   OPEN_ALWAYS, FILE_ATTRIBUTE_NORMAL, NULL);

// Create a file-mapping object for the file.
HANDLE hFileMapping = CreateFileMapping(hFile, NULL, PAGE_WRITECOPY,
   0, 0, NULL);

// Map a copy-on-write view of the file; the system will commit
// enough physical storage from the paging file to accommodate
// the entire file. All pages in the view will initially have
// PAGE_WRITECOPY access.
PBYTE pbFile = (PBYTE) MapViewOfFile(hFileMapping, FILE_MAP_COPY,
   0, 0, 0);

// Read a byte from the mapped view.
BYTE bSomeByte = pbFile[0];
// When reading, the system does not touch the committed pages in
// the paging file. The page keeps its PAGE_WRITECOPY attribute.

// Write a byte to the mapped view.
pbFile[0] = 0;
// When writing for the first time, the system grabs a committed
// page from the paging file, copies the original contents of the
// page at the accessed memory address, and maps the new page
// (the copy) into the process' address space. The new page has
// an attribute of PAGE_READWRITE.

// Write another byte to the mapped view.
pbFile[1] = 0;
// Because this byte is now in a PAGE_READWRITE page, the system
// simply writes the byte to the page (backed by the paging file).

// When finished using the file's mapped view, unmap it.
// UnmapViewOfFile is discussed in the next section.
UnmapViewOfFile(pbFile);
// The system decommits the physical storage from the paging file.
// Any writes to the pages are lost.

// Clean up after ourselves.
CloseHandle(hFileMapping);
CloseHandle(hFile);

Step 4: Unmapping the File's Data from the Process' Address Space 用base Address of the return region flushViewOFFile可以強迫把東西寫進去

如果map file is on remote disk, 要傳FILE_FLAG_WRITE_THROUGH給createFile

Steps 5 and 6: Closing the File-Mapping Object and the File Object

HANDLE hFile = CreateFile(...);
HANDLE hFileMapping = CreateFileMapping(hFile, ...);
CloseHandle(hFile);
PVOID pvFile = MapViewOfFile(hFileMapping, ...);
CloseHandle(hFileMapping);

// Use the memory-mapped file.

UnmapViewOfFile(pvFile);

Processing a Big File Using Memory-Mapped Files

如果要寫一個大檔, 要怎麼處理?

可以先map一小段, 在unmap, 接著map後面另一段....

__int64 Count0s(void) {

   // Views must always start on a multiple
   // of the allocation granularity
   SYSTEM_INFO sinf;
   GetSystemInfo(&sinf);

   // Open the data file.
   HANDLE hFile = CreateFile(TEXT("C:\\HugeFile.Big"), GENERIC_READ,
      FILE_SHARE_READ, NULL, OPEN_EXISTING, FILE_FLAG_SEQUENTIAL_SCAN, NULL);

   // Create the file-mapping object.
   HANDLE hFileMapping = CreateFileMapping(hFile, NULL,
      PAGE_READONLY, 0, 0, NULL);

   DWORD dwFileSizeHigh;
   __int64 qwFileSize = GetFileSize(hFile, &dwFileSizeHigh);
   qwFileSize += (((__int64) dwFileSizeHigh) << 32);

   // We no longer need access to the file object's handle.
   CloseHandle(hFile);

   __int64 qwFileOffset = 0, qwNumOf0s = 0;

   while (qwFileSize > 0) {

   // Determine the number of bytes to be mapped in this view
   DWORD dwBytesInBlock = sinf.dwAllocationGranularity;
   if (qwFileSize < sinf.dwAllocationGranularity)
      dwBytesInBlock = (DWORD) qwFileSize;

   PBYTE pbFile = (PBYTE) MapViewOfFile(hFileMapping, FILE_MAP_READ,
         (DWORD) (qwFileOffset >> 32),          // Starting byte
         (DWORD) (qwFileOffset & 0xFFFFFFFF),   // in file
         dwBytesInBlock);                       // # of bytes to map

      // Count the number of 0s in this block.
      for (DWORD dwByte = 0; dwByte < dwBytesInBlock; dwByte++) {
         if (pbFile[dwByte] == 0)
         qwNumOf0s++;
       }

      // Unmap the view; we don't want multiple views
      // in our address space.
      UnmapViewOfFile(pbFile);

      // Skip to the next set of bytes in the file.
      qwFileOffset += dwBytesInBlock;
      qwFileSize -= dwBytesInBlock;
   }

   CloseHandle(hFileMapping);
   return(qwNumOf0s);
}

Memory-Mapped Files and Coherence

不保證多個file mapping obj下每個人的view依樣 建議用read only

Specifying the Base Address of a Memory-Mapped File

baseAddress可以拿來做share data,但address要在process的user mode partition

Implementation Details of Memory-Mapped Files

一個file mapping 不同的兩個view,但是他們可能是被allocate很遠的位子, 雖然map view只是下一個section

int WINAPI _tWinMain (HINSTANCE, HINSTANCE, PTSTR, int) {

   // Open an existing file--it must be bigger than 64 KB.
   HANDLE hFile = CreateFile(pszCmdLine, GENERIC_READ | GENERIC_WRITE,
      0, NULL, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, NULL);

   // Create a file-mapping object backed by the data file.
   HANDLE hFileMapping = CreateFileMapping(hFile, NULL,
      PAGE_READWRITE, 0, 0, NULL);

   // Map a view of the whole file into our address space.
   PBYTE pbFile = (PBYTE) MapViewOfFile(hFileMapping,
      FILE_MAP_WRITE, 0, 0, 0);

   // Map a view of the file (starting 64 KB in) into our address space
   PBYTE pbFile2 = (PBYTE) MapViewOfFile(hFileMapping,
     FILE_MAP_WRITE, 0, 65536, 0);

   // Show that the two views are not 64 KB away from each other
   // in the address space, meaning that there is no overlap.
   int iDifference = int(pbFile2 - pbFile);
   TCHAR szMsg[100];
   StringCchPrintf(szMsg, _countof(szMsg),
      TEXT("Pointers difference = %d KB"), iDifference / 1024);
   MessageBox(NULL, szMsg, NULL, MB_OK);

   UnmapViewOfFile(pbFile2);
   UnmapViewOfFile(pbFile);
   CloseHandle(hFileMapping);
   CloseHandle(hFile);

   return(0);
}

Using Memory-Mapped Files to Share Data Among Processes

  1. createFile to open .exe file on disk
  2. createfilemapping
  3. mapviewofFileEx with SEC_IMAGE
  4. new mapViewOfFileEx // use map 多個view來達到這個笑過

map同一個file到不同的addr來做到share data

Memory-Mapped Files Backed by the Paging File

如果沒有要丟到disk, ....????? 因為CreateFileMapping的error是NULL, 所以一定要在create file後檢查handle invalid When CreateFileMapping is called, INVALID_HANDLE_VALUE is passed in the hFile parameter, which causes the system to create a file mapping using storage from the paging file instead of the intended disk file

Sparsely Committed Memory-Mapped Files

if need to share a big spreadsheet

CreateFileMapping(invalid_handle_value, ..... SEC_RESERVE...);
LPVOID lpMVF = MapViewOfFile(...);
LPVOID lpVM = VirtualAlloc(lpMVF , MEM_COMMIT, PAGE_READWRITE);
unmaViewOfFile(lpMVF);
VirtuaF(lpVM , size, MEM_DECOMMIT);

只要記得SEC_RESERVE ＆ MEM_COMMIT

results matching ""

    No results matching ""