Bulat_Ziganshin Цитата: http://www.haskell.org/bz/arc-255m.7z - используй -di+$
Win64 - Гы работает в ручном режиме -lc- -ld-
На автомате Win32, Win64 - почему-то выставляется лимит 1920m?
(WinXP 64bit)2.5Гб+2Гб(файл подкачки)
Arc050.exe a a.arc -lc- -ld- -mXXX -di -di+$ 2207.tar
пределы
lzma:255m
Virtual Memory
Peak Private Bytes 2762700кб=2698Мб=2.63Гб
Virtual Size 2822684кб=2756Мб=2.69Гб
lzma:511m:fast
Virtual Memory
Peak Private Bytes 3423444кб=3343Мб=3.26Гб
Virtual Size 3482140кб=3400Мб=3.32Гб
ppmd:2047m
Virtual Memory
Peak Private Bytes 2114344кб=2064Мб=2.02Гб
Virtual Size 2175696кб=2125Мб=2.07Гб
rep:2047m:h28
Virtual Memory
Peak Private Bytes 3165176кб=3091Мб=3.02Гб
Virtual Size 3224276кб=3149Мб=3.07Гб
lzp:1649m:h26
Virtual Memory
Peak Private Bytes 1705848кб=1666Мб=1.63Гб\3660392кб=3575Мб=3.49Гб
Virtual Size 1767120кб=1726Мб=1.69Гб\3717848кб=3631Мб=3.55Гб
[more=full_log..]
(WinXP 64bit)2.5Гб+2Гб(файл подкачки)
Arc050.exe a a.arc -lc- -ld- -mXXX -di -di+$ 2207.tar
пределы
lzma:255m
Virtual Memory
Peak Private Bytes 2762700кб=2698Мб=2.63Гб
Virtual Size 2822684кб=2756Мб=2.69Гб
FreeArc 0.50 beta (Jan 13 2008) There is 15 percent of memory in use.
There are 2559 total Mbytes of physical memory.
There are 2167 free Mbytes of physical memory.
There are 4449 total Mbytes of paging file.
There are 4220 free Mbytes of paging file.
There are 4095 total Mbytes of virtual memory.
There are 4039 free Mbytes of virtual memory.
There are 0 free Mbytes of extended memory.
Test memory allocation using VirtualAlloc::MEM_RESERVE
Allocated 2047 mb, addr=7FFF0000
Allocated 1667 mb, addr=02CB0000
Allocated 104 mb, addr=6B2D0000
Allocated 73 mb, addr=78B90000
Allocated 69 mb, addr=71C20000
Allocated 19 mb, addr=7DCE0000
Allocated 15 mb, addr=76250000
Allocated 11 mb, addr=78000000
Allocated 3 mb, addr=777B0000
Allocated 3 mb, addr=772C0000
Allocated 2 mb, addr=6B050000
Allocated 1 mb, addr=7D6F0000
Creating archive: a.arc using lzma:255mb:normal:bt4:32
Memory for compression 2683mb, decompression 255mb, cache 1mb
Started: 0.00 secs
Found 1 files, 0 archives: 0.00 secs
Sorted 1 files: 0.00 secs
Joined filelists: 0.00 secs
2207.tar ["$incompressible","$incompressible","$incompressible"]There is 1
5 percent of memory in use.
There are 2559 total Mbytes of physical memory.
There are 2167 free Mbytes of physical memory.
There are 4449 total Mbytes of paging file.
There are 4209 free Mbytes of paging file.
There are 4095 total Mbytes of virtual memory.
There are 4023 free Mbytes of virtual memory.
There are 0 free Mbytes of extended memory.
Test memory allocation using VirtualAlloc::MEM_RESERVE
Allocated 2047 mb, addr=7FFF0000
Allocated 1649 mb, addr=03E80000
Allocated 104 mb, addr=6B2D0000
Allocated 73 mb, addr=78B90000
Allocated 69 mb, addr=71C20000
Allocated 19 mb, addr=7DCE0000
Allocated 15 mb, addr=76250000
Allocated 11 mb, addr=78000000
Allocated 3 mb, addr=777B0000
Allocated 3 mb, addr=772C0000
Allocated 2 mb, addr=03240000
Allocated 2 mb, addr=6B050000
Allocated 1 mb, addr=7D6F0000
Compressing 1 file of 2.314.722.816 bytes: 0.02 secs
Using lzma:255mb:normal:bt4:32
Memory for compression 2683mb, decompression 3.4%
lzma:511m:fast
Virtual Memory
Peak Private Bytes 3423444кб=3343Мб=3.26Гб
Virtual Size 3482140кб=3400Мб=3.32Гб
FreeArc 0.50 beta (Jan 13 2008) There is 15 percent of memory in use.
There are 2559 total Mbytes of physical memory.
There are 2160 free Mbytes of physical memory.
There are 4449 total Mbytes of paging file.
There are 4211 free Mbytes of paging file.
There are 4095 total Mbytes of virtual memory.
There are 4039 free Mbytes of virtual memory.
There are 0 free Mbytes of extended memory.
Test memory allocation using VirtualAlloc::MEM_RESERVE
Allocated 2047 mb, addr=7FFF0000
Allocated 1667 mb, addr=02CB0000
Allocated 104 mb, addr=6B2D0000
Allocated 73 mb, addr=78B90000
Allocated 69 mb, addr=71C20000
Allocated 19 mb, addr=7DCE0000
Allocated 15 mb, addr=76250000
Allocated 11 mb, addr=78000000
Allocated 3 mb, addr=777B0000
Allocated 3 mb, addr=772C0000
Allocated 2 mb, addr=6B050000
Allocated 1 mb, addr=7D6F0000
Creating archive: a.arc using lzma:511mb:fast:hc4:32
Memory for compression 3327mb, decompression 511mb, cache 1mb
Started: 0.00 secs
Found 1 files, 0 archives: 0.00 secs
Sorted 1 files: 0.00 secs
Joined filelists: 0.00 secs
2207.tar ["$incompressible","$incompressible","$incompressible"]There is 1
5 percent of memory in use.
There are 2559 total Mbytes of physical memory.
There are 2160 free Mbytes of physical memory.
There are 4449 total Mbytes of paging file.
There are 4201 free Mbytes of paging file.
There are 4095 total Mbytes of virtual memory.
There are 4023 free Mbytes of virtual memory.
There are 0 free Mbytes of extended memory.
Test memory allocation using VirtualAlloc::MEM_RESERVE
Allocated 2047 mb, addr=7FFF0000
Allocated 1649 mb, addr=03E80000
Allocated 104 mb, addr=6B2D0000
Allocated 73 mb, addr=78B90000
Allocated 69 mb, addr=71C20000
Allocated 19 mb, addr=7DCE0000
Allocated 15 mb, addr=76250000
Allocated 11 mb, addr=78000000
Allocated 3 mb, addr=777B0000
Allocated 3 mb, addr=772C0000
Allocated 2 mb, addr=03240000
Allocated 2 mb, addr=6B050000
Allocated 1 mb, addr=7D6F0000
Compressing 1 file of 2.314.722.816 bytes: 0.00 secs
Using lzma:511mb:fast:hc4:32
Memory for compression 3327mb, decompression 2.6%
ppmd:2047m
Virtual Memory
Peak Private Bytes 2114344кб=2064Мб=2.02Гб
Virtual Size 2175696кб=2125Мб=2.07Гб
FreeArc 0.50 beta (Jan 13 2008) There is 15 percent of memory in use.
There are 2559 total Mbytes of physical memory.
There are 2170 free Mbytes of physical memory.
There are 4449 total Mbytes of paging file.
There are 4223 free Mbytes of paging file.
There are 4095 total Mbytes of virtual memory.
There are 4039 free Mbytes of virtual memory.
There are 0 free Mbytes of extended memory.
Test memory allocation using VirtualAlloc::MEM_RESERVE
Allocated 2047 mb, addr=7FFF0000
Allocated 1667 mb, addr=02CB0000
Allocated 104 mb, addr=6B2D0000
Allocated 73 mb, addr=78B90000
Allocated 69 mb, addr=71C20000
Allocated 19 mb, addr=7DCE0000
Allocated 15 mb, addr=76250000
Allocated 11 mb, addr=78000000
Allocated 3 mb, addr=777B0000
Allocated 3 mb, addr=772C0000
Allocated 2 mb, addr=6B050000
Allocated 1 mb, addr=7D6F0000
Creating archive: a.arc using ppmd:10:2047mb
Memory for compression 2047mb, decompression 2047mb, cache 1mb
Started: 0.00 secs
Found 1 files, 0 archives: 0.00 secs
Sorted 1 files: 0.00 secs
Joined filelists: 0.00 secs
2207.tar ["$incompressible","$incompressible","$incompressible"]There is 1
5 percent of memory in use.
There are 2559 total Mbytes of physical memory.
There are 2169 free Mbytes of physical memory.
There are 4449 total Mbytes of paging file.
There are 4212 free Mbytes of paging file.
There are 4095 total Mbytes of virtual memory.
There are 4023 free Mbytes of virtual memory.
There are 0 free Mbytes of extended memory.
Test memory allocation using VirtualAlloc::MEM_RESERVE
Allocated 2047 mb, addr=7FFF0000
Allocated 1649 mb, addr=03E80000
Allocated 104 mb, addr=6B2D0000
Allocated 73 mb, addr=78B90000
Allocated 69 mb, addr=71C20000
Allocated 19 mb, addr=7DCE0000
Allocated 15 mb, addr=76250000
Allocated 11 mb, addr=78000000
Allocated 3 mb, addr=777B0000
Allocated 3 mb, addr=772C0000
Allocated 2 mb, addr=03240000
Allocated 2 mb, addr=6B050000
Allocated 1 mb, addr=7D6F0000
Compressing 1 file of 2.314.722.816 bytes: 0.02 secs
Using ppmd:10:2047mb
Memory for compression 2047mb, decompression 0.5%
rep:2047m:h28
Virtual Memory
Peak Private Bytes 3165176кб=3091Мб=3.02Гб
Virtual Size 3224276кб=3149Мб=3.07Гб
FreeArc 0.50 beta (Jan 13 2008) There is 15 percent of memory in use.
There are 2559 total Mbytes of physical memory.
There are 2157 free Mbytes of physical memory.
There are 4449 total Mbytes of paging file.
There are 4209 free Mbytes of paging file.
There are 4095 total Mbytes of virtual memory.
There are 4039 free Mbytes of virtual memory.
There are 0 free Mbytes of extended memory.
Test memory allocation using VirtualAlloc::MEM_RESERVE
Allocated 2047 mb, addr=7FFF0000
Allocated 1667 mb, addr=02CB0000
Allocated 104 mb, addr=6B2D0000
Allocated 73 mb, addr=78B90000
Allocated 69 mb, addr=71C20000
Allocated 19 mb, addr=7DCE0000
Allocated 15 mb, addr=76250000
Allocated 11 mb, addr=78000000
Allocated 3 mb, addr=777B0000
Allocated 3 mb, addr=772C0000
Allocated 2 mb, addr=6B050000
Allocated 1 mb, addr=7D6F0000
Creating archive: a.arc using rep:2047mb:h28
Memory for compression 3071mb, decompression 2047mb, cache 1mb
Started: 0.00 secs
Found 1 files, 0 archives: 0.00 secs
Sorted 1 files: 0.00 secs
Joined filelists: 0.00 secs
2207.tar ["$incompressible","$incompressible","$incompressible"]There is 1
5 percent of memory in use.
There are 2559 total Mbytes of physical memory.
There are 2156 free Mbytes of physical memory.
There are 4449 total Mbytes of paging file.
There are 4199 free Mbytes of paging file.
There are 4095 total Mbytes of virtual memory.
There are 4023 free Mbytes of virtual memory.
There are 0 free Mbytes of extended memory.
Test memory allocation using VirtualAlloc::MEM_RESERVE
Allocated 2047 mb, addr=7FFF0000
Allocated 1649 mb, addr=03E80000
Allocated 104 mb, addr=6B2D0000
Allocated 73 mb, addr=78B90000
Allocated 69 mb, addr=71C20000
Allocated 19 mb, addr=7DCE0000
Allocated 15 mb, addr=76250000
Allocated 11 mb, addr=78000000
Allocated 3 mb, addr=777B0000
Allocated 3 mb, addr=772C0000
Allocated 2 mb, addr=03240000
Allocated 2 mb, addr=6B050000
Allocated 1 mb, addr=7D6F0000
Compressing 1 file of 2.314.722.816 bytes: 0.00 secs
Using rep:2047mb:h28
Memory for compression 3071mb, decompression 17.7%
lzp:1649m:h26
Virtual Memory
Peak Private Bytes 1705848кб=1666Мб=1.63Гб\3660392кб=3575Мб=3.49Гб
Virtual Size 1767120кб=1726Мб=1.69Гб\3717848кб=3631Мб=3.55Гб
FreeArc 0.50 beta (Jan 13 2008) There is 16 percent of memory in use.
There are 2559 total Mbytes of physical memory.
There are 2148 free Mbytes of physical memory.
There are 4449 total Mbytes of paging file.
There are 4195 free Mbytes of paging file.
There are 4095 total Mbytes of virtual memory.
There are 4039 free Mbytes of virtual memory.
There are 0 free Mbytes of extended memory.
Test memory allocation using VirtualAlloc::MEM_RESERVE
Allocated 2047 mb, addr=7FFF0000
Allocated 1667 mb, addr=02CB0000
Allocated 104 mb, addr=6B2D0000
Allocated 73 mb, addr=78B90000
Allocated 69 mb, addr=71C20000
Allocated 19 mb, addr=7DCE0000
Allocated 15 mb, addr=76250000
Allocated 11 mb, addr=78000000
Allocated 3 mb, addr=777B0000
Allocated 3 mb, addr=772C0000
Allocated 2 mb, addr=6B050000
Allocated 1 mb, addr=7D6F0000
Creating archive: a.arc using lzp:1649mb:64:h26
Memory for compression 3554mb, decompression 3554mb, cache 1mb
Started: 0.02 secs
Found 1 files, 0 archives: 0.02 secs
Sorted 1 files: 0.02 secs
Joined filelists: 0.02 secs
2207.tar ["$incompressible","$incompressible","$incompressible"]There is 1
6 percent of memory in use.
There are 2559 total Mbytes of physical memory.
There are 2148 free Mbytes of physical memory.
There are 4449 total Mbytes of paging file.
There are 4185 free Mbytes of paging file.
There are 4095 total Mbytes of virtual memory.
There are 4023 free Mbytes of virtual memory.
There are 0 free Mbytes of extended memory.
Test memory allocation using VirtualAlloc::MEM_RESERVE
Allocated 2047 mb, addr=7FFF0000
Allocated 1649 mb, addr=03E80000
Allocated 104 mb, addr=6B2D0000
Allocated 73 mb, addr=78B90000
Allocated 69 mb, addr=71C20000
Allocated 19 mb, addr=7DCE0000
Allocated 15 mb, addr=76250000
Allocated 11 mb, addr=78000000
Allocated 3 mb, addr=777B0000
Allocated 3 mb, addr=772C0000
Allocated 2 mb, addr=03240000
Allocated 2 mb, addr=6B050000
Allocated 1 mb, addr=7D6F0000
Compressing 1 file of 2.314.722.816 bytes: 0.02 secs
Using lzp:1649mb:64:h26
Memory for compression 3554mb, decompression 74.6%
[/more]
Цитата: помимо прочего, я переделал lzma. сейчас он выделяет память трёмя кусками (8x, 1x и 1.5x)
В документации надо привести пример на практике, как правильно в ручную рассчитывать пределы, размер словаря\размер хэша и т.п.
Добавлено: Цитата: На автомате Win32, Win64 - почему-то выставляется лимит 1920m?
уточнее - автолимиты памяти
(WinXP 32bit)2.5Гб+2Гб(файл подкачки)
Arc050.exe a a.arc -mXXX:XXXXm -di -di+$ 2207.tar
Allocated 1562 mb, addr=10070000
Allocated 218 mb, addr=02570000
Allocated 86 mb, addr=71AC0000
Allocated 72 mb, addr=77FE0000
Allocated 45 mb, addr=7C9C0000
Allocated 7 mb, addr=7F7F0000
Allocated 4 mb, addr=77610000
Allocated 2 mb, addr=01970000
Allocated 1 mb, addr=77250000
ppmd:3000m
Using ppmd:8:1920mb
Memory for compression 1920mb, decompression 1920mb
ERROR: Can't allocate memory required for (de)compression in ppmd:8:1920mb
lzp:3000m
Using lzp:2208mb:64:h18
Memory for compression 321mb, decompression 321mb
ERROR: Can't allocate memory required for (de)compression in lzp:2208mb:64:h18
rep:3000m - 1280m (rep:1gb)
lzma:3000m - 1348m (lzma:128m)
ppmd:2047m
Using ppmd:10:2050476kb
Memory for compression 2002mb, decompression 2002mb
ERROR: Can't allocate memory required for (de)compression in ppmd:10:2050476kb
lzp:2047m
Compressing 1 file of 2.314.722.816 bytes: 0.02 secs
Using lzp:982528kb:64:h18
Memory for compression 1920mb, decompression 41.9%
ERROR: Can't allocate memory required for (de)compression in lzp:982528kb:64:h18
rep:2047m - 1280m (rep:1gb)
lzma:2047m - 1348m (lzma:128m)
rep
- предел 1305m (1306m-1536m не хватает памяти, начиная с 1537m авто срабатывает до rep:1gb)
Using rep:1536mb
Memory for compression 1792mb, decompression 1536mb
ERROR: Can't allocate memory required for (de)compression in rep:1536mb
rep - предел 1562m:h25
ppmd - предел 1562m
lzp - предел 780m:h25
lzma
- предел lzma:164m (lzma:165m-188m не хватает памяти, начиная с 189m авто срабатывает до lzma:128m)
Using lzma:188mb:normal:bt4:32
Memory for compression 1918mb, decompression 188mb
ERROR: Can't allocate memory required for (de)compression in lzma:188mb:normal:bt4:32