__________________
A MacPro 2.66 Xeon, an iMac G5, a Uni MB, a wee PowerBook 12" 1.5, an eMac, a couple of cameras, a lens collection and a wonderful iPod. Oh, and a delicious wife.
"Words are sacred. They deserve respect. If you get the right ones in the right order, you can nudge the world a little." Tom Stoppard
“No tears in the writer, no tears in the reader".
Robert Frost
That is really neat. Sounds like Apple is keeping customers from wasting money by going to 4GB when most of that last GB (~700MB) would be wasted anyways.
It's interesting, that Apple, a premium-priced company would be more $$-savvy for it's customers than HP or Dell, the more sales-priced companies.
As I understand it, some people feel that memory above 3GB isn't useful with or without a hardware limit. They suggest 3GB is all you need because the applications we use so far have what amount to 2GB maximums. Thus 3GB is fine, leaving a gig for the rest of the system.
With regard to the MacBookPro, that limit appears to be hardware limited; the last Gig or RAM nets you about 300 MB of useable memory, because the Intel controller chip assigns those addresses already.
It does this, presumably, because it's limited to 4GB because it's a 32-bit device, yet it needs to assign those addresses somewhere. So, it uses the last 700 MB or so in the addresses it has available, since it's least likely for most users that they will have real RAM in that address space as well. Of course, if you install more than 3GB, well, you're one of those guys who are not like most users, and that address space is not available to you. Thus, 3.3GB is all you can use.
If there was no hardware limit, then we come to the software limits, where again some suggest whopping RAM installations are unnecessary, because applications won't use more than 2GB themselves.
However, the OS still needs memory, there is still advantages to reducing OSX's SwapFile pageouts and pageins, and there is still advantages with storage of large data sets (applications that use their own swapfile regimen, like PhotoShop and FinalCutPro, caches, and clipboard storage/export/import).
One test on a Mac Pro desktop:
with a 108MB image, performance leveled off at 1.5 GB
with a 192.6 MB image, performance showed gains with each incremental increase up to 4GB (almost 2x as fast as 2GB). It leveled off at 4GB (407 seconds vs 400 seconds with 8GB). I'm sure more demanding use would reveal larger improvements with installations with more than 4GB, although I suppose the emphasis should be on "demanding". 3D maybe.
Remember, this is with an application that will only recognize something less than 2GB (in this test, it used a maximum of 1807MB when set to use 2GB and with 4GB installed; it needed a little less, 1678MB, when set to use 2GB and with 8GB installed) <i>for the application itself</i> but does use it for swapfiles, caches, and clipboard storage.
If one were to use a RAMDisk, I would expect further improvements. I use a small one for all Safari caches and it works wonders; on the desktop and I will be trying it on the laptop soon (Esperance DV: freeware).
As always, depends on what you do. Those looking for hard and fast rules will once again be disappointed. I would suggest that if you leave everything in default configurations the 3GB limit is probably a good rule of thumb. If you optimize for a high performance hardware setup, which you should anyway if you have it, then more gains are indeed available to you with more memory. Bang-for-the-buck seems to be 2GB, or perhaps even 1GB if you don't work with large files.
Last edited by gordguide; Oct 31st, 2006 at 10:01 AM.
Thanks for the detailed post. I am always interested in learning ways to tweak performance, so I appreciate the input.
__________________
A MacPro 2.66 Xeon, an iMac G5, a Uni MB, a wee PowerBook 12" 1.5, an eMac, a couple of cameras, a lens collection and a wonderful iPod. Oh, and a delicious wife.
"Words are sacred. They deserve respect. If you get the right ones in the right order, you can nudge the world a little." Tom Stoppard
“No tears in the writer, no tears in the reader".
Robert Frost
Its too bad that the intel mobile platforms dont properly support more than 3GB of RAM because...
...heres what I have seen first hand. The more RAM you have the more applications you can keep open without disk-swapping. Screw all those 2GB app limits and only this much is addressable and whatever....
With 4-5GB of RAM OSX and hungry apps will eat up all your RAM in a second. OSX eventually will move everything it has into main memory if you let it. I would love to have 8GB of RAM because its VERY EASY for OSX + Aperture + Photoshop + Mail + Safari + iChat + iPhoto to use 4GB of RAM. Screw performance benchmarks!!! benchmarks dont account for pagefile activity, they benchmark the application that is fully loaded in memory, and most often its the only app running on the machine.
If im quickly switching between aperture and photoshop and the app has to be pulled from the pagefile then I have to wait while the app loads to main memory from the pagefile on disk. If its a big app then it takes time and you can hear the disk thrashing away.
anyways, THE MORE RAM THE BETTER FOR OSX! If you are a pro then I would say 4GB MINIMUM! And if you are a serious mac using pro then 8GB or higher depending on the apps you run.
__________________
Hi-diddly-ho neighbourino!
Last edited by saxamaphone; Oct 31st, 2006 at 12:53 PM.
Reason: relate back to thread topic
Funny, that's not my experience at all. With 1.5 GB on a G4 my swapfile has zero or near zero page-outs, and that's while doing graphics, video, DVD authoring, real-time audio recording at 24/96, etc. I often have a dozen or more apps running simultaneously.
Generally very little, if any, is written to the page files when you have adequate RAM (very broadly speaking, with OSX and non-intensive use, 768MB is plenty to reduce swap to a minimum).
You may be thinking the swapfile is large when it is really not; and the reason is pretty much the same reasons why you can't access the RAM above 3.3 GB with the Intel laptops, as it turns out.
There are many resources that are shared across many applications but are seldom accessed by them. One example is shared libraries; if they were treated as "regular" data by the VM system they would be written to the swapfile once for each application that accessed them, and with certain common libraries that could be every application.
But, they are not. They are mapped to the memory space but not actually loaded off the HD to the swap directory. Just like the PCIe bus doesn't really use RAM but needs to be accounted for as if it did (and thus is mapped to the top 700 MB of maximum RAM like in the MacBook Pro) the "mapped files" are accounted for as if they were written to VM but they are not.
It is not unusual to find 80% or more of what is allocated as VM to be mapped files; in other words only 20% or less of what is reported as VM is actually written to the swap file. Mapping resources to memory spaces is just one example of how computers work; it doesn't mean that any memory is actually used.
There is also little use in placing mapped files into RAM rather than VM, because they are seldom accessed and therefore provide only incremental speed boosts and only for brief periods of time; basically they won't speed anything up, really.
Look at the page-ins (data actually written from RAM to the swapfile) and pageouts (data written at least once from RAM to the swapfile that is needed a second time by an open application). That's where you see how much VM you are using. Pageins are normal; any time data in RAM hasn't been used for a while it will be written to VM (a page-in).
If you run short sessions (say 18 hours or less) I would expect to see very few pageouts; if you have long uptimes (logged in for days or months) you would naturally expect to see a lot more, in particular if you run a series of seldom used applications during that session.
As an example, given adequate RAM to run PhotoShop and given that you took the time to configure PhotoShop to use that RAM (if you don't, it will use a few hundred MB and use OSX's swapfile just like any other application) you should see zero or near zero activity in OSX's swapfile from using PhotoShop. Extra RAM over and above what PhotoShop uses and what OSX and the rest use (ie roughly anything over 3GB) , will be used by PhotoShop to reduce it's own VM swapfile for images; if it can it will load it all into free RAM. Considering that can need more than 100 MB for each level of undo when you work with a 100 MB file, that can be quite a bit. All the while, OSX's own swapfile will have few pageins and outs and even though it will report shared libraries as being in swap, leading you to believe there are hundreds of MB in the swapfile, actual data written to the /swap partition may be negligible.
Currently my memory data is (via Activity Monitor)
1.5 GB RAM installed; used 1.13 GB
That 1.13 includes 528 MB used as a RAMDisk and therefore unavailable to the System or other Applications, yet the OS seems to be fine; having 300+ MB of free, unused RAM.
Threads: 161 Processes: 55
VM size: 5.92 GB
Pageins: 42006
Pageouts: 117
Uptime: 37949 seconds (from the Snort NIDS)**
As you can see, in 11 hours I needed to re-use from VM something that had at one time been in RAM a total of 117 pages worth. Each page is 4096 bytes, and I would venture to guess that it would be rare indeed if all 4K was actual data; chances are there is a lot of unallocated space on that page. Not very much at all, and it's obvious that, with 300 MB of physical RAM that's never been touched yet, that there was no RAM shortage. Whatever prompted the pageouts was probably totally insignificant; it was probably just convenient to get it from there at the time.
Looking at a process more carefully (choosing the one with the most VM use listed):
Safari: Real Memory: 39.28 MB Virtual Memory: 208.80
Sounds pretty bad. But what exactly is going on with VM?
Let's pop into the terminal and see what Safari has in VM: vmmap Safari
... a bunch of stuff, scroll to the bottom ... ==== Summary for process 935
ReadOnly portion of Libraries: Total=98656KB resident=44112KB(45%) swapped_out_or_unallocated=54544KB(55%)
Writable regions: Total=194452KB written=34384KB(18%) resident=57500KB(30%) swapped_out=0KB(0%) unallocated=136952KB(70%)
So, in 11 hours, with up to a dozen apps running simultaneously, with as many as 30 browser windows open simultaneously, today Safari has actually used no VM at all: swapped out=0KB(0%)
And, most of the reserved space for VM is empty (unallocated: 55% and 70%). Of the 220 MB it reports Safari using, 54544KB and 136952KB are unallocated: that's almost 200 MB right there. Marked as used, but in reality, it's empty.
What actually is in VM is mostly shared libraries, which are never actually written to the swapfile.
There is exactly 34384KB, or roughly 34MB, of actual swap data, and none of it was ever needed (swapped_out=0KB(0%), which is confirmed in Process Viewer's memory page for Safari: Pageouts = 0.
Now, I'm not saying this was a particularly trying day on the old QuickSilver. But, don't think that large amounts of VM reported in Process Viewer actually means you are using a lot of VM to swap out data from RAM. With lots of physical memory, even with demanding applications, you probably use much, much less than you think.
In fact, at any time, go ahead and check the actual swap file. It's even more enlightening; since it records the actual amount of data held in VM. You could find more than one; OSX creates a new one when the first one gets to be 80MB. Right now mine is ... wait for it ... 67, 108, 864 bytes. A single swap file of a whopping 64 MB.
[ To see yours, but please please please don't screw with it, go to Finder: Go: and type in the path:
/var/vm
And Please, please, please don't screw with anything you find in that folder, or anything else in /var. You will be sorry if you do. You can, however, do a Get Info on the file to see how big it is. Then get the hell out of there. It's hidden for a reason].
Forget the 6GB I found reported in Process Viewer: that includes all of the total memory being used right now of every kind, whether real, physical RAM, or virtual memory or shared resources that isn't in VM or any memory resource that has been allocated but not used and probably never will be used; kind of like just housekeeping by keeping a room clean in case someone visits. It's a 6GB hotel with one 64MB virtual room full, and a big blinking neon sign outside saying "no vacancy".
That's not to say it's unnecessary; UNIX needs to do it's housekeeping and it needs to know it has a place to put critical stuff if it ever needs to. Don't confuse it with actual, used resources, that's all.
Both Apeture and PhotoShop have their own, proprietary Virtual Memory systems for the files they have open. This dedicated VM to a large extent holds multiple copies of the open file for undo's (which always is many times more than the size of the open file alone), for clipboard support, and for use by plugins.
That's what you hear from the hard drive when you switch from one to the next; it most certainly isn't Apple's VM. It's up to the developers of these applications to use RAM in place of their own proprietary VM systems (which they do).
I never said more memory isn't better, but beyond 3GB it's very unlikely if the actual application themselves can use it. Smarter apps will use it for open files themselves over caching systems, which most every high end application employs, but they are unrelated to Apple's VM or memory management; they use the unused RAM after OSX and open applications are happy and have enough.
PhotoShop, Apeture, FinalCut Pro ... these are very intensive applications and thier own swap files are sophisticated and work very well with extra memory, but in the grand scheme of things, they are exceptions. Relatively few applications cache open files, undos, etc at all. Of those that do, almost certainly some won't have implemented moving those caches to RAM when there is extra available. More memory won't help these apps at all; they already are using all they know how to use.
I will say it one more time: if you use demanding applications, and those applications are written so that they can take advantage of the extra memory, more power to you. Gobs of memory will make noticeable improvements under certain, but not all, circumstances.
If they are not so written, or if you do not use resource intensive applications and data sets, it won't speed up anything beyond a certain point. Nor will you significantly reduce pageouts from Apple's VM, under most circumstances once you approach 1GB RAM you will be at zero or near zero pageouts. That point is probably somewhere (depending on what you're doing) between 768MB and 3 GB.
** NIDS = Network Intrusion Detection System
Last edited by gordguide; Oct 31st, 2006 at 09:42 PM.
Gordguide said...
"
If they are not so written, or if you do not use resource intensive applications and data sets, it won't speed up anything beyond a certain point. Nor will you significantly reduce pageouts from Apple's VM, under most circumstances once you approach 1GB RAM you will be at zero or near zero pageouts. That point is probably somewhere (depending on what you're doing) between 768MB and 3 GB.
"
I can only speak from observation and experience, but I do not agree with what you say. Great theory though, I wish it was true.
OSX seems to behave differently when it has more than 2GB of memory. It caches things that it normally wouldnt even try to cache. I saw this with my powermac g5 before and after memory upgrades. When i open the display pref pane and select a photo from iphoto for my desktop it works great with very little memory (running 512 on dual G5). But when I installed 4GB of memory all of a sudden OSX was aggressively cacheing all these iphotos into main memory. I can just sit there and watch as OSX caches over a 1GB of photo thumbnails (~7500 photos). You can hear it load from the drive, you can watch the page-ins in activity monitor, you can watch the hard drive activity in activity monitor. Next thing you know OSX is gobbling up 2GB or more of main memory. Same thing with other applications, it seems they are more aggressive on what they load to main memory when there is more memory available. It must be the OSX memory manager. I find it very easy to fill up 4GB of real memory and as soon as you open another application, BANG - the page-outs happen. you would think 4GB would be fine but there is still disk action as OSX shifts around what is stored in main memory.
That is my practical experience working on my powermac G5 and my Mac Pro.
I assume that the desktop photo thing was just an example. Normally I would never ask the Finder to redraw a screen from a large image selected from a large iPhoto library if I expected to maximize performance. Similarly, selecting list view with few options (no "calculate sizes' etc) will all speed up any computer of any vintage if you need horsepower for some reason. The Finder is an application like any other and asking it to be busy and hog resources when it could just as easily be relaxing is going to hurt performance, VM or no VM.
With OSX, simply putting another 8GB of physical RAM in the machine will automatically cause VM to reserve another 8GB of VM; it doesn't mean it actually has written an 8GB swapfile to disk or even a 1GB swapfile. It does mean it will expect a VM space equal to another 8GB, though. That's just how VM works. I guarantee that if you go by what Process Viewer reports, you will be wildly misled as to the true amount of swap you are using.
But, I am curious. Assuming you do have your desktop picture selected as one image from your iPhoto library, and your library is 7000+ images, could you check your /var/vm folder and see how big the swapfile folder is under these conditons?
[From the Finder: Go: and enter the pathname /var, and look for the vm folder, select it and enter Command-I or Control/Right Click and select Get Info].
Last edited by gordguide; Nov 1st, 2006 at 05:31 PM.
.
Linear Mode
