Tmpfs considered harmful(rwmj.wordpress.com)
rwmj.wordpress.com
Tmpfs considered harmful
https://rwmj.wordpress.com/2012/09/12/tmpfs-considered-harmful/#content
84 comments
Why on earth would anybody ever use tmpfs for /tmp? If I'm writing a program that deals with a big file, and I want to temporarily write something to the filesystem, it is because I dont want it in memory. Why would you ever write to /tmp for any reason other than to get something out of ram for a while?
I have been using tmpfs for /tmp for many years (since 2005 IIRC). Note that I do have large amounts of swap space (typically 64GB).
One simple benefit is that files there do not get caught in backups - they are temporary detritus so I don't want backup space wasted by them.
The major benefit is that fsync is really slow on Linux. Typically it turns into a sync, and a lot of programs like to ensure you don't lose data by being fsync happy. For temporary stuff that is even more painful - you don't care about data loss - that is why it is in tmp in the first place.
TLDR: I prefer filesystem operations on transient/temporary files and data to run at the speed of RAM, and not end up in backups or waiting for syncs.
One simple benefit is that files there do not get caught in backups - they are temporary detritus so I don't want backup space wasted by them.
The major benefit is that fsync is really slow on Linux. Typically it turns into a sync, and a lot of programs like to ensure you don't lose data by being fsync happy. For temporary stuff that is even more painful - you don't care about data loss - that is why it is in tmp in the first place.
TLDR: I prefer filesystem operations on transient/temporary files and data to run at the speed of RAM, and not end up in backups or waiting for syncs.
Because tmpfs is RAM cached disk (swap) space just like any other filesystem?
Is there some better place than swap to accommodate /tmp?
Is there some better place than swap to accommodate /tmp?
Because a third party binary expects posix FS semantics for some data that does not require strong durability?
This is very popular amongst people just working on desktop setups. They usually don't need /tmp to house anything too large. Arch Linux actually switched recently to mounting /tmp as tmpfs recently.
I use /tmp all the time. I find myself needing to interact with things for which the filesystem/shell interface is appropriate, but for which I don't want to worry about having to clean out the directory before it gets too large.
Having it sit in RAM is great, because I get the speed without subjecting my disk to unnecessary writes - for certain operations, this can be a huge bonus.
Having it sit in RAM is great, because I get the speed without subjecting my disk to unnecessary writes - for certain operations, this can be a huge bonus.
tmpfs is useful for embedded systems where the primary filesystem is mounted read-only. Generally /tmp and /var may be tmpfs backed in that situation. Not sure why a desktop system would want this though.
Here, let me man 7 hier that for you:
/tmp This directory contains temporary files which may be deleted with no notice, such as by a regular job or at system boot up.
/tmp This directory contains temporary files which may be deleted with no notice, such as by a regular job or at system boot up.
Running out of swap and memory is one thing, although unlikely considering how much of that stuff we've got nowadays. I for instance have 8gb of main memory and 10gb of swap space. 300mb are used after system boot (still too much). That leaves me with plenty of space to "waste" on temporary files in a tmpfs.
The Linux FHS explicitly states "Programs must not assume that any files or directories in /tmp are preserved between invocations of the program." That's not only reboots. It's not meant to be used to store drafts or anything that you might want to reopen at a later time. That's what we've got /var/tmp for. The FHS specifies as follows: "The /var/tmp directory is made available for programs that require temporary files or directories that are preserved between system reboots. Therefore, data stored in /var/tmp is more persistent than data in /tmp".
Tmpfs isn't harmful because of this, it's the default mutt configuration which is broken. If developers would have adhered to the standard the problem wouldn't exist. Since programs should adhere to the standard, because decisions like the one in question are made upon it, they should be reviewed and their default configurations should be changed in order to fix this issue.
The Linux FHS explicitly states "Programs must not assume that any files or directories in /tmp are preserved between invocations of the program." That's not only reboots. It's not meant to be used to store drafts or anything that you might want to reopen at a later time. That's what we've got /var/tmp for. The FHS specifies as follows: "The /var/tmp directory is made available for programs that require temporary files or directories that are preserved between system reboots. Therefore, data stored in /var/tmp is more persistent than data in /tmp".
Tmpfs isn't harmful because of this, it's the default mutt configuration which is broken. If developers would have adhered to the standard the problem wouldn't exist. Since programs should adhere to the standard, because decisions like the one in question are made upon it, they should be reviewed and their default configurations should be changed in order to fix this issue.
My theory on this is that it's intended as a step to avoid SSDs writes.
Exactly - this is a major benefit of using tmpfs on /tmp
The fact is, is that not every app that uses /tmp does so in a safe manner.
As long as that is true, it's not a good idea to use tmpfs for /tmp.
It's perfectly reasonable, however, to use tmpfs for, say, /var/local/tmp and let individual users point $TMP at that. Want to live fast and dangerously? Set $TMP.
As long as that is true, it's not a good idea to use tmpfs for /tmp.
It's perfectly reasonable, however, to use tmpfs for, say, /var/local/tmp and let individual users point $TMP at that. Want to live fast and dangerously? Set $TMP.
This assumes processes use $TMP.
I'd disagree with the more global point (in the title) that tmpfs is harmful: it's actually quite handy for all sorts of things.
As an amusing example, the SHM (shared memory) system in linux is actually implemented on top of tmpfs (/dev/shm is a tmpfs mount).
Other uses include CoW shadow mounting (when unioned to an immutable filesystem).
That said, the point that /tmp should not be a tmpfs mount is actually a decent one, just because we're used to the idea that /tmp is neither space-constrained (it's scratch space) nor extra performant.
As an amusing example, the SHM (shared memory) system in linux is actually implemented on top of tmpfs (/dev/shm is a tmpfs mount).
Other uses include CoW shadow mounting (when unioned to an immutable filesystem).
That said, the point that /tmp should not be a tmpfs mount is actually a decent one, just because we're used to the idea that /tmp is neither space-constrained (it's scratch space) nor extra performant.
I'm surprised no one is chanting "CADT" and linking to "What killed Linux on the desktop". This sort of subtle breakage over time is absolutely goddamn maddening.
Solaris has done this for many years; it's not the end of the world as the author believes.
tmpfs is awesome to run rails from (i.e. for development).
IO bound becomes CPU bound so you're getting as good speed as you can get.
IO bound becomes CPU bound so you're getting as good speed as you can get.
9 times out of 10, tmpfs is explicitly not for use mounted as /tmp. The only obvious exception I can think of are cluster nodes where the workload is specialized and the node has no local storage whatsoever.
I guess it only takes one silly person to connect "tmp" in the name to "/tmp", and the rest is history.
I guess it only takes one silly person to connect "tmp" in the name to "/tmp", and the rest is history.
sysfs is mounted on /sys
debugfs is mounted on /sys/kernel/debug
securityfs is mounted on /sys/kernel/security
So tmpfs is mounted on /tmp? Can see that...
debugfs is mounted on /sys/kernel/debug
securityfs is mounted on /sys/kernel/security
So tmpfs is mounted on /tmp? Can see that...
It makes sense in certain cases to use tmpfs for /tmp. However, I'm not sure this is on desktop machines, which (generally) have lower RAM than servers thrashing their /tmp directories.
Even in that case, using tmpfs on /tmp sounds like band aid. If there are applications that suffer performance because stuff it puts in /tmp , then change those applications - possibly by using an application specific tmpfs , or a new system wide tmpfs that doesn't overload something as ubiquitous /tmp
The systemwide tmpfs already exists: /dev/shm.
Minor disagreement about the following: "Everyone must worry about whether their files need to survive a reboot"
I believe there was never such expectation of /tmp. On many systems it's being wiped periodically or even when the user's last session ends. The example app(s) that use /tmp for drafts need(s) to be fixed.
I believe there was never such expectation of /tmp. On many systems it's being wiped periodically or even when the user's last session ends. The example app(s) that use /tmp for drafts need(s) to be fixed.
Totally agree, /tmp should be wiped at the end of each session.
The key is that it may be wiped. Files stored in /tmp are not guaranteed to survive boot. Whether or not they are there or not is a bit of a gamble (a regularly losing one should you choose tmpfs).
Agreed; this prevents establishment of expectations otherwise.
Also prevents it filling up your disk.
This is exactly what I use it for on my desktop. I have any app that I don't want to maintain longterm history for it's internal files (ex: Firefox browser cache) pointed to /tmp (mounted as tmpfs). As an added bonus if you're running your OS on an SSD (which you should be) you don't have it thrashed by constant writes/deletes.
The memory size issue is basically a non-issue at this point (pair of desktop 16 GB DIMMS are ~$80 and laptop isn't that much more).
The memory size issue is basically a non-issue at this point (pair of desktop 16 GB DIMMS are ~$80 and laptop isn't that much more).
The memory size issue is very much still an issue. I happen to have globs of memory on my machine, but for the very reason that I have software to use it. If large tmp files start eating into the resources available to my programs, its an issue for me. Also, older hardware that has less memory available, and even newer hardware that can have more memory installed but with non-tech users who don't know how to upgrade it or even that its possible (or can't afford $80).
If it's an issue for you, you can simply umount the tmpfs and remove the entry for fstab. I think this is a good default - on all mystems these days, it's disk-io that's the killer, not running out of memory.
I can verify this - on my Fedora 12 box, I accidentally set up a test MySQL database into /tmp. The test database grew into my blog, and it worked great till I rebooted. Then I learned lots about recovering data. (I make regular backups, of coures, but who backs up /tmp?)
Sure, but if the power goes down, there's at least a chance that I may be able to find that long email I was writing. With /tmp on tmpfs, it's gone.
That's why the GP considers mutt broken. Drafts should be stored in ~/.mutt or /var, not /tmp. Files in /tmp should not be expected to survive a process.
This is exactly the point of the posting. Every program has to be examined for this problem, and it forces a complex decision about the storage hierarchy onto every programmer.
Every program has had to be designed with this in mind for decades or be broken on any number of systems. This may be new behaviour for Fedora, but it's not a new thing on Linux, or for Unix-like systems in general.
It's not just systems that delete on reboot, but also systems set up with cron-jobs to regularly wipe /tmp, or sysadmins that will (rightfully) see /tmp as something that can be wiped at pretty much time.
It's not just systems that delete on reboot, but also systems set up with cron-jobs to regularly wipe /tmp, or sysadmins that will (rightfully) see /tmp as something that can be wiped at pretty much time.
That's like saying "RAM considered harmful", because every time a developer has to think "do I just store this in memory, or do I sync it to disk" "it forces a complex decision about the storage hierarchy": /tmp is simply RAM that has the API of a file; it used to be implemented using a horrible kludge with files on disk that were periodically deleted, but with tmpfs the dream became a reality.
/tmp is simply RAM that has the API of a fileAccording to the comments on the original article, that location is /var/tmp. Apparently it's some sort of convention/standard.
First time I've heard about /var/tmp.
First time I've heard about /var/tmp.
/tmp. If it's tmpfs, it can expand into swap space.
Swap is usually no bigger than two or three times your RAM. Disk in general is 10 to 100 times the size of your RAM.
You should take all of the space you previously felt acceptable to provide /tmp and allocate it to swap space. The history and semantics of that folder and how it relates to RAM, especially on systems that could not implement swap space, actually make this make sense: if you upgrade to tmpfs from "periodic deletion of /tmp" and are not increasing the size of your swap partition accordingly, you are doing something wrong.
If you know you're going to need more space, you can just create a transient swapfile for the duration. Or use /var/cache, if you suspect you know better than tmpfs and the swap allocator.
RAM really is a harmful kludge. Neither Turing machines nor lambda calculus call for some of an algorithm's state to suddenly be obliterated. The only reason we put up with it is that disk and SSD are so much slower.
Speed is not the only benefit of RAM. The internal data structures of most programs are gnarly messes of intertwined pointers, fragile, easily corrupted, hard to upgrade in place, almost impossible to fix when they break. The division between RAM and disk forces everything to be serialized at intervals to clean storage formats that can be reread for a fresh start. As a practical matter, that's a big enough benefit to justify enforcing the division at an API level even if disk were as fast as RAM.
The industry might produce less worthless crap without the crutch of "it's not quite unusable if you start over often enough".
I've never understood the obsession with Turing machines. It's like basing one's core belief of what a car is off one of Da Vinci's crazy wind-up things.
Ram is just state. Turing machines require a non-tape state machine mechanism. You could consider ram a part of that, 4 gigs corresponding to many many possible states.
Some people (I do not agree with this view) believe that applications should be universally persistent: so that they never "stop running" and never "lose data" even when the machine restarts or even if the programs are transferred between systems.
Do you consider distros that rm -rf /tmp on startup to be broken? If not, why not?
Firstly Fedora (<= 17) doesn't do this. It has a tmpwatch cronjob which deletes files after 10 days (/tmp) and 30 days (/var/tmp).
Secondly, no I don't think it's broken. But it doesn't in any way affect the fact that as a last resort I can reboot in single user mode and grab the precious mutt file before it gets deleted.
Secondly, no I don't think it's broken. But it doesn't in any way affect the fact that as a last resort I can reboot in single user mode and grab the precious mutt file before it gets deleted.
Every program has to be examined for this problem anyway, tempfs or otherwise, so I don't get the point.
I'd vote for ~/.cache
I can't help but find that to be an issue of the MUA rather than that of the filesystem.
Although I agree with your points, I find that the design of one application alone cannot be a justifiable point.
Although I agree with your points, I find that the design of one application alone cannot be a justifiable point.
No, because /tmp is supposed to be assumed to be cleared on start up. If you want something to survive between reboots, /tmp is the last place you should put it, even if you're not using tmpfs.
I've never had a problem with this using mutt, but if I cared, I'd switch it to use /var/tmp.
Either way, if there's a problem, it's with the program, not the filesystem - no need to use a cannon to kill a fly.
I've never had a problem with this using mutt, but if I cared, I'd switch it to use /var/tmp.
Either way, if there's a problem, it's with the program, not the filesystem - no need to use a cannon to kill a fly.
I remember myself creating cron jobs to free wasted /tmp space
"Everyone must now be careful never to store a file in /tmp that might grow large"
If you're going to trash my /tmp with gigantic files please don't. Really
"it’s better to fix the filesystem to make it faster"
Yes, let's spend adding more complexity to the file system to work out every /tmp abuser out there. But his original point was avoiding adding complexity in the first place
So basically, he contradicts himself.
Mounting tmp as a memory backed fs is great (to which there may be better options than tmpfs), avoids (potentially) spinning up disks covered in rust for minor tasks.
If you're going to trash my /tmp with gigantic files please don't. Really
"it’s better to fix the filesystem to make it faster"
Yes, let's spend adding more complexity to the file system to work out every /tmp abuser out there. But his original point was avoiding adding complexity in the first place
So basically, he contradicts himself.
Mounting tmp as a memory backed fs is great (to which there may be better options than tmpfs), avoids (potentially) spinning up disks covered in rust for minor tasks.
>If you're going to trash my /tmp with gigantic files please don't. Really
I write statistical programs that often generate temporary intermediate files of multiple GB size, which need to persist at the end of runs for trouble-shooting, partial re-runs etc. i.e. the software shouldn't delete them when it's finished, but they don't need to be kept long term. I kept forgetting to delete them when I finished a session and my disks kept filling up, so I decided to use /tmp and let the OS manage the space. Is there a better way to manage large tmp files than this that I'm missing?
I write statistical programs that often generate temporary intermediate files of multiple GB size, which need to persist at the end of runs for trouble-shooting, partial re-runs etc. i.e. the software shouldn't delete them when it's finished, but they don't need to be kept long term. I kept forgetting to delete them when I finished a session and my disks kept filling up, so I decided to use /tmp and let the OS manage the space. Is there a better way to manage large tmp files than this that I'm missing?
Given that they should persist, the better way would be /var/tmp I would think, at least if persistence means "should survive reboot".
Even before shifting to tmpfs I'm pretty sure Fedora was erasing the contents of /tmp at startup (and I know Gentoo/OpenRC is).
There might be an even more appropriate FHS-compliant directory (/var/run perhaps?)
Even before shifting to tmpfs I'm pretty sure Fedora was erasing the contents of /tmp at startup (and I know Gentoo/OpenRC is).
There might be an even more appropriate FHS-compliant directory (/var/run perhaps?)
Apologies, by persist I meant just beyond the end of that run of the software (i.e. the software itself couldn't manage/delete the files), to see what went "wrong". Deleting at reboot is perfectly fine.
/var/run "This directory contains system information data describing the system since it was booted" which doesn't seem to fit my use case that well. Not to say there isn't another directory that does fit, but I haven't found it.
/var/run "This directory contains system information data describing the system since it was booted" which doesn't seem to fit my use case that well. Not to say there isn't another directory that does fit, but I haven't found it.
Depends on your use case and how much of that complexity you wish to manage
In your case, what could work is having the new run use the same file names as the old run, or coming up with a rotation schedule (so you would keep the latest N runs)
But if having them on /tmp as it is working for you, great, no need to complicate things.
There are also speed considerations, if you're using these files as actual intermediate storage (as opposed to a log/debug dump) it may be interesting to have them written to a different disk (like an SSD)
In your case, what could work is having the new run use the same file names as the old run, or coming up with a rotation schedule (so you would keep the latest N runs)
But if having them on /tmp as it is working for you, great, no need to complicate things.
There are also speed considerations, if you're using these files as actual intermediate storage (as opposed to a log/debug dump) it may be interesting to have them written to a different disk (like an SSD)
In my case, the runs are often run in parallel and in batches of varying sizes, so "same name" or rotation schedules wouldn't work well. They are a kind of hybrid log/storage file (they contain intermediate data sectioned/logged into the actions that created them, like an extremely verbose log). Dedicated SSDs would be very nice, but out of the price budget (i.e. its a zero cost project, run on existing scavanged hardware!).
As you say, /tmp is working well for me, my example was to perhaps illustrate that there are (what I think are) legitimate uses for large files in /tmp.
As you say, /tmp is working well for me, my example was to perhaps illustrate that there are (what I think are) legitimate uses for large files in /tmp.
Sounds like a job for /var/cache.
> Mounting tmp as a memory backed fs is great (to which there may be better options than tmpfs), avoids (potentially) spinning up disks covered in rust for minor tasks.
I mount /tmp as a tmpfs for almost exactly this reason. I have an always-on home server/HTPC that runs the OS off of a small (40 GB) SSD, and I don't want to wear down the SSD with writes to /tmp, /var/tmp, /var/run, /var/log, etc. In theory, I don't want any writes at all, unless the thing is actually doing something useful. So I simply threw in 2GB extra RAM and mounted all these temporary/logging directories as tmpfs. Periodically (and on reboot) I zip the logs up and copy them to a backup directory just in case something fails.
Of course this runs the risk of losing temporary data and logs whenever the server goes down, but that's a trade-off I'm willing to make. It's been running just fine for over 2 years now. I'd do the same thing straight away if I ever need to set up some similar system.
I mount /tmp as a tmpfs for almost exactly this reason. I have an always-on home server/HTPC that runs the OS off of a small (40 GB) SSD, and I don't want to wear down the SSD with writes to /tmp, /var/tmp, /var/run, /var/log, etc. In theory, I don't want any writes at all, unless the thing is actually doing something useful. So I simply threw in 2GB extra RAM and mounted all these temporary/logging directories as tmpfs. Periodically (and on reboot) I zip the logs up and copy them to a backup directory just in case something fails.
Of course this runs the risk of losing temporary data and logs whenever the server goes down, but that's a trade-off I'm willing to make. It's been running just fine for over 2 years now. I'd do the same thing straight away if I ever need to set up some similar system.
Yep - make your box do what you want.
Nothing stops you from changing fstab to make any mount tmpfs or disk.
Nothing stops you from changing fstab to make any mount tmpfs or disk.
The /tmp on your system is not 'your' /tmp. The home directory is yours. /tmp exists pretty much solely as a place for programs to fill up with a mess of files, with the understanding that it will get automatically cleaned out and they aren't messing up the system proper. If a program needs to trash a place with large files, /tmp is where it should do it.
> If you're going to trash my /tmp with gigantic files please don't.
Where else are temporary files supposed to go, exactly? It's no abuse to put in /tmp temporary "scratch" data which have no reason to survive a process restart let alone a machine reboot, that's the mount point's explicit purpose.
Where else are temporary files supposed to go, exactly? It's no abuse to put in /tmp temporary "scratch" data which have no reason to survive a process restart let alone a machine reboot, that's the mount point's explicit purpose.
>Yes, let's spend adding more complexity to the file system to work out every /tmp abuser out there. But his original point was avoiding adding complexity in the first place
Better to add complexity once, in the filesystem, than in every program that writes to it.
Better to add complexity once, in the filesystem, than in every program that writes to it.
Linkbait headline. Thought it was going to be about a gotcha with tmpfs in general, but it's only about one specific use of tmps in one specific -not even release- version of one linux distribution.
I get that the guy want's attention for this life-and-death issue, but linkbaiting is just mean :(
I get that the guy want's attention for this life-and-death issue, but linkbaiting is just mean :(
One of the hardest parts of writing a compiler is register allocation.
Not quite. There are many much more difficult problems in a competitive optimizing compiler before hitting that step.
You can get reasonable results from even a linear scan allocator, which is simple.
Not quite. There are many much more difficult problems in a competitive optimizing compiler before hitting that step.
You can get reasonable results from even a linear scan allocator, which is simple.
It really depends on the architecture. There are some architectures with only 1 register, and others with many thousands of registers.
Current x86 implementations do a good job of being fast even when registers spill.
I've seen dual-issue Power cores (32ish registers) that only have 16k of 4-way dcache, which means register spills can easily kill performance.
Current x86 implementations do a good job of being fast even when registers spill.
I've seen dual-issue Power cores (32ish registers) that only have 16k of 4-way dcache, which means register spills can easily kill performance.
What's wrong with swapfiles?
It should be mentioned that swapfiles are accessed directly, going around the filesystem. They can suffer from fragmentation if you don't create them when the filesystem is fresh, however.
Oh, i hope they revert this change!
The big trouble for me would be "Everyone must now be careful never to store a file in /tmp that might grow large". I often use /tmp for all kinds of temporary storage. Be it a small text file or a GB big tar file.
The big trouble for me would be "Everyone must now be careful never to store a file in /tmp that might grow large". I often use /tmp for all kinds of temporary storage. Be it a small text file or a GB big tar file.
Rare use case...
I remember CD burning softwares and torrent clients asking for special temp directories to be used for that.
Allowing every day applications to create GBs of tmp files would mean you would need to worry once your partition has less than 100 GB free.
However, nobody stops you from mounting /tmp anywhere else or even disable tmpfs.
I remember CD burning softwares and torrent clients asking for special temp directories to be used for that.
Allowing every day applications to create GBs of tmp files would mean you would need to worry once your partition has less than 100 GB free.
However, nobody stops you from mounting /tmp anywhere else or even disable tmpfs.
Very common for me, not so common for you. ;)
And: when is use /tmp on my laptop, it gets cleared after reboot, so that's perfect for _temporary_ files..
[deleted]
> "Everyone must now be careful never to store a file in /tmp that might grow large"
Be warned that when he says this, he doesn't mean it will eat all your memory, it will only grow upto a maximum size (by default half of your ram). When this is reached, it acts like any other filesystem that is full: you get write errors. There is actually another implementation called ramfs, which doesn't have such a size restriction. However, tmpfs is what is actually being implemented here. The ramfs itself is more of a 'toy' filesystem, also being quite limited, including not being able to use swap space (unlike tmpfs).
Be warned that when he says this, he doesn't mean it will eat all your memory, it will only grow upto a maximum size (by default half of your ram). When this is reached, it acts like any other filesystem that is full: you get write errors. There is actually another implementation called ramfs, which doesn't have such a size restriction. However, tmpfs is what is actually being implemented here. The ramfs itself is more of a 'toy' filesystem, also being quite limited, including not being able to use swap space (unlike tmpfs).
Bad title. tmpfs is good. Fedora maintainers assuming that all of /tmp should be in tmpfs is bad.
Apps are already in the habit of writing a lot of expendable garbage to /tmp that doesn't require the fast I/O of tmpfs. Clogging RAM with this junk is bad idea.
Apps are already in the habit of writing a lot of expendable garbage to /tmp that doesn't require the fast I/O of tmpfs. Clogging RAM with this junk is bad idea.
Example of exactly when to use hard disk and when to use tmpfs:
User browsing the web for a month without rebooting. Flash stores temporary files in /tmp/. For a month straight, Pandora is pumping out songs to /tmp/ as a cache. 43829 minutes in a month, imagine an average of 3 minutes per song that's 14609 songs, at an average 1.5 megabytes per song that's 7304 megabytes that's 7.13 gigabytes. Probably shouldn't be stored in RAM.
On the other hand, you have ganglia clusters of about 2,000 hosts. You have multiple gmetad's collecting data from different data sources and pooling it into your web interface box. It's updating ~10,000 tiny files every 30 seconds. Your i/o wait and system time is through the roof. You move the place gmetad writes to a tmpfs mount, and rsync to the disk every minute. Suddenly the i/o and system times are at 0.01%.
Tmpfs for /tmp/ is considered harmful, but extremely helpful in other cases.
User browsing the web for a month without rebooting. Flash stores temporary files in /tmp/. For a month straight, Pandora is pumping out songs to /tmp/ as a cache. 43829 minutes in a month, imagine an average of 3 minutes per song that's 14609 songs, at an average 1.5 megabytes per song that's 7304 megabytes that's 7.13 gigabytes. Probably shouldn't be stored in RAM.
On the other hand, you have ganglia clusters of about 2,000 hosts. You have multiple gmetad's collecting data from different data sources and pooling it into your web interface box. It's updating ~10,000 tiny files every 30 seconds. Your i/o wait and system time is through the roof. You move the place gmetad writes to a tmpfs mount, and rsync to the disk every minute. Suddenly the i/o and system times are at 0.01%.
Tmpfs for /tmp/ is considered harmful, but extremely helpful in other cases.