“PSD is not a good format”(code.google.com)
code.google.com
“PSD is not a good format”
https://code.google.com/p/xee/source/browse/XeePhotoshopLoader.m?r=f16763d221dfca6253983824b470adf553a19e06#108
52 comments
I don't get that response at all. The rant was obviously written in frustration, I wonder why it touched such a nerve at Adobe. If that's the classed up version for a corporate blog, how did they really feel?
Maybe I've had ego about products beaten out of me by tough internal code reviews of every line I've ever written, but I'm not afraid to admit to the parts of our product that suck, or shipped "good enough." The psd format hardly seems like a thing worth defending. And it's basically an admission the rant is right that attempts to soften it with some excuses.
Maybe I've had ego about products beaten out of me by tough internal code reviews of every line I've ever written, but I'm not afraid to admit to the parts of our product that suck, or shipped "good enough." The psd format hardly seems like a thing worth defending. And it's basically an admission the rant is right that attempts to soften it with some excuses.
The rant in the Xee source is IMO ignorant and not very useful. The Photoshop format has some warts, but for what it is, it’s documented well enough (better than many similar app-specific serialization formats), and most of the complexity is necessary to deal with various Photoshop features.
The origin of PSD’s complexity is the low-level organization of Photoshop’s abstractions, which dates from the late 1980s and has become more and more unwieldy as Photoshop has gradually accreted features over its 25-year history.
Complaints about PSD in this style basically amount to complaints that Adobe hasn’t ever tossed out backwards compatibility and done a rewrite from scratch on a cleaner set of abstractions. But there are obvious business reasons for them not to do that.
I’ve had some experience parsing and working with the data inside the PSD format, and honestly, it’s not nearly so bad as this rant makes it sound. (Disclaimer: that’s coming from a baseline of 15 years of heavy experience as a Photoshop user.)
The origin of PSD’s complexity is the low-level organization of Photoshop’s abstractions, which dates from the late 1980s and has become more and more unwieldy as Photoshop has gradually accreted features over its 25-year history.
Complaints about PSD in this style basically amount to complaints that Adobe hasn’t ever tossed out backwards compatibility and done a rewrite from scratch on a cleaner set of abstractions. But there are obvious business reasons for them not to do that.
I’ve had some experience parsing and working with the data inside the PSD format, and honestly, it’s not nearly so bad as this rant makes it sound. (Disclaimer: that’s coming from a baseline of 15 years of heavy experience as a Photoshop user.)
As a heavy Photoshop user you should know that the Photoshop format has seen several backwards incompatible upgrades over the years. And Adobe probably had good business reasons for doing that. Since they were doing that anyway, they might as well have improved the format in the process, I have no idea why they did not do that.
Old PSD files open with all later versions of Photoshop in every example I’ve seen. Newer PSD files include features that aren’t available in old versions of Photoshop, so for obvious reasons those newer files aren’t completely compatible with old Photoshop versions.
But overall, I think you’re misunderstanding what I’m saying. In order to fundamentally change what needs to be stored in a PSD file – which is basically a serialization of the Photoshop document that lives in memory while you work with Photoshop – you would need to fundamentally re-think the very low level abstractions that Photoshop is built in, which would mean completely changing the Photoshop user interface and every level of the technical stack, basically rewrite it all from scratch.
Given the features Photoshop supports, and the way they are built, the technical details of the PSD format are actually pretty straightforward and reasonable. If you spent a few months learning how all the Photoshop features interact and getting a loose sense of how they’re implemented, and then you read the spec for the PSD format, I suspect that you would find the file format to mostly make good sense.
The reason that PSD is complex and hard to implement for third parties (especially for someone not intimately familiar with Photoshop’s UI or structure) is that Photoshop is a very complex piece of software with many moving parts, worked on by hundreds of people over 25 years, not that Adobe’s engineers are incompetent or made the format needlessly arcane just to be jerks.
But overall, I think you’re misunderstanding what I’m saying. In order to fundamentally change what needs to be stored in a PSD file – which is basically a serialization of the Photoshop document that lives in memory while you work with Photoshop – you would need to fundamentally re-think the very low level abstractions that Photoshop is built in, which would mean completely changing the Photoshop user interface and every level of the technical stack, basically rewrite it all from scratch.
Given the features Photoshop supports, and the way they are built, the technical details of the PSD format are actually pretty straightforward and reasonable. If you spent a few months learning how all the Photoshop features interact and getting a loose sense of how they’re implemented, and then you read the spec for the PSD format, I suspect that you would find the file format to mostly make good sense.
The reason that PSD is complex and hard to implement for third parties (especially for someone not intimately familiar with Photoshop’s UI or structure) is that Photoshop is a very complex piece of software with many moving parts, worked on by hundreds of people over 25 years, not that Adobe’s engineers are incompetent or made the format needlessly arcane just to be jerks.
FBX has been quite successful for 3d content. The format itself is proprietary but the FBX SDK is quite easy to use. Pretty much all tools support it and it pretty much completely solves a wheel that was being re-invented thousands of times per year. Now you can just export as FBX and move on with your life.
I'm not sure how FXG was every going to replace PSD given than FXG is for vector based graphics. I suppose it could replace PSD for some types of content but stuff like digital painting it doesn't seem appropriate at first glance?
I'm not sure how FXG was every going to replace PSD given than FXG is for vector based graphics. I suppose it could replace PSD for some types of content but stuff like digital painting it doesn't seem appropriate at first glance?
Wouldn’t it be better if all apps could read/write just one common interchange format?
As an aside this is what FBX is trying to be for the CAD world.
http://www.autodesk.com/products/fbx/overview
As an aside this is what FBX is trying to be for the CAD world.
http://www.autodesk.com/products/fbx/overview
https://news.ycombinator.com/item?id=6410821
https://news.ycombinator.com/item?id=575122
and sundry others: https://hn.algolia.com/?q=psd#!/story/forever/0/psd%20format
https://news.ycombinator.com/item?id=575122
and sundry others: https://hn.algolia.com/?q=psd#!/story/forever/0/psd%20format
Looks like a total of 5: https://hn.algolia.com/?q=psd#!/story/forever/prefix/0/XeePh...
But yeah, I definitely remember seeing this in the past. Feels slightly like a case of http://xkcd.com/1053/, though?
And I don't see anything in the guidelines about re-posts. (https://news.ycombinator.com/newsguidelines.html) You've got more HN experience than me, though, so I could very easily be wrong. :)
But yeah, I definitely remember seeing this in the past. Feels slightly like a case of http://xkcd.com/1053/, though?
And I don't see anything in the guidelines about re-posts. (https://news.ycombinator.com/newsguidelines.html) You've got more HN experience than me, though, so I could very easily be wrong. :)
It's in https://news.ycombinator.com/newsfaq.html. The rule of thumb is that reposts are ok after a year or so.
Was PSD designed to be a portable format? My understanding is that it was only meant to be used directly by Adobe Photoshop, much like the original DOC was meant to be used only by Microsoft Word. As such, it's a little unfair to call it out as a bad format because it was never meant to be reverse engineered and reimplemented.
As far as i'm aware PSD was never supposed to be a portable format. I believe Adobe did release some limited information on the format in the past and that's why there's a lot of apps that can read and write the format.
It's a complicated format but that i imagine is from the fact that it's highly backwards compatable. Not always with 100% success but i believe it's possible to open new PSD files in older versions of photoshop. It's rare to see compatability work both ways like that.
It's a complicated format but that i imagine is from the fact that it's highly backwards compatable. Not always with 100% success but i believe it's possible to open new PSD files in older versions of photoshop. It's rare to see compatability work both ways like that.
You seem to confuse "portable file format" with "horrible designed format". :-)
So, this is the justification for having a horrible file format?
Photoshop 1.0 is from 1988. This was the era of the Macintosh SE [1], the default configuration of which came with two floppy drives, but if you paid a lot more, you could a hard drive that had as much as 40 MB. And it could have an entire 4 MB of RAM.
There are watches today that are 100x more powerful than Macs of that era. Almost anything that was good in a file format from that era would be bad today. Every novel and innovative application feature from Photoshop 1.0 would be seen today as trivial. Many of today's features would have been inconceivable.
If you think you can do better, then I'd encourage you to try. Just start shipping an app with a file format today, get a lot of users, take it through 15 major versions [2], and come back to us in 25 years. Then you can lecture those of us left alive on the right way to do file formats.
[1] http://en.wikipedia.org/wiki/Macintosh_SE [2] http://en.wikipedia.org/wiki/Adobe_Photoshop_version_history
There are watches today that are 100x more powerful than Macs of that era. Almost anything that was good in a file format from that era would be bad today. Every novel and innovative application feature from Photoshop 1.0 would be seen today as trivial. Many of today's features would have been inconceivable.
If you think you can do better, then I'd encourage you to try. Just start shipping an app with a file format today, get a lot of users, take it through 15 major versions [2], and come back to us in 25 years. Then you can lecture those of us left alive on the right way to do file formats.
[1] http://en.wikipedia.org/wiki/Macintosh_SE [2] http://en.wikipedia.org/wiki/Adobe_Photoshop_version_history
It's not that long ago. GIF certainly predates it. I think ZIP does as well, and probably EPS, just to name a few for scale.
None of these does exactly what PSD does of course, so it's impossible to compare them.
A better example is perhaps TeX, which predates it by far, has been extended a lot during the years, and can still render both original documents as well as modern ones.
None of these does exactly what PSD does of course, so it's impossible to compare them.
A better example is perhaps TeX, which predates it by far, has been extended a lot during the years, and can still render both original documents as well as modern ones.
GIF, EPS, and ZIP do the same thing as they did decades ago. PSD does radically different things. Things that were, as I said, completely unpredictable at the time the format was started.
If I were to take up your challenge, I would probably just use sqlite. Is that cheating?
Consider that one of the challenges to early Photoshop was be doing graphics manipulation that is right at the edge of what the hardware at the time is capable of, I don't think that stuffing a relational database engine between the raw bits and the drive would be helpful.
The challenge needs to add the condition of the app stretching the hardware capabilities so hard that a whole pile of not-so-clean optimization tricks are necessary to get the performance up to the acceptable level.
The challenge needs to add the condition of the app stretching the hardware capabilities so hard that a whole pile of not-so-clean optimization tricks are necessary to get the performance up to the acceptable level.
SQLite versions 1, 2 and 3 have completely different file formats as well as differing semantics especially around typing. Later versions cannot even read earlier versions.
The file format is also well documented and fairly complex. https://www.sqlite.org/fileformat.html
The file format is also well documented and fairly complex. https://www.sqlite.org/fileformat.html
Considering SQLite is only 15 years old and implements a rigorous specification instead of an evolving set of product requirements, you're just over halfway to what he talked about.
Go to Adobe board room, tell them about your great format that will empower artists to use their documents in competing applications.
When engineering a file format, you have multiple competing priorities you need to balance, such as size of the resulting file, reading and writing efficiency, backward compatibility, etc. Since Adobe didn't anticipate using PSD as a generic file format for graphics applications, it wasn't necessary to design it that way, and I'm sure that other priorities prevailed instead. It's also a pretty old format, probably designed not just once but many times over the years by many people resulting in the inconsistencies the article talks about.
Looks like a typical rant from a junior developer. More experienced developers realize why decades-old file formats are they way they are and when needing to work with them, adapt themselves to it.
Yeah it's easy to say "I could've done this better" without thinking about it too much.
I'm pretty much that the things guys like this are programming now (assuming they are going mainstream which I HIGHLY doubt) will be seen as cryptic and really naive when a programmer from 2036 looks into them.
I'm pretty much that the things guys like this are programming now (assuming they are going mainstream which I HIGHLY doubt) will be seen as cryptic and really naive when a programmer from 2036 looks into them.
Your comment sort of looks a bit like a senior developer who hasn't, you know, really achieved as much as they might have striking a pose against these pesky juniors who whatever else you think of them, really get some things done.
When you battle the darkness and smite it to get something working pretty well you have a right to do a little bit of editorializing in a comment. If you choose not to and save the rant for the bar on Friday evening or a blog post, that's ok too.
Of course looks can be deceptive and either or both of us could be wrong about how things "look."
Me? I appreciate a heartfelt "here there be dragons" in source with some reasoning and description, I may be too junior for your tastes. ;-)
When you battle the darkness and smite it to get something working pretty well you have a right to do a little bit of editorializing in a comment. If you choose not to and save the rant for the bar on Friday evening or a blog post, that's ok too.
Of course looks can be deceptive and either or both of us could be wrong about how things "look."
Me? I appreciate a heartfelt "here there be dragons" in source with some reasoning and description, I may be too junior for your tastes. ;-)
Mwahaha. Kiddie stuff. Go try and read RTF some time. Try to convert it to HTML. Try to figure out how to extract tables. Oh, bugger.
The best I can say about that experience is (a) I learned a lot about how legacy file formats grow, especially when they are supposed to be "open" but are really treated as proprietary ground for developers to stomp all over in metal-soled booties, and (b) the company that paid me to do that work is long, long out of business. The misery is long past.
The best I can say about that experience is (a) I learned a lot about how legacy file formats grow, especially when they are supposed to be "open" but are really treated as proprietary ground for developers to stomp all over in metal-soled booties, and (b) the company that paid me to do that work is long, long out of business. The misery is long past.
> The misery is long past.
With legacy apps, your miserable past is usually somebody else's nightmarish present...
I had a funny - though unrelated - experience with RTF production a couple decades ago. To make things more interesting, we were using KnowledgeWare ObjectView for the job - a 16-bit RAD tool with a custom simil-VisualBasic syntax, and collecting data from a UNIVAC 1100. I moved to a different company about four years later, and later discovered ObjectView... oops, just ceased to exist. Phewwww!
http://en.wikipedia.org/wiki/KnowledgeWare
With legacy apps, your miserable past is usually somebody else's nightmarish present...
I had a funny - though unrelated - experience with RTF production a couple decades ago. To make things more interesting, we were using KnowledgeWare ObjectView for the job - a 16-bit RAD tool with a custom simil-VisualBasic syntax, and collecting data from a UNIVAC 1100. I moved to a different company about four years later, and later discovered ObjectView... oops, just ceased to exist. Phewwww!
http://en.wikipedia.org/wiki/KnowledgeWare
I wrote an XSLT transformation for SharePoint that emitted an RTF doc that included tables a long while ago. Damn, I write a lot of that in anger. Another one for DocBook to RTF, for another client. Also written in anger. RTF is painful to try and reproduce rationally.
Meh. It's old and complicated.
I wrote a PSD dumper in 2013 so our artists could use Photoshop as a level editor. Interestingly, the Photoshop Javascript automation engine is so slow, that it was faster to parse the file in Python. Like a minutes-long export using Javascript in Photoshop took a second or so when dumping with Python for a file with 100 layers.
I wrote a PSD dumper in 2013 so our artists could use Photoshop as a level editor. Interestingly, the Photoshop Javascript automation engine is so slow, that it was faster to parse the file in Python. Like a minutes-long export using Javascript in Photoshop took a second or so when dumping with Python for a file with 100 layers.
The code (and comment) is five years old. Any updates on how the PSD format has changed (if at all) since then?
Also, the title used here isn't actually a quote from the link. I'd opt for the closing sentence "PSD is not my favourite file format" instead ...
Also, the title used here isn't actually a quote from the link. I'd opt for the closing sentence "PSD is not my favourite file format" instead ...
In modern PSD files a lot of information is stored in "descriptors"; they are all parsed in an uniform way. Some of the information (like font data) is in undocumented "Raw Data" descriptors though.
The official PSD specs are now online: http://www.adobe.com/devnet-apps/photoshop/fileformatashtml/.
Specs don't provide information about all PSD features, but they are a good start. As I recall, an initial version of https://github.com/kmike/psd-tools took a weekend; I was following specs, using PIL (Pillow) for pixels handling; also, I was checking other PSD readers when something was unclear - there is a lot of open-source PSD readers. Earlier readers are from 2005 or maybe earlier, they have been available long before the code with this "famous" comment was written.
The official PSD specs are now online: http://www.adobe.com/devnet-apps/photoshop/fileformatashtml/.
Specs don't provide information about all PSD features, but they are a good start. As I recall, an initial version of https://github.com/kmike/psd-tools took a weekend; I was following specs, using PIL (Pillow) for pixels handling; also, I was checking other PSD readers when something was unclear - there is a lot of open-source PSD readers. Earlier readers are from 2005 or maybe earlier, they have been available long before the code with this "famous" comment was written.
As far as I know PSD hasn't had a "reset", but only incremental features, and compatibility has been good, so I'd expect that not much has changed. Only user perspective though, I haven't looked at the format itself.
Since compatibility is quite important in this case, I don't really expect big changes though. Adobe has a working code base and documentation for it, so there likely is no incentive to risk change.
Since compatibility is quite important in this case, I don't really expect big changes though. Adobe has a working code base and documentation for it, so there likely is no incentive to risk change.
Is this problem being solved the best way? Is it not possible to build a plugin for Photoshop that exports an interim format, something where there are things like layers with text elements in something CSS friendly so that those people with familiarity with things like imagemagick can automate what they need doing in some sensible way? This need be a one-way export so those of us that have to do something with other people's Photoshop files can just get on with it. The import of the interim format (to Photoshop) need not be necessary for many workflows.
That still requires you to have Photoshop to do the initial export, at least until your new export format is an accepted standard (Photoshop files are de-facto exchange format in many areas, since everybody uses Adobe products anyways)
If you already know that your workflow will leave Photoshop at some point and you control it fully I agree, that would be a solution (and might already be possible, depending on your exact needs)
> Why, for instance, did it suddenly decide that these particular chunks should be aligned to four bytes, and that this alignement should not be included in the size? Other chunks in other places are either unaligned, or aligned with the alignment included in the size.
This sounds like a spec that grew organically, with multiple people contributing to it over the years; the alignment might've come from PowerPC/68k-based Macs, as also evidenced by the use of big-endianness throughout.
This sounds like a spec that grew organically, with multiple people contributing to it over the years; the alignment might've come from PowerPC/68k-based Macs, as also evidenced by the use of big-endianness throughout.
Previously: https://news.ycombinator.com/item?id=8831172
And previously: https://news.ycombinator.com/item?id=4746787
And previously: https://news.ycombinator.com/item?id=4746787
[deleted]
“Most are the gradual result of discovering better ways to do things over 20 years, while staying compatible with older applications.” (PSD apologist Tim Wright)
There is no force more destructive.
There is no force more destructive.
Calling someone a "X apologist" is silly.
You disagree with someone, awesome. That's not a reason to try to ridicule that person imo.
You disagree with someone, awesome. That's not a reason to try to ridicule that person imo.
I don't see that as ridiculing them. An apologist is someone who argues in defense of something. In this case, the PSD format. It seems apt here.
I guess I gotta wave my "descriptive not prescriptive" flag a bit. =) You are, technically, correct. Hermes Conrad aside, technically correct is not the best kind of correct--"apologist" has nearly universally negative connotations in, well, modern English, here, now. It is used almost exclusively to characterize a position that the user of the word views as negative. And people who don't fall into that generality often are using the word in "defiance" of that generality, which kind of makes you wonder why, when words are used to communicate. It's also worth noting that at least some dictionaries characterize "apologist" as a defender of something controversial, which is a nod towards the real-world use of the term if you read into what they mean by controversial a bit.
The world isn't an SAT test. Context always always always matters. (And, for extra context-matters, if you are now compelled, at the end of this post, to ask what a "Standard Aptitude Test test" is, I invite you to take a good long look at your life and ask yourself why you want to be That Guy because nobody likes That Guy.)
The world isn't an SAT test. Context always always always matters. (And, for extra context-matters, if you are now compelled, at the end of this post, to ask what a "Standard Aptitude Test test" is, I invite you to take a good long look at your life and ask yourself why you want to be That Guy because nobody likes That Guy.)
Christian writers are called "apologists" and it's not a negative connotation, it's just what they do
Sure. But the use of "apologist" in a Christian context is, today in 2015, much more niche than, say, calling the news organ of a particular political stripe "liberal apologia"; it would be rare to see the New York Times use that phrase except in irony but much more common for conservative sources to do so, because it's commonly understood to be more negative than neutral.
It's much the same as getting mad when somebody uses "hacker" to mean something other than "train and/or computer nerd". You don't get to lay a prescriptive claim to truth.
It's much the same as getting mad when somebody uses "hacker" to mean something other than "train and/or computer nerd". You don't get to lay a prescriptive claim to truth.
If I may I'd like to just reinforce that point.
I am an apologist for the ideas of democracy, the rule of law and equality before it. As I am for certain aspects of attempts to achieve such. I am not any kind of apologist for facism, totalitarianism and ideologies that lead to such.
From a slightly different angle that may make the example clearer:
The so called "2 party" democracy has it's drawbacks, one of which is often labeled "partisanship" or something like that. I am an apologist for that, while acknowledging its imperfections and problems. The concept of the loyal opposition is the thing that seems to have worked best so far, I would argue it has worked best by such a huge margin that alternatives are scarcely worth considering other than as the enemy of freedom.
I am an apologist for the ideas of democracy, the rule of law and equality before it. As I am for certain aspects of attempts to achieve such. I am not any kind of apologist for facism, totalitarianism and ideologies that lead to such.
From a slightly different angle that may make the example clearer:
The so called "2 party" democracy has it's drawbacks, one of which is often labeled "partisanship" or something like that. I am an apologist for that, while acknowledging its imperfections and problems. The concept of the loyal opposition is the thing that seems to have worked best so far, I would argue it has worked best by such a huge margin that alternatives are scarcely worth considering other than as the enemy of freedom.
I am pretty sure it's tongue-in-cheek humor. The idea of a file format apologist is silly, a Hitler apologist not so much.
> Calling someone a "X apologist" is silly.
Not if you know what the word "apologist" means (it has nothing to do with ridicule). Try using Google.
Edit: dang has criticised my last sentence there, which is fair enough. It was meant as a dig. However, if you are unsure of the meaning of a word and are using either Firefox or Chrome, you can select the word, right-click, and choose "Search with Google" and Google will actually give you the definition for that word. It's a (relatively) new feature they've added. Also, search with "etymology" to find the origin of a word.
Not if you know what the word "apologist" means (it has nothing to do with ridicule). Try using Google.
Edit: dang has criticised my last sentence there, which is fair enough. It was meant as a dig. However, if you are unsure of the meaning of a word and are using either Firefox or Chrome, you can select the word, right-click, and choose "Search with Google" and Google will actually give you the definition for that word. It's a (relatively) new feature they've added. Also, search with "etymology" to find the origin of a word.
> Try using Google
This comment would be strictly better without that final swipe. Please don't do that on HN.
This comment would be strictly better without that final swipe. Please don't do that on HN.
One quote worth picking out: "Wouldn’t it be better if all apps could read/write just one common interchange format? That’s just what Adobe is developing with FXG." Come 2015, and FXG appears to be dead and buried.