Why Pixar uses Microsoft's Azure(blog.seattlepi.com)
blog.seattlepi.com
Why Pixar uses Microsoft's Azure
http://blog.seattlepi.com/microsoft/archives/226427.asp
16 comments
[deleted]
Green Button is a startup already doing this:
http://www.greenbutton.net/
They even use Azure. They don't specifically support Renderman, though. (At least, not yet.)
Like other posters here, I would be curious to know why these companies are using Azure instead of some other cloud platform.
They even use Azure. They don't specifically support Renderman, though. (At least, not yet.)
Like other posters here, I would be curious to know why these companies are using Azure instead of some other cloud platform.
There was a similar system (on demand render farm "in the cloud") on WAM!Net in... 1999? the more it changes, the more it's the same :)
If a job posting to work at a Pixar server farm is anything to go by, they actually use Linux for their films.
This article does not address the technical reasons why Pixar chose to demonstrate this proof of concept using Azure.
Mad props to roadnottaken though for this other gem: http://news.ycombinator.com/item?id=1744214
Mad props to roadnottaken though for this other gem: http://news.ycombinator.com/item?id=1744214
Huh... was sort of hoping that the article would answer the question that it asked... "it'll be around for a while" isn't a great answer considering Microsoft's history with some of it's service platforms.
Anyone have a sense of who the market is for cloud-based 3D rendering? It seems like even for a small-to-medium-sized shop it's likely to be worthwhile to have a pretty solid baseline capacity in house...
I would think it's the opposite: for a small-to-medium shop it makes all kinds of sense to concentrate on your core (design, animation, client development) and just rent "rendering as a service".
a) If you have a small headcount, a dedicated server wrangler's salary would be better used to pay for a person with a more product-oriented profile. That's what makes the difference between you and your competition: you aren't writing your own renderers anyway.
b) When maintaining your own serverfarm it can be difficult for you to have the jobs pipeline full at all times, and you can either have a lot of spare capacity sitting around between jobs, or need more horsepower during peaks.
c) This also allows small-to-medium shops to tackle problems bigger than they could otherwise from a pure cashflow point of view. If you are renting rendering-as-a-service, your clients are paying for the hardware their jobs need, no need to invest, amortize, etc. Just pay an invoice coming in and an cash an invoice going out.
It seems to me that these are exactly the kind of problems SaaS is designed to solve, and that small-to-medium shops can benefit from this solution more than big studios who can:
a) amortize their server wranglers over bigger installations, so their sysadmin/creative ratio is lower.
b) keep the rendering farm working efficiently, or even rent it out if they have lulls in work.
c) afford big financial investments.
a) If you have a small headcount, a dedicated server wrangler's salary would be better used to pay for a person with a more product-oriented profile. That's what makes the difference between you and your competition: you aren't writing your own renderers anyway.
b) When maintaining your own serverfarm it can be difficult for you to have the jobs pipeline full at all times, and you can either have a lot of spare capacity sitting around between jobs, or need more horsepower during peaks.
c) This also allows small-to-medium shops to tackle problems bigger than they could otherwise from a pure cashflow point of view. If you are renting rendering-as-a-service, your clients are paying for the hardware their jobs need, no need to invest, amortize, etc. Just pay an invoice coming in and an cash an invoice going out.
It seems to me that these are exactly the kind of problems SaaS is designed to solve, and that small-to-medium shops can benefit from this solution more than big studios who can:
a) amortize their server wranglers over bigger installations, so their sysadmin/creative ratio is lower.
b) keep the rendering farm working efficiently, or even rent it out if they have lulls in work.
c) afford big financial investments.
[deleted]
The headline is very misleading, they aren't using it at all. It was a proof of concept only.
They also mention that Pixar was co founded by Steve jobs instead of purchased from George Lucas.
Pixar doesn't use Azure. Pixar demonstrated a proof of concept that involved using Azure instances as virtual processors for a cloud-based Renderman farm that MIGHT be attractive for CG shops that can't afford, or don't need, a fulltime render farm. Maybe there's a shop somewhere that would be useful for, but it's certainly not going to be a Pixar or an ILM.
The biggest issue with a cloud farm like this is going to be bandwidth. For cinematic CG the models, textures and other assets are absolutely huge - it's not unusual for a single shot to come to hundreds of gigabytes. And you tweak and render, and tweak and render. Often the size of the data is such that when there is a need to collaborate with a different FX shop the files are sent over Sneakernet - HDD via courier or overnight FedEx.
If I was to guess, I would say this is basically virtualization technology developed for in-house use (which is very useful) that has been turned into a cloud demo for Azure. Why I don't know, but Pixar won't be switching to Azure anytime soon (or ever).
The biggest issue with a cloud farm like this is going to be bandwidth. For cinematic CG the models, textures and other assets are absolutely huge - it's not unusual for a single shot to come to hundreds of gigabytes. And you tweak and render, and tweak and render. Often the size of the data is such that when there is a need to collaborate with a different FX shop the files are sent over Sneakernet - HDD via courier or overnight FedEx.
If I was to guess, I would say this is basically virtualization technology developed for in-house use (which is very useful) that has been turned into a cloud demo for Azure. Why I don't know, but Pixar won't be switching to Azure anytime soon (or ever).
"For cinematic CG the models, textures and other assets are absolutely huge - it's not unusual for a single shot to come to hundreds of gigabytes."
I recall reading an article, IIRC in CineFX, about the Godzilla model in Gozilla 2000 (the Devlin and Emmerich film). The textures for their Godzilla model alone were something like 900 MB.
Honestly I wouldn't be surprised if Pixar introduced a renderfarm hosted in Azure.
Although the amount of data is very large, you don't general end up taking the bandwidth hit all at once. If you have your renderfarm set up on Azure, you could set up a plugin for your animation software, e.g. Maya, that uploads the assets for the scene you're working on and renders it on an Azure virtual machine configured with multiple processors, then download the rendered image data. One you upload the assets once, you will rarely if ever have to upload the whole lot of them again if the render management software is smart enough to keep track of which files you've modified since the last time you rendered. So you've modified a mesh but none of its textures, it only needs to upload the mesh in order to re-render.
It's hardly trivial, and for the larger shops like WETA Digital that are creating massive, virtual sets with massive, virtual armies modeled and rendered with Massive (couldn't resist :)), they can afford an in-house renderfarm -- and the costs will favor it.
For a smaller shop, that might not be so easy, BUT there are shops that are pulling off some pretty amazing animations without buying the pricey software... outfitting a renderfarm with Photorealistic RenderMan or Mental Ray or Final Render licenses can get quite expensive -- when last I checked, it was something like $2000 or $2500 per machine for a Mental Ray rendernode license. PhotoRealistic RenderMan was in the same price range.
I recall reading an article, IIRC in CineFX, about the Godzilla model in Gozilla 2000 (the Devlin and Emmerich film). The textures for their Godzilla model alone were something like 900 MB.
Honestly I wouldn't be surprised if Pixar introduced a renderfarm hosted in Azure.
Although the amount of data is very large, you don't general end up taking the bandwidth hit all at once. If you have your renderfarm set up on Azure, you could set up a plugin for your animation software, e.g. Maya, that uploads the assets for the scene you're working on and renders it on an Azure virtual machine configured with multiple processors, then download the rendered image data. One you upload the assets once, you will rarely if ever have to upload the whole lot of them again if the render management software is smart enough to keep track of which files you've modified since the last time you rendered. So you've modified a mesh but none of its textures, it only needs to upload the mesh in order to re-render.
It's hardly trivial, and for the larger shops like WETA Digital that are creating massive, virtual sets with massive, virtual armies modeled and rendered with Massive (couldn't resist :)), they can afford an in-house renderfarm -- and the costs will favor it.
For a smaller shop, that might not be so easy, BUT there are shops that are pulling off some pretty amazing animations without buying the pricey software... outfitting a renderfarm with Photorealistic RenderMan or Mental Ray or Final Render licenses can get quite expensive -- when last I checked, it was something like $2000 or $2500 per machine for a Mental Ray rendernode license. PhotoRealistic RenderMan was in the same price range.
WETA Digital uses Ubuntu Server on their server farm and desktops:
http://jordanopensource.org/freeplanet/article/ubuntu-linux-...
http://jordanopensource.org/freeplanet/article/ubuntu-linux-...
I'd hazard a bet that the type of servers deployed in cloud systems like Azure isn't really suited to large number crunching operations (such as rendering).
A good example of a big FX shop is WETA Digital (they did LOTR and Avatar) who own supercomputers at 279, 280, 281, 282, 283 and 413 on the Top 500 Supercomputer List. I'd be amazed if Azure had anywhere near that kind of power (in any kind of suitable configuration).
A good example of a big FX shop is WETA Digital (they did LOTR and Avatar) who own supercomputers at 279, 280, 281, 282, 283 and 413 on the Top 500 Supercomputer List. I'd be amazed if Azure had anywhere near that kind of power (in any kind of suitable configuration).
Too bad they didn't use EC2 to handle the Renderman Farm. They could have made it a bit easier by using Amazon Web Services own version of a sneakernet, http://aws.amazon.com/importexport/
Pixar doesn't use Azure for it's production render pipeline, this is a deceptive marketing article showing proof of concept. THIS WOULD BE AMAZING if it worked, bandwidth is not an issue because each frame element texture and footage would never really top 1GB, the time is doing the Mental Ray/IBL CPU intense render. Here is what all of us VFX people want: Amazon EC2 render farms for Maya, Mental Ray, Nuke, 3DS, and whatever we can throw at it. Also a strict privacy policy- these renders could be one of the last steps in the composites......
The uploader does a diff between what was uploaded before and what you're uploading now and only sends what's needed.
Would cloud farms be practical if your assets lived on the cloud as well? That could solve the bandwidth problem both with the render cycle and collaboration with other shops.
Or would getting the assets there in the first place be impractical?
Or would getting the assets there in the first place be impractical?
You're off the rails. If two things are "in the cloud" that can still mean they're in different data centers in opposite ends of the earth. The "cloud" just means outside your local lan. So it's the 99.999999...% of the internet you don't have any control of.
Meaning you'd still have bottle necks and bandwidth issues in getting them somewhere. That's why large projects just send HDDs.
This is just a proof of concept.
Meaning you'd still have bottle necks and bandwidth issues in getting them somewhere. That's why large projects just send HDDs.
This is just a proof of concept.
Just have creation tools that allow you to capture and replay input. Then to send that multi-gig mesh or texture, you just send the input events from the artist and replay it in another instance of that tool (think doom2 demo file; as a YouTube video it is hundreds of megs, as a demo file it is a few kb).
There's no way that would be at all productive. It's not like the only thing that the artist is doing is typing characters on a screen. There's also a lot more going on than just echoing pixels like in an IDE.
For a character animator, there's an IK rig which, depending on the character, can be quite complicated, and IK involves a lot of math. Then there's the mesh deformation and blended IK/FK. And then things like hair and cloth simulation. And texture rendering. Doing that on a cloud would have latencies that would make it excruciating to work with.
Compositing would be just as bad; the amount of data involved is larger, and as you add layers and masks, filters that affect those layers and masks, you can pretty much throw any hope of interactivity out the window. Now imagine trying to paint out safety wires over a cloud...
Digital art requires a LOT more computing power than creating software.
Try playing a first person shooter on a laggy network, you'll get a sense for how frustrating and counter-productive what you're suggesting would end up being, though I suspect that it's because you're vastly underestimating how much goes on in digital artists' tools more than anything else.
For a character animator, there's an IK rig which, depending on the character, can be quite complicated, and IK involves a lot of math. Then there's the mesh deformation and blended IK/FK. And then things like hair and cloth simulation. And texture rendering. Doing that on a cloud would have latencies that would make it excruciating to work with.
Compositing would be just as bad; the amount of data involved is larger, and as you add layers and masks, filters that affect those layers and masks, you can pretty much throw any hope of interactivity out the window. Now imagine trying to paint out safety wires over a cloud...
Digital art requires a LOT more computing power than creating software.
Try playing a first person shooter on a laggy network, you'll get a sense for how frustrating and counter-productive what you're suggesting would end up being, though I suspect that it's because you're vastly underestimating how much goes on in digital artists' tools more than anything else.
It was claimed that assets couldn't be sent to the cloud because they use too much bandwidth. I pointed out that sending a replay of events used to create assets takes very little bandwidth. This has nothing to do with the latency you are talking about here.
I think he's implying that the bottleneck won't be an issue if (when?) we eventually have web-based software powerful enough to create those assets entirely within the same "cloud" that would eventually scale out to do the rendering.
When things are in the cloud, they don't have to be in different data centers. Cloud providers are starting to provide tools that give you control over proximity for exactly these sorts of performance issues.
Does anyone know who actually deploys Windows based cloud/Internet-facing systems? All the big corporate IT shops I've consulted for only used Windows servers to support Windows clients and that kind of stuff doesn't move to the cloud well. All the big Internet companies (Google, Facebook, etc) deploy on Linux as far as I've seen. Most startups I've heard about deploy on Linux VPS/Heroku/AppEngine type infrastructures. I also though most of the 3D rendering was being done on Linux but maybe that is not the case.
The only example I've seen of windows deployment was dpreview.com, and the comments I read from them were in the line of "it's not as bad as everyone says it is and we're familiar with the stack so we use it".
Is windows a large percentage of EC2 for example? I tried to google for that but couldn't find anything.
The only example I've seen of windows deployment was dpreview.com, and the comments I read from them were in the line of "it's not as bad as everyone says it is and we're familiar with the stack so we use it".
Is windows a large percentage of EC2 for example? I tried to google for that but couldn't find anything.
This might help: http://www.microsoft.com/windowsazure/evidence/
I cannot see the content on that page at all without updating my Silverlight (that I just installed last month) to a new version. I love Microsoft.
Nothing in the article says anything about Pixar using Azure other than a proof of concept and gives no answer to the leading question.
Pixar is founded by George Lucas then bought and nurtured by Steve Jobs though :)
Pixar is founded by George Lucas then bought and nurtured by Steve Jobs though :)
I could see Pixar working with Microsoft to make Azure-hosted Renderman available for other groups doing rendered projects. For a small film project, I'd think that could be really useful.
Then again, Photorealistic Renderman was in NeXTSTEP 3.0, and you could split a rendering job among local NeXT machines as easily as choosing render hosts from a GUI list. That didn't mean Pixar was rendering their Listerine commercials on 25Mhz 68040 boxes.