Linux at Industrial Light and Magic 285
Nicholas DePetrillo writes "Linux Journal has a big story about how LINUX is being used in hollywood, specificly at Industrial Light and Magic with some GREAT screenshots
and a very descriptive article with configuration details." Word has it that their rendering farms have gotten even bigger since this article was published.
Article Text (Score:3, Informative)
The Computers of ILM
ILM says they have rarely seen artists get excited by hardware, but artists fought to get the new Linux workstations--Dell single-CPU P4s with NVIDIA Quadra 2 Pro graphics cards. The question became, ``Where's my Linux box?''
Production Engineering Manager Ken Beyer says
More than 350 Linux boxes were deployed during Episode II. Animators and modelers got their workstations first, then compositors. The first group had flat panels because animators lack the desk space for monitors. There were problems with monitor calibration under Red Hat 7.1. We used flat panels to get Linux out there. Last to get workstations were TDs. They push the envelope of what they ask for. An issue was how quickly we could get things ported for them.
``We've changed over quite a bit of our plant here to Linux--half of our desktops and about 30% of our 2,000 CPU renderfarm is now Linux'', says ILM Director of Research and Development Andy Hendrickson. ``We've got 700-plus O2 machines'', adds Beyer. ``But it isn't affordable to replace those with Octanes.'' SGI is recognized for producing high-end workstations and servers but has abandoned competing with commodity PC hardware. SGI seems to be rebounding in the military market but less so in entertainment.
``Our renderfarm towers carry the Deathstar logo'', points out Beyer. A render tower is a stack of 1RU 2-CPU units connected together with inexpensive 100Base-TX. He says:
These are 1RU, 2-CPU P4 units. If we lose a unit it is more convenient now that it is just two CPUs rather than four or eight with SGI 2800. For Episode II we had to double available capacity and power. It's 512 processors. We use dual 225 kVA UPS systems, and have three AC systems that rotate. Power goes out often in the San Rafael area. We can run on UPS for 15 minutes then [on a] diesel generator.
An unexpected snag arose during the upgrade: all the PC fans had to be replaced because they were defective.
Systems R&D Group Manager Mike Kiernan reports a few problems with Linux:
Sometimes when I arrive in the morning a quarter of the Linux cluster is locked up. Fortunately, it doesn't happen too often. VM problems in the 2.4 kernel appear to be at the root of our kernel lockup problem. Recent improvements in the 2.4 kernel may resolve that. Things look promising.
But he adds that ``Linux needs work on NFS big time.''
We won't be going to Linux for our NFS servers. I wish we could replace NFS, but none of the document management systems is flexible enough. And the ones that are flexible have a rather high integration cost. When AFS is distributed natively for all the client platforms we need to support, perhaps we'll consider it.
ILM is comfortable with multiple platforms. Its 1,400 employees use a variety of operating systems. The art department has Macs, with the rotoscopers and painters transitioning to OS X. Hendrickson sees OS X as a possible player. ``What attracts us is the BSD-like Darwin core and network compatibility.'' ILM has few Windows boxes, besides those on business side. ``There's no advantage to a Windows conversion for us'', says Hendrickson. ``We're a UNIX shop and probably always will be.''
R&D Principal Engineer Phil Peterson says ILM chose the Red Hat distro because it seemed easier to go with what's popular. ``At ILM the 2.4.9 kernel is deployed, and 2.4.17 or 2.4.18 is in test. We tweak the kernel--things like shared memory size, number of file descriptors, default stack size--nothing dramatic.'' Open Motif 2.1 did a good job maintaining the look-and-feel of IRIX, so ILM didn't try LessTif. ILM workstations include limited installations of GNOME and KDE. ``No special effort was spent to strip machines down'', says Peterson. ``We just left out unused portions of the full install. We're pretty vanilla.''
An unusual aspect of the ILM Linux workstation configuration is the replacement of the MESA libs with the SGI open source, OpenGL implementation. ``MESA is behind compared to the SGI version in aspects such as libGLU'', explains Peterson. Other studios haven't experienced the best stability using Maya on Linux with NVIDIA drivers. It seems that may be due to MESA and not Maya, NVIDIA or Linux, as previously thought. ILM has replaced the MESA libraries with a combination of NVIDIA's core OpenGL and libraries from the SGI open-source sample implementation.
``Chances are you will not find solutions in any documentation'', notes Peterson.
We don't have a support line to call. We fix things and extend. It introduces a layer of maintenance we're not used to. We had to use open-source drivers with tablets. With calibrating monitors, the work is ongoing. Still, we've had an easy road. Our artists are technically savvy, able to endure pain. Having the best testers in the world around the corner from you provides quick feedback.
Hendrickson concurs that Linux support can be a problem. He says, ``As we get into Linux we're not finding one company to hand-hold. IBM and HP aren't there, yet. But, before Linux it was out of our control and out of control. [Now] we own our Linux problems.''
Is it possible for Linux to be too fast? ``Due to the speed of Linux, for the first time in my life, 15 years in the business, I'm starting to feel some RSI [repetitive strain injury]'', says Technical Director Robert Weaver. ``Usually you are working the machine, but Linux is so fast it can overwork you.'' Weaver has to remember to take breaks because with Linux he doesn't get any breaks waiting for the machine anymore.
Looks bad for Alias Wavefront and SGI (Score:4, Informative)
Compute performance is a real problem though for all but the high end SGI machines. The O2 went EOL this year for all but OEM (Medical and Video apps).
What I find interesting is that the studios are building their own tech to suit their needs. They started this process on SGI machines in order to leverage the rich toolset found in IRIX. Now they are able to build on the open tools found under Linux.
Alias Wavefront (owned by SGI) used to package and sell critical technology to the Studios along with a number of services delivered under NDA. They still do this, but their days are numbered.
Open Source is enabling a new trend toward in-house technology development. Given the high cost of A|W and SGI toolsets plus the added services and consulting required to make them work in creative ways, I am not surprised to find the studios able to just build what they want cheaper. They also are able to create the features that make their workflow better.
Seems like the studios want nothing to do with any sort of lock in with any vendor. Talk about lowering TCO!
Not only does their in-house development allow them to make the most of their time and creative energy, but they get to take advantage of new platforms in the future --without having to re-purchase tools.
They are smart enough to develop common tools that they (and us) can all use while keeping those things that make them special in-house. Too bad more of the mainstream businesses are not able to see this yet. When they do --look out!
So SGI moves back into their niche with IRIX, high end visualization, and parallel computing and serving. What of Alias Wavefront?
They look doomed to me. Maya now is cheap and runs on Linux. Its users develop their own tools with it so A|W maybe gets consulting, but little else. On the Industrial Design side, their Studio Tools package really only appeals to the high end Automotive styling group. Everyone else can use either MCAD tools, or tools like Rhino to get their work done.
This was a great article that does a lot to illuminate just how computing can change under an open platform.
ILM OSes (Score:4, Informative)
ILM is comfortable with multiple platforms. Its 1,400 employees use a variety of operating systems. The art department has Macs, with the rotoscopers and painters transitioning to OS X. Hendrickson sees OS X as a possible player. ``What attracts us is the BSD-like Darwin core and network compatibility.'' ILM has few Windows boxes, besides those on business side. ``There's no advantage to a Windows conversion for us'', says Hendrickson. ``We're a UNIX shop and probably always will be.''
Nice to see ILM is keeping with the times. When Phantom Menace came out, SGI had promotional info up about SGI [origin?] servers and EP I. Fast forward three years and we have come upon another case of Linux and [relativel] commodity hardware changing the heart of a big Iron SGI all-star. ILM did have a JEDI Pact [sgi.com] with SGI not too long ago, but as was inferred in the article, its really hard to compete with free (as in beer) in the shrinking-margin world of SFX.
FWIW, On the Ep I DVD Making of Documentary, OS 9 was visible durinag a photoshopping session, Windows (or a GUI clone) for Motion capture and unix (presumably IRIX) for the rest.
Mirror with pictures (Score:3, Informative)
wget -p --convert-links http://www.linuxjournal.com/article.php?sid=6011
Re:CBDTPA .... Linux ... the irony... (Score:2, Informative)
Alternate mirror (better bandwidth) (Score:3, Informative)
Here [tjs.org].
Re:Great News (Score:1, Informative)
Isn't this called SDL? Anytime I need to write code that even hints at needing multimedia capability I pull up the SDL man pages. Its fast, supports 2D and 3D (via OpenGL), Open Source, does hardware and software rendering, and handles keyboard/mouse/joystick input. Oh yeah, its also platform independent.