Interview: Brian Paul Answers

Okay, here they are: Brian Paul's answers to your questions. Even if you've never heard of Brian or Mesa, his comments give insight into how an "essential but unsung" Open Source project runs, and why Brian has devoted endless time to Mesa. (More below.)

Roadmap?
by ink
Could you please explain how all this fits together?

• Precision Insight
• XFree86 4.0
• Mesa
• Glide/3dfx acceleration
• Other accelerations
• GLX
• DGA
• SVGAlib/GGI/KGI and 3D acceleration

Many are confused. ;)

Brian:

Precision Insight, Inc. is the company I work for. We're an open-source development company focusing on 3D graphics drivers for XFree86 on Linux. We're funded by 3D hardware vendors and others (such as Red Hat and SGI) to implement the infrastructure and drivers for specific hardware. Go to www.precisioninsight.com for more information.

XFree86 4.0 will be the basis for our 3D hardware acceleration work. It will include the DRI/DRM technology we've developed.

Mesa is my OpenGL work-alike library. It provides the top- level interface between applications and graphics hardware.

Glide is a thin, low-level interface to the 3dfx Voodoo hardware. Recently, Glide was open-sourced by 3dfx. The first popular graphics hardware to be supported by Mesa was the Voodoo hardware. The existance of Glide on Windows and Linux made that possible. Otherwise, we would have needed register-level hardware documentation.

GLX is actually three things: an extension to the X server which integrates OpenGL with X, it's a wire protocol for remote rendering, and it's an API for X-based OpenGL programs.

DGA is XFree86's Direct Graphics Architecture. It's a method for direct access to your graphics hardware, bypassing Xlib. It also lets you switch display resolutions and depths on the fly. At this time, DGA is not part of our DRI-based driver work.

SVGAlib/GGI/KGI are non-X-centric approaches to graphics on Linux (and other OSes, I believe). They're not components in our DRI system.

One important component you missed is the DRI (Direct Rendering Infrastructure). This is a component developed by PI to solve the problems of resource management, coordination, and secure hardware access. In some ways you can think of this as the glue holding all the other components together.

The way that XFree86, the DRI, GLX, Mesa and the hardware drivers fit together is pretty complicated. In fact, that's a big reason why it's taken so long for 3D accelleration to happen on Linux. It's a big, complicated problem.

What's Missing?
by handorf

We've seen Mesa and X come pretty far in the last year or so. We've got nVidia, SGI, Precision Insight, 3dfx and others kicking in support for the efforts. Willing developers aren't too difficult to come by either.

So what are we missing? Will X 4.0 solve our issues with 3d lagging behind MS? What do we need to go to get 3d chip makers bundling Linux drivers with their cards?

Basically, where do you see the Linux/3d systems' weaknesses? We're moving ahead, but are we going to catch up or are we always going to be playing "Johhny-come-lately"?

Brian:

In the past, the weaknesses were lack of hardware specifications and lack of manpower/expertise.

The hardware vendors now recognize the value of the Linux market and want to satisfy it. So now they're funding us to develop the needed drivers and infrastructure. When you send in the product registration card for your new 3D hardware, be sure you indicate that your OS of choice is Linux. That'll help affirm the presence of the Linux market.

As I said before, implementing 3D hardware acceleration is a really big and complicated problem. You need skilled engineers to work on a bunch of components including the kernel, the X server, the GLX interface, 3D software (Mesa) and the hardware drivers. In the past it was almost impossible to find the manpower to do all this. The people with the know-how were already employed full-time with other jobs. Thanks to IHV funding and the creation of Precision Insight, we've now got a strong group of skilled engineers working full-time on the project.

I don't think we're missing anything now. It's just a matter of time before 3D hardware is well-supported on Linux. As for weaknesses, I don't see any obvious ones. We have the know-how to produce high quality, high performace 3D solutions on Linux.

I could ask you...
by aheitner

...about dry performance tradeoffs and clever ways of making GL blazing fast. But I've got a better question.

A long time ago, you said you didn't want to be paid for your efforts on Mesa. Rather than sending you checks, you requested that people show their appreciation by sending you a six-pack of an interesting local beer.

I imagine all these years later you had the opportunity to sample quite a few brews. Which have been your favorites, and why? Have you received beer from any interesting people?

Brian:

I think I've had 8-10 people send me beer. Unfortunately, I didn't keep a list of them and don't remember specific ones. I've enjoyed them all though! Probably the most "interesting" experience was when someone from Finland (I think) sent me a case of beer. I figured I just had to go down to the shipper's office in Boston to pick it up. Wrong! I had to go to 3 different agencies scattered around town: first the shipping agent, then the US customs office and finally to the British Airways warehouse. It took the whole afternoon to track down that one case of beer and I must have signed my name to a dozen forms along the way! It was worth it though!

Distributed Mesa?
by Lord Kano

Are there any plans to develop a "professional" tool based on Mesa for "professional quality" 3d rendering?

In a nut shell I'm thinking of an app designed to run on a cluster of machines to decreas rendering time for movies.

Brian:

I think most of the commercial 3D vendors provide render farm support. SoftImage, for example, uses Mental Ray as its (distributed) renderer. However, these renderers are not based on OpenGL/Mesa. They're typically scan-line renderers which don't need or use graphics hardware. All you need is a lot of memory, disk, bandwidth and CPU power.

A few people have experimented with parallelizing Mesa either on multi-cpu systems or on network clusters. I used to have the URL for one such project but have misplaced it. I'm sure that if you do a web search for "Mesa" and "parallel" you'll find something. I don't have plans to personally work on that sort of thing- I've got too many other things to do.

3D Engine Design Question
by Effugas

I've been banging around a theory for a while, one that (in my mind) explains why Direct3D drivers end up being ubiquitous(I mean beyond the fact that *MS* is behind it).

Would it be fair to say that OpenGL Driver Development is much more difficult than Direct3D Driver Development because Direct3D is a much, much simpler 3D description language?

In other words, game/app programmers can expect much richer functionality out of the OpenGL API, but at the expense of the chipset designer needing to implement that same extremely rich functionality? In other words, do game programmers have to reinvent a wheel that perfectly suits their needs in Direct3D, whereas chipset designers have to reinvent a wheel that perfectly suits their hardware for OpenGL?

Brian:

I'm not 100% clear on your question but I'll take a stab at it.

I'm not an expert on Direct3D but I'd expect that writing a high-quality driver for Direct3D takes about as much time as for OpenGL.

For 3D game developers, I think the hardware really determines which features are going to be used, regardless of the API. Only those features which the hardware accelerates will be used and I'd expect the same hardware features to be exposed by both the Direct3D and OpenGL drivers (with some assorted exceptions, of course).

For general 3D applications performance might not always be the top concern. Portability, uniform, rich feature set and good documentation can make the API decision (hopefully OpenGL).

In the early generations of PC 3D hardware, the chip makers only implemented the features which were essential to satisfying the needs of gamers. The competitive nature of the business has forced the IHVs to increase performance, increase image quality and/or expand the feature set. I don't think targetting a particular API is their top priority. It's industry competion which drives their designs.

However, I do think the hardware designers have found the OpenGL specification to be pretty useful. It defines a pretty comprehensive set of features and a concrete order of operations. If if didn't exist I bet there'd be a lot of painful inconsistencies across the available hardware.

A quick question
by jd

What do you see as the advantage of Mesa over other GL implementations, or even other 3D approaches?

In other words, what does Mesa/GL have, that makes it a system you want to develop?

Brian:

Mesa's open source. Because of that, Mesa has been ported to dozens of operating systems. Now, there's hardly an OS in existance that doesn't have an OpenGL-compatible library. That's great for application developers. Otherwise, you'd be forced to use proprietary graphics libraries on different systems. And some systems wouldn't have a 3D graphics library at all.

Modesty aside, I have to wonder how popular OpenGL might be if it weren't for Mesa. A lot of people have learned 3D graphics programming with Mesa who otherwise wouldn't have had access to the technology. And Mesa has allowed application developers to deploy OpenGL applications on platforms they might have had to omit otherwise.

Personally, I _really_ like the clean design of the OpenGL API and pipeline. It's been fun to implement it. Which leads to...

Implementing existing standards
by shawnhargreaves

All these years you've been doing such great work on MESA, youve been concentrating on implementing the existing OpenGL API. Does it ever get frustrating having to stick to that standard, and do you ever wish you were able to do things a bit differently? If you could, what things about OpenGL would you most like to change?

Brian:

Some little corners of the OpenGL spec have been a pain to implement. At times it was tempting to take short cuts and leave things out that I didn't like but I realized pretty early on that that would be a very bad move. If I didn't follow the spec, lots of users would eventually find the inconsistencies and report bugs and maybe even abandon Mesa. It could have also made a bad impression of OpenGL on people.

I also realize that OpenGL was designed from the beginning for hardware implementation. That makes a software implementation a bit clumsy sometimes.

If I could change the spec though, there's a few things I'd change:

• omit texture borders
• omit glEdgeFlag
• omit glPolygonMode
• omit glColorMaterial
• require homogeneous use of glVertex, glColor, etc calls between glBegin/glEnd.
• overhaul glRasterPos semantics, add glWindowPos.
• minor changes to glDrawPixels

These would simplify my life as an OpenGL API implementor but probably not dramatically. The fact that I don't have any big complaints with the API goes to show how good I think it is.

Legal Hassles?
by Effugas

You've been the implementor of quite the massive reverse engineering project -- something rather similar to what we might have expected out of the Qt-clone Harmony project had the tea leaves turned out differently.

Have you had any legal hassles from SGI, before they turned so powerfully towards the Open Source ethic? Similarly, how do you see your relationship with SGI now that they're no longer incompatible with their culture?

Brian:

I might have had legal problems with SGI if I wouldn't have been so careful. Before releasing Mesa I asked SGI for their position on an open source implementation of the spec. I worked with them to define the disclaimer and legal terms of the library. I think they appreciated that. If I'd have just sprung Mesa on the world without checking with them first I might have upset them.

Next, I think that they appreciate the fact that I try to be fully compliant with the spec. If I'd have written an incomplete or buggy library they might not be too happy. Users would be unhappy and confused too -- witness the MiniGL drivers of recent years.

Finally, as I said before, Mesa has allowed a lot of people to learn and use OpenGL who otherwise wouldn't have access to an official implementation. I think that Mesa's helped to promote OpenGL, not threaten it.

Question about OpenGL licensing...
by Rexifer

Is there any chance that Mesa3D will eventually become an OpenGL licensee? At work, the higher-ups have officially discounted Linux as a viable option for developing some of our visualization tools because Mesa isn't officially OpenGL, despite the many protests of the developers... :(

With Microsoft backing off Open GL, how badly does this affect game developers developing for different OS's out there?

Brian:

There's a chance that Mesa could become an official OpenGL. Really all that's required is a license. However, there's issues about conformance testing the many Mesa drivers on the many Mesa platforms that makes this complicated. I can't give any firm predictions on what might happen or when.

Despite Mesa not being an official OpenGL I am able to run the conformance tests on it (provided by SGI under strict terms). I'm currently only testing with software Xlib rendering and some tests don't pass yet. But from my personal experience with many other OpenGL implementations I can say that Mesa is at least as good as the average OpenGL from other vendors.

Finally, Microsoft isn't backing away from OpenGL. The Windows 2000 / OpenGL story wasn't factual and misled many people. You can find clarifications from Microsoft in several forums.

SGI involvement
by shawnhargreaves

Now that SGI are getting involved with Linux, have you had much to do with their original OpenGL developers? Do you think they will be contributing code to MESA, or are they more likely to come up with their own alternative implementations? I would imagine it feels quite strange to be suddenly getting so much interest from the "big official company" that started the whole thing in the first place!

On a related note, now that SGI are making noises about Linux support, is there any chance of MESA becoming officialificated so that we can actually call it OpenGL? Not that this makes any practical difference, but it would be nice if we could finally stop having to call it "something that works exactly like OpenGL, but isn't OpenGL for legal reasons" :-)

Brian:

I've known several of the SGI OpenGL engineers for years now. We get along great. Some former SGI people have contributed features to Mesa over the years.

I've said all I can about Mesa becoming an official OpenGL in the preceeding answer.

Where do you go now?
by asparagus

Back a few years ago, Mesa was the only choice for 3d (OpenGL-esque, at least...don't get me started on QD3D) on the macintosh platform. It worked great! [When apple finally released their version, it took 10 minutes for me to relink libaries and recompile my code. Nice work.]

However, now that Apple's released/purchased OpenGL for their platform, development for mesa has lagged. (Can't compete, unfortunately.) I'm certain that this has happened on more than one of the Mesa3D platforms. And so, my question:

Where do you see development of Mesa going once a particular platform has licensed opengl?

Brian:

On some platforms, like Windows, there isn't a really big need for Mesa. There's lots of other (hardware accelerated) OpenGL drivers available. By that reasoning, there may certainly be a number of platforms on which Mesa isn't relevant.

However, the fact that Mesa is open source and that IHVs are releasing their technical specs changes this a bit. The open source community has proven its ability to produce large scale, efficient, high quality products. In many cases an open source project can produce better results than a closed, commercial development team.

In the way that Linux is outpacing commercial Unix products, so might Mesa outpace commercial OpenGL drivers. Mesa's got enough momentum behind it now that this could happen. I'm not betting on it but I won't be surprised if it happens on a few platforms.

Unfortunately, Mesa has lagged on Windows and MacOS in comparison to Linux and similar OSes. It's really a supply/demand thing. If there's a big enough demand for Mesa support on a particular platform, eventually some of those interested will step up to the plate and put in the work needed to make it useful.

[I'll also answer a few more questions which I thought were interesting:]

Mesa on MacOS, MacOS X Server
by Anonymous Coward

I do most of my computer graphics work on SGI IRIX/OpenGL, and sometimes I try to recompile some code, just for fun, on my Mac, using Apple's OpenGL SDK.

Mesa does exist for the Mac, but it's not distributed as part of the main Mesa distribution. I would like to see that change. So here is my question: why are MacOS and MacOS X Server versions of Mesa not part of the main Mesa distribution? Is this ever going to happen?

Thanks - and thanks for Mesa! Besides everything else, it's a GREAT educational tool having the source code available - I can show it to my comp.graphics students and say: see how it's implemented? See what it means?

Brian:

Mesa 3.1 (to be really very soon) will include the latest Mac code. I haven't used it so I can't say how well it works.

Motivation and Mark Kilgard
by Midnight Warrior

Brian,
Let me start off by saying I have followed you since before v1.2.8 and you have saved my company large sums of money by making us not port OpenGL to X ourselves (when IrisGL died). My question is this: When Mark Kilgard (of GLUT and the GLX green book fame) left SGI, he seems to have completely left the OpenGL scene. Or at least publicly. Do you envy his newfound privacy? If not, what keeps you going through all the "when are you going to support the xxxx card?" questions? Especially since you yourself have stated that the easy stuff is done. It's all optimization now - paths I'm sure you've already completely explored. That and 3d support is something you've almost always allowed others to build from the ground up (ie glide support). Thanks. Sorry I never sent you any beer from Ohio.

Brian:

I think Mark's just busy working really hard on NVIDIA's OpenGL drivers. He's contributed to several conferences lately so he hasn't left the OpenGL scene. GLUT could be called "done". There's not a whole lot needed there.

Somedays I get tired of dealing with Mesa bugs and random newbie questions but I'm still having fun working on the project so I put up with it. I somtimes wonder how much longer I'll have to energy to keep it going. Putting Mesa into CVS and getting more developers on board has helped me a lot.

What parts of OpenGL were fun to implement?
by foolishj

What were your favorite parts of OpenGL to implement, and which ones were your least favorite? Were there any areas with bugs that seemed to take forever to stomp out? Any new features you're looking forward to trying out?

Brian:

Favorite parts: designing the overall architecture. I also get some enjoyment from occasionally tearing up existing parts of the code, rewriting it to be more efficient, do new things, and making it easier to understand.

Least favorite: fixing nasty bugs. Occasionally I'll get a bug report that takes several days to track down and fix. Those aren't fun but you do feel good when you finally solve them.

Looking forward to: improving Mesa's device driver interface to better support 3D hardware.

In terms of the big picture, I'm looking forward to the day when I can go down to the local computer store, buy one or more of the latest graphics cards, find Linux drivers in the box, plug them all into my Linux box and seeing fast, full-featured, multi-head applications (games, VR sims, etc). And knowing that I had a hand in making it happen.

Response to John Carmack's call
by _jthm

What are your feelings, and responses, to John Carmack's recent comments in his .plan requesting an "Independant OpenGL conformance nazi" - "I think there is a strong need for a proactive, vendor-neutral OpenGL watchdog, or even a small group, especially in the linux space."

Brian:

I replied to this on the glx-dev mailing list. I agree with him. We need to strive for quality.

Engineering Applications
by EXTomar

I know that most people are interested in MesaGL for their gaming needs, but I want you to know that there are some of us who use your team's work for engineering applications and I might say, it works rather well. :-) How do you feel about any company who uses your team's work for these engineering applications, especially since they are often close source and usually have a high price tag?

ps. If Mr. Paul or any other MesaGL team members are interesting in seeing some of this work, please let me know. I would be more than happy to show you some examples. :-)

Brian:

Personally, I'm more interested in 3D applications like visualization, CAD, visual sim, etc. than games. My first real job was in scientific visualization at the U. of Wisconsin. It was really rewarding to work on software used by scientists trying to better understand the world. I think games are great but I get a lot more satisfaction from hearing about scientists and professionals using Mesa to do their work than people using Mesa to play games.

And I do appreciate the game industry for pushing the 3D technology. If it weren't for huge gaming market we wouldn't have the amazingly fast and cheap hardware we have now.

I'd also like to point out that if games were the only real use of 3D we might all be happy with simple full-screen hardware drivers. It's the non-game applications which demand 3D in an X window.

The Next Step in 3D Building Blocks
by StormCaller

A little while back, John Carmack made a comment in Next Generation about future 3D technologies, discussing alternates to NURBS such as displacement mapping and subdivision surfaces. Basically a polygons vs NURBS vs voxels etc sort of thing and which technology would be the next big thing. What's your take on that, specifically, are NURBS the future, or are there other technologies that would be just as viable?

Brian:

NURBS? Yawn. I think image-based rendering is where it's at. If you've been following developments in the graphics field for the past few years you'll be familiar with this.

Basically, image-based rendering takes images captured from the real world and processes them so they may be resynthesized into virtual environments. Paul Debevec, for example has shown some really amazing results at recent SIGGRAPHs.

I think hardware accelerated image-based rendering is in our future. I don't have time to elaborate but I've had a few ideas about what could be done. It should be really exciting.