Intel Releases 5,000 Pages of Open-Source Haswell Documentation 111
An anonymous reader writes "Intel has ended out 2013 by publishing 5,000 pages of new GPU documentation about their latest generation 'Haswell' graphics hardware. The new documentation complements their longstanding open-source Linux graphics driver that has supported Haswell HD / Iris Graphics since last year. The new documentation covers the hardware registers and special information for 3D, video acceleration, performance counters, and GPGPU programming."
I received them earlier ... (Score:4, Funny)
... thanks Snowden!
is it complete? (Score:5, Funny)
Does it include APIs for the NSA backdoors?
Re: (Score:1)
Does it include APIs for the NSA backdoors?
Bloody idiot.
Re: (Score:2, Insightful)
Re: (Score:2)
he's talking to an audience that still mindlessly throws in a "beowulf" joke or whatever.
Then Hrothgar departed, his earl-throng attending him,
Folk-lord of Scyldings, forth from the building;
The war-chieftain wished then Wealhtheow to look for,
The queen for a bedmate.
?
Re: (Score:1)
Doesn't matter either way. Most of the open source fanboys out there talk a good game but most don't know shit. I bet out of everyone reading this here that maybe 1 in 100 could make any amount of sense of what the documents contain and about 1 in 10,000 will actually do something with the information.
Slashdot is just a fanboy paradise of flame baiting and a bunch of would be nerds tugging on each others dicks. Why do you think people don't talk about real tech around here anymore?
Re: (Score:2)
Only issue w/ that theory is that every FOSS project that ever came up should be a rip-roaring success like MS Office or Angry Birds, and no project should ever have a shortage of volunteers.
In reality, few people can afford to work for $0.00 - particularly the ones w/ the skills to be of any use to these projects.
Re: (Score:1)
Re:is it complete? (Score:5, Funny)
what a relief to find that the arguments (of the api calls) only need rot13 applied twice in succession. its not at all cpu-intensive, so that's a relief.
alarmingly, the result is returned in plaintext. waiting for nsa api v1.1 for the fix to that one.
Awesome! (Score:1)
Way to go Intel! Fully documented hardware is essential to the future of Free Software.
Re: (Score:2)
Dear Nvidia... (Score:5, Insightful)
Re: (Score:1)
What I wonder is what really makes it harder / impossible for Nvidia or whomever to do it but works for Intel? If anything.
Re:Dear Nvidia... (Score:5, Interesting)
What I wonder is what really makes it harder / impossible for Nvidia or whomever to do it but works for Intel? If anything.
The standard rumor is that they they all violate bogus patents rampantly and only by keeping their code secret (and possibly backdoored) can they stay afloat, in face of the patent trolls.
A deep cynic might claim that Intel can survive more of these trolls than nVidia could so this could be a competitive move. IIRC Intel and nVidia had a cross-licensing deal that involved Intel staying out of the discrete market - maybe that's due to expire soon.
Re: Dear Nvidia... (Score:1)
Discreet is dead. a 5 year old video card can run most modern games. Unless there's a breakthrough like ray tracing the majority won't bother.
Re: (Score:2)
Yeah, yeah. Discrete is dead, and IGP is good enough now... as long as you don't want hardware-accelerated realtime raytracing:
(drool)
http://www.siliconarts.co.kr/gpu-ip [siliconarts.co.kr]
(/drool)
When I can have Windows or Linux with full eye candy and zero performance hit (vs Win2k or something XFCE-like) at 2560x1920@240fps, I'll accept current GPUs as "good enough".
When I can open a pdf document on my phone or tablet and effortlessly fling through it without any perceptible lag waiting for the fonts to render, curr
Re: (Score:2)
(drool)
http://www.siliconarts.co.kr/gpu-ip [siliconarts.co.kr]
(/drool)
When I can have Windows or Linux with full eye candy and zero performance hit (vs Win2k or something XFCE-like) at 2560x1920@240fps, I'll accept current GPUs as "good enough".
I checked out that link and it looked like I was stepping back into the 90s. That image on the home page looks like it's a 256 colour GIF! Where's the specular mapping? Everything in those shots looks dead, like a bad phong highlighted raytrace.
Re: (Score:2)
There's much more impressive stuff going on with path tracing on conventional GPUs [youtube.com] - something that, at least for me, is making a definite case for ungodly improvements in processing power for GPU hardware.
Re: (Score:2)
Re: (Score:1)
Don't forget GPGPU. Discrete AMD cards still slaughter anything short of a Xeon Phi for GPGPU.
Re: (Score:3)
The standard rumor is that they they all violate bogus patents rampantly and only by keeping their code secret (and possibly backdoored) can they stay afloat, in face of the patent trolls.
The other reason would be that if NVidia engineers could read the documentation needed to write ATI drivers and vice versa, they would figure out some clever ideas that their competitor had and reproduce them. If you make more and more powerful cards, you always need new clever ideas how you turn more transistors into more speed. You run into bottlenecks that weren't bottlenecks when you had a quarter of the transistors. Since Intel integrated graphics is quite a bit behind in that respect, Intel is probabl
Re:Dear Nvidia... (Score:4, Insightful)
The other thing is, well, the documentation simply doesn't exist. Having worked with quite a few ASIC vendors, inside and out, I can tell you the modern SoC if heavily undocumented. For a good chunk of the blocks, unless they're external IP (from say, Synopsys, Cadence or ARM), there IS no documentation other than a register list.
Yes, the documentation may just be a register list. Nothing to tell you how you should drive the hardware, if the registers have to be programmed in a certain way, or if there's tricks and other things that are important to know. Or even how some data structures need to be laid out in memory for hardware (you may get a brief layout, but that's it). Any my favorite - how the hardware reacts to an error. Sometimes it just plain locks up requiring a full reset. Other times it sets some strange error bit that must be cleared in order to resume functionality. If you're lucky to know how to clear the error.
Oh, and how does a software developer find out such information? Easy, they ask the ASIC designer how they envisioned the thing to work. Sometimes they're helpful and tell you exactly what you need to know, but they can be terse and expect you to figure out their thinking. And sometimes what you want to do LOOKS possible from the registers, only to find out that no, you just cannot program the bits that way and no, you shouldn't try.
That's probably why Intel took so long to get the documentation out - Haswell has been out since what, April? And only now they release the documentation? Well, it's probably because Intel was getting the whole whack together from random designer notes, software design, etc. Then bringing all that together (Intel creates some of the best documentation around) and editing it and all that.
For the most part, nVidia and AMD probably DON'T have much in the way of documentation on the latest GPUs. Intel does, and only because they spend time and effort doing it - but Intel's also huge and has a lot of cash to have a bunch of engineers sitting around writing documentation exclusively.
If you think OSS people hate writing documentation - it's no better when it's commercial development - again, the project usually doesn't allow for much documentation to be created. And things like GPUs which change frequently, well, by the time the documents are written, it's obseleted 3 times over.
Re: (Score:1)
They get the ideas from analyzing the bottlenecks on their own GPU's by running performance analysis and using instrumentation like timers and event counters. Whatever FIFO is filling up first, which cache is always swapping the most, which bus is running to full capacity, how to avoid wasted unused reads and writes. It's non-stop. Every new feature adds new methods.
Re: (Score:2)
The standard rumor is that they they all violate bogus patents rampantly and only by keeping their code secret (and possibly backdoored) can they stay afloat, in face of the patent trolls.
That rumor would persist only because most people don't know anything about how lawsuits work. You don't need to see someone's source code before filing a lawsuit; you get to see their source code after you file the lawsuit.
Re: (Score:2)
The patents are in the OpenGL implementation, not in the hardware specifications.
Re: (Score:2)
It's not the code that is kept secret from the patent trolls, executable code can always be disassembled, precompiled shaders can be disassembled - usually there's even a free disassembler thrown in with most development kits. The secret bits are the comments; explanations of techniques, todos, for-the-future, optimize this, that will be done in the next cycle. It's enough to mention two buzzwords together in one line, and the patent-trolls will be jumping up and down shout patent violation and wanting paym
Re:Dear Nvidia... (Score:5, Interesting)
I'm still waiting for Intel drivers that are on par with their Nvidia counterpart.
Despite all of the noise made about Intel's cooperation, this is the first time we've actually had full disclosure from them. Prior to today what they offered was incomplete. It was all empty promises despite of all of the rhetoric from the political purists about how Intel does things better.
Someday, this might lead to a proper driver. Although Intel hardware will probably still be just as lame then.
Re: (Score:3)
I doubt that. The firmware, maybe, but probably not drivers. Normally the difference between high-priced and low-priced models is that the low-priced models have some internal fuses blown, so that some of the cores are disabled. Sometimes those cores were defective, other times they disable cores just to meet the demand. It could be that they disable the cores with firmware instead of fuses, and somehow the drivers could reenable cores in the latter cases, but my guess is that the people who give the orders
Re: (Score:2)
If that firmware is loaded by the driver, then it's essentially the same thing, isn't it?
Re: (Score:2)
Re: (Score:1)
At least earlier you flashed the firmware onto the card, I think that's the case still as people have been tinkering with GeForce 600-series cards trying to make them similar to the 700-series ones as far as clocking goes.
Re: (Score:2)
I doubt that. The firmware, maybe, but probably not drivers. Normally the difference between high-priced and low-priced models is that the low-priced models have some internal fuses blown, so that some of the cores are disabled. Sometimes those cores were defective, other times they disable cores just to meet the demand. It could be that they disable the cores with firmware instead of fuses, and somehow the drivers could reenable cores in the latter cases, but my guess is that the people who give the orders simply think of their precious architecture details as information that needs to be kept secret, in case the competition gets too many ideas from those details.
drivers and/or jumpers
http://www.eevblog.com/forum/chat/hacking-nvidia-cards-into-their-professional-counterparts/ [eevblog.com]
Re: (Score:3)
Market disagrees. Vast majority of PCs sold, both desktop and laptop run intel's intergrated graphics. Most of them are the older GMA chipsets, but quite a few desktops nowadays run on intel's HD GPUs that come with the CPU.
Re: (Score:3)
That's a trap. The mobo comes with integrated intel, yes. But in most cases the end user also has a discrete card. You can guess which one is actually used.
Re: (Score:1)
That's a trap. The mobo comes with integrated intel, yes. But in most cases the end user also has a discrete card. You can guess which one is actually used.
Depends if its WebGL in a browser nvidia locks you to intel: Option to select the preferred graphics processor is greyed out for IE, Chrome, and Firefox. [custhelp.com] and https://www.scirra.com/blog/ashley/7/nvidia-hobbles-webgl-performance-on-laptops [scirra.com]
Re: (Score:2)
GMA comes on a mobo. More modern HD comes on CPU.
Re: (Score:2)
Latest Intel Iris Pro can easily compete with mid-range discrete solutions from Nvidia.
How the Intel fanboys can think that this is bragging material is beyond me...
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
That's nice.
We're still talking about huge market of gaming GPUs, not the tiny market of compute GPUs.
Re: (Score:2)
Re: (Score:2)
I see what you did there. We're all talking about nvidia and intel and you decided that discussion doesn't really matter at all and went on to talk about powervr.
Since we're on the line of being pants on the hat stupid, let's talk about pants. I'm pretty sure they're worn on the legs!
Re: (Score:2)
Re: (Score:2)
NVidia on the other hand has to fight for every dollar they make. There are dozens of ARM chip designers who would sell their souls for their documentation. NVidia doesn't have a single product or technology which others wouldn't love to compete against. With companies like Qualcomm
Re: (Score:2)
Re: (Score:1)
Please take notice, this is how to support GPU hardware correctly.
Considering Nvidia are the leaders in Linux performance drivers. You should send that letter to AMD instead.
AMD couldn't be bothered to consider Linux as a "profitable return" in the past. Hence it took them so bloody long to add simple GPU acceleration to its driver.
Nvidia was always the leader in Linux GPU performance and support, way before AMD.
Yes, i'd love to see Nvidia release their source code. But to be honest, i'd rather AMD released theirs first, it needs alot more work to bring it upto Nvidia's q
Code. (Score:4, Insightful)
Documentation is ONE PART. It says what the design was supposed to be like.
Then you have errata and variations - when some of the hardware doesn't correspond to the documentation and acts differently.
Then you have examples - where someone shows you how to, e.g. draw a simple triangle using the documented opcodes and all of the boilerplate and set up necessary.
And then you have actual working code. Where you give away, for example, a complete implementation that conforms to a higher, standardised API and issues instructions to the hardware to perform those actions.
Out of all of those, documentation is the easiest thing to do. You can just (for example, just flicked through a PDF from that site) say that instruction X transposes a matrix. No idea of performance, whether that's the recommended way, what it contends with, how it works, whether the Intel drivers use that themselves, whether it's a legacy function, whether it has huge constraints on its use.
Without some code, it's all just fancy tech sheets. Sure, better than nothing, but a long way from actual co-operation. I'm not saying Intel don't co-operate in other areas, but documentation like this? That's the "quick reference" stuff for when your thousands of lines of existing example code don't act like you expect when you tweak them and you look up what that operand is supposed to do and how.
Put a hardware driver author in front of a documentation pack and a compiler, and tell him to write a driver, and he'll tell you to fuck off.
Put a hardware driver author in front of many working examples of device, with debug-level access, with example source (that he can't just copy due to licensing), errata, a direct line to cooperative hardware engineers AND this documentation and he'll start.
This is why I've never been that bothered by documentation releases, or even unmaintained source-drops. Supposedly Broadcom did something similar for the RPi's graphics chips. I think we're still waiting on anything that's not a binary driver there. And we have this sort of stuff for some ancient 3D graphics cards - it's just not as easy as reading it all and then sitting down to write a driver.
Intel, nVidia, ATI: Give us drivers with code that have no reliance on "black box" information/code, and we'll be happy. Until then, it's just lip-service. And you know that. That's why you don't release this kind of stuff for graphics chips, and nor does anyone else. Because you can drop this in someone's lap and years later STILL end up being pestered to the ends of the earth for an open-source driver (or assistance to help write one) because it doesn't exist.
Code is a lot more than writing things to perform a protocol described in the documentation. If only it were that easy.
Re: (Score:2)
To cite a simple TL;DR analogy:
Documentation is like giving someone a dictionary to a foreign language they don't know.
Getting a working driver is like asking them to write the laws of the country in that language, and give a speech to inspire the majority of people who can understand it.
Re:Code. (Score:5, Insightful)
You make a good point, however you are incorrect. As an author of a handful of drivers, and contributor to a handful more - we like specs. If you are incapable of taking an RFC or spec and outputting a working driver, then you aren't quite the programmer you think you are. Specs are often all a driver author ever receives, and you should be able to produce working code with nothing else. It's *nice* if the vendor sends best practices or additional notes about deprecation of certain methods...but it's not the norm, and should not be expected. Being a good developer means being able to benchmark methods described in specs and determine what performs best, and when that applies.
Re: (Score:2)
This. I'm not especially prolific or talented, but even I generally tend to write code directly from the spec.
(GP)
Then you have examples - where someone shows you how to, e.g. draw a simple triangle using the documented opcodes and all of the boilerplate and set up necessary.
These are usually pretty useless, involve horrible paradigms only used by crazy people like a buinch of access macros for some sublanguage-of-the-week that the authors thought was chic, and don't yield any information beyond what the documentation says.
However, the GP's point about documentation like "opcode foo: does foo" stands. That said, if the documentation did tell me how efficient a gi
Re: (Score:1)
I write drivers for a living. All I want is good specs.
Have you seen the quality of the code written by the hardware design guys?
Re: (Score:3)
Have you seen the quality of the code written by the hardware design guys?
Yep. Atrocious. Topped only by BIOS coders. I have a laptop that you have to limit your BIOS sessions to under 1 minute or it will overheat. Animals.
Re:Code. (Score:4, Insightful)
About 15 years ago, my company (a producer of VMEbus and CompactPCI boards) designed a video module. We used a Trident mobile graphics chip. Unfortunately, we were attempting to use it with a PowerPC, not an x86 CPU. We had the big user manual for the chip, but when we programmed all the registers according to the published configuration info, it refused to initialize.
We then were given the BIOS object code from the factory (they wouldn't share the x86 assembly source code). We disassembled the code. It was such a tangled web of spaghetti that we never did figure out how to get the part initialized, and the factory app engineering team was unable to tell us how to do so either.
We eventually dumped the part and used an Intel part with C source code available. It worked just fine.
Re: (Score:2)
Re:Code. (Score:5, Informative)
lots of blah blah blaa
The Haswell GPU driver source code has been in the upstream kernel and userspace parts for maybe a year now.
Re: (Score:2)
Yes, it would be nice if we could get entire stack - documentation, working code, test examples, free support accounts, testing hardware, source repository access, Intel engineers payed to work on our favorite project, board of directors meeting memos and our own Santa but that is not going to happen. Documentation and some support is probably all the community would get, but that should be enough. The community usually had to work with a lot less and it was still capable of making useful code.
Re: (Score:2)
Documentation is ONE PART. It says what the design was supposed to be like.
Then you have errata and variations - when some of the hardware doesn't correspond to the documentation and acts differently.
Then you have examples - where someone shows you how to, e.g. draw a simple triangle using the documented opcodes and all of the boilerplate and set up necessary.
And then you have actual working code. Where you give away, for example, a complete implementation that conforms to a higher, standardised API and issues instructions to the hardware to perform those actions.
Out of all of those, documentation is the easiest thing to do. You can just (for example, just flicked through a PDF from that site) say that instruction X transposes a matrix. No idea of performance, whether that's the recommended way, what it contends with, how it works, whether the Intel drivers use that themselves, whether it's a legacy function, whether it has huge constraints on its use.
Without some code, it's all just fancy tech sheets. Sure, better than nothing, but a long way from actual co-operation. I'm not saying Intel don't co-operate in other areas, but documentation like this? That's the "quick reference" stuff for when your thousands of lines of existing example code don't act like you expect when you tweak them and you look up what that operand is supposed to do and how.
Put a hardware driver author in front of a documentation pack and a compiler, and tell him to write a driver, and he'll tell you to fuck off.
Put a hardware driver author in front of many working examples of device, with debug-level access, with example source (that he can't just copy due to licensing), errata, a direct line to cooperative hardware engineers AND this documentation and he'll start.
This is why I've never been that bothered by documentation releases, or even unmaintained source-drops. Supposedly Broadcom did something similar for the RPi's graphics chips. I think we're still waiting on anything that's not a binary driver there. And we have this sort of stuff for some ancient 3D graphics cards - it's just not as easy as reading it all and then sitting down to write a driver.
Intel, nVidia, ATI: Give us drivers with code that have no reliance on "black box" information/code, and we'll be happy. Until then, it's just lip-service. And you know that. That's why you don't release this kind of stuff for graphics chips, and nor does anyone else. Because you can drop this in someone's lap and years later STILL end up being pestered to the ends of the earth for an open-source driver (or assistance to help write one) because it doesn't exist.
Code is a lot more than writing things to perform a protocol described in the documentation. If only it were that easy.
It's been a few years since I worked in hardware, but even when you build it for a commercial company; not open source, you don't get the things you (poster) think are important.
In the past I found AMD far more helpful than Intel, and I was building high end workstations then (Motorola and MIPS based, but lots of AMD chips). MIPS and AMD at least gave you the documentation for free if they thought you were serious: that was LOTS of thick books. Sometimes they helpd you figure out if things didn't work p
Re: (Score:2)
Put a hardware driver author in front of a documentation pack and a compiler, and tell him to write a driver, and he'll tell you to fuck off.
Put a hardware driver author in front of many working examples of device, with debug-level access, with example source (that he can't just copy due to licensing), errata, a direct line to cooperative hardware engineers AND this documentation and he'll start.
You don't want a hardware driver author, you want a technical writer. They will have the experience to draw prett
Re: (Score:3)
Put a hardware driver author in front of a documentation pack and a compiler, and tell him to write a driver, and he'll tell you to fuck off.
Put a hardware driver author in front of many working examples of device, with debug-level access, with example source (that he can't just copy due to licensing), errata, a direct line to cooperative hardware engineers AND this documentation and he'll start.
So in short you're saying the open source community is a bunch of liars and nVidia is right when they say "Well if we gave you X you'd ask for Y and Z until we're really doing all the job anyway, so why give you the little finger when you'll chew off our arm? If you're going to leave it all to us anyway, use the binary." For sure, there are hardware bugs and maybe undocumented ones too because the closed source driver never tripped it but this is like saying you can't program an x86 processor because Intel
Re: (Score:2)
Unfortunately, this is another example where software developers are not a profession.
One could argue that Intel isn't going to spend the time and money to do all those things. They require man power to fund, and so they're not likely to do them.
Microsoft was/is one of the better companies in this regard and they mainly did it because you were locked into their platform and APIs. Their whole money making scheme was based around software being made for their platform.
But ultimately, this gap is what differen
Re: (Score:2)
Put a hardware driver author in front of a documentation pack and a compiler, and tell him to write a driver, and he'll tell you to fuck off.
My how things have changed. I remember being handed register documentation for StarPort 16-, 32- and 64-port serial cards and being asked to write FOSSIL drivers for them. And I had to supply my own compiler and logic analyser.
Re: (Score:2)
I remember being handed register documentation for StarPort 16-, 32- and 64-port serial cards and being asked to write FOSSIL drivers for them. And I had to supply my own compiler and logic analyser
I remember being asked to waste my time on things too, to reinvent wheels that had already been invented, and to reverse engineer things that we were presumably supposed to know how worked.
GP is right, you say 'fuck off' when you get handed a task like that. Maybe, it doesn't go anywhere, and you really do have
Re: (Score:2)
No, *you* say fuck off. The rest of us are competent.
Re: (Score:2)
The rest of us are competent.
Just because I can reinvent the wheel, doesn't mean I should waste my time doing it.
Being competent doesn't make it any less a waste of time. If a month of development time can be eliminated by simply talking to an engineer who built the X and knows it, then only an idiot would brag about how he wasted the weeks figuring it out for himself... because, you know, "competence".
I can waste 10 weeks too, if I have to. Sometimes that IS the only way. But unlike you I seem to be able t
Re: (Score:2)
"If a month of development time can be eliminated by simply talking to an engineer who built the X "
If you are waiting until the X is built, you've already lost the market.
Re: (Score:2)
"Out of all of those, documentation is the easiest thing to do."
If documentation is so easy, why is there so little of it and why is it so bad?
Re: (Score:2)
Other implications for manufacturers. (Score:1)
This release would make it completely possible for a manufacturers to easily create completely supported devices without the need for the Windows OS. Given the long stability and versatility of GNU and OSS. There is no reason why manufacturers could not jump ship. The average joe does not care which OS is on a device. It is only those who hold Windows as something akin to a religious experience on their computing device that bleat away against OSS operating systems.
Chrome books, android devices, even the i
Re: (Score:2)
Thank you, Intel. . . (Score:1)
Re: (Score:2)
Because it uses much, much less power?
Good luck putting a 300 watt GPU in a tablet.
those id10ts @ slashdot (Score:2)
Dear samzenpus,
Please don't include any link directly to the documentation.
Regards
The Community.
Re: (Score:2)
Re: (Score:1)
Re: (Score:2)
Re: (Score:1)