Open Source ARM userland(raspberrypi.org)
raspberrypi.org
Open Source ARM userland
http://www.raspberrypi.org/archives/2221
18 comments
I disagree strongly. The value is that if APIs change (which they will), the drivers can be ported forward. Unlike all those poor routers stuck on Linux 2.4 forever, Raspberry Pi can now be maintained by the community.
The purpose of open source isn't always to learn. Sometimes the purpose is just to make your computer work.
(It's interesting that hackers were opposed to softmodems and such for many years but now people are taking the opposite stance in this thread.)
The purpose of open source isn't always to learn. Sometimes the purpose is just to make your computer work.
(It's interesting that hackers were opposed to softmodems and such for many years but now people are taking the opposite stance in this thread.)
Is there no value to a computer beyond graphics and 3D?
It's fascinating to see people who would seem to be aligned with the RPi's goals getting hostile toward the RPi Foundation. Why is this? My (probably wrong) guess on the psychology at work is that it is not the lack of much forward progress in this project (unquestionably, they have made _some_ forward progress by making the RPi, regardless of whether it is "enough"), but it is the idea of "stealing someone else's fire". It threatens people whose devoted purpose is to fight for what they perceive as the impossible: a totally open device or those whose mission it is only to complain about the lack of openness in devices (cf. actually willing to make sacrafices to use an open device). It steals their fire. The more open things become, even if only a little more open, the less they have to fight for or complain about. Their devoted purpose starts to deflate. It's less important.
And I would guess there are also those who just can't stand to see any sort of change. When the "impossible" starts to seem eventually possible, it's disorienting for them.
My understanding of this announcement is that it's directed at people who are trying to port other OS's to the RPi. Having this code makes their work easier. What if I do not use Linux much? What if I am not insistent on using my computer only as a TV set or a game console? What if my usage is text-based? What if I just want to run another OS on the Pi? Would I consider making this userland code available a definite step forward?
It's fascinating to see people who would seem to be aligned with the RPi's goals getting hostile toward the RPi Foundation. Why is this? My (probably wrong) guess on the psychology at work is that it is not the lack of much forward progress in this project (unquestionably, they have made _some_ forward progress by making the RPi, regardless of whether it is "enough"), but it is the idea of "stealing someone else's fire". It threatens people whose devoted purpose is to fight for what they perceive as the impossible: a totally open device or those whose mission it is only to complain about the lack of openness in devices (cf. actually willing to make sacrafices to use an open device). It steals their fire. The more open things become, even if only a little more open, the less they have to fight for or complain about. Their devoted purpose starts to deflate. It's less important.
And I would guess there are also those who just can't stand to see any sort of change. When the "impossible" starts to seem eventually possible, it's disorienting for them.
My understanding of this announcement is that it's directed at people who are trying to port other OS's to the RPi. Having this code makes their work easier. What if I do not use Linux much? What if I am not insistent on using my computer only as a TV set or a game console? What if my usage is text-based? What if I just want to run another OS on the Pi? Would I consider making this userland code available a definite step forward?
Boilerplate? The title is "Open source arm user land". Which is exactly what the code is. With all the available code you can expose VideoCore services to bare metal or new OS ports.
The OpenGL ES implementation is accessible via the interface. The VideoCore firmware has not been open sourced and that claim hasn't been made.
The OpenGL ES implementation is accessible via the interface. The VideoCore firmware has not been open sourced and that claim hasn't been made.
You aren't the only person to think this:
http://airlied.livejournal.com/76383.html
"really what they've released is totally useless to anyone who uses or develops this stuff."
http://airlied.livejournal.com/76383.html
"really what they've released is totally useless to anyone who uses or develops this stuff."
Wow, this is big. Embedded vendors are particularly precious about their GPU drivers.
Hopefully this is the start of a trend.
Hopefully this is the start of a trend.
Absurdly precious in this case, given that it's just an RPC wrapper around the closed-source blob running on the GPU processor core which contains the real OpenGL ES driver. This reveals literally no technical details about the actual GPU hardware that the existing API didn't.
Having open drivers is further enhanced by the limited hardware models, it means more time to focus on getting the most out of the hardware instead of getting the stack to be compatible with different hardware all the time.
From what I can see in the referenced talk, Samsung will still only provide binaries for the graphics.
https://www.youtube.com/watch?v=LyYXh1V8ov8#t=30m40s
https://www.youtube.com/watch?v=LyYXh1V8ov8#t=30m40s
That's a shame. It's something at least...
Binary drivers or binary GPU firmware? If the latter then Pi is in much the same place. (I would watch the video but am at work...)
Binary drivers or binary GPU firmware? If the latter then Pi is in much the same place. (I would watch the video but am at work...)
I'm curious how much of the OpenGL stack lives on the GPU. Does anyone know how low- or high-level the interface to the Videocore is? (Yes, I could just dive into the official source code release and the unofficial reverse engineering project, but that would be an inefficient use of potentially hours, when someone in the know might be able to answer in minutes)
The GL calls are marshalled across to the VideoCore firmware. The VideoCore firmware has essentially a function for each GL entry point. The shader compiler is in the VideoCore firmware. The QPU code fragments are generated by a pull on a data flow representation in the firmware.
Thanks! I suppose that's not too surprising; I've yet to hear of anybody open sourcing their core OpenGL implementation (is there even an open implementation other than Mesa?). It would be fun, in a sense, to have raw access to the VideoCore components without any existing firmware. Judging from this comment[0] of yours that I found on a Google search, I'm not the first to think so.
Now, as I've said before, if the RPi gets solid USB host and isochronous transfer support (for e.g. Kinect, USB audio devices), then it will be unbeatable.
[0] http://www.raspberrypi.org/phpBB3/viewtopic.php?f=33&t=6...
Now, as I've said before, if the RPi gets solid USB host and isochronous transfer support (for e.g. Kinect, USB audio devices), then it will be unbeatable.
[0] http://www.raspberrypi.org/phpBB3/viewtopic.php?f=33&t=6...
It's possible to splice into the current blob by using /dev/mem to access the GPU partitioned memory. This lets you hot patch the VideoCore firmware (quite tricky as its running ThreadX OS), or you can hunt down QPU fragments generated by shaders.
I have a plan to release an Architecture/Programming guide and will be recruiting more hands shortly for tools like LLVM etc. But today let's celebrate the great work the foundation have done opening up the user land!
I have a plan to release an Architecture/Programming guide and will be recruiting more hands shortly for tools like LLVM etc. But today let's celebrate the great work the foundation have done opening up the user land!
Just commenting to say thanks for working on this. A barebones open source firmware would be very cool to see.
It certainly appears so, though that didn't stop Liz from spreading misinformation as usual when one of the developers of an actual open-source driver for an ARM GPU correctly pointed this out: http://www.raspberrypi.org/archives/2221#comment-34981
The RasPi foundation is staying as classy as ever.
The RasPi foundation is staying as classy as ever.
Sour grapes from Luc I would say. The Lima project is tainted by breaches of the Mali license agreement and side channel information leaked out despite NDA. They need to start again with a Chinese wall.
(Edit: According to rumours in the pubs around ARM headquarters)
I think everyone who announces that their hardware is "fully open-source" should answer the "Stallman test".
That is, answer the question "Would Stallman use this?".
That is, answer the question "Would Stallman use this?".
Instruction set is licensed, fails instantly!
Thanks for using BSD license so everyone can benefit.
> Aside from being exciting to FOSS enthusiasts for philosophical reasons
Freedom is not a philosophical reason.
Freedom is not a philosophical reason.
Philosophical doesn't mean arbitrary.
"Philosophical" means related to wisdom or knowledge. Philosophy inquires about the nature of things and the boundaries of nature itself.
In that sentence, «philosophical reasons», philosophical meant abstract, remote, detached from practical concerns. The freedoms sought by FLOSS people are very much practical (see https://www.gnu.org/philosophy/free-sw.html ) and with many concrete effects on your daily life. Engineering problems (problems with your video driver? Want to update the OS of your 2-year old tablet?) and social problems (ever heard of somebody whose books have been deleted remotely by a FLOSS app?).
In that sentence, «philosophical reasons», philosophical meant abstract, remote, detached from practical concerns. The freedoms sought by FLOSS people are very much practical (see https://www.gnu.org/philosophy/free-sw.html ) and with many concrete effects on your daily life. Engineering problems (problems with your video driver? Want to update the OS of your 2-year old tablet?) and social problems (ever heard of somebody whose books have been deleted remotely by a FLOSS app?).
You are mostly referring to epistemology and ontology, two rather theoretic branches of philosophy that probably aren't that relevant to software.
The philosophy of free software (even your URL contains the P-Word ;)) makes use of ethical arguments and political philosophy, for examples the ideals of enlightenment.
The age of enlightenment was kickstarted by philosophers, some of whom went to prison for clearly voicing thoughts that seem quite common to us today.
I want to argue that, while modern academic philosophy may be a bit of an ivory tower, philosophy in general isn't synonymous with "detached from practical concerns".
I want to argue that, while modern academic philosophy may be a bit of an ivory tower, philosophy in general isn't synonymous with "detached from practical concerns".
> philosophy in general isn't synonymous with "detached from practical concerns".
I strongly agree with you on this; I did not make my point clear enough. What I do not like is the use that the article's author did of the term "philosophical". I do not think that philosophy is synonymous with "detached from practical concerns" but it feels like the author of the article do:
> Aside from being exciting to FOSS enthusiasts for philosophical reasons, it’s also going to make it much easier for third party developers to (for instance) implement Wayland EGL client and EGL server support, or to provide better integration of GLES/VG with X.Org.
To me it reads like "Aside finally shouting the mouth of these FOSS do-gooders, this change has practical positive implications". As if the people pushing the FOSS philosophy where not doing that because of their interest in practical (long term) benefits.
I strongly agree with you on this; I did not make my point clear enough. What I do not like is the use that the article's author did of the term "philosophical". I do not think that philosophy is synonymous with "detached from practical concerns" but it feels like the author of the article do:
> Aside from being exciting to FOSS enthusiasts for philosophical reasons, it’s also going to make it much easier for third party developers to (for instance) implement Wayland EGL client and EGL server support, or to provide better integration of GLES/VG with X.Org.
To me it reads like "Aside finally shouting the mouth of these FOSS do-gooders, this change has practical positive implications". As if the people pushing the FOSS philosophy where not doing that because of their interest in practical (long term) benefits.
Well, that was unexpected. Bravo!
http://www.h-online.com/open/news/item/Raspberry-Pi-opens-it...
Has some more details.
Has some more details.
OK good, now the blob?
This is a pretty dumb request. What are you going to do with a chunk of bits for a processor for which you don't even have a toolchain. There is nothing more to open source other than what is already there.
Asking for 'the blob' is akin to asking for the firmware source to a hard disk. Even if you had it you could not do much with it.
Asking for 'the blob' is akin to asking for the firmware source to a hard disk. Even if you had it you could not do much with it.
Though I agree that the parent comment was unhelpful and a bit ungrateful, there is at least one good reason to want source code, especially for one-of-a-kind GPU processors. It's just plain interesting.
Asking for 'the blob' is akin to asking for the firmware source to a hard disk. Even if you had it you could not do much with it.
Play. Is that not enough? I'd love to experiment with hard drive firmware, toy with different caching algorithms, learn about the algorithms that convert between the analog signals at the heads and the data sent to the host, etc.
Asking for 'the blob' is akin to asking for the firmware source to a hard disk. Even if you had it you could not do much with it.
Play. Is that not enough? I'd love to experiment with hard drive firmware, toy with different caching algorithms, learn about the algorithms that convert between the analog signals at the heads and the data sent to the host, etc.
You could do a lot with hackable HD firmware. For example recover data from bricked disks or find bugs or make sure that disk you're about to RMA is infact fully wiped, etc etc. That would be especially useful now that SSD's have "turn into brick" failure modes and frequent firmware bugs that trigger them. Other ideas: adding crypto or RAID-friendly knobs, making the filesystem and the on-SSD flash
fs aware of each other and parametrize accordingly, etc etc.
In fact sounds like Apple's latest Fusion Drive is a thing that requires disk firmware and filesystem to cooperate.
The Alphamosaic processor (branded VideoCore in its Broadcom incarnation) has also been marketed as a network and security accelerator in its previous life... It might well be useful for all kinds of stuff.
The Alphamosaic processor (branded VideoCore in its Broadcom incarnation) has also been marketed as a network and security accelerator in its previous life... It might well be useful for all kinds of stuff.
Not really, IMHO.
What are you going to do with it? Who knows, we don't know what the capabilities are. I also have an instinctive dislike of the odd GPU-driven boot process where something open and hackable like u-boot could have been used.
What are you going to do with it? Who knows, we don't know what the capabilities are. I also have an instinctive dislike of the odd GPU-driven boot process where something open and hackable like u-boot could have been used.
[deleted]
What does the "userland" subset encompass? What's left in closed source blobs?
Nothing AFAIK. The kernel part has already been open source in the Raspberry Pi kernel fork: https://github.com/raspberrypi/linux/tree/rpi-3.2.27/drivers...
The userland part is the implementation of hardware accelerated OpenGL ES 2 and OpenMAX (video acceleration), at least.
The userland part is the implementation of hardware accelerated OpenGL ES 2 and OpenMAX (video acceleration), at least.
AFAIK everything that runs on the ARM is now open source. The firmware of the GPU itself could still be a blob.
That's very cool, I hope NVidia follows suit wishful thinking.
Thanks to Nouveau reverse engineering, you can actually run a number of recentish OpenGL GPUs with a completely open source stack including even the GPU firmware itself.
Now, I guess NVidia could put a high-powered processor on their GPUs and move their entire OpenGL driver stack onto it as "firmware" so that they could claim their drivers were open source, but that doesn't seem like something they'd do.
Now, I guess NVidia could put a high-powered processor on their GPUs and move their entire OpenGL driver stack onto it as "firmware" so that they could claim their drivers were open source, but that doesn't seem like something they'd do.
I certainly know about Nouveau. I even contributed to it, albeit indirectly (I'm the guy that wrote decuda). But reverse engineering GPUs is a thankless process, which has to be repeated for every new release of the hardware (in a certain sense, even for every stepping). Of course the tools are better these days so a part of reverse engineering can be done automatically, but still...
And sure, having the userspace and kernel driver open source is only a start. But having everything that runs on the CPU open source is great from a separation-of-concerns perspective.
Open sourcing the firmware and even the hardware itself would be cool, but is a battle for another day... And that'd no more be a matter of the just the GPU, but of all hardware that has firmware, including the CPU, network equipment, 3G radios...
And sure, having the userspace and kernel driver open source is only a start. But having everything that runs on the CPU open source is great from a separation-of-concerns perspective.
Open sourcing the firmware and even the hardware itself would be cool, but is a battle for another day... And that'd no more be a matter of the just the GPU, but of all hardware that has firmware, including the CPU, network equipment, 3G radios...
The raspberrypi looks better every day. The last major item on the to-do list appears to be the supply side.
Even the supply is getting better. They have been increasing the production rates and the delivery timings are getting shorter.
Just wanted to mention that the supply from Element 14 in Australia is outstanding. I put in an order for 2x model B's two weeks ago and received them both (512mb model) last thursday ... approx. an 8 day wait.
I actually got my Pi from Element 14 Singapore in just two days. I’m surprised when I got the package. Maybe it's because the demand in my area (Hong Kong) is low.
This mirrors my experience for Element 14 in the UK; ordered on the 11th and received on the 18th.
The supply has gotten really bad again since the switchover to 512MB base RAM has been made public knowledge. You ordered yours just in time.
Apparently Element14 does not ship to Japan. Even if you select "others" in the first country selection list. I have no idea why...
Although this is good news am I correct in assuming the boot loader is still closed source? I was hoping that this would enable an official OpenBSD port but if the boot loader is still closed source this probably won't happen.
To all those complaining about computers becoming "too closed", remember this.
What a fantastic amount of work that this must've required.
What a fantastic amount of work that this must've required.
Wow, that's huge. Can't wait to get hacking on processing and openframeworks. Being able to do real accelerated computer vision stuff with the Pi will be slick.
Does it mean that we will soon see many cheap arm based computers built in China? (Similar to android effect)
You mean like all the Allwinner boards? Closest you're going to come to the price is ~50USD. I doubt the Chinese have the VideoCore GPU available to them (they use Mali) so, no.
My hope is that it would at least get ARM to open up more of the Mali stuff, but based on my interpretation (and experience) there is nothing extremely special between this announcement and every other ARM board. The meaty GPU bits that are the really fun and interesting bit are still locked away in a firmware blob.
My hope is that it would at least get ARM to open up more of the Mali stuff, but based on my interpretation (and experience) there is nothing extremely special between this announcement and every other ARM board. The meaty GPU bits that are the really fun and interesting bit are still locked away in a firmware blob.
[deleted]
No shader compiler => lame.
good source.
With such an open stack, could the Pi be used as part of a GPU password cracking cluster?
An exceedingly slow part. Ten Pi units together costing $350 would be absolutely dusted by a single $350 full-scale GPU.
A $350 video card would have upwards of 2000 shaders and would run at a much higher clock-speed. It's simply orders of magnitude faster at a price that's only one order of magnitude higher.
A $350 video card would have upwards of 2000 shaders and would run at a much higher clock-speed. It's simply orders of magnitude faster at a price that's only one order of magnitude higher.
Not any more easily than before. Previously you had the full power of the OpenGL ES API to program the GPU with, but if you wanted to do anything that didn't fit into the API you were SOL. Now you're in exactly the same situation, because the "high-level interface" to the GPU firmware is just OpenGL ES with the context set-up and buffer management replaced by something nastier.
With such an open stack, could the Pi be used as part of a GPU password cracking cluster?
From what hermanhermitage said above, I'd guess no. You've now got full access to the OpenGL API, but you don't have full access to the hardware running the binary blob which is what is running the OpenGL code.
From what hermanhermitage said above, I'd guess no. You've now got full access to the OpenGL API, but you don't have full access to the hardware running the binary blob which is what is running the OpenGL code.
I write software for ARM system on chips for living so I was naturally interested in what would a competitor's software stack look like. And I was very skeptical that they would have actually revealed anything of importance. And, unfortunately, I was right. There's nothing important here (at least in the 3d part).
The OpenGL implementation is still somewhere in a binary blob that is not accessible.
The graphics drivers would reveal crucial information about the hardware design and give an advantage to the competitors who can use that information and the software itself to make chips cheaper.
I'm not talking about established competitors in the high end market but the anonymous small chip companies that do chips for the $100 Android tablets you can get from Asia.
Note: I would really like that everyone in the SoC business would open source everything they had and and the competition would be about who designs and manufactures the badassest silicon chips. But that's just not the way it works and I can't do anything about it.