Linked by Thom Holwerda on Thu 7th Mar 2013 11:43 UTC
Debian and its clones "When you buy a Raspberry Pi, the $35 computer doesn't come with an operating system. Loading your operating system of choice onto an SD card and then booting the Pi turns out to be pretty easy. But where do Pi-compatible operating systems come from? With the Raspberry Pi having just turned one year old, we decided to find out how Raspbian - the officially recommended Pi operating system - came into being. The project required 60-hour work weeks, a home-built cluster of ARM computers, and the rebuilding of 19,000 Linux software packages. And it was all accomplished by two volunteers."
Order by: Score:
Nice to see !
by Kochise on Thu 7th Mar 2013 14:54 UTC
Kochise
Member since:
2006-03-03

Re-inventing the wheel everytime for the challenge is something that should be rewarded at least with a little fame ;) So much time spent (not said wasted) on redoing/adapting stuff is impressive.

Kochise

Reply Score: 3

RE: Nice to see !
by Alfman on Thu 7th Mar 2013 15:27 UTC in reply to "Nice to see !"
Alfman Member since:
2011-01-28

Kochise,

Is that sarcasm ;)


This thing happens quite a lot. You start recompiling custom kernels, then recompiling the latest userspace tools one by one, make your own init system and before you know it, you have your own distro.

Sometimes we ask why do it? There's no doubt that it is reinventing the wheel. Having been there, for me it's party to prove to oneself that one can do it and to gain a more in dept understanding of everything. Sometimes there's something specific that you'd like to change, such as the directory hierarchy. Is it worth it? Well obviously there need to be some people to do it, but after a point it becomes totally redundant. Most of us will never be commercially viable. On the other hand there is personal learning, gratification and bragging rights.

Reply Score: 2

RE[2]: Nice to see !
by Lennie on Thu 7th Mar 2013 17:53 UTC in reply to "RE: Nice to see !"
Lennie Member since:
2007-09-22

How is this reinventing the wheel ?

Debian is the Linux distro which supports the most hardware of all. But they didn't support the floating point unit of this type of ARM-processor line yet.

So they recompiled everything optimized for that floating point unit.

Edited 2013-03-07 17:56 UTC

Reply Score: 4

RE[3]: Nice to see !
by Alfman on Thu 7th Mar 2013 18:27 UTC in reply to "RE[2]: Nice to see !"
Alfman Member since:
2011-01-28

"How is this reinventing the wheel ?"

The article didn't go into details, but it really reads as though the bulk of the work was re-invention and not adding math support. If you are opposed to labeling it re-invention, then how about we call it extraneous side-work, which is only tangentially related to the main task of recompiling the packages with new math support.


If and when debian decides to support the RaspPi's math processor, will they really reproduce this guy's hardware rig and use his toolchain to produce new packages? Or will they modify their existing toolchain to support the math processor?


Edit:
I've placed my finder on it, this reminds me of a rube goldberg invention for software!

Edited 2013-03-07 18:42 UTC

Reply Score: 2

RE[4]: Nice to see !
by Lennie on Thu 7th Mar 2013 21:41 UTC in reply to "RE[3]: Nice to see !"
Lennie Member since:
2007-09-22

Looking at the list below and the article the Debian ARM-port they based it on has not yet reached official status yet:

http://www.debian.org/ports/index.en.html

So it can't yet be integrated in Debian, but it might still happen ?

Reply Score: 2

RE: Nice to see !
by jockm on Thu 7th Mar 2013 16:10 UTC in reply to "Nice to see !"
jockm Member since:
2012-12-22

Reinventing the wheel? It is Debian, so they hardly reinvented something, they added much needed support for ARMv6 hard floating point, which no other distro did.

And I am personally very glad they did — I am doing voice recognition, and other compute based tasks on the Pi, without hardfloating point I don't think I would be able to use the Pi for this project at all...

Edited 2013-03-07 16:14 UTC

Reply Score: 11

RE[2]: Nice to see !
by Alfman on Thu 7th Mar 2013 16:32 UTC in reply to "RE: Nice to see !"
Alfman Member since:
2011-01-28

jockm,

Reading the article, it's seems pretty clear to me he did a lot of reinventing well beyond the floating point unit. It looks like he had a lot of fun building his ARM cluster and converting packages the way he did, but he could have avoided a great deal of work by cross compiling the packages in a more conventional way.

Edited 2013-03-07 16:37 UTC

Reply Score: 3

RE[3]: Nice to see !
by Kochise on Thu 7th Mar 2013 17:43 UTC in reply to "RE[2]: Nice to see !"
Kochise Member since:
2006-03-03

That's the point I made, with a little pinch of sarcasm, yeah. It's strange that an open source distro never implemented ARMv6's FPU in the first place, while the specs are pretty much open.

But the fun fact is all the stuff surrounding this event of adding FPU support. Reminds me this :

http://www.mini-itx.com/projects/cluster/

Kochise

Reply Score: 3

Comment by ilovebeer
by ilovebeer on Thu 7th Mar 2013 17:58 UTC
ilovebeer
Member since:
2011-08-08

While I can appreciate the commitment, I fail to be impressed. Generally speaking there are a lot of talented people out there who rather than focus their efforts on things with potentially far greater benefit in general, take on pet projects that don't have much of a shelf life and really only have limited interest to begin with. What I'm saying, in simple terms, is that I think a lot of talent gets wasted.

Before anyone bothers to ramble on specifically about the "wonders" of the Raspberry Pi, keep in mind I'm simply stating my own personal opinion which is neither right nor wrong. I'm not trying to convince anyone of anything. Agree or disagree, my opinion is what it is and everyone is welcome to their own.

Reply Score: 1

RE: Comment by ilovebeer
by Laurence on Fri 8th Mar 2013 07:21 UTC in reply to "Comment by ilovebeer"
Laurence Member since:
2007-03-26

While I can appreciate the commitment, I fail to be impressed. Generally speaking there are a lot of talented people out there who rather than focus their efforts on things with potentially far greater benefit in general, take on pet projects that don't have much of a shelf life and really only have limited interest to begin with. What I'm saying, in simple terms, is that I think a lot of talent gets wasted.

Everyone is entitled to a hobby. Sometimes those hobbies teach us employable skills, sometimes those hobbies are purely for fun. Either way I wish more people had interesting (if arguably pointless) hobbies than those who just sit in front of the TV watching "reality" shows.

Reply Score: 4

Comment by Antartica_
by Antartica_ on Thu 7th Mar 2013 21:29 UTC
Antartica_
Member since:
2012-12-28

There are some points that IMHO are being misundertood.

WARNING! Wall of text follows...

First point, about wasting talented people time:

In the Ars Technica article it was explained that this project was started as a "learning Journey", that is, one individual wanted to understand better the process of porting a distribution to a new architecture.

If you want to master something, you have to do this type of projects. If you want to know it inside-out, you have to do this sort of thing. [Unrequited car analogy follows] Car aficionados used to dismantle a car and rebuild it part by part. This is the same but with software.

This people just did it in a way that created something useful. AFAIK, this skill is normally studied using the "Linux From Scratch" guides.

Second point, about reinventing the wheel:

This is just an optimized port of a distribution. Just like in the past the i386 Linux distributions had versions optimized for i586 and later to i686.

It is something valuable, not "reinventing the wheel for the sake of it".

And the code is mostly there, the difficult part is bootstraping: having the initial "core" packages that allows you to build the rest of the packages. They smartly piggybacked on the armhf port and compiled from there the versions for raspbian.

This is not the first time someone has done a port of debian to a platform that has suboptimal characteristics with current arm ports. For example someone did an "armeb" Debian port, that is, arm big endian. This was to have a port with compatibility with big endian ARM binaries, in contrast with "arm" and "armel" that were little endian.

Third point, about the lack of this specific port in Debian proper:

Debian already had various ARM ports: the old "arm", the new "armel" and the newer "armhf" (see http://wiki.debian.org/ArmPorts ). And if I'm not mistaken, both "arm" and "armel" official ports should work in the pi (albeit slower that raspbian).

I will explain this a little.

There are a lot of configurations of ARM in the wild. The initial Debian ARM port ("arm") was for armv4t with "old ABI".

This Debian used hardfloat with floating point accelerator opcodes which can be emulated in the kernel (so Debian worked in hardware like the ancient StrongARM).

Later there was the migration to EABI (the "new ABI", debian port named "armel"), which had a lot of advantages, such as being able to use in the same system binaries with hardware floating point support and binaries without it (softfloat), so you can have special versions of some packages optimized for a specific floating point unit. A specific one? Yes, there are hardware with several incompatible hardware floating point implementations (see http://wiki.debian.org/ArmEabiPort ).

Even later, a new port was started targeting modern arm cores (armv7, that is Cortex A8 and compatibles; port name "armhf").

Just for reference, Debian is brewing a new port for 64bit ARM (although that hardware is not available yet!).

So, it is not that Debian didn't had a port for ARM hardware, rather that there are so many variations of ARM that they had to choose the subset to target. As they wanted to be "universal", they chose the ports which cover most hardware, at the expense of some performance.

Thank you for reading up to here, and excuse my grammar (I'm not a native english speaker).

Edited 2013-03-07 21:46 UTC

Reply Score: 9

RE: Comment by Antartica_
by Alfman on Thu 7th Mar 2013 22:03 UTC in reply to "Comment by Antartica_"
Alfman Member since:
2011-01-28

Thank you for posting. Are you connected to the project or are you just reading the article too?

That ARM ABI switch affected me quite directly as I witnessed it on a Buffalo Linkstation NAS device.

http://buffalo.nas-central.org/wiki/Category:LSPro


For my embeded NAS, I was able to recompile software to the new ABI without any of the shenanigans the article alludes to. Why go about solving the problem by building a completely unnecessary cluster comprised of underpowered target processors except as a personal challenge?

I'm not trying to be critical of that, I said in my first post that many of us do it for fun and do learn this way. But it still looks like most of the work was overkill and not necessary to achieve the end result. It will probably get thrown away if debian officially adds support.

Reply Score: 3

RE[2]: Comment by Antartica_
by Antartica_ on Fri 8th Mar 2013 11:22 UTC in reply to "RE: Comment by Antartica_"
Antartica_ Member since:
2012-12-28

Yes, the information about them is only from the article and comentaries. I'm not connected to the raspbian project.

On the other hand, I've been a long time using Debian, both for hobby projects and for work. I've been using it continuously from version 1.3.1 onwards, and in embedded scenarios in the MIPS and several ARM variants.

About the compile farm: Debian is not setup for cross-compiling the operating system (in contrast to Angstrom, for example).

Not so long ago, you could cross-compile Debian using scratchbox (based on qemu emulating ARM, patched so that it calls the native cross-compiler when necessary), but as Nokia abandoned Maemo I suspect that scratchbox is not being updated anymore, so not usable with Debian Wheezy.

Reply Score: 3

Title misleads
by ronaldst on Thu 7th Mar 2013 21:32 UTC
ronaldst
Member since:
2005-06-29

I was expecting a new OS. Not just a retrofit from the Linux world.

Reply Score: 4

RE: Title misleads
by DeepThought on Fri 8th Mar 2013 23:29 UTC in reply to "Title misleads"
DeepThought Member since:
2010-07-17

Just my thought. Why call just another Linux a "new" OS?

Reply Score: 2

The power of two
by benali72 on Fri 8th Mar 2013 00:02 UTC
benali72
Member since:
2008-05-03

Engaged individuals who really care can accomplish a lot! I wonder if corporate America will ever learn this? Doubt it.

Reply Score: 1

RE: The power of two
by Kochise on Fri 8th Mar 2013 07:51 UTC in reply to "The power of two"
Kochise Member since:
2006-03-03

With a good salary, many people would care. But the salary level being what it is (pyramidal) and the productive people being at the bottom, they get an average salary. Plus the pressure of a corporate expecting to return 100x on their investment, that renders these kind of project only feasible on spare time, by volunteers.

Or join a corporate with a good R&D policy like IBM, Google or Microsoft, that invest some time and/or money giving their employees the ability to work on promising projects that some may succeed and many not.

Kochise

Reply Score: 3