Linked by jockm on Tue 19th Aug 2014 21:31 UTC
General Unix

Modern microcontrollers are becoming quite beefy. The Microchip PIC32 line is actually an implementation of the MIPS32 4K architecture - and with 512K of flash and 128K of RAM you can even run Unix! RetroBSD is a port of BSD 2.11 for the PIC32. You might not be able to run X11, but it is still very useful and a great reminder of how small Unix used to be - and how far it has come.

Order by: Score:
Next up...
by TemporalBeing on Tue 19th Aug 2014 21:51 UTC
TemporalBeing
Member since:
2007-08-22

Someone making a 4k unix cluster with these things...

Reply Score: 5

kernel
by Dano on Tue 19th Aug 2014 22:46 UTC
Dano
Member since:
2006-01-22

While theoretically you could run a UNIX on this part, it would not really make sense. It would be better to run a standard RTOS and save memory for your application code. I'm not a fan on these parts over the ARM microcontrollers due to lack of on board pull resistors.

Reply Score: 0

RE: kernel
by Vanders on Tue 19th Aug 2014 23:49 UTC in reply to "kernel"
Vanders Member since:
2005-07-06

I think this story is more in the spirit of "People who want a full MIPS32 core with some decent memory to run a retro UNIX on" than "People who want a UNIX to run on a micro controller", if you get my meaning.

Reply Score: 3

v RE[2]: kernel
by Dano on Wed 20th Aug 2014 00:00 UTC in reply to "RE: kernel"
RE[3]: kernel
by r_a_trip on Wed 20th Aug 2014 09:12 UTC in reply to "RE[2]: kernel"
r_a_trip Member since:
2005-07-06

Hey spoilsport, whatever happened to "Because we can!"?

Reply Score: 10

RE[4]: kernel
by Dano on Wed 20th Aug 2014 14:58 UTC in reply to "RE[3]: kernel"
Dano Member since:
2006-01-22

Wish that I could mod up your comment because it's just plain funny.

Reply Score: 2

RE[3]: kernel
by Vanders on Wed 20th Aug 2014 10:23 UTC in reply to "RE[2]: kernel"
Vanders Member since:
2005-07-06

Articles like this give people the dumb idea that clusters can or should be built with microcontrollers.

Why the hell shouldn't someone do it, if that's what they want to do? Hell, I'm tempted to do it myself just to annoy you.

No doubt everything you do has a purpose and a full economic impact statement prepared in advance.

Reply Score: 5

RE[4]: kernel
by Dano on Wed 20th Aug 2014 12:22 UTC in reply to "RE[3]: kernel"
Dano Member since:
2006-01-22

The PIC32 is a microcontroller not a microprocessor. It's a microprocessor with Microchip peripherals wrapped around it to do microcontroller type applications. I am not even sure how a general computing "cluster" could even be implemented. Would love to know how this could be pulled off, still laughing at the concept.

Yes, I am corrected on the MX, the uC does have pullup resistors on it after re-reviewing the datasheet.

I like the ARMs better because there is a lot more support for the family development tool-wise. The ARMs have a virtual monopoly on 32 bit microcontrollers. The STM32 being my favorite family. Not really sure what advantage the PIC32 would have over something like the STM32 or ATSam.

Also most RTOS like Micrium, CoOS or FreeRTOS all can run under 10K of RAM, not 128K. 128K is a lot of consumption on a uC and is not realy suited for real world applications.

Edited 2014-08-20 12:27 UTC

Reply Score: 1

RE[5]: kernel
by Alfman on Wed 20th Aug 2014 14:21 UTC in reply to "RE[4]: kernel"
Alfman Member since:
2011-01-28

Dano,

The PIC32 is a microcontroller not a microprocessor. It's a microprocessor with Microchip peripherals wrapped around it to do microcontroller type applications. I am not even sure how a general computing "cluster" could even be implemented. Would love to know how this could be pulled off, still laughing at the concept.


It would clearly be possible to attach many many micro-controllers to one or more interlink buses with peripherals, storage, etc. However I think what you might be missing is that it would be for boasting rights more than anything else. If someone built a cluster of these things, I think it would be osnews-worthy. Not because it's particularly powerful mind you (a 4k cluster of pics still has a fraction of the ram and storage in my laptop, LOL), but it would be impressive just because someone managed to do it.

Don't take it too seriously, look at it more as a project to have fun with. Additionally, don't underestimate what people can do with primitive technology.

A good example is NASA's original tech for the apallo missions. Their tech was extremely primitive, your run of the mill office worker today has much more computing power the the entirety of NASA back then, but it remains so fascinating because of what they managed to do with it:
http://arstechnica.com/science/2012/10/going-boldly-what-it-was-lik...

Another example is this 8088 demo was posted on osnews a while back:
http://trixter.oldskool.org/2014/06/19/8088-domination-post-mortem-...


Now there may not be much commercial merit in going back and doing things using primitive tools, but it doesn't have to be commercially viable in order for us to recognize the technical skills of whoever took up the challenge.

Reply Score: 4

RE[6]: kernel
by Dano on Wed 20th Aug 2014 14:46 UTC in reply to "RE[5]: kernel"
Dano Member since:
2006-01-22

It's just for geeks and Unix zealots. Total waste of time and life. It could be done with a lot of coding to run the bus, but you are right, would not make any sense capability-wise. Even then, you might not be able to run actual Unix applications on such an animal. I just don't understand why people jump to the conclusion that a microcontroller is the same as a processor. I can see the device driver routines with a bunch of TRIS statements in them already...I am well aware of what can be done with simple technology as I program microcontrollers daily. Funny but CoOS actually takes up less than 1K and is a base RTOS!

Revisiting the datasheets...the PIC32 is actually a fine product. Since you can compile most of your C code to other microcontrollers pretty easily, the difference between using ARM vs. MIPS is pretty small.

Edited 2014-08-20 15:00 UTC

Reply Score: 1

RE[7]: kernel
by Alfman on Wed 20th Aug 2014 15:49 UTC in reply to "RE[6]: kernel"
Alfman Member since:
2011-01-28

Dano,

It's just for geeks and Unix zealots.


I do a lot of things that only geeks enjoy, but so what? This isn't really about unix zealotry. It could be any OS and still be interesting, IMHO.

Total waste of time and life.


Sure, some people enjoy legacy computers, some people photograph pots, some people fly model airplanes, some people collect dolls, some people watch TV... arguably it's all a "total waste of time and life". But if it's what people enjoy doing, then who are you to judge? Seriously, if old stuff is so uninteresting to you then why even bother posting to an article titled "retrobsd"? Find something that actually does interest you and spend your time enjoying it! Fair enough?

Reply Score: 5

RE[7]: kernel
by Drumhellar on Wed 20th Aug 2014 20:59 UTC in reply to "RE[6]: kernel"
Drumhellar Member since:
2005-07-12

"STOP LIKING WHAT I DON'T LIKE!"

-Dano

Reply Score: 7

RE[7]: kernel
by richarson on Sat 23rd Aug 2014 19:37 UTC in reply to "RE[6]: kernel"
richarson Member since:
2014-05-24

It's just for geeks and Unix zealots.


Yeah, and that's exactly what the majority of us read OSNews for. You know Operating Systems News.

Total waste of time and life.


Um, then stop wasting your precious time on this stupid thread?

OK, that was harsh, but when did you get to decide what other people can do with their time and lives?

They want to do it, thay have fun doing it, why can't they do it?

Reply Score: 1

RE[5]: kernel
by jockm on Thu 21st Aug 2014 03:51 UTC in reply to "RE[4]: kernel"
jockm Member since:
2012-12-22

The PIC32 is a microcontroller not a microprocessor.


It is both. All microcontrollers are microprocessors, but not all microprocessors are microcontrollers.

A microcontroller is a microprocessor that includes memory (ie RAM) and I/O on a single die. Most microcontrollers also include Flash or [E][E][P]ROM to hold the program, but not all do. The XMOS chips load their program from an SPI Flash chip into memory at power up.

Reply Score: 3

RE[5]: kernel
by jockm on Thu 21st Aug 2014 04:45 UTC in reply to "RE[4]: kernel"
jockm Member since:
2012-12-22

The ARMs have a virtual monopoly on 32 bit microcontrollers.


I am not sure I would go that far. ARM is very popular to be sure, but there are a whole lot of MIPS cores out there. It is probably the most common more used in routers, and those chips would qualify as microcontrollers. PowerPC microcontrollers remain very popular in telecommunications, etc.

ARM has a very large marketshare, but it isn't quite to virtual monopoly yet.

Also most RTOS like Micrium, CoOS or FreeRTOS all can run under 10K of RAM, not 128K. 128K is a lot of consumption on a uC and is not realy suited for real world applications.


First off lets not focus only on RTOS's as that is only one segment of the overall Embedded OS world.

Yes there are plenty of tiny barebones embedded OSs out there, but that doesn't mean they rule the roost or solve all problems.

If I am working on a resource limited chip and need to be close to the metal, then I will use an OS like the ones you mentioned; but there is a reason why they are so small: they do almost nothing.

But if you need more advanced scheduling, installable device drivers, networking, POSIX support (so you can use advanced libraries like Pocket Sphinx, etc), memory management, hardware security, etc; then you need a more capable OS, and that means larger footprints and beefier hardware.

That is why we have Prex, VxWorks, uCLinux, Unison, etc etc etc.

It isn't a one size fits all solution. I can personally see using RetroBSD on the right project. I have learned never to say never...

Reply Score: 3

RE[6]: kernel
by Dano on Thu 21st Aug 2014 12:49 UTC in reply to "RE[5]: kernel"
Dano Member since:
2006-01-22

Are you going to compile and debug all of these advanced libraries under MPLAB? It would be challenging to do all that from PIC32-gcc via command line. Running POSIX applications on a PIC32 not compiled into the application code is something that I would love to see in person on how it would be done. Also seeing all this fit into RAM our Flash would be interesting and difficult. You would need a file system storage and driver.

Also if you look at the major 32 bit volume vendors, they are mostly ARM based... And if they are not then customers are starting to reject their proprietary platforms... This is what is happening to Renesas. The only reason Renesas introduced theIr ARM series is because they are having trouble selling the RX. Besides Microchip, the volume vendors are all ARM... This includes ST, NXP, Atmel (they pretty much have up on the AVR32, which is an excellent product), Cypress, Luminary, TI, Freescale and several others. In the future we are going to see many other older proprietary platforms drop off. PIC32 will probably one of the only popular non ARM micro in the future because the large 8 bit base of microchip developers are sticking with a brand name they are familiar with, but some of them are also seeing benefits in other lines and are questioning Microchips choice of MIPS over ARM considering differences in the two architectures... Context switching being one of them.

Edited 2014-08-21 12:56 UTC

Reply Score: 1

RE[7]: kernel
by jockm on Thu 21st Aug 2014 15:06 UTC in reply to "RE[6]: kernel"
jockm Member since:
2012-12-22

Are you going to compile and debug all of these advanced libraries under MPLAB? It would be challenging to do all that from PIC32-gcc via command line.


#1 why not? and #2 not everyone likes MPLAB. I think it is fine, but I know more than a couple people who just use makefiles and the command line. It isn't that hard.

Running POSIX applications on a PIC32...


You just made up that requirement. Remember I was talking about cases where you would need a lot of resources, and a much more capable OS than COoS, or FreeRTOS.

When I did the very thing I described, I was using a MIPS core, and uCLinux — where I had an embedded filesystem, and could run multiple applications... not just one that is compiled into the kernel.

Though I should point out that embedded OSs like Unison and DSPnano allow you to have a full filesystem that is then compiled into a single executable image, which can include multiple executables.

not compiled into the application code is something that I would love to see in person on how it would be done. Also seeing all this fit into RAM our Flash would be interesting and difficult. You would need a file system storage and driver.

The world of embedded OS's and embedded apps is vast and wide. It goes from CPUs may have only a handful of bytes of ram, all the way up to 64bit cores and gigs of RAM. It isn't just one thing, and there is cause for lots of different approaches to the OSs, including something like RetroBSD...

Also if you look at the major 32 bit volume vendors, they are mostly ARM based..


So you are saying that because the majority of "32 bit volume vendors) have ARM in their catalogs, ARM is dominant? I would suggest that you take a look at unit sales.

By that same logic, one would think that AVR and PIC had a virtual duopoly over the market, except that when you look at sales, the venerable 8051 sells more units.

The number of 8051's shipped inside of SD cards is shocking in and of itself.

And if they are not then customers are starting to reject their proprietary platforms... [elided] Freescale...


Have you looked at Freescale's catalog? It is filled with 68XX variants, 680X0 variants (ie ColdFire, and they are embedded in Freescale's accelerometers), PowerPC, etc.

Atmel makes AVR and ARM sure, but they also make 8051s and SPARC cores because the market in those things is too big to be ignored.

ARM is widely popular, and I use it a lot, but that doesn't mean it eclipse's everything else. You should go off and look up actual sales numbers.


and several others. In the future we are going to see many other older proprietary platforms drop off. PIC32 will probably one of the only popular non ARM micro in the future because the large 8 bit base of microchip developers are sticking with a brand name they are familiar with, but some of them are also seeing benefits in other lines and are questioning Microchips choice of MIPS over ARM considering differences in the two architectures... Context switching being one of them. [/q]

Reply Score: 3

RE[3]: kernel
by charlieg on Wed 20th Aug 2014 19:31 UTC in reply to "RE[2]: kernel"
charlieg Member since:
2005-07-25

I wonder what you think of people making data storage in Minecraft?

http://imgur.com/a/NJBuH

Reply Score: 3

RE[4]: kernel
by WereCatf on Wed 20th Aug 2014 19:47 UTC in reply to "RE[3]: kernel"
WereCatf Member since:
2006-02-15

Inherently useless, but at the same time delightfully creative?

Reply Score: 3

RE[3]: kernel
by JAlexoid on Wed 20th Aug 2014 21:59 UTC in reply to "RE[2]: kernel"
JAlexoid Member since:
2009-05-19

PIC32 is far from a microprocessor.


PIC32 is as much a microprocessor as Apple A7.

Reply Score: 4

RE[4]: kernel
by Dano on Wed 20th Aug 2014 22:18 UTC in reply to "RE[3]: kernel"
Dano Member since:
2006-01-22

A7 is a SOIC...even more advanced peripherals like a high end video card, wrapped into one part with an ARM core. PIC32 is a MIPS core wrapped with simpler Microchip peripherals and onboard RAM & Flash. MIPS processors are just the core itself in a part, put on a bus used in a PC with external RAM. Totally different target applications for each.

Great MCU's like the PIC16 series?

Edited 2014-08-20 22:34 UTC

Reply Score: 1

RE[5]: kernel
by jockm on Fri 22nd Aug 2014 02:56 UTC in reply to "RE[4]: kernel"
jockm Member since:
2012-12-22

A7 is a SOIC


I think you mean SoC — System On a Chip. SOIC is a IC package type. While the A7 is definitely a SoC it is most assuredly not a SOIC ;)

Reply Score: 3

RE[6]: kernel
by Dano on Fri 22nd Aug 2014 20:55 UTC in reply to "RE[5]: kernel"
Dano Member since:
2006-01-22

"A7 is a SOIC


I think you mean SoC — System On a Chip. SOIC is a IC package type. While the A7 is definitely a SoC it is most assuredly not a SOIC ;)
"

Thanks for spell checking my Windows Phone keyboard!

Good luck making your SPI cluster. Hope that you run the SPI interface in fast mode at a whopping 400 kHz. Better over clock the interface and add buffers to accommodate fanout interfacing those 4000 PIC32s!

Reply Score: 2

RE[7]: kernel
by jockm on Fri 22nd Aug 2014 22:16 UTC in reply to "RE[6]: kernel"
jockm Member since:
2012-12-22

Good luck making your SPI cluster. Hope that you run the SPI interface in fast mode at a whopping 400 kHz. Better over clock the interface and add buffers to accommodate fanout interfacing those 4000 PIC32s!


Dano—

I genuinely don't want to sound mean, or hostile; but most of your comments sound like you looked at the summary, made some unfounded assumptions, and are sticking to them.

For example: RetroBSD isn't just a kernel it is a full distribution, the kernel needed 128K (128k is what you need for the kernel, userland, and enough space to run apps), the PIC32 doesn't have internal pull up/down resistor, and now the speed of SPI.

If you bothered to glance at the datasheet for the PIC32 you would see the maximum SPI speed is 25Mhz.

Indeed I have yet to find a microcontroller whose SPI speed wasn't in the MHz range. Even the AVRs work at 4Mhz.

I2C however typically works at around 400khz, but comes in MHz variants as well. The PIC32 supports 1 Mhz I2C. It also supports 20Mbs serial, and 10/100 ethernet.

Considering that the Transputer's link speed topped out at 20Mbps, and I personally saw real time 640x480 rendering demos done at the time. I don't know how many you would need, but it is theoretically possible.

But back to rendering — which wasn't my suggestion, but makes an interesting thought experiment — 25Mhz SPI provides almost enough bandwidth to transmit a 640x480 8-bit image uncompressed in 1 second. Apply some mild compression to that and it should fit fairly easily.

To render in 24bit colors, you would need to diivide the problem across three PIC32s and you can handle all all three color channels.

We need 30fps so we would need at 30 rendering pipelines of 3 PIC32s each, for a total of 90. Of course I haven't factored in rendering time, just transmission time. So that is really one frame every two seconds. So add in another 90 cores group A can be transmitting while group B is rendering.

You might need some extra in there for managing the pipelines, etc so call it 200 tops.

Just like the transputer you would probably use a traditional computer on either side of the pipeline to send the data in, and collect the accumulated data and display it.

No one claims it would be easy, no one claims it would be useful, there are a lot of assumptions (for example the scene can be rendered in one second for example); but the point of all of this is that it is theoretically possible. Nothing more. No one is running out to spend $1600 for 200 PIC32s ($8 each at QTY 100).

It is like you are complaining that people are having fun the wrong way...

Reply Score: 2

RE[3]: kernel
by jockm on Thu 21st Aug 2014 20:34 UTC in reply to "RE[2]: kernel"
jockm Member since:
2012-12-22

You understand that this is the full distribution of BSD 2.11 right? Shell, games, awk, etc?

So you can do anything that people used BSD on a PDP-11 or early VAX could; which was a lot. It is smaller than uCLinux, smaller than Prex, but supports a very similar feature set. It is very mature and well understood.

You keep calling it big, but have you actually looked at its memory footprint? From what I can see it is roughly on par with VxWorks "new" microcontroller Kernel. And of course you don't have to use the full BSD distro if you were using it in an embedded project.

Reply Score: 3

RE: kernel
by smashIt on Wed 20th Aug 2014 00:30 UTC in reply to "kernel"
smashIt Member since:
2005-07-06

I'm not a fan on these parts over the ARM microcontrollers due to lack of on board pull resistors.


pic32mx do have pull-up and pull-down resistors

Reply Score: 5

RE[2]: kernel
by JAlexoid on Wed 20th Aug 2014 21:58 UTC in reply to "RE: kernel"
JAlexoid Member since:
2009-05-19

Exactly. That statement did not make any sense... Microship has been making great MCUs for a long time.

Reply Score: 3

RE: kernel
by jockm on Wed 20th Aug 2014 08:07 UTC in reply to "kernel"
jockm Member since:
2012-12-22

I'm not a fan on these parts over the ARM microcontrollers due to lack of on board pull resistors.


Would you care to elaborate on that? I mean it is a nice convenience, but there are plenty of advantages to various ARM microcontrollers that can outweigh this fairly minor feature.

I am not married to a particular CPU architecture, I tend to use the best option for the job.

Reply Score: 3

RE[2]: kernel
by zima on Sun 24th Aug 2014 16:39 UTC in reply to "RE: kernel"
zima Member since:
2005-07-06

What is that feature anyway? :>

Reply Score: 2

RE[3]: kernel
by jockm on Sun 24th Aug 2014 16:56 UTC in reply to "RE[2]: kernel"
jockm Member since:
2012-12-22

Look at my quoted text. Dano presumed (without checking) that the PIC32 didn't support internal pull up or pull down resistors. It does BTW but that isn't the point.

While I know some engineers who refuse to use internal pull resistors; Dano is the first person I encountered that would refuse to use one that didn't have them.

Reply Score: 2

RE[4]: kernel
by zima on Sun 24th Aug 2014 20:11 UTC in reply to "RE[3]: kernel"
zima Member since:
2005-07-06

Oh I realised the context; I just wonder what actually are pull up or pull down resistors... :p (and prefer to ask than to google; I might still have to do the latter, this thread is probably going to close soon :p )

Edited 2014-08-24 20:11 UTC

Reply Score: 2

RE[5]: kernel
by jockm on Sun 24th Aug 2014 21:32 UTC in reply to "RE[4]: kernel"
jockm Member since:
2012-12-22

A pull up resistor is one that is attached to a A GPIO pin and positive voltage. This was it is being "pulled up" and requires the circuit to pull them down. Pull down resistors are the inverse: the resistor it attached to ground, and requires the circuit to provide positive current.

They are used for various things but buttons are a common case. When the button isn't closed (IE pressed) the GPIO pin is "floating" and may return 0 or 1 until it is pressed. To resolved this you use pull up or pull down resistors.

Internal pull resistors are built into the chip itself and can be enabled via software. It is a a nice to have, but far from essential. Some engineers don't like to trust the internal resistors for things like buttons because they can be fairly week, and so sometimes you can get false positives or false negatives depending.

Reply Score: 2

RE[5]: kernel
by Alfman on Sun 24th Aug 2014 21:50 UTC in reply to "RE[4]: kernel"
Alfman Member since:
2011-01-28

zima,

I'm not an EE, but basically these resisters are there to pull up or down the voltage by default (when no transistors/latches are active) - to prevent the pin from being in a floating state that can produce erratic behavior with other components.

Consider without the resister, the pin's on/off states might be 5v/unknown. But by adding a pulldown resister (internally or externally) we can make the pin's on/off states be 5v/0v. The resistance needs to be chosen such that it's strong enough to pull the voltage down to 0v, but weak enough to allow the transistor to pull it back up to 5v.

Sometimes you want a floating state to implement tri-state logic (on/off/unset), and in such cases a pull down resister is bad. Consider a 485/1wire/etc bus where you can hook up dozens of devices together on the same wires. If each device had a resister pulling down the voltage to 0 on the same line, the sum of all those would exceed the transistor's power to pull it back up to high voltage. So in these cases two transistors are used to explicitly set on/off states on exactly one device at a time.

Edit: jockm beat me to it

Edited 2014-08-24 22:07 UTC

Reply Score: 2

RE[6]: kernel
by jockm on Sun 24th Aug 2014 21:57 UTC in reply to "RE[5]: kernel"
jockm Member since:
2012-12-22

To be fair, I think your explanation is better ;)

Reply Score: 2

RE: kernel
by zima on Sun 24th Aug 2014 16:37 UTC in reply to "kernel"
zima Member since:
2005-07-06

"Theoretically"? Look at the news, it runs.

Reply Score: 2

Moar RAM
by agentj on Wed 20th Aug 2014 09:55 UTC
agentj
Member since:
2005-08-19

RAM is not really big issue with pic32. You can connect external SRAM with external memory interface with pic32mz for example.

Reply Score: 5

Great and ....
by ameasures on Wed 20th Aug 2014 09:58 UTC
ameasures
Member since:
2006-01-09

It is a fun accomplishment and having looked at the product offerings, several go beyond the 128kB mentioned.
Still wondering what might usefully be done with these? Or how their suitability would stack up against rivals in those spaces?

Reply Score: 3

RE: Great and ....
by jockm on Wed 20th Aug 2014 15:07 UTC in reply to "Great and ...."
jockm Member since:
2012-12-22

Rather well in terms of features with one area: threading. I might be wrong about that, but I don't think I am.

So you would have to use more heavyweight processes. But otherwise it would compare well on features. More resource hungry that Contiki, etc but far from useless.

If I had a use where I needed two or more processes that only needed coarse ground IPC; then I might well consider RetroBSD on PIC32.

Reply Score: 2

RE[2]: Great and ....
by Dano on Wed 20th Aug 2014 15:52 UTC in reply to "RE: Great and ...."
Dano Member since:
2006-01-22

You can do efficient threading, queuing and semaphores with any small RTOS on PIC32. Microchip's own "Harmony" library has a port of FreeRTOS in it, which takes up only a few K of memory.

Reply Score: 1

RE[3]: Great and ....
by WereCatf on Wed 20th Aug 2014 15:58 UTC in reply to "RE[2]: Great and ...."
WereCatf Member since:
2006-02-15

I do not think you just understand the idea of doing things simply for the fun and challenge of it.

Reply Score: 3

RE[2]: Great and ....
by Alfman on Wed 20th Aug 2014 16:08 UTC in reply to "RE: Great and ...."
Alfman Member since:
2011-01-28

jockm,

Rather well in terms of features with one area: threading. I might be wrong about that, but I don't think I am.


Perhaps a 4k pic cluster could be used for raytracing, it would be an impressive feat. Although I suspect the limited ram of the pics would pose problems even for relatively moderate scenes. Protein folding applications might be feasible. If nothing else, it could do bitcoin mining.

Of course anyone tackling these problems seriously would opt for high powered GPGPUs or custom designed FPGA/ASIC. Using a pic cluster is just a neat challenge.

Reply Score: 2

RE[3]: Great and ....
by Dano on Wed 20th Aug 2014 16:16 UTC in reply to "RE[2]: Great and ...."
Dano Member since:
2006-01-22

Ray tracing? What would the output be presented on? You guys are on crack and I am sure that you don't understand the depth of the problem.

Reply Score: 1

RE[4]: Great and ....
by WereCatf on Wed 20th Aug 2014 16:26 UTC in reply to "RE[3]: Great and ...."
WereCatf Member since:
2006-02-15

Ray tracing? What would the output be presented on?


Well, the output could just be relayed to a device with an actual display with the microcontrollers just handling the actual calculations. I do not see what the problem is.

Reply Score: 2

RE[5]: Great and ....
by Dano on Wed 20th Aug 2014 16:46 UTC in reply to "RE[4]: Great and ...."
Dano Member since:
2006-01-22

The fact that you don't see the problem is the problem. You have to have a kernel running on every part in the cluster, have drivers and hardware to tie them all together, have enough onboard memory to run user code, have an interface to launch user code along with a hundred other issues including clocking and bus arbitration issues that would slow a cluster to a halt even if you had enough ram and glue to get it all working. Has anyone here actually programmed a microcontroller before? Even after all of this you would need custom app code and perhaps a custom compiler to write the app code to make a program that actually runs on the platform.

Edited 2014-08-20 16:49 UTC

Reply Score: 1

RE[6]: Great and ....
by WereCatf on Wed 20th Aug 2014 17:00 UTC in reply to "RE[5]: Great and ...."
WereCatf Member since:
2006-02-15

And? All that goes with the challenge of it. As others have tried to tell you, it's not about it being efficient or a useful system or anything, but you just don't get it.

Reply Score: 3

RE[6]: Great and ....
by TemporalBeing on Wed 20th Aug 2014 17:01 UTC in reply to "RE[5]: Great and ...."
TemporalBeing Member since:
2007-08-22

The fact that you don't see the problem is the problem. You have to have a kernel running on every part in the cluster, have drivers and hardware to tie them all together, have enough onboard memory to run user code, have an interface to launch user code along with a hundred other issues including clocking and bus arbitration issues that would slow a cluster to a halt even if you had enough ram and glue to get it all working. Has anyone here actually programmed a microcontroller before? Even after all of this you would need custom app code and perhaps a custom compiler to write the app code to make a program that actually runs on the platform.


Yes, I have done microcontroller programming - solving a real-world, real-time problem.

And no, this wouldn't be that bad if the cluster master (a normal system) and the cluster were designed right - in fact, it'd just be a really really large version of the Cell Processor used by Sony for the PS3.

Yes, the Cell was a PITA to program for, but it did work. Now scale that out to 4k nodes with about the same amount of RAM. A PITA but doable.

Reply Score: 2

RE[7]: Great and ....
by Dano on Wed 20th Aug 2014 17:25 UTC in reply to "RE[6]: Great and ...."
Dano Member since:
2006-01-22

The cell is a multi core processor with an internal bus to make all of the cores interconnect. You don't have anything like this available to interconnect thousands of PIC32s. The software that would run on a theoretical PIC cluster would not be a standard UNIX application. You won't have enough IO and clocks to actually make it run efficiently. As a matter of fact you would have a limit as to how many pics you could actually interconnect if you even wanted to die it just because of IO.

Reply Score: 1

RE[4]: Great and ....
by Alfman on Wed 20th Aug 2014 17:33 UTC in reply to "RE[3]: Great and ...."
Alfman Member since:
2011-01-28

Dano,

Ray tracing? What would the output be presented on? You guys are on crack and I am sure that you don't understand the depth of the problem.


With all due respect, you can dismiss the practical utility all you want, but if you are trying to dismiss the technical feasibility then I think you lack vision and ingenuity.

Standard definition TV has roughly 704*240 pixels at 60FPS (interlaced), that's 10137600 pixels/second (assuming every single pixel gets computed every single frame). With a ~4k cluster, or 2534.4 pixels/processor/second. Depending on pic32 model, we might get 100-330MIPS, lets choose conservatively: 100MIPS. That's 39,457 instructions per pixel. Let's assume 75% of that is lost to various overhead, so about 10K instructions are conservatively available for raytracing a single ray. It certainly seems this could produce some stunning realtime ray tracing.

Granted I haven't done it, and there may be pitfalls (I suspect the lack of RAM will limit scene complexity), but that gets solved as you come to it. Fractal/procedurally generated scenery can produce infinite detail without infinite ram, etc. The trick is to optimize, optimize, optimize... Creativity can overcome huge obstacles, but only if we allow ourselves to envision it first instead of being dismissive.

Edited 2014-08-20 17:36 UTC

Reply Score: 2

RE[5]: Great and ....
by Dano on Wed 20th Aug 2014 17:53 UTC in reply to "RE[4]: Great and ...."
Dano Member since:
2006-01-22

Your math dismisses bus arbitration and software overhead dramatically, even if you could even make the connection on a bus. And you know that it's not really even feasible with this family.

Edited 2014-08-20 18:12 UTC

Reply Score: 1

RE[6]: Great and ....
by Alfman on Wed 20th Aug 2014 18:48 UTC in reply to "RE[5]: Great and ...."
Alfman Member since:
2011-01-28

Dano,

Your math dismisses bus arbitration and software overhead dramatically. I'd you could even make the connection on a bus. And you know that it's not really even feasible with this family.


Of course it's naive to stick 4k controllers on a massive bus, but this is part of the problem that you need to solve creatively. It seems you have strong preconceived ideas about the limitations of micro-controllers stemming from the designs you may have already worked with. But try to think creatively, other more scalable designs will pop out that are better suited to the problem at hand.

The internet doesn't croak with millions of hosts online because it's subdivided into a hierarchies. Our bus doesn't need to be flat, by applying appropriate network hierarchies we can increase parallelism while reducing I/O contention.

However if you think about raytracing for a second, you should realize that one raytracing controller has very little reason to communicate to any of it's peers, so we can use this to our advantage in designing a bus. So right there alot of your theoretical problem goes away. For the most part they could be write-only, take rendering instructions and then output the rendered results into a separate buffer into the DAC. For that matter, since controllers will be rendering the exact same scene at a tiny offset, we might even get away with just broadcast the same scene instructions to all controllers simultaneously and allow them to render the scene on their own with no further I/O at all.

This is exactly what I mean. When we take some time to think about the problem, we find that the problems we thought were intractable (ie bus I/O overhead), can be optimized away if we dare to embrace the challenge. That's all part of the fun. If we jump the gun and simply assume it's not feasible, then not only does this attitude hold back our ingenuity, it also prevents us from enjoying the thrill of solving the challenge.

Reply Score: 3

RE[6]: Great and ....
by Vanders on Wed 20th Aug 2014 21:28 UTC in reply to "RE[5]: Great and ...."
Vanders Member since:
2005-07-06

The problems of designing a bus that can connect 4k micro controllers together are the same as the problems that existing HPC interconnects have solved. Ever heard of the Transputer, out of interest?

Reply Score: 3

Wow
by mikeanderson on Sun 24th Aug 2014 12:21 UTC
mikeanderson
Member since:
2014-08-22

Interesting project! Well done.

Reply Score: 1