Linked by Thom Holwerda on Mon 16th Apr 2012 02:08 UTC
In the News "Modern science relies upon researchers sharing their work so that their peers can check and verify success or failure. But most scientists still don't share one crucial piece of information - the source codes of the computer programs driving much of today's scientific progress." Pretty crazy this isn't the norm yet.
Thread beginning with comment 514316
To view parent comment, click here.
To read all comments associated with this story, please click here.
RE: Yes and no
by looncraz on Mon 16th Apr 2012 07:43 UTC in reply to "Yes and no"
looncraz
Member since:
2005-07-24

During peer review, the code would be checked to verify any unexpected results.

With open code, any meaningful problem would be found and solved, and old studies could be easily re-run and verified or discarded.

With closed code, the bugs are never found, and the authors have no reason to repair it if they get what they think are sound results.

--The loon

Reply Parent Score: 10

RE[2]: Yes and no
by kwan_e on Mon 16th Apr 2012 08:44 in reply to "RE: Yes and no"
kwan_e Member since:
2007-02-18

With closed code, the bugs are never found, and the authors have no reason to repair it if they get what they think are sound results.

--The loon


It doesn't matter because it's the published results that matter, and if the results are wrong, someone can verify it independently when its published. If you use the original source code, to verify, it's no longer independent.

In a research organization where there are hundreds of people pulling in open source code, you cannot guarantee someone did not pull code from the original base, leading to a compromised verification of the data.

Reply Parent Score: 1

RE[2]: Yes and no
by thomasg76 on Mon 16th Apr 2012 15:00 in reply to "RE: Yes and no"
thomasg76 Member since:
2012-04-16

Well, it can also work quite easily in the opposite. The source code is taken, with little or no review, and new data are run through it, confirming the original result.

I am sure that this happens. Not too long ago I run in to this issue while looking at studies done in the field of psychology. They run most there studies through SPSS to make a factor analysis, do get something out of the data. Everybody using the same software the same way of conducting the study, of course they confirm the result of others. Most of the conclusions drawn are just simply wrong, because less than half of the data is actually supporting the result.
Now since most psychologist aren't statisticians, they just take the work of others as template for their own. And you propagate a wrong method / software.

The same is going to happen with opening the source code for all research. If the code is critical to the research than it should be implemented independently to confirm the results, based on the same data. If the code is auxiliary to the problem, then who cares anyway.

Also I know of Professors that stopped publishing all together because of that requirement. Now what do you gain?
The good thing from all the published work is, we KNOW that certain things work/exist, so they can be re-discovered and independently verified.

Reply Parent Score: 1