Recently, some sharp criticism has been displayed against the OS reviews posted on OSNews by third parties. In an effort to make the reviews more substantial, I have put together some guidelines in addition to the generic guidelines OSNews already provides for the readers who want to have their voices heard on this site.I do not want the guidelines to be taken as rules but as a guide to a well structured review. No one
will be forced to use these guide lines. The more material the author of the review provides the more informed the reader will be in trying out the reviewed product. Suggestions are encouraged to make this guide
lines document better.
Operating System or Ditribution Review Guide Lines
Document version: 0.01
Experience of Reviewer
Describe the level of experience you have had with computers.
a) Novice. Very little experience with computers. Possibly just
bought a computer in the last six months. Never installed an operating
system before.
b) Intermediate. Use computers at work. Have owned a computer for
more than six months but less than two years. Installed programs but not
an operating system.
c) Expert. Have used several different operating systems. Have owned
a computer for more than two years. Possibly own more than one
computer. Possibly a software developer or system administrator.
The Computer Hardware
Description of the computer the operating system or distribution is on.
a) Maker and model number or
b) Specific items such as CPU, amount of RAM, video card, sound card, keyboard and mouse {PS2 or USB}, and the size of hard drive.
Operating System or Distribution Install
Maker, name and version of the operating system or distribution. Was the operating system already installed or did you install it?
If you installed it,
a) Did you do a typical or custom install?
b) Partition the hard drive?
c) Was all hardware detected?
d) How much space did the software installed take up on the hard drive? Did the installer give you that number?
e) How long did the install take?
f) Were there any problems?
Software Installed
List some of the software that was installed either by default or
through the custom install. If a piece of software that you use regularly
was not installed by default or you could not install it from the CDs,
please make a note of those as well.
a) Internet software such as email, ftp, web browser, and instant
messaging
b) Office type software such as office suites, text editors, graphics,
and faxing.
c) Entertainment software such as music, video, and games.
d) Specialty software such as system utilities, network utilities,
software development, and web development .
Daily Use
Give an overview of how the operating system or distribution performed on a daily bases.
a) How responsive was the operating system and software? How powerful the OS is? What can it do and what can’t it?
b) Did you install extra software off the CD, from the Internet? How
easy was it?
c) Is there a software update feature?
d) Is there documentation? How well written is it?
e) How stable, fast, easy to use, bug-free the OS is? Make sure you write all the good points of the product, as well as its bad points, objectively.
Reviewer’s Opinion
Compare this operating system or distribution to any or all of the competing products you have used before.
a) Likes
b) Dislikes
c) Over All Rating or Opinion
Finally, spellcheck and proof read your article!
As always, OSNews would like to hear from you and publish your articles. More community activity to this site is always a good thing!
I only browsed through it, but I missed a “special features of the os” section. If you would review BeOS, one would mention the database-style filesystem, gentoo would have it’s portage, .. The thing that the Operating system sets appart from the rest of the heap.
Well, a person may have owned a computer for more than 2 years but can’t be considered an expert.
It depends on a person’s motives to learn and discover new things in order to become an expert.
“”Experience of Reviewer
Describe the level of experience you have had with computers.
a) Novice. Very little experience with computers. Possibly just bought a computer in the last six months. Never installed an operating system before.
b) Intermediate. Use computers at work. Have owned a computer for more than six months but less than two years. Installed programs but not an operating system.
c) Expert. Have used several different operating systems. Have owned a computer for more than two years. Possibly own more than one computer. Possibly a software developer or system administrator.””
I think you need a few more levels between b and c. There are going to be many people that have been using computers for years maybe 10 but only installed windows. Or only ever ran a mac. They probably have done installs and such and worked way up through. But these people would not consider themselves expert I would think in most cases.
Another would be person who has been using for a long time and has installed many OS’s. But many of those would hardly consider themselves experts. I’ve been using computers a good long time now and messed with probably 1/2 dozen OS’s but never would consider myself expert.
Just a thought, I think your on the right track though. On the flip side though many will look at your “resume” as it is and see you don’t meet some “requirement” in there mind to have a clue what your doing and thus still find one’s review bad, or bias or something. If one could come up with a fair and perfect baseline for review OS’s in a way that everyone would agree with and never complain about a review that person might be able to create world peace quickly.
Just realised that my wife,who could write a great book on how to crash any system in less than one day, qualifies as an expert.
Used computers for 7 years, crashed more than 14 OS’s. We have around 6 or so computers in house. (various stages of togetherness)
And don’t tell me to try the BSD’s she crashed FreeBSD 4.7 in less than 2 hours.
It is something about women. I find bugs ALL the time on ALL the OSes I am sent to review! And I get back to these companies, we debug, and we indeed find the bug most of the times! But I heard that other women also find bugs that the whole QA team on some companies haven’t ever met!
I am not sure if this is a gift from God, or a curse for me. From the one side I find obscure bugs that I can reproduce most of the time easily, others are showstoppers and I help the companies to fix them afterwards, while on the other side, I get flamed by people when I am obligated to write about my experiences with that software (so I can be fair to our readership that need to know about possible problems of the reviewed product).
It has to be something about women and computers!
Hmm maybe I’ve found her a new career. Pro system crasher. I can have a system up for weeks using it hard. Research multiple desktops and too many programs open. No problems she walks over and turns on the calculator function with everything else off and crash.
There should certainly be another level between intermediate and expert. I’ve owned computers for more than 20 years and I’ve installed numerous OSes. But apart from RISC OS, Mac OS and Windows I’ve mainly played with them, rather than using them for anything practical. IME installing OSes tends to be the easiest part, it’s usually just a matter of following instructions. Unlike a lot of documentation, I find that installation instructions tend to be well written and heavily tested. Actually using the OS day to day, upgrading it, installing new software and hardware drivers, etc. is what requires real understanding of the system.
My limited programming ability is pretty much worthless and I’ve no experience in system administration. So it would seem a bit of an insult to real software developers and system administrators if I called myself an expert.
2 years is not long enough to be an expert. You won’t know enough about the behaviour of different OS’s and their daily use in only 2 years. It should be more like 5 or 10 years. How about saying the usage of different platforms as well? If you only ever used a PC or Mac then your view will not be as broad as someone who has used different platforms/architectures.
I would say you could have two more levels between Intermediate and Expert. I would say “Power User” and “Advanced Power User”.
A “Power User” is likely to have more experience tweaking and installing programs that require a level of technical understanding. People that become heavy gamers often grow into this category. You begin to understand network speeds, package conflicts as well as hardware limits, etc.
An “Advanced Power User” is that guy/gal that everyone else thinks is an expert except himself/herself, but he/she is nearly there. They would probably have a very strong understanding of computing in general and most certainly an intimate understanding of at least one OS. Perhaps even expertise with certain programs or various areas of computing.
That’s just how I see things. I feel I am somewhere between an Advanced Power User and an Expert. I have successfully done System/Network Adminstration on many different levels, as well as QA and some engineering. However, I have worked with some remarkable people. People that have forgotten more about computing than I currently know. they are ,by all means, experts and I feel that I have not earned the right to stand next to them (metaphorically speaking).
To be an expert in any field, especially one as vast as computing, is not something just anyone (no matter how bright) can do.
Just my $.02
To describe yourself as a novie, expert, or power user will be relative to what a person feels confortable calling him/herself. I do find it a nice idea to have the user’s level of experience in a review. It takes some effort for any one to try to review software through the eyes of a person on a different experience level. Some of the reviewers have been trying to look at operating systems as Joe/Jane User would see them. May be in version 0.02 of the document there will be more levels of experience with better descriptions.
Review guide-lines are a good thing. They’ll provide greater consistency. But maybe whats needed is stronger posting guide-lines as well.
Most of the complaints I read on this site are violently irrational and portray the poster as a dysfunctional little freak. Cripes, some repeat themselves verbatim every time. Kinda scary. They must have a macro programmed – hit one key and post to every thread on OSnews.
My first vote would be for unique User names. There must be at least 12 Anynonymous’s.
Little off topic but wanted to say *I* think you do a great job Euginia.
Rather than using “experience of reviewer”, why not use “Targetted Reader Level” …
Beginner/Newbie ;
Intermediate ;
Advance …
I totally agree we need to differentiate more the author skill level, and also possibly state the suggested audience experience level.
On another matter, I’m not convinced that the installation of an OS qualifies per se as a ‘review’ of it… I’d suggest that the minimum reviewing time *after* an installation should be in the week range, for qualifying the article as an ‘OS review’, instead of an ‘installer review’.
We all appreciate having a review as soon as possible, but reviewing the installation process alone is like going to a restaurant and eating the menu…
Corrado
I think “Experience of Reviewer” should be refined into two sections: “Experience of Reviewer with (desktop) computers” and “Experience of Reviewer with the OS being reviewed”. For example, one of the last Mandrake reviews was written by somebody who obviously only had experience in Windows. That doesn’t make him a “computer novice”, but it certainly makes him a “GNU/Linux novice”.
…9th word, 2nd sentence. Should be “substantial”, not “substancial”.
It is about a time to set some standards here as most of the reviews I have seen here were simply put crappy particularly linux ones.
I would be nice to see opinions about security too:
– open ports (why open, how easy was to close them)
– simple firewall setup
– how easy it is to control services and such
I am not sure about this whole user gradation think.
Here I have seen reviewers that identifed themselfs as “experienced” but based on their mistakes for example during install routine I would say that they were just beginners. Years of experience are important, but three years of administration of heterogenic network is worth more than 10 years of home networking (usually). And professional DTP user will have more experience than any wanabe on this field and so on.
If reviewer is really honest, then he/she should simply mention what he/she (or she/he?) is the exparience (OS and apps) how many years but without gradation (expert or not)
MP
I think it´s up to the editor (Eugenia) to select the articles that she wants to publish and to determine what constitutes a true review. Some of the articles published in OSNews under the heading of Review of Mandrake Linux 9.1 were just emails from readers describing their personal experience with Mandrake 9.1.
Now, whereas I agree that every review is a personal experience, I would say that not every single personal experience is a review. Quite obvious, isn´t it?
Guidelines are a good idea, but they can´t replace the common sense that an editor should exercise when selecting articles for publishing.
I can’t see the installation of an OS as saying anything about a user’s skill. My Amiga 1000 may have had it’s OSes (1.1, 1.2, 1.3 and 2.04) installed over a hundred times. I bought my Amiga 3000 with the OS installed and never had to reinstall it once. However, I downloaded and test tons of extention software for it. MacOS 5.xx to 8.xx installs with a dozen mouse clicks at most. BeOS installs so easy I can’t consider it’s installtion to even rank you as a beginner.
The fact the Linux is harder to install says nothing about the skills of people who install other OSes. The number of diffirent OSes a person used may say something about thier skills, the number of OSes they have installed says very little.
On the issue of how many categories of experience should be included, I think the easy answer is: none at all.
Rather than having a writer put the word “expert” or “novice” at the top of their review, it would be more useful to have a paragraph briefly describing their previous experience. For example,
“I have used Windows (insert version) for (x) years. The software I’ve used on it is (x,y,z). I’ve installed it approximately (x) times. I’ve also set up (small, large) networks, etc. I’ve had limited experience using linux on my home desktop; I’ve installed it once and used it only for email, web browsing, and word processing, and have not done any real system administration.”
A paragraph like this required at the beginning of any review will help readers place the experience level and general knowledge of the reviewer much better than a category which only tells us that the reviewer thinks of him/herself as an expert, novice, etc.
Shouldn’t the editors know how to spell substantial? How can we expect to hold third parties to a “higher standard” when the editors have trouble with basic English?
I’m guessing that the reason for stating whether you are a novice is in response to some of the reactions from intolerant Linux users who shout down anyone who says they had difficulties with Linux. In my opinion, those reviews from “novice” users are perfectly valid because:
1. Many Linux desktop distributions promote themselves as easy-to-use and easy-to-install. Criticizing the reviewer for not RTFM is not always fair.
2. These “novice” users (novices with Linux) are exactly the type of person that Linux distrubutors want to attract. If these usesr have difficulties with installation and use, this is a far better reflection of the difficulties other, similiar users will face.
I’m not going to nit-pick about details, but rather thank you for the information.
I’d like to write for OSN at some point and I had wondered what was required. Now if I can just find a subject before it is written to death elsewhere. MDK9.1 for example.
I would also like to see less about installation and more about daily use. Any installation comments beyond three paragraphs means that the installer or hardware detection is broken, IMHO.
A long term test would catch many of the “why the heck did they do THAT?” problems. Unfortunately, by the time a long term test is done, the distro is old news and nobody seems to care.
As for the novice, expert, etc debate, new users confident enough to write an article will tend to overestimate themselves and real experts may underestimate themselves. A little guidance in this area was needed.
Mutiny
Eugenia the Bugsquasher: It is something about women. I find bugs ALL the time on ALL the OSes I am sent to review! And I get back to these companies, we debug, and we indeed find the bug most of the times! But I heard that other women also find bugs that the whole QA team on some companies haven’t ever met!
Hm, looks like a women’s advantage in natural selection
I can remember the now-legendary GNOME ladybughunter Telsa Gwynne’s ability to find the most hidden bugs known to date. I’m sure there are other ladybughunters out there.
It is something about women. I find bugs ALL the time on ALL the OSes I am sent to review! And I get back to these companies, we debug, and we indeed find the bug most of the times! But I heard that other women also find bugs that the whole QA team on some companies haven’t ever met!
I am just like this. I find bugs all the time, in all software, operating systems, etc. The kind that are not recognized by developers and the kind that even most users never seem to notice.
I also recommend the following additional item to your excellent list: “Writing Experience of Reviewer: beginner, intermediate, experienced, professional”
I especially love your addition of “Proofread and spell check!” Cheers!!