Linked by Thom Holwerda on Wed 24th Jun 2015 23:10 UTC
Mac OS X

It is showing hidden files (that have names starting with a dot) when invoked by root and doesn’t show them (as expected) when running as a normal user. This differs from what ls on Linux (the one coming from coreutils) does.

Why does ls behave this way?

Very interesting answer. I love stuff like this.

Order by: Score:
Comment by Drumhellar
by Drumhellar on Thu 25th Jun 2015 01:25 UTC
Drumhellar
Member since:
2005-07-12

MacOS X has a mostly BSD userland, and anybody that uses any of the BSDs on a regular basis would recognize that behavior.

Reply Score: 5

RE: Comment by Drumhellar
by shotsman on Thu 25th Jun 2015 05:37 UTC in reply to "Comment by Drumhellar"
shotsman Member since:
2005-07-22

so it is really a non visible story then? ;)

the use of .{name} on Unix/Linux is well known.

The thing I find really annoying is the way APPDATA and PROGRAMDATA directoies are hidden away on windows. The USER.APPDATA in particular can grow to silly sizes if you copy stuff into and out of a VM{ware} setup. Making more people aware of this would IMHO be more worthy of an article.

Reply Score: 3

RE: Comment by Drumhellar
by dpJudas on Thu 25th Jun 2015 07:52 UTC in reply to "Comment by Drumhellar"
dpJudas Member since:
2009-12-10

MacOS X has a mostly BSD userland, and anybody that uses any of the BSDs on a regular basis would recognize that behavior.

Yes, but the answer goes much further than that. It tries to examine the history and reason for why the two OSes diverge on this issue.

There is something fascinating that some single developer 38 years ago felt that root should see everything and it stays that way even today (with probably no further justification than that he could commit this change).

Also, another comment further below hints that the dot-is-hidden notation was initially a bug that became a feature as people started abusing it. No idea if that is actually true, but I find it funny if it is. Especially with the many people bragging about how well designed Unix is. ;)

Reply Score: 2

RE[2]: Comment by Drumhellar
by sergio on Thu 25th Jun 2015 08:24 UTC in reply to "RE: Comment by Drumhellar"
sergio Member since:
2005-07-06

There is something fascinating that some single developer 38 years ago felt that root should see everything and it stays that way even today (with probably no further justification than that he could commit this change).


Another justification: It makes sense.

I'm not a BSD fanboy, in fact, I prefer GNU userland, but I think BSD design choices are always pretty smart and effective... maybe that's why they don't need to change things so often. They have a culture.

Reply Score: 3

RE[3]: Comment by Drumhellar
by moltonel on Fri 26th Jun 2015 09:34 UTC in reply to "RE[2]: Comment by Drumhellar"
moltonel Member since:
2006-02-24

"There is something fascinating that some single developer 38 years ago felt that root should see everything and it stays that way even today (with probably no further justification than that he could commit this change).

Another justification: It makes sense.
"

Call me set in my ways, but it doesn't make sense to me (and to the stackexchange OP):

* Hiding dotfiles is a usability feature, I want it regardeless of who I'm loggued in as.
* Other tools only change their behaviour depending on what the kernel allows them to do; there's no reason that ls should be an exception.
* Calling it a security feature is very naive, but if it was it should apply to all admins, not just uid==0.
* There's no option to reverse "-A", which makes the behaviour even more annoying.

Reply Score: 1

RE[4]: Comment by Drumhellar
by AWdrius on Mon 29th Jun 2015 10:48 UTC in reply to "RE[3]: Comment by Drumhellar"
AWdrius Member since:
2006-07-18


* There's no option to reverse "-A", which makes the behaviour even more annoying.


There's an -I (uppercase i) options. From man pages:
Prevent -A from being automatically set for the super-user.

Reply Score: 1

RE[2]: Comment by Drumhellar
by gotocaca on Thu 25th Jun 2015 12:14 UTC in reply to "RE: Comment by Drumhellar"
gotocaca Member since:
2014-02-22

Hiding files, for the unique reason that some user would find them annoying is a concept that would work 38 years ago. Hidden files are still accessible, it's a concept totally divergent from what could be the concept of an app space today : a user centric file repository and some access to storage space from the application. Hidden, Archive, and all the access rights from 38 years ago should not be something to consider when talking about an operating system. Operating systems like microsoft Windows re-built from scratch many times would want to consider also the money you can put to have access to your file. Something is wrong with the architecture of what is called 'computer science' as it is more like an old bible and NO ONE would re-consider what is written. Scientists from all over the world re-consider even the piss they take every day as a matter of consciousness. Why Oses would'nt take the change from old republic free of charge mamouth to fast and clean, driver switchable and open architecture micro-kernel based virtualization server. Google did realize they could use their browser as a full virtualization mecanism. Their supposed kernel has nothing to do with what could be the last replication of unix CLI system. People made many progress in Human interfaces such as speaking, touching. What could be a real progress in OS today remain the fact to leave the drivers to manufacturer and have an open architecture for software, interfaces, design and use case. Neat World please respond.

Reply Score: 0

RE[3]: Comment by Drumhellar
by acobar on Thu 25th Jun 2015 14:16 UTC in reply to "RE[2]: Comment by Drumhellar"
acobar Member since:
2005-11-15

Hidden, Archive, and all the access rights from 38 years ago should not be something to consider when talking about an operating system.

From a server point of view they are still very useful. You can claim your "user centric computing" trend as much as you want, it does not matter that much as we are seeing things moving to a sophisticated server interface that was not possible years ago (basically, I think, because deficiencies on server processing power, storage, bandwidth and availability).

Something is wrong with the architecture of what is called 'computer science' as it is more like an old bible and NO ONE would re-consider what is written. Scientists from all over the world re-consider even the piss they take every day as a matter of consciousness.

Give it time. A young field always struggle to establish the best tools to accomplish its goals. Math took millennium and only on 19 century it got an "acceptable" foundation (from mathematicians POV). Physics had huge shakedowns on 19 and 20 centuries. Shakedowns happened also on Biology, Chemistry and all other knowledge fields way older than computing.

And lets not forget that most of us, when complain about "computer science" are actually complaining about "software engineering", which is not really science, as are not also civil, electrical and mechanical engineering, even though they all use the tools science provides. You would be surprised by how many things are done in such form on these engineering fields just because they where that way a lot of years ago.

Edited 2015-06-25 14:18 UTC

Reply Score: 2

RE[4]: Comment by Drumhellar
by Alfman on Thu 25th Jun 2015 14:51 UTC in reply to "RE[3]: Comment by Drumhellar"
Alfman Member since:
2011-01-28

acobar,

You would be surprised by how many things are done in such form on these engineering fields just because they where that way a lot of years ago.


Haha, reminds me of this where the space shuttle boosters were engineered to fit on train tracks:
http://www.astrodigital.org/space/stshorse.html

Some history:
http://www.nasa.gov/mission_pages/shuttle/flyout/railroad.html

Edited 2015-06-25 14:57 UTC

Reply Score: 3

RE[5]: Comment by Drumhellar
by acobar on Thu 25th Jun 2015 15:37 UTC in reply to "RE[4]: Comment by Drumhellar"
acobar Member since:
2005-11-15

Best exemplification ever ! + ∞ (if it was possible)

Reply Score: 2

RE[6]: Comment by Drumhellar
by Drumhellar on Fri 26th Jun 2015 23:25 UTC in reply to "RE[5]: Comment by Drumhellar"
Drumhellar Member since:
2005-07-12

Best exemplification ever ! + ∞ (if it was possible)


Also completely untrue, at least as far as the Roman Chariot connection.

https://standards.nasa.gov/documents/RomanChariots.pdf

Reply Score: 2

RE[7]: Comment by Drumhellar
by acobar on Sat 27th Jun 2015 08:45 UTC in reply to "RE[6]: Comment by Drumhellar"
acobar Member since:
2005-11-15

Happy that we have a better reason and are not that crazy but, at same time, sad that lost a very good and funny telling.
:) + ;)

Thank you, anyway, no matter how, truth any day (but, after some pondering will reconsider, will keep the thing going to have some fun - of course, will have to mark it as an urban legend at the end).

Reply Score: 2

RE[2]: Comment by Drumhellar
by Bill Shooter of Bul on Thu 25th Jun 2015 13:40 UTC in reply to "RE: Comment by Drumhellar"
Bill Shooter of Bul Member since:
2006-07-14

Also, another comment further below hints that the dot-is-hidden notation was initially a bug that became a feature as people started abusing it. No idea if that is actually true, but I find it funny if it is. Especially with the many people bragging about how well designed Unix is. ;)


Well, Rob Pike kind of admits fault. So, yeah its true. In light of that, I wonder if the showing dot files as root was a way for someone to try and fix that bug in a way as to not upset users/applications that may be depending on it.

Reply Score: 2

RE[3]: Comment by Drumhellar
by dpJudas on Thu 25th Jun 2015 16:06 UTC in reply to "RE[2]: Comment by Drumhellar"
dpJudas Member since:
2009-12-10

Well, Rob Pike kind of admits fault. So, yeah its true. In light of that, I wonder if the showing dot files as root was a way for someone to try and fix that bug in a way as to not upset users/applications that may be depending on it.

My guess is he felt nothing should be invisible to root (by the same logic that I always disable hidden folders in Windows Explorer - you want to know its there, it could be important).

Whatever the rationale, his change didn't cause any protests and is now part of BSD. In the GNU land someone else got tired of 'ls * -la' not working and fixed that. Unix tools work they way they do a lot more due to whoever is maintaining them than some great masterplan of perfection. ;)

Reply Score: 2

Bill Shooter of Bul Member since:
2006-07-14

Edit: pointless comment removed.

Edited 2015-06-25 16:36 UTC

Reply Score: 2

RE[4]: Comment by Drumhellar
by gotocaca on Fri 26th Jun 2015 11:01 UTC in reply to "RE[3]: Comment by Drumhellar"
gotocaca Member since:
2014-02-22

Some scientists of my laboratory did research about walking in a collection. They stated that three procedures should be available from one node to another : first, next and foreach in an atomic interface. Walking through a tree would consider one more concept : the hierachical up and down. they also stated that the two movement should be conceptually possible. In term of procedure, we have two PROGRAMMING LANGUAGE concepts : sibling and children. The parent concept is considered as Framework-related. As it could interfere in structural representation of a data, it should be considered like first, next and foreach or shade or run. Parallel programming is something very different from usual OOP. We are talking about parallel programming compatible languages.

As a matter of usability usage for a storage search, users could benefit from metadata facet browsing, like KDE 4 did present lately. It is the proved best way to find a document in a storage space. the noise is reduced by far from classic research, and the user effort in term of action with the machine is also reduced. Listing a directory would consider listing a dimension of metadata, like day of use, project, status, of version.

Edited 2015-06-26 11:09 UTC

Reply Score: 1

Bill Shooter of Bul Member since:
2006-07-14

I'm guessing this was supposed to be in reply to a different post. Interesting, though. Theory is often bastardized by implementation.

Reply Score: 2

RE[2]: Comment by Drumhellar
by Alfman on Thu 25th Jun 2015 14:13 UTC in reply to "RE: Comment by Drumhellar"
Alfman Member since:
2011-01-28

dpJudas,

Also, another comment further below hints that the dot-is-hidden notation was initially a bug that became a feature as people started abusing it. No idea if that is actually true, but I find it funny if it is. Especially with the many people bragging about how well designed Unix is.



Hiding the "." and ".." entries makes sense, but I actually wish they never existed in any of the dir listings in the first place (such that they would not have needed to be hidden). Now in order to recurse a directory, we must have exceptions to handle "." and "..", otherwise the code will enter an infinite loop.

sub Walk {
($path) = @_;
$dir = opendir($path) or die($!);
while($entry = readdir($dir)) {
if ($entry eq '.' || $entry eq '..') next; # damn you exception!
$path2 = "$path/$entry";
if (-d $path2) {
print "$path2\n";
Walk($path2);
} # if
} # while
closedir($dir);
} # sub


I've never seen any value whatsoever in including the current and parent directories in the current directory listing. It's just a shame because we need this exceptional pattern in virtually 100% of directory walking code, when it could have easily been avoided.

Reply Score: 2

RE[3]: Comment by Drumhellar
by acobar on Thu 25th Jun 2015 15:10 UTC in reply to "RE[2]: Comment by Drumhellar"
acobar Member since:
2005-11-15

Hum, I always thought about them, '.' and '..', as a practical way to have a correspondence between the theoretical control structures, trees, and their storage counterparts. So '.' and '..' should be the equivalent of the current node (self reference) and parent pointer addresses, respectively, optimization for "fast access" POV. Do you devise a better approach than that?

Also, I would like to point out that their existence makes data recovery easier on cases of severe data corruption and disgraceful lack of proper backup.

Reply Score: 2

RE[4]: Comment by Drumhellar
by Alfman on Thu 25th Jun 2015 16:04 UTC in reply to "RE[3]: Comment by Drumhellar"
Alfman Member since:
2011-01-28

acobar,

Hum, I always thought about them, '.' and '..', as a practical way to have a correspondence between the theoretical control structures, trees, and their storage counterparts.


I'm not suggesting that they shouldn't exist in the file system structure, or in the kernel (if it's helpful for the sake of the structure). However I question the need to expose them this way to the user space API. Is there really value in telling the caller that "." exists? And since ".." is present even for the root directory, it's existence is equally uninformative. As such I've never seen a single use case for these in directory listings. Even if we are talking about the metadata, I see no reason a normal "stat" couldn't work.

Also, I would like to point out that their existence makes data recovery easier on cases of severe data corruption and disgraceful lack of proper backup.


Having them in the structure is fine, but I don't see a reason it should have a bearing on the API. For comparison: in Javascript a DOM node has node.childNodes(), which returns only the children of a node. But it's the same abstraction and could have been exposed the same way with node.selfAndParentAndChildNodes(). This way every time the function got used, programmers would have to explicitly filter out "self" and "parent" from the results before using it. Of course it would work, but what would be the justification given that what we are really seeking are the children?

I guess the way I see it is if the majority of users of an API have to add exceptional code in order to use it correctly, then the API is at least somewhat ill-conceived. IMHO a directory listing should list children only and not oneself nor one's parent. Hypothetically, can you imagine anybody saying "boy I wish these directory functions would return the current and parent directories in addition to the child directories"?

As an exercise, can anyone find code where the "." entries are used?

Edited 2015-06-25 16:13 UTC

Reply Score: 2

RE[5]: Comment by Drumhellar
by acobar on Thu 25th Jun 2015 17:19 UTC in reply to "RE[4]: Comment by Drumhellar"
acobar Member since:
2005-11-15

As an exercise, can anyone find code where the "." entries are used?


Perhaps, you are on future and have a lot of holographic clones in a room trying to capture some dangerous alien creature and every one of you respond to an unique name, and some of you would like to know who is the master so to avoid the chance that all die by risking the original ..

thereal=$( ls -ai . | grep -e "[ \t]\.$" | sed -e "s=^[ \t]*\([0-9]\+\).*=\1=" ); ls -ai .. | grep -e "^[ \t]*$thereal\b"

PS.: forgot to add, you are a directory.

Edited 2015-06-25 17:25 UTC

Reply Score: 2

RE[6]: Comment by Drumhellar
by Alfman on Thu 25th Jun 2015 17:56 UTC in reply to "RE[5]: Comment by Drumhellar"
Alfman Member since:
2011-01-28

acobar,

Haha, you may want to patent that ;)
Also, I'll patent this...

pwd | sed -e "s|^.*/||"

... Profit!

But even in bash I would argue that "stat" is the superior method for getting what you want (which is the inode).

stat -tc"%i" .
654082

VERUS

ls -ai . | grep -e "[ \t]\.$" | sed -e "s=^[ \t]*\([0-9]\+\).*=\1="
654082


As a side note, I'm not sure how reliable inodes are for this purpose across mount points. On my system "/" and "/mnt/box" are both inode 2. A few contrived mounts and voila, those clones don't stand a chance.

/mnt# ls -lai
total 24
915713 drwxr-xr-x 6 root root 4096 Jun 25 13:59 .
2 drwxr-xr-x 23 root root 4096 May 14 15:19 ..
2 drwsr-sr-x 19 root root 4096 Feb 25 12:29 a
2 drwsr-sr-x 19 root root 4096 Feb 25 12:29 b
2 drwsr-sr-x 19 root root 4096 Feb 25 12:29 box
2 drwxr-xr-x 4 root root 4096 May 14 15:19 c


I didn't want to mess up my root directory, but it would be possible to contrive a scenario where every inode in the directory is equal.

BTW, here is the output under this scenario:
/mnt/c# thereal=$( ls -ai . | grep -e "[ \t]\.$" | sed -e "s=^[ \t]*\([0-9]\+\).*=\1=" ); ls -ai .. | grep -e "^[ \t]*$thereal\b"
2 ..
2 a
2 b
2 box
2 c


Edited 2015-06-25 18:05 UTC

Reply Score: 2

RE[7]: Comment by Drumhellar
by acobar on Thu 25th Jun 2015 18:27 UTC in reply to "RE[6]: Comment by Drumhellar"
acobar Member since:
2005-11-15

As a side note, I'm not sure how reliable inodes are for this purpose across mount points.


They aren't. The only thing you have for sure is that if you get '.' and '..' with the same inode you reached the lowest level of your ext# partition (not of your file system) and, indeed you can actually use this to check if some ext# partition is mounted under some directory. I suspect this may work also with other "traditional" unix fs but did not check. For ext#, it must be 2.

This trick does not work with mounted ntfs, though.

Reply Score: 2

RE[5]: Comment by Drumhellar
by alisonken1 on Sun 28th Jun 2015 12:07 UTC in reply to "RE[4]: Comment by Drumhellar"
alisonken1 Member since:
2006-03-20

As an exercise, can anyone find code where the "." entries are used?


Not sure about daily usage other than with tarballs.

If you "tar -czvf ./*" (like slackware packages), then you can untar the files in any location rather than the hard-coded place that some people seem to prefer.

Reply Score: 1

RE[2]: Comment by Drumhellar
by grat on Fri 26th Jun 2015 17:46 UTC in reply to "RE: Comment by Drumhellar"
grat Member since:
2006-02-02

Considering the proliferation of .*rc files and similar configuration files and directories, if they weren't already hidden, some form of hiding would be created anyway.

Reply Score: 3

Comment by p13.
by p13. on Thu 25th Jun 2015 07:17 UTC
p13.
Member since:
2005-07-10

my guess is /etc/profile isn't read with sudo, and the default options for ls aren't set.

EDIT: Whoops, should have RTFA. Will leave my answer here for shame.

Edited 2015-06-25 07:22 UTC

Reply Score: 6

Comment by calden
by calden on Thu 25th Jun 2015 18:54 UTC
calden
Member since:
2012-02-02

This is kind of silly story, anyone who has ever used a nix command line would know this. Which I guess nowadays is becoming rare. Even half the people running Linux would be absolute clueless if their machines ever started up in a console.

Reply Score: 1