Linked by Thom Holwerda on Wed 15th Nov 2017 23:59 UTC
BeOS & Derivatives

Haiku's GUI is in principle entirely scriptable. You can change a window's position and size and manipulate pretty much every widget in it. The tool to do this is hey. It sends BMessages to an application, thus emulating what happens if the user clicks on a menu, checkbox, or other widgets.

Order by: Score:
Security risk?
by jgfenix on Thu 16th Nov 2017 00:22 UTC
jgfenix
Member since:
2006-05-25

Am I mistaken or this is the kind of behavior that Google is worried about in it's accessibility API?

Reply Score: 4

RE: Security risk?
by Brendan on Thu 16th Nov 2017 02:32 UTC in reply to "Security risk?"
Brendan Member since:
2005-11-16

Hi,

Am I mistaken or this is the kind of behavior that Google is worried about in it's accessibility API?


I'm not entirely sure; but I don't think so.

The main problem Google would be worried about is "man in the middle snooping" - things like keyloggers that intercept user's input and send it to a third party so they can scrape out your passwords, etc.

Hiaku's "Hey" scripting looks like it doesn't intercept anything from the user (and can't be used for "man in the middle snooping"). It's more like (a GUI equivalent of) batch files in MSDOS, or shell scripts in Unix, or...

Also note that the "inability" to write scripts for GUI applications is one of the common complaints that command line users bring up in "command line vs. GUI" arguments; even though there's been many approaches to GUI scripting for a very long time (dating back to things like AppleScript and VBScript in the 1990s).

- Brendan

Reply Score: 7

RE[2]: Security risk?
by codifies on Thu 16th Nov 2017 08:26 UTC in reply to "RE: Security risk?"
codifies Member since:
2014-02-14

so... you couldn't create a new text box widget with hey, hover it over one for a password in an application, log passwords and pass them onto the original application... hmmm....

Reply Score: 2

RE[3]: Security risk?
by The123king on Thu 16th Nov 2017 08:41 UTC in reply to "RE[2]: Security risk?"
The123king Member since:
2009-05-28

No, but you could script an application that requires a password to automatically input a password and initiate login without doing it yourself

Reply Score: 5

RE[4]: Security risk?
by BlueofRainbow on Thu 16th Nov 2017 19:50 UTC in reply to "RE[3]: Security risk?"
BlueofRainbow Member since:
2009-01-06

This would be useful for automated tests of the user interface of an application.

However, I would not do this for daily use as the script would not necessarily be encrypted.

Reply Score: 4

Comment by DefineDecision
by DefineDecision on Thu 16th Nov 2017 01:54 UTC
DefineDecision
Member since:
2017-10-09

Haiku always gets appreciation from me.

That being said, the examples seem a bit.... clunky. I think AppleScript could have been shining the beacon w.r.t. GUI app automation - applications expose less a UI surface, more a UI, and recording a UI workflow really repeats the actions behind the scenes, not the UI events. It was the classic Mac OS' alternative to shell-based scripting in an environment that had absolutely no concept of a command line.

However, the AppleScript language was weird, so it never got a lot love from programmers. Automator was a late addition to help encourage AS development, but didn't help in the end, JS was added later on, but by then, Apple had gutted the division.

Reply Score: 4

RE: Comment by DefineDecision
by hhas on Thu 16th Nov 2017 22:32 UTC in reply to "Comment by DefineDecision"
hhas Member since:
2006-11-28

That being said, the examples seem a bit.... clunky. I think AppleScript could have been shining the beacon w.r.t. GUI app automation


The key to “AppleScript” automation is that it isn’t about mimicking users’ GUI interactions. An Apple event API is a complete, dedicated View/Controller layer that sits alongside the app’s GUI View/Controller, exposing an interface to the app’s data and behaviors specifically tailored for remote programmatic control.

(Incidentally, one of Apple event IPC’s most distinctive features is that it’s not based on the now-common Object-Oriented interaction model you see in the likes of DOM, but rather on RPC plus simple first-class relational queries, more akin to SQL+RDBs; a potential killer USP if today’s Apple had the wits to combine it with their query-building Siri UI… which they don’t as they’ve long since slipped from Thrilling Market Innovator to Just Another Corporate Bureaucracy.)

GUI Scripting, on the other hand, is an unforgivable atrocity that should be shot the moment it attempts to raise its head. Its only value is in writing GUI test scripts; any ingenious idea to use it for other purposes is Already Failed.


However, the AppleScript language was weird, so it never got a lot love from programmers. Automator was a late addition to help encourage AS development, but didn't help in the end


Automator was never about extending AS; it was a Cocoa API through and through. ObjC-based Action plugins could use AEs to communicate with other processes, but the APIs for that were even more crap than AS. And for most App developers the cost of implementing Automator Actions far outweighed any ROI, so it never scaled well.


JS was added later on, but by then, Apple had gutted the division.


Nope, not only did Apple expand the Automation team in order to develop JavaScript for Automation (from 2 developers to 3), they even took the JavaScriptCore project that the JSCore/WebKit folks had announced at WWDC13 as an AS alternative and handed it to the Automation team instead. The fundamental failure was PEBKAC: specifically, Sal Soghoian, head of the Automation team, was a fucking terrible Project Manager (a talented Automation evangelist promoted way beyond his level of competence; see: Peter Principle) who in his last decade at Apple effectively ran Mac Automation into the ground.

Result, the Automation team shipped JXA half-baked and half-broken in 10.10, then not only did they not try to fix it but didn’t even bother to document, promote, or support it. Unsurprisingly, JXA completely failed to capture any market, despite having millions of sitting JS users right there for the taking, with an incredibly powerful, widespread Automation infrastructure that should by all rights be Utterly Irresistable Industrial-Strength Catnip For Geeks™. That incredible failure is what got Sal’s ass belatedly sacked and his team disbanded.

If you’re curious, this is what an Apple event bridge for JavaScript should look like:

https://bitbucket.org/hhas/nodeautomation

Documentation’s a bit shit, as it’s decade-old recycled mess that I’ve never bothered to rewrite. But at this point I only really write these things for shits and giggles, and to prove other people wrong. (If you’re really curious, here’s the “OSA-packaged JSCore” reference implementation I sent Sal before 10.10 shipped: https://sourceforge.net/projects/appscript/files/ – bit messy/buggy as it was done in a huge rush, but still better’n the crap they shipped, and even has a nifty automagical AS-to-JS translation tool for turning AS commands into their equivalent JS syntax which alone would’ve gone a long way to making it a popular success.)

But anyway, I digress…TL;DR, Apple event IPC = Awesome Automation infrastructure, designed right and still the benchmark by which all others should be judged, screwed by being shackled to a shit language-cum-brand name and squandered by a vast not-nearly-as-bright-as-you-think corporation that after 20 years is quietly preparing to throw it away for good. (If FOSS/Linux had an ounce of sense, they’d steal all its concepts and philosophy and bat it out the park with nominal effort…eh but who are we kidding.[sad])

Reply Score: 6

RE[2]: Comment by DefineDecision
by zima on Fri 17th Nov 2017 14:17 UTC in reply to "RE: Comment by DefineDecision"
zima Member since:
2005-07-06

So... it seems like you might be the local MacOS automation expert? ;) I have a related question...

Some time ago I got a Connectix QuickCam, the 1st webcam, connected to ~classic Macs through the ADB port. And... what else to do with it than to put it on the web! ;) Some caveats though...

I do not have the drivers/software, I must yet to hunt it down on the web. And, having little experience with classic MacOS, I do not know how OS-version specific it is - would the driver be "universal" across 68k and PPC, or across OS 7.x, 8.x, 9.x ...or not?

Also, I do not yet have a required Macintosh. It would have to be a machine with ADB and Ethernet ports, so probably some PPC Mac running MacOS 8.x or 9.x...

The way I want to use it, is to take a photo every minute, give it a name in the YYYYMMDDHHMM format, then upload it to a server for the world to see ...is learning AppleScript my only option on such setup? ;) (what resources would you recommend?)

Reply Score: 3

RE[4]: Comment by DefineDecision
by zima on Sun 19th Nov 2017 13:55 UTC in reply to "RE[3]: Comment by DefineDecision"
zima Member since:
2005-07-06

Oh joy, probably everything I could hope for (planning to get into Python anyway / it's not a dead tech). Thank you. ;)

Reply Score: 2

No Link?
by Brendan on Thu 16th Nov 2017 02:15 UTC
Brendan
Member since:
2005-11-16

Hi,

I think someone forgot to include a link to the article being mentioned here.

That link is probably: https://www.haiku-os.org/blog/humdinger/2017-11-05_scripting_the_gui...

- Brendan

Reply Score: 4

Haiku's GUI is from another age
by gotocaca on Thu 16th Nov 2017 12:12 UTC
gotocaca
Member since:
2014-02-22

Haiku's GUI is aging, device-dependant, bitmap based, pixel-wide surface with à lack of color freedom.

The script API IS hard to read, closed to this solely plateform, ready to use from à monthly-basis usage course.

Thé design pattern used in this framework are litteraly from another age, based on IOT principeles, unhealthy procédures and hard-to-find documentation.

The running processus itself lack of nowadays graphism effects, made to enjoy framed Gray buttons.

I AM sure Haiku's développent team can make better with nowadays tools and design principles. Receipes from web and mobile GUI. With Todays desktop like KDE we can play with colorfull widgets and graphism effects. Enjoy drag'n'drop collections, Device-independant relative surfaces and a meta-framework with interactive event-based patterns.

Thanks for the sake of Haïku, it is completely original and out of frame designed piece of art.

Edited 2017-11-16 12:13 UTC

Reply Score: 0

The123king Member since:
2009-05-28

And some of us don't want such superfluous crap. All i want my window manager to do is manage windows. I don't need them transparent, or rainbow coloured. Haiku's Window manager does things right. All the "innovation" has been focused on making "managing windows" as powerful as possible, and not on crap like aesthetics. Sure, it looks like it came out of the 90's, but how many other OSes allow you to stack and tile a whole combination of applications?

Edited 2017-11-17 11:00 UTC

Reply Score: 1

RE: Haiku's GUI is from another age
by gfx1 on Fri 17th Nov 2017 14:26 UTC in reply to "Haiku's GUI is from another age"
gfx1 Member since:
2006-01-20

Yes it is BeOS from the late nineties but back then it was an improvement on Windows and xwindows. The documentation was not bad, at least it was included instead of paying a lot of $ for the RKM's

Reply Score: 2

Plasma
by kwan_e on Thu 16th Nov 2017 20:12 UTC
kwan_e
Member since:
2007-02-18

Isn't a scriptable GUI also the goal of KDE Plasma? You have to use QML though.

Reply Score: 4

RE: Plasma
by The123king on Fri 17th Nov 2017 11:02 UTC in reply to "Plasma"
The123king Member since:
2009-05-28

Haiku has had Hey for years, I remember playing with it in the alpha 1/2 days. Maybe KDE should hurry the f*** up.

Reply Score: 1

RE[2]: Plasma
by kwan_e on Sat 18th Nov 2017 03:13 UTC in reply to "RE: Plasma"
kwan_e Member since:
2007-02-18

Haiku has had Hey for years, I remember playing with it in the alpha 1/2 days. Maybe KDE should hurry the f*** up.


What?

KDE Plasma already exists and you can write stuff for it in QML.

Reply Score: 2