Linked by Thom Holwerda on Wed 25th Jan 2012 22:05 UTC, submitted by twitterfire
Internet & Networking "Google's efforts to improve Internet efficiency through the development of the SPDY (pronounced 'speedy') protocol got a major boost today when the chairman of the HTTP Working Group (HTTPbis), Mark Nottingham, called for it to be included in the HTTP 2.0 standard. SPDY is a protocol that's already used to a certain degree online; formal incorporation into the next-generation standard would improve its chances of being generally adopted."
Thread beginning with comment 504754
To view parent comment, click here.
To read all comments associated with this story, please click here.
RE[2]: no binary protocols please
by Neolander on Thu 26th Jan 2012 17:41 UTC in reply to "RE: no binary protocols please"
Neolander
Member since:
2010-03-08

Text also has advantages beyond human readability though, like the availability of many quality parsers and the fact that text-based protocols abstract away endianness issues in the underlying text transmission protocol.

Reply Parent Score: 2

Alfman Member since:
2011-01-28

Neolander,

"Text also has advantages beyond human readability though, like the availability of many quality parsers"

My opinion is that binary protocols don't need to be difficult to implement, sometimes they're even easier to implement than the text parsers. It could go either way: SMTP is very difficult due to the complexity of ASN encoding, which is designed to serialize arbitrary hierarchical objects, others like Modbus borderline on trivial.

One of the problems with pure text is that we have to scan strings of unknown length which is far less efficient than knowing string lengths up front. It's the difference between pascal strings and c-strings. With terminated strings, every single byte has to be checked. When the length is specified, data can be copied/compared at least 8 bytes at a time on many architectures.

Also, text based protocols like HTTP have to accept superfluous whitespace and 3 different types of newline encodings, all of which makes parsing even less efficient.

All this says nothing of unicode support, which if required makes the delimiter scans even more difficult due to erroneous byte matches within multi-byte unicode sequences.


"and the fact that text-based protocols abstract away endianness issues in the underlying text transmission protocol."

Yes, endianness is a problem, however let me point out that it would be equally problematic for text-represented numbers if humans had not standardized on big-endian numeric representation for themselves. There's nothing stopping binary protocols from standardizing a byte order (network byte order) in the same way.

Reading/writing textual numbers requires multiplying/dividing decimal digits one at a time. When using binary numbers, they can be processed in their entirety with a possible opcode for byte swap. Also, since every binary byte represents a valid number, no ASCII digit tests are required.


All in all, binary protocols are more efficient, but the question goes back to how large is text overhead compared to the payload itself? Probably not much. Only once HTTP is used for highly asynchronous/multiplexed communication will it's overhead start to overtake the payload.


I think there is (legitimate) fear that binary protocols would be extended in proprietary & undocumented ways to make it incompatible, and that would be a damn shame. This is why it'd be crucial for the standard to mandate clients/servers break the connection on non-standard requests, it would force developers to get their act together with regards to total standard compliance.

Reply Parent Score: 2

Lennie Member since:
2007-09-22

Did you really mean SMTP when you talked about ASN.1 ?

Because I think you are mistaken, I've never seen it in SMTP. SMTP is an example of a text protocol.

All I could find about ASN.1 is a bug in the GSSAPI (authentication) of Microsoft Exchange.

Maybe you meant SNMP ?

Reply Parent Score: 2

Zifre Member since:
2009-10-04

Have you ever written parsers? It is vastly easier to parse binary protocols than text. Even JSON, with a relatively simple grammar, is much harder to parse than BSON, for example.

Endianness is not really an issue, if the protocol simply defines it to be one way or the other. Also, some binary protocols are byte-based, like UTF-8.

Reply Parent Score: 2

Alfman Member since:
2011-01-28

Zifre,

This is the first time I've heard of BSON. I see the mongodb project came up with it.

http://bsonspec.org/#/specification

I read the spec, and to be honest I don't like some of the design choices they made. Some of the datatypes are fixed size, some strings are zero terminated, others have a length prefix, yet another has both a length prefix and is zero terminated (I haven't the foggiest idea why?).



Whereas JSON encapsulates only primary generic datatypes, BSON has started enumerating application specific datatypes like MD5, RegEx, and UUID.

{"Name": "Alfman", "MD5": (string)"md5bytes"}
{"Name": "Alfman", "MD5": (Generic)"md5bytes"}
{"Name": "Alfman", "MD5": (User)"md5bytes"}
{"Name": "Alfman", "MD5": (MD5)"md5bytes"}
{"Name": "Alfman", "MD5": (RegEx)"md5bytes"}
{"Name": "Alfman", "MD5": (UUID)"md5bytes"}

In my opinion this is fundamentally flawed:

Problem 1: Application specific datatypes don't deserve special treatment in a generic transport protocol. Why is SHA not present? What about RSA keys? What about a JPEG datatype?

Problem 2: The *meaning* of a variable is implicitly known by the code which uses it. If it's *expecting* the MD5 field to contain an MD5 hash, there's no need for the transport protocol to tell it that the raw bytes are an MD5 datatype. JSON/XML work fine this way, I can't think of many applications that would benefit from overloading the MD5 field into different datatypes, which is probably always going to be an error.


So in my opinion, BSON is not a good model for a binary HTTP protocol. But I do think a simple binary name/value collection could work nicely.

Edit: it has occurred to me that you didn't actually propose BSON would be suitable for HTTP, but merely used it as an example of a binary container. So maybe my reply is out of context to what you were thinking, but I'll leave my comments anyway.

Edited 2012-01-27 03:27 UTC

Reply Parent Score: 2

Neolander Member since:
2010-03-08

Have you ever written parsers? It is vastly easier to parse binary protocols than text. Even JSON, with a relatively simple grammar, is much harder to parse than BSON, for example.

Just to make it clear : what I meant was not that it is easy to write a text parser, but that thanks to the UNIX world, there are several quality general-purpose text parsing code and algorithms around the web, ready to be tuned for specific uses. I am not sure that the same can be said of binary parsers, where it seems to me that one is more likely to find one parser for each specific protocol/file format.

Endianness is not really an issue, if the protocol simply defines it to be one way or the other. Also, some binary protocols are byte-based, like UTF-8.

It is arguable that UTF-8, like ASCII, is more of the minimal binary support that any text-based protocol needs, though. This protocol is pretty much only good at transmitting text.

Edited 2012-01-27 05:59 UTC

Reply Parent Score: 3