Multimedia, AV Archive

Remastering Star Trek: Deep Space Nine with machine learning

As a little side-project, I have been working on putting the artificial neural networks of AI Gigapixel to the test and having them upscale another favorite thing of mine… Star Trek: Deep Space Nine (DS9). Just like Final Fantasy 7, of which I am upscaling the backgrounds, textures, and videos in Remako mod, DS9 was also relegated to a non-HD future. While the popular Original Series and The Next Generation were mostly shot on film, the mid 90s DS9 had its visual effects shots (space battles and such) shot on video. While you can rescan analog film at a higher resolution, video is digital and can’t be rescanned. This makes it much costlier to remaster this TV show, which is one of the reasons why it hasn’t happened. Fascinating methodology, and the results speak for themselves. Amazing work.

Why vinyl records survive in the digital age

The entire experience of vinyl helps to create its appeal. Vinyl appeals to multiple senses—sight, sound, and touch—versus digital/streaming services, which appeal to just one sense (while offering the delight of instant gratification). Records are a tactile and a visual and an auditory experience. You feel a record. You hold it in your hands. It’s not just about the size of the cover art or the inclusion of accompanying booklets (not to mention the unique beauty of picture disks and colored vinyl). A record, by virtue of its size and weight, has gravitas, has heft, and the size communicates that it matters. Anyone who says vinyl sounds objectively better – using the same amplifier and speaker hardware as modern media – can hardly be taken seriously, but that doesn’t mean vinyl can’t sound subjectively better. When it comes to older music from the ’60s and ’70s, I enjoyed listening to it on vinyl records (I don’t have a record player at this moment), but that had nothing to do with sound quality, and everything to do with the more archaic, unique experience of listening to a vinyl record.

Magic Lantern

Magic Lantern is a software enhancement that offers increased functionality to the excellent Canon DSLR cameras. We have created an open framework, licensed under GPL, for developing extensions to the official firmware. Magic Lantern is not a “hack”, or a modified firmware, it is an independent program that runs alongside Canon’s own software. Each time you start your camera, Magic Lantern is loaded from your memory card. Our only modification was to enable the ability to run software from the memory card. ML is being developed by photo and video enthusiasts, adding functionality such as: HDR images and video, timelapse, motion detection, focus assist tools, manual audio controls much more. What a fascinating project. I knew you could put custom ROM images on digital cameras, but this seems like a far safer and less warranty-breaking way of extending and improving the functionality of your camera.

PC speaker to eleven

«System Beeps» is a music album in shape of an MS-DOS program that features original music composed for PC Speaker using the same basic old techniques like ones found in classic PC games. It follows the usual retrocomputing demoscene formula — take something rusty and obsolete, and push it to eleven — and attempts to reveal the long hidden potential of this humble little sound device. You can hear it in action and form an opinion on how successful this attempt was at Bandcamp, or in the video below. The following article is an in-depth overview of the original PC Speaker capabilities and making of the project, for those who would like to know more. What an amazing work of art, and I love the detailed description of how it was made using nothing but the PC speaker. This article is quite detailed, and the project itself is released under the CC-BY license.

VLC 4.0 to get new user interface, better Wayland support, more

VLC 4.0 is on the way, and the VLC developers have listed what they have in store for this major new release. The most obvious new user-facing feature is brand new user interfaces for each platform the media player supports, such as KDE, Gnome, Windows, macOS, and more. Work on the new VLC 4.0 user-interface is progressing, there will be GNOME and KDE adaptations, support for both server-side and client-side decorations, and great support for Wayland as well as X11 — including support for macOS, Windows, etc. With VLC 4.0, they intend to gut out support for Windows XP/Vista as well as bumping the macOS, iOS, and Android requirements. On the Linux front, they intend to require OpenGL acceleration for this media player. There’s no information yet on when this new release will be made available.

The VU meter and how it got that way

Given its appearance in one form or another in all but the cheapest audio gear produced in the last 70 years or so, you'd be forgiven for thinking that the ubiquitous VU meter is just one of those electronic add-ons that's more a result of marketing than engineering. After all, the seemingly arbitrary scale and the vague "volume units" label makes it seem like something a manufacturer would slap on a device just to make it look good. And while that no doubt happens, it turns out that the concept of a VU meter and its execution has some serious engineering behind that belies the really simple question it seeks to answer: how loud is this audio signal?

I love analog VU meters, and I'm kind of sad regular, non-professional music equipment has done away with them entirely.

Codec2: a whole podcast on a floppy disk

In a previous blogpost we talked about the Opus codec, which offers very low bitrates. Another codec seeking to achieve even lower bitrates is Codec 2.

Codec 2 is designed for use with speech only, and although the bitrates are impressive the results aren’t as clear as Opus, as you can hear in the following audio examples. However, there is some interesting work being done with Codec 2 in combination with neural network (WaveNets) that is yielding great results.

Game Boy Camera Canon EF lens mount

I designed and built a Canon EF Mount for my Game Boy Camera. The GBC has a sensor size of about 3.6mm² which seems equivalent to a 1/4" sensor. This gives the GBC a crop factor of about 10.81. With my 70-200 f4 mounted on a 1.4x extender, this gives me a max equivalent focal distance of about 200x1.4x10.81=3,026.8mm.

I always wanted a Game Boy Camera when I was a kid. It still looks like magic to me today.

FFmpeg 4.0 released

FFmpeg 4.0 has been released, and it's a major one. Since this particular subject matter - and its changelog - are way beyond the scope of my capabilities, I'll just leave you with the generic description of the project (in case you live under a rock).

FFmpeg is the leading multimedia framework, able to decode, encode, transcode, mux, demux, stream, filter and play pretty much anything that humans and machines have created. It supports the most obscure ancient formats up to the cutting edge. No matter if they were designed by some standards committee, the community or a corporation. It is also highly portable: FFmpeg compiles, runs, and passes our testing infrastructure FATE across Linux, Mac OS X, Microsoft Windows, the BSDs, Solaris, etc. under a wide variety of build environments, machine architectures, and configurations.

The fight for patent-unencumbered codecs is nearly won

Apple joining the Alliance for Open Media is a really big deal. Now all the most powerful tech companies - Google, Microsoft, Apple, Mozilla, Facebook, Amazon, Intel, AMD, ARM, Nvidia - plus content providers like Netflix and Hulu are on board. I guess there's still no guarantee Apple products will support AV1, but it would seem pointless for Apple to join AOM if they're not going to use it: apparently AOM membership obliges Apple to provide a royalty-free license to any "essential patents" it holds for AV1 usage.

It seems that the only thing that can stop AOM and AV1 eclipsing patent-encumbered codecs like HEVC is patent-infringement lawsuits (probably from HEVC-associated entities).

I can barely believe this is still a thing, and that it seems like a positive outcome.

Bluetooth won’t replace the headphone jack – walled gardens will

Nilay Patel on the further disappearance of the headphone jack, and its replacement, Bluetooth:

To improve Bluetooth, platform vendors like Apple and Google are riffing on top of it, and that means they’re building custom solutions. And building custom solutions means they’re taking the opportunity to prioritize their own products, because that is a fair and rational thing for platform vendors to do.

Unfortunately, what is fair and rational for platform vendors isn’t always great for markets, competition, or consumers. And at the end of this road, we will have taken a simple, universal thing that enabled a vibrant market with tons of options for every consumer, and turned it into yet another limited market defined by ecosystem lock-in.

This is exactly what's happening, and it is turning something simple and straightforward - get headphones, plug it in literally every single piece of headphones-enabled audio equipment made in the last 100 years, and have it work - into an incompatibility nightmare. And this incompatibility nightmare is growing and getting worse, moving beyond just non-standard Bluetooth; you can't use Apple Music with speakers from Google or Amazon, and Spotify doesn't work on the Apple Watch.

Removing the headphone jack was a user-hostile move when Apple did it, and it's still a user-hostile move when Google does it.

Update on HTML5 video for Netflix

About four years ago, we shared our plans for playing premium video in HTML5, replacing Silverlight and eliminating the extra step of installing and updating browser plug-ins.  

Since then, we have launched HTML5 video on Chrome OS, Chrome, Internet Explorer, Safari, Opera, Firefox, and Edge on all supported operating systems.  And though we do not officially support Linux, Chrome playback has worked on that platform since late 2014.  Starting today, users of Firefox can also enjoy Netflix on Linux. This marks a huge milestone for us and our partners, including Google, Microsoft, Apple, and Mozilla that helped make it possible.

It wasn't that long ago we barely dared to imagine HTML5 video taking over from Flash and Silverlight.

MiniDisc: an appreciation

In this video you'll see the first machine and the last machine as well as some in-between. There's talk about MD-LP, Net-MD and HiMD. It's a personal retrospective of a format that was loved by many people around the world but one that is all too often is judged purely on its lack of performance in the US market.

Great video by a great channel.

I'm one of those MiniDisc people. MiniDisc was fairly successful in The Netherlands, and quite a few people around me were MiniDisc users as well. I've had countless machines over the years, and I was still using HiMD well into the smartphone era - and carried both a smartphone and my HiMD player for quite a while. Even though the world had long ago moved on to MP3 players and then smartphones, I was still using MD.

I've long wondered why, and this video finally made it dawn on me: rituals. Since prerecorded MiniDiscs were rare and incredibly expensive, you copied CDs onto MiniDiscs instead. Especially before the advent of NetMD and later HiMD, you did this without the help of a computer. You'd get a new album, listen to it, enjoy it - and then, to make sure you could listen to it on the go, you plugged one end of an optical cable into your CD player, the other end into your portable MD recorder, and copy the CD in real time. Once it was done, neat freaks like me would even enter all the track information using the little dial on the recorder, track by track, letter by letter. Painstaking doesn't even begin to describe it.

Even listening to your MiniDiscs - they were satisfying to hold, the loading and unloading was deeply mechanical, the spring-loading trays were a delight. It was just an endless array of rituals that, while pointless and cumbersome to others, were deeply enriching and soothing to me. I guess it must be similar to people still using vinyl today.

To me, MiniDisc was one of the greatest formats - not because it was better or more advanced (even though during the 90s and early 2000s, it actually was), but because it was full of little delights and rituals. Just one of those irrational things that only few of us will ever fully understand.

A really obscure forgotten audio format: “Talking rubber”

While common magnetic tape uses very thin, plastic-coated iron oxide, "talking rubber" uses rubber impregnated with iron oxide. Iron oxide (a form of rust) is ferromagnetic, which means in the presence of a magnetic field, the electrons in the iron oxide magnetically line up and stay that way even after the magnetic field is turned off. This allows cassette tapes to create a “track” of magnetically aligned iron oxide when the electromagnet in a cassette recorder creates a magnetic field.

But with magnetic rubber, the iron oxide is actually mixed into the rubber material; the whole band becomes ferromagnetic, instead of just the coating. According to that Bell System Journal article, this “talking rubber” could be around 1/16 or 1/8 of an inch think, whereas magnetic tape was (even in the '50s) already much thinner at 1/1000 of an inch thick.

More obscure audio formats!

Forgotten audio formats: Elcaset

Back before all-digital music, back before the Digital Compact Cassette, back before even the Digital Audio Tape existed, there was a strange audio device that briefly captured the imagination of Hi-Fi freaks across the world. The Elcaset, as it was called, was an enlarged cassette that started in Japan, wove its hidden, spinning spools around the world, and then finished, appropriately enough, in Finland.

As someone who swore by MiniDisc up until quite recently, I love obscure audio formats. This article is from the summer of last year, but I only came across it just now thanks to Atlas Obscura.

H.264 is magic

H.264 is a video compression codec standard. It is ubiquitous - internet video, Blu-ray, phones, security cameras, drones, everything. Everything uses H.264 now.

H.264 is a remarkable piece of technology. It is the result of 30+ years of work with one single goal: To reduce the bandwidth required for transmission of full-motion video.

Technically, it is very interesting. This post will give insight into some of the details at a high level - I hope to not bore you too much with the intricacies. Also note that many of the concepts explained here apply to video compression in general, and not just H.264.

Netflix to try to stop users using proxies

Some members use proxies or "unblockers" to access titles available outside their territory. To address this, we employ the same or similar measures other firms do. This technology continues to evolve and we are evolving with it. That means in coming weeks, those using proxies and unblockers will only be able to access the service in the country where they currently are. We are confident this change won't impact members not using proxies.

Good luck with that.

Netflix’ new video compression strategy

Netflix is working on a new video compression strategy, and they've published a very detailed blog post about it.

We've spent years developing an approach, called per-title encoding, where we run analysis on an individual title to determine the optimal encoding recipe based on its complexity. Imagine having very involved action scenes that need more bits to encapsulate the information versus unchanging landscape scenes or animation that need less. This allows us to deliver the same or better experience while using less bandwidth, which will be particularly important in lower bandwidth countries and as we expand to places where video viewing often happens on mobile networks.

The technical details go way over my head, but the basic premise seems to make sense to a layman such as I.

The doomed effort to make videos go vinyl

Few people even remember that such a medium as vinyl movies existed, but for a brief, doomed period in the early 1980s, home video was available on CEDs. While CED players were not released to consumers until 1981, the development of the system dates back to the 1960s. The idea was that they could encode sound and video information to a vinyl disc if they could only get the grooves small enough.

Fascinating. I had no idea they ever tried to do this. Better yet, that they succeeded.