General Development Archive

Along with the broader industry trend of transitioning security-sensitive code to memory-safe languages like Rust, there has been an effort to write a Rust-based replacement to GNU Coreutils. For nearly a year that Rust Coreutils has been able to run a basic Debian system while more recently they have been increasing their level of GNU Coreutils compatibility and in some cases now even outperforming the upstream project. For someone like me, who isn’t a programmer, it’s difficult to really say anything meaningful when it comes to the pros and cons of individual programming languages, but on the face of it, with my limited understanding, modern languages like Rust do seem like a safer, more modern, more robust choice.

Modern languages such as Go, Julia and Rust don’t need complex garbage collectors like the ones use by Java C#. But why? To explain why, we need to get into how garbage collectors work and how different languages allocate memory in different ways. However, we will start by looking at why Java in particular needs such a complex garbage collector. Good info on how Go deals with memory versus how Java, mainly, handles memory. The most interesting start of a rabbit hole is the mention of research work around memory allocators.

Here, I will try to present another huge benefit of using Flutter desktop: the ability to build an app with a user interface that matches the underlying platform’s design standards. As you can see, Shortcut Keeper is built to be an adaptive app for desktops, boasting a different UI design for macOS and Windows, while using a single codebase. This has always been the holy grail of cross-platform development, and the screenshots here are relatively convincing.

Today, we are launching a technical preview of GitHub Copilot, a new AI pair programmer that helps you write better code. GitHub Copilot draws context from the code you’re working on, suggesting whole lines or entire functions. It helps you quickly discover alternative ways to solve problems, write tests, and explore new APIs without having to tediously tailor a search for answers on the internet. As you type, it adapts to the way you write code—to help you complete your work faster. Sounds like a cool and useful feature, but this does raise some interesting questions about the code it generates. Sure, generated code might be entirely new, but what about possible cases where the code it “generates” is just taken from the existing projects the AI was trained on? The AI was trained on open source code available on GitHub, including a lot of code licensed under, for instance, the GPL. GitHub says in the Copilot FAQ: GitHub Copilot is a code synthesizer, not a search engine: the vast majority of the code that it suggests is uniquely generated and has never been seen before. We found that about 0.1% of the time, the suggestion may contain some snippets that are verbatim from the training set. Here is an in-depth study on the model’s behavior. Many of these cases happen when you don’t provide sufficient context (in particular, when editing an empty file), or when there is a common, perhaps even universal, solution to the problem. We are building an origin tracker to help detect the rare instances of code that is repeated from the training set, to help you make good real-time decisions about GitHub Copilot’s suggestions. That 0.1% may not sound like a lot, but that’s misleading – another way to put it is that out of every 1000 suggestions Copilot makes, 1 is copy/pasted code someone has written and selected a license for, and that license must, of course, be respected. On top of that, it’s hard to argue that code generated from a set of existing open source code doesn’t constitute a derivative work, and is thus covered by the copyright open source licenses are based on. I am not a lawyer, so I’m not going to argue Copilot is definitively a massive GPL violation, but as a layman, on the face of it, it definitely feels like a tool that’s going to strip a lot of code from their licenses – without consent and permission of the code’s authors.

Rust developers have repeatedly raised concerned about an unaddressed privacy issue over the last few years. Rust has rapidly gained momentum among developers, for its focus on performance, safety, safe concurrency, and for having a similar syntax to C++.  StackOverflow’s 2020 developer survey ranked Rust first among the “most loved programming languages.” However, for the longest time developers have been bothered by their production builds leaking potentially sensitive debug information. I’ll leave this one for you folks to figure out, but from a layman’s perspective, it looks like a really dumb thing to keep paths from the developer’s machine like this in compiled binaries? At least after countless years, the Rust developers seem committed to fixing it, finally.

Well, I’ll not tell a long story, how I debug, but come directly to the bug mentioned in the title. I tracked his existence down to BASIC 2.0 as used in the VIC-20, C64 and the early PET/CBM series and it seems, that it was never detected, documented or fixed. It is related to temporary strings, the stack of descriptors for temporary strings, that has a size of 3, and the so called “garbage collection”, which in reality doesn’t collect garbage, but does a defragmentation of string storage. Fixing an ancient bug like this must be a weirdly satisfying experience.

This is the heart of the conflict: Rust (and many other modern, safe languages) use LLVM for its relative simplicity, but LLVM does not support either native or cross-compilation to many less popular (read: niche) architectures. Package managers are increasingly finding that one of their oldest assumptions can be easily violated, and they’re not happy about that. But here’s the problem: it’s a bad assumption. The fact that it’s the default represents an unmitigated security, reliability, and reproducibility disaster. I’m sure this will go down well.

Go 1.16 has been released. The new embed package provides access to files embedded at compile time using the new //go:embed directive. Now it is easy to bundle supporting data files into your Go programs, making developing with Go even smoother. You can get started using the embed package documentation. Carl Johnson has also written a nice tutorial, “How to use Go embed”. Go 1.16 also adds macOS ARM64 support (also known as Apple silicon). Since Apple’s announcement of their new arm64 architecture, we have been working closely with them to ensure Go is fully supported; see our blog post “Go on ARM and Beyond” for more. More details can be found in the release notes.

Petit FatFs is a sub-set of FatFs module for tiny 8-bit microcontrollers. It is written in compliance with ANSI C and completely separated from the disk I/O layer. It can be incorporated into the tiny microcontrollers with limited memory even if the RAM size is less than sector size. Also full featured FAT file system module is available here. Fascinating little project.

This release enables quite a lot of new things to appear in const fn, two new standard library APIs, and one feature useful for library authors. See the detailed release notes to learn about other changes not covered by this post. Well, not much for me to add.

Shells have been around forever and, for better or for worse, haven’t changed much since their inception. Until NuShell appeared to reinvent shells and defy our muscle memory. It brought some big changes, which include rethinking how pipelines work, structured input/output, and plugins. We wanted to learn more about NuShell so we interviewed both of its creators: Jonathan Turner and Yehuda Katz.

This release makes great progress in the C++20 language support, both on the compiler and library sides, some C2X enhancements, various optimization enhancements and bug fixes, several new hardware enablement changes and enhancements to the compiler back-ends and many other changes. There is even a new experimental static analysis pass. GCC is already 33 years old. That’s one heck of a legacy.

The Cidco MailStation is a series of dedicated e-mail terminals sold in the 2000s as simple, standalone devices for people to use to send and receive e-mail over dialup modem. While their POP3 e-mail functionality is of little use today, the hardware is a neat Z80 development platform that integrates a 320×128 LCD, full QWERTY keyboard, and an internal modem. After purchasing one (ok, four) on eBay some months ago, I’ve learned enough about the platform to write my own software that allows it to be a terminal for accessing BBSes via its modem or as a terminal for a Unix machine connected over parallel cable. A year old story, but come on, this is timelessly cool.

Dropbox is a big user of Python. It’s our most widely used language both for backend services and the desktop client app (we are also heavy users of Go, TypeScript, and Rust). At our scale—millions of lines of Python—the dynamic typing in Python made code needlessly hard to understand and started to seriously impact productivity. To mitigate this, we have been gradually migrating our code to static type checking using mypy, likely the most popular standalone type checker for Python. (Mypy is an open source project, and the core team is employed by Dropbox.) This post tells the story of Python static checking at Dropbox, from the humble beginnings as part of my academic research project, to the present day, when type checking and type hinting is a normal thing for numerous developers across the Python community. It is supported by a wide variety of tools such as IDEs and code analyzers. I recently came across an article complaining about Python’s dynamic typing and couldn’t quite believe this was still the case. As it turns out, nowadays there is indeed a standardized way to do write type annotations and to type-check prior to runtime using mypy, all the while being driven forward by the good folks at Dropbox (which includes Python’s Benevolent Dictator for Life Guido van Rossum). This article provides a fascinating insider insight into the history of type-checking in Python and how it evolved in symbiosis with Dropbox’s codebase.