Caligo

I've had close to 10 years of history with Shadow. It all started with my own interest in streaming a computer over the network to play games and avoid turning my own house into a datacenter, and eventually turned into "why not do business with this company?" I've spent my free time over the years helping improve the product, made countless projects to help users on Linux and Mac, made my own CLI launcher in the early days, and put together tons of documents with information, gathered details, captures, analysis, and a possible technical roadmap. Up until two weeks ago, when I decided: "why not try to fully reimplement the client now?"

So I ran my LLM over all my documents and started drafting a battle plan. I started with the Radxa Rock 5B's most bleeding-edge features in mind and consolidated the plan:

• SDL3, because we don't want to reinvent the wheel here, and it runs on any platform
• Vulkan as the default renderer, because we want to leverage some cool features (DMABUF, Shaders)
• FFmpeg (v4L2request, vaapi, cuda, videotoolbox, mediacodec for H264/H265/AV1)
• libdatachannel, because I wanted WebRTC support from day one at same level as native UDP and TCP

I started implementing a basic launcher. My old work was already obsolete for a lot of things, protocols had changed and evolved but I managed to implement basic Auth, then dove into the unknown and implemented the other protocols, tirelessly, one by one, enabling a new feature each time, until I had all the basic features working, unoptimized.

At that point, I started a new round of optimization on each feature, with one thing in mind: how can I make it better, faster, and able to run on as many devices as possible, easily. After a week, I ended up with 1920x1080@120Hz, with Overwatch fully playable.

I was happy, so I had a few contacts test my build, and I was suddenly facing all the issues Shadow has had to overcome over the years and still struggles with some of. Not all users have a good network, we don't all do the same things, and I ran into new bugs :)

Back to hardening the implementation. The most challenging part was Input (HID). I'm still doing some optimization on it dealing with fast, efficient acquisition, OS specifics, scancodes, transport, and Shadow's own streamer protections, without the source code, specs, or any details, was quite the challenge. But I've managed to get something stable now, and even implemented a few features I was missing from the official Shadow client.

With that working nicely on Linux (Wayland native, but it also supports X11 and even DRM/KMS, GBM), why not make a macOS client? Then a Windows client? An Android TV client? Well, it's done, clearly untested and unoptimized, but it's being built in my CI and it "works."

Yes, I only have two hands :)

At that point I only had basic fragment shaders, and I started wondering: can we convert those to compute shaders, do a lot more, and leverage more Vulkan and Metal features? Turns out yes, it works now, and CPU usage stays under 13% on an RK3588, with the GPU barely affected, while doing this fun stuff:

• Teamwork with on-chip memory (cooperative chroma reconstruction)
• Subpixel centering (no "fuzzy" blur)
• Smart sharpening (AMD CAS)
• Color conversion (YUV → RGB)

Now, the "cursor shape" and "locked/unlocked" still a challenge, especially on macOS. Getting a solid implementation of this was hard, but I've managed to land a solid one, plus a workaround most dual-screen users know ("mouse trail" configured in Windows). It's not perfect, but I'm leveraging a couple of interesting tricks: a mouse-shape cache, and only updating the shape when needed, to save a few CPU cycles (yes, we need to think about laptop users too).

I've also added a copy-paste feature and microphone support. The microphone works nicely, with a few options to control mic gain and noise reduction, though it's still pretty basic for now. I have more plans for it that I'm saving for later. File transfer is also done, but not enabled yet, because I have more ideas for that feature and other, more fun subjects to attend to right now :)

So, why "Caligo"? Caligo is a rare, primordial figure in Roman mythology who represents dark mist, fog, or profound obscurity. I immediately thought that would be a cool name for this PoC :)

What are my plans? For now, none, I'm just having fun with this project, and it's showing me things I'm not used to. I've also always wanted to bring old devices back to life (Steam Link, the Freebox Delta Player from Devialet) and see their limits with this project. Maybe I'll also try implementing it on Tizen or other platforms after I'm done with all the remaining issues, UX, GUI........

Stay tuned.