Experiment

Hugo is a static site generator. However sometimes we want to serve dynamic contents. We can embed javascript to achieve what we want, but why not use templates when they’re more convenient? Caddy Template Engine Caddy comes with a powerful template engine that is based on golang standard library. Some of the sites (including Caddy’s official website) are built this way. The list of template functions/actions can be viewed here. In addition, caddy also support all Sprig functions. ...

Google released BBR v3 nearly two months ago. To try it out, we have to use a kernel which is compiled from a source that has this patch. We can compile the kernel ourselves or be lazy and use a kernel compiled by others. But what fun is there if we take the easy way? About GitHub Actions GitHub Actions can be used to automate software workflows, from building to deploying. It’s free of charge for standard GitHub-hosted runners in public repositories and has 2000 minutes limit per month for private repositories. Their runners have decent hardware resources for building linux kernels. ...

In a previous post, I mentioned using yt-dlp as a service to extract YouTube information. The method I used has one serious problems: it uses a custom protocol that serves one request per connection and dumps the error as a string if any. To find out if there is an error, we have to parse the result first and if it’s not a valid json string, we can be sure there’s an error. ...

Previous post shows how to create a mpd manifest from the result of yt-dlp and play it using various players. However, there are some problems when we use the manifest with other tools such as ffmpeg and other video players rather than shaka-player and dash.js, mainly in the form of slow buffering. Determine the Cause of Slow Buffering First we determine how a browser requests YouTube assets. Open devtools: Request Query Parameters and Payload We can find the request is a POST request with query parameter range and payload x\n. After some testing it can be determined that range parameter controls which part of the assets the browser is requesting just like http range requests. ...

Previous post shows how to extract YouTube video assets using yt-dlp and implement a dynamic reverse proxy. This post details what the format YouTube is using. First we’ll find out what formats YouTube uses to stream high quality videos in a browser. Naive Solution There is an audio html element. We could try to synchronize an audio and a video element. However, it’s easier said than done as we have to consider not only play, pause, seek states, but also video and audio buffering, and it’s very easy to for these two elements to fall out of sync. ...

This series of posts details how to proxy YouTube videos. Unlike using tools to download YouTube videos, we’re proxying them so there is no need for a large storage system. This can be used to avoid ads and tracking and for other media streaming purpose. First we’ll learn how to extract YouTube video urls and write a simple dynamic reverse proxy. Extracting YouTube Video Urls If we want to proxy YouTube assets, first we’ll need to know the urls of the video assets. We can use yt-dlp to retrieve urls: ...

Continuing from previous post, this post is about how to handle ttyd using golang. Overview The golang handler consists of 3 parts: sending frontend, sending auth token and handling websocket connection. Sending Frontend Golang can embed static assets in the compiled binary so there is no need to specify a path to the file. The frontend can be extracted using devtools. //go:embed ttyd.html var ttydHtml string func index(writer http.ResponseWriter, request *http.Request) { _, _ = io.WriteString(writer, ttydHtml) } Sending Auth Token We just write a static json string with an empty token since we don’t check the token in our implementation. ...

Golang comes with a webdav server implementation. It only supports webdav operations and nothing more. To manager a group of users, we need to implement our own handling logic. hacdias’s webdav server associates each user with his/her own webdav.Handler, and each authenticated request goes to their respective handler. rclone can generate dynamic webdav backends using auth proxy. By implementing auth proxy, rclone can serve different contents based on users. However, both of them suffer from lock states not shared, meaning when multiple users share the same directory, some changes may be lost if they try to modify its resources concurrently. ...

This post first demonstrates http1, http2, http3 like akamai, then shows how the demo works. Demonstrations Info Firefox has a bug/feature that it will cache images even though instructed not to. Use another browser or hard refresh to rerun the tests. ...

Previously I have written a post about ttyd that describes how to authenticate requests for ttyd sessions. The problem is that vanilla ttyd only supports running a predefined command, and it’s nearly impossible to customize what command to run based on the authenticated user. To solve this problem we need to port ttyd to another language. But first we need to understand how ttyd works. Request Waterfall When a browser connects to a ttyd instance, below is the order in which resources are requested. ...