Compare commits

...

2 Commits

Author SHA1 Message Date
Matthew Tran
bf74966191 add coreos migration post 2025-05-22 01:12:14 -07:00
Matthew Tran
f36aa2316c update dependencies 2025-05-21 21:49:56 -07:00
6 changed files with 126 additions and 75 deletions

View File

@ -1,11 +1,7 @@
source "https://rubygems.org" source "https://rubygems.org"
# remove once added to jekyll dependencies gem "jekyll", "~> 4.4"
gem "csv" gem "minimal-mistakes-jekyll", "4.27.1"
gem "base64"
gem "jekyll", "~> 4.0"
gem "webrick"
gem "faraday-retry" gem "faraday-retry"
group :jekyll_plugins do group :jekyll_plugins do

View File

@ -7,63 +7,75 @@ GEM
bigdecimal (3.1.9) bigdecimal (3.1.9)
colorator (1.1.0) colorator (1.1.0)
concurrent-ruby (1.3.5) concurrent-ruby (1.3.5)
csv (3.3.2) csv (3.3.4)
em-websocket (0.5.3) em-websocket (0.5.3)
eventmachine (>= 0.12.9) eventmachine (>= 0.12.9)
http_parser.rb (~> 0) http_parser.rb (~> 0)
eventmachine (1.2.7) eventmachine (1.2.7)
faraday (2.12.2) faraday (2.13.1)
faraday-net_http (>= 2.0, < 3.5) faraday-net_http (>= 2.0, < 3.5)
json json
logger logger
faraday-net_http (3.4.0) faraday-net_http (3.4.0)
net-http (>= 0.5.0) net-http (>= 0.5.0)
faraday-retry (2.2.1) faraday-retry (2.3.1)
faraday (~> 2.0) faraday (~> 2.0)
ffi (1.17.1) ffi (1.17.2)
ffi (1.17.1-aarch64-linux-gnu) ffi (1.17.2-aarch64-linux-gnu)
ffi (1.17.1-aarch64-linux-musl) ffi (1.17.2-aarch64-linux-musl)
ffi (1.17.1-arm-linux-gnu) ffi (1.17.2-arm-linux-gnu)
ffi (1.17.1-arm-linux-musl) ffi (1.17.2-arm-linux-musl)
ffi (1.17.1-arm64-darwin) ffi (1.17.2-arm64-darwin)
ffi (1.17.1-x86-linux-gnu) ffi (1.17.2-x86-linux-gnu)
ffi (1.17.1-x86-linux-musl) ffi (1.17.2-x86-linux-musl)
ffi (1.17.1-x86_64-darwin) ffi (1.17.2-x86_64-darwin)
ffi (1.17.1-x86_64-linux-gnu) ffi (1.17.2-x86_64-linux-gnu)
ffi (1.17.1-x86_64-linux-musl) ffi (1.17.2-x86_64-linux-musl)
forwardable-extended (2.6.0) forwardable-extended (2.6.0)
google-protobuf (4.29.3) google-protobuf (4.31.0)
bigdecimal bigdecimal
rake (>= 13) rake (>= 13)
google-protobuf (4.29.3-aarch64-linux) google-protobuf (4.31.0-aarch64-linux-gnu)
bigdecimal bigdecimal
rake (>= 13) rake (>= 13)
google-protobuf (4.29.3-arm64-darwin) google-protobuf (4.31.0-aarch64-linux-musl)
bigdecimal bigdecimal
rake (>= 13) rake (>= 13)
google-protobuf (4.29.3-x86-linux) google-protobuf (4.31.0-arm64-darwin)
bigdecimal bigdecimal
rake (>= 13) rake (>= 13)
google-protobuf (4.29.3-x86_64-darwin) google-protobuf (4.31.0-x86-linux-gnu)
bigdecimal bigdecimal
rake (>= 13) rake (>= 13)
google-protobuf (4.29.3-x86_64-linux) google-protobuf (4.31.0-x86-linux-musl)
bigdecimal
rake (>= 13)
google-protobuf (4.31.0-x86_64-darwin)
bigdecimal
rake (>= 13)
google-protobuf (4.31.0-x86_64-linux-gnu)
bigdecimal
rake (>= 13)
google-protobuf (4.31.0-x86_64-linux-musl)
bigdecimal bigdecimal
rake (>= 13) rake (>= 13)
http_parser.rb (0.8.0) http_parser.rb (0.8.0)
i18n (1.14.6) i18n (1.14.7)
concurrent-ruby (~> 1.0) concurrent-ruby (~> 1.0)
jekyll (4.3.4) jekyll (4.4.1)
addressable (~> 2.4) addressable (~> 2.4)
base64 (~> 0.2)
colorator (~> 1.0) colorator (~> 1.0)
csv (~> 3.0)
em-websocket (~> 0.5) em-websocket (~> 0.5)
i18n (~> 1.0) i18n (~> 1.0)
jekyll-sass-converter (>= 2.0, < 4.0) jekyll-sass-converter (>= 2.0, < 4.0)
jekyll-watch (~> 2.0) jekyll-watch (~> 2.0)
json (~> 2.6)
kramdown (~> 2.3, >= 2.3.1) kramdown (~> 2.3, >= 2.3.1)
kramdown-parser-gfm (~> 1.0) kramdown-parser-gfm (~> 1.0)
liquid (~> 4.0) liquid (~> 4.0)
mercenary (>= 0.3.6, < 0.5) mercenary (~> 0.3, >= 0.3.6)
pathutil (~> 0.9) pathutil (~> 0.9)
rouge (>= 3.0, < 5.0) rouge (>= 3.0, < 5.0)
safe_yaml (~> 1.0) safe_yaml (~> 1.0)
@ -83,9 +95,11 @@ GEM
rubyzip (>= 1.3.0, < 3.0) rubyzip (>= 1.3.0, < 3.0)
jekyll-sass-converter (3.0.0) jekyll-sass-converter (3.0.0)
sass-embedded (~> 1.54) sass-embedded (~> 1.54)
jekyll-sitemap (1.4.0)
jekyll (>= 3.7, < 5.0)
jekyll-watch (2.2.1) jekyll-watch (2.2.1)
listen (~> 3.0) listen (~> 3.0)
json (2.9.1) json (2.12.0)
kramdown (2.5.1) kramdown (2.5.1)
rexml (>= 3.3.9) rexml (>= 3.3.9)
kramdown-parser-gfm (1.1.0) kramdown-parser-gfm (1.1.0)
@ -94,8 +108,15 @@ GEM
listen (3.9.0) listen (3.9.0)
rb-fsevent (~> 0.10, >= 0.10.3) rb-fsevent (~> 0.10, >= 0.10.3)
rb-inotify (~> 0.9, >= 0.9.10) rb-inotify (~> 0.9, >= 0.9.10)
logger (1.6.5) logger (1.7.0)
mercenary (0.4.0) mercenary (0.4.0)
minimal-mistakes-jekyll (4.27.1)
jekyll (>= 3.7, < 5.0)
jekyll-feed (~> 0.1)
jekyll-gist (~> 1.5)
jekyll-include-cache (~> 0.1)
jekyll-paginate (~> 1.1)
jekyll-sitemap (~> 1.3)
net-http (0.6.0) net-http (0.6.0)
uri uri
octokit (4.25.1) octokit (4.25.1)
@ -103,57 +124,55 @@ GEM
sawyer (~> 0.9) sawyer (~> 0.9)
pathutil (0.16.2) pathutil (0.16.2)
forwardable-extended (~> 2.6) forwardable-extended (~> 2.6)
public_suffix (6.0.1) public_suffix (6.0.2)
rake (13.2.1) rake (13.2.1)
rb-fsevent (0.11.2) rb-fsevent (0.11.2)
rb-inotify (0.11.1) rb-inotify (0.11.1)
ffi (~> 1.0) ffi (~> 1.0)
rexml (3.4.0) rexml (3.4.1)
rouge (4.5.1) rouge (4.5.2)
rubyzip (2.4.1) rubyzip (2.4.1)
safe_yaml (1.0.5) safe_yaml (1.0.5)
sass-embedded (1.83.4) sass-embedded (1.89.0)
google-protobuf (~> 4.29) google-protobuf (~> 4.31)
rake (>= 13) rake (>= 13)
sass-embedded (1.83.4-aarch64-linux-android) sass-embedded (1.89.0-aarch64-linux-android)
google-protobuf (~> 4.29) google-protobuf (~> 4.31)
sass-embedded (1.83.4-aarch64-linux-gnu) sass-embedded (1.89.0-aarch64-linux-gnu)
google-protobuf (~> 4.29) google-protobuf (~> 4.31)
sass-embedded (1.83.4-aarch64-linux-musl) sass-embedded (1.89.0-aarch64-linux-musl)
google-protobuf (~> 4.29) google-protobuf (~> 4.31)
sass-embedded (1.83.4-aarch64-mingw-ucrt) sass-embedded (1.89.0-aarch64-mingw-ucrt)
google-protobuf (~> 4.29) google-protobuf (~> 4.31)
sass-embedded (1.83.4-arm-linux-androideabi) sass-embedded (1.89.0-arm-linux-androideabi)
google-protobuf (~> 4.29) google-protobuf (~> 4.31)
sass-embedded (1.83.4-arm-linux-gnueabihf) sass-embedded (1.89.0-arm-linux-gnueabihf)
google-protobuf (~> 4.29) google-protobuf (~> 4.31)
sass-embedded (1.83.4-arm-linux-musleabihf) sass-embedded (1.89.0-arm-linux-musleabihf)
google-protobuf (~> 4.29) google-protobuf (~> 4.31)
sass-embedded (1.83.4-arm64-darwin) sass-embedded (1.89.0-arm64-darwin)
google-protobuf (~> 4.29) google-protobuf (~> 4.31)
sass-embedded (1.83.4-riscv64-linux-android) sass-embedded (1.89.0-riscv64-linux-android)
google-protobuf (~> 4.29) google-protobuf (~> 4.31)
sass-embedded (1.83.4-riscv64-linux-gnu) sass-embedded (1.89.0-riscv64-linux-gnu)
google-protobuf (~> 4.29) google-protobuf (~> 4.31)
sass-embedded (1.83.4-riscv64-linux-musl) sass-embedded (1.89.0-riscv64-linux-musl)
google-protobuf (~> 4.29) google-protobuf (~> 4.31)
sass-embedded (1.83.4-x86_64-cygwin) sass-embedded (1.89.0-x86_64-darwin)
google-protobuf (~> 4.29) google-protobuf (~> 4.31)
sass-embedded (1.83.4-x86_64-darwin) sass-embedded (1.89.0-x86_64-linux-android)
google-protobuf (~> 4.29) google-protobuf (~> 4.31)
sass-embedded (1.83.4-x86_64-linux-android) sass-embedded (1.89.0-x86_64-linux-gnu)
google-protobuf (~> 4.29) google-protobuf (~> 4.31)
sass-embedded (1.83.4-x86_64-linux-gnu) sass-embedded (1.89.0-x86_64-linux-musl)
google-protobuf (~> 4.29) google-protobuf (~> 4.31)
sass-embedded (1.83.4-x86_64-linux-musl)
google-protobuf (~> 4.29)
sawyer (0.9.2) sawyer (0.9.2)
addressable (>= 2.3.5) addressable (>= 2.3.5)
faraday (>= 0.17.3, < 3) faraday (>= 0.17.3, < 3)
terminal-table (3.0.2) terminal-table (3.0.2)
unicode-display_width (>= 1.1.1, < 3) unicode-display_width (>= 1.1.1, < 3)
unicode-display_width (2.6.0) unicode-display_width (2.6.0)
uri (1.0.2) uri (1.0.3)
webrick (1.9.1) webrick (1.9.1)
PLATFORMS PLATFORMS
@ -186,16 +205,14 @@ PLATFORMS
x86_64-linux-musl x86_64-linux-musl
DEPENDENCIES DEPENDENCIES
base64
csv
faraday-retry faraday-retry
jekyll (~> 4.0) jekyll (~> 4.4)
jekyll-feed jekyll-feed
jekyll-gist jekyll-gist
jekyll-include-cache jekyll-include-cache
jekyll-paginate jekyll-paginate
jekyll-remote-theme jekyll-remote-theme
webrick minimal-mistakes-jekyll (= 4.27.1)
BUNDLED WITH BUNDLED WITH
2.6.3 2.6.9

View File

@ -12,7 +12,7 @@ linkedin_username: matthewlamtran
permalink: /:year/:month/:title/ permalink: /:year/:month/:title/
show_excerpts: true show_excerpts: true
remote_theme: "mmistakes/minimal-mistakes@4.26.2" theme: minimal-mistakes-jekyll
minimal_mistakes_skin: dark minimal_mistakes_skin: dark
plugins: plugins:
- jekyll-include-cache - jekyll-include-cache

View File

@ -42,3 +42,11 @@ In the future, it would be good to buffer the packets going to and coming from T
Since the initial completion of this project, I've continued work on it and it has become something I'm quite proud of. Using task notifcations instead of binary semaphores improves speed, although I haven't measured by exactly how much. I did a massive refactor of the code to decouple the TinyUSB application from the Ethernet layer, creating portable `Eth` and `OASPI` classes in the process. To minimize copying, packets are now allocated and deallocated from a fixed size pool of maximum length packets. Doing this also made buffering packets between TinyUSB and the Ethernet layer easy. Since WSL doesn't come with RNDIS/ECM drivers, I also switched to NCM for true driverless use everywhere. Even more, I modified the HID interface to allow arbitrary OASPI register options, which even allows a webpage to access it using WebHID. As a proof of concept, I also implemented my `Eth` class in macOS using `libpcap`. Since the initial completion of this project, I've continued work on it and it has become something I'm quite proud of. Using task notifcations instead of binary semaphores improves speed, although I haven't measured by exactly how much. I did a massive refactor of the code to decouple the TinyUSB application from the Ethernet layer, creating portable `Eth` and `OASPI` classes in the process. To minimize copying, packets are now allocated and deallocated from a fixed size pool of maximum length packets. Doing this also made buffering packets between TinyUSB and the Ethernet layer easy. Since WSL doesn't come with RNDIS/ECM drivers, I also switched to NCM for true driverless use everywhere. Even more, I modified the HID interface to allow arbitrary OASPI register options, which even allows a webpage to access it using WebHID. As a proof of concept, I also implemented my `Eth` class in macOS using `libpcap`.
Currently I'm limited to about 6Mbps due to my use of USB FS. If a MAC-PHY with topology discovery ever becomes publicly available, I'll definitely build a new board with it and USB HS. If I get paid to do it though, it might have to be closed-source 😧. Currently I'm limited to about 6Mbps due to my use of USB FS. If a MAC-PHY with topology discovery ever becomes publicly available, I'll definitely build a new board with it and USB HS. If I get paid to do it though, it might have to be closed-source 😧.
## update 5-21-25
Well looks like `$DAYJOB` needed an in-house USB to 10BASE-T1S dongle. The full design will never be open-source, but I'll discuss high level details here. For the hardware, I switched to a microcontroller from the STM32U5 family since it has USB HS with a built-in PHY and USB PD. The not-yet-publicly-available MAC-PHY importantly supports topology discovery. To support powering 48V/5A devices, I added a beefy PoDL injector circuit and associated HSD. From a raw parts perspective, that board is one of my most expensive.
On the firmware side, the first change was supporting multiple microcontrollers, board configurations, and applications. I then expanded the HID RPC significantly to support the HSD and topology discovery controls. I also fixed the macOS `BIOCPROMISC` issue which was a capabilities advertisement issue. As you might guess, the intention of the `Eth` class was to also implement it on Windows, macOS, and Linux so I could build applications that run cross-platform. Without revealing too much, I can say that the custom protocol library I'm working on at `$DAYJOB` works seamlessly across embedded and desktop OSes.
In terms of performance, I can hit 9.5Mbps in CSMA/CD mode which is about the maximum one can expect for 10BASE-T1S. Topology discovery also works well after tweaking the PoDL circuit inductance a bit to fix ringing issues with some MAC-PHYs (Onsemi cough-cough). As my first Altium/mouse board in almost 4 years, I'm quite proud of it. If I ever leave `$DAYJOB` I might talk more about it, but this is where the open-source side of the story ends for now.

View File

@ -0,0 +1,30 @@
---
title: Server CoreOS Migration
date: 2025-5-21
categories: projects
excerpt: After my server's SSD died, I figured out how to use Fedora CoreOS to make redeployments almost trivial.
header:
teaser: /assets/img/2025/coreos.webp
---
[dragonlock2/matthewtran.com](https://github.com/dragonlock2/matthewtran.com)
I can't believe it's been almost nine months since my last post. Nine months without designing another open-source board. Nine months of procrastination since I made the decision to block future projects on forcing myself to update my resumé and LinkedIn first. For the first few months, I can say that I was moving between places, busy with self-inflicted work at `$DAYJOB`, and coming to terms with the election results. For the last few months, I can't really explain why I've just been frozen. Perhaps it's the political climate. It's terrifying how backwards we seem to be moving and how idiotic and/or self-serving a significant portion of the population is. Perhaps it's that some of those who voted for the current administration were friends and family that I now have to keep a distance from. Regardless, it's time for a change, I hope.
To kick off some of this change, I'll do a post about software. It'll be short, but it'll at least be something. Generally speaking, I don't consider pure software worth writing about but perhaps I'll change my mind.
## Ubuntu Era
Upon moving this blog to Github Pages, my server was finally free for experimentation. If you trace back the commits, it all started as a fresh Ubuntu Server install and learning how to containerize services, except for WireGuard. Under one Docker Compose instance, I added multiple services including NGINX, Monero, and Minecraft. It might not have been particularly secure nor easy to backup and restore, but it worked and services would come up on boot. At some point, I got all containers to stop gracefully so I could restart things faster.
After moving to my new place, it was time to re-evaluate everything. To focus the server on just containerized services, I switched to an OpenWrt router and moved WireGuard onto it. This also had the benefit of being able to script the entire configuration of a freshly wiped router. It was also at that point that I learned about IPv6 delegated prefixes and could finally correctly allow IPv6 connections to my services.
From there, I focused on hardening my server's security. Since Ubuntu Server didn't support TPM-backed FDE, I had to use Ubuntu Desktop. Since Docker runs containers as actual root by default, I had to update each container to run as a non-root user and dropped all root capabilities. In case of container breakout, I also mapped each non-root user to a non-existent user on the host. To keep file permissions sane, the container users would map to an actual group on the host. It was also at this time that I added an SMB share to get Time Machine backups working. Once everything was working, I wiped everything to check that my setup scripts worked.
## CoreOS Era
A few months in, my server failed to reboot properly. It took a bit too long to realize the SSD was faulty. Seeing the mess that was my setup script, I decided to look into better options. I came upon Fedora CoreOS which promised provisioning of an entire server from a single Ignition configuration file. Even more, I could reinstall the OS without touching additional partitions and drives if needed. Recovering from any drive failure would be almost trivial. I started by generating the Ignition file that would properly configure and mount encrypted LUKS drives. Since Ignition doesn't currently support BTRFS RAID, I had to use mdadm RAID.
Instead of Docker Compose, CoreOS prefers Podman and Quadlets. Podman runs non-root by default and each user gets their own set of containers and network. Quadlets essentially allow control of containers through systemd. Podman's default security model makes what I was attempting to accomplish with Docker Compose much easier. Instead of making a network for each group and manually messing with UIDs and GIDs, I could simply add a user for each group of services and their networks would be isolated by default. I also don't have to mess with running as non-root in the containers as Podman maps the root user in the container to the user that started it. As a last barrier, folder permissions isolate users from accessing other user's data. All of this ensures that even in the event of container takeover and breakout, there isn't much an attacker can do.
Last but not least, I added scripts to update container images and also backup and restore data. As a final test, I wiped everything, reprovisioned, and restored the backup. At the end of the day, I finally have a system I can be sure is not only secure but also easy to restore when needed.

BIN
assets/img/2025/coreos.webp Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 20 KiB