Apparently this morning's project is wget’ing a bunch of retired web pages from an old WNYC show (Studio 360), pulling out the links to mp3s, and downloading them. All because the podcast feed only goes back three years instead of covering all 20+ years.
Why? Because there are interesting conversations.
Looks like The Fishko Files are after that.
If you care about game preservation this non-profit organization that works on preserving games in Japan really needs money.
Website: https://www.gamepres.org/en/
Downloading a bunch of crap today.
* TGOD Floppy Collection from archive.org (https://archive.org/details/TGODFloppyCollection)
* Kiwix ZIMs of Wikipedia, Gutenberg, iFixit, and a bunch of Stack Exchanges
Surprising just how *small* a lot of these are.
I've got 20 HDDs connected to 5 backplanes. The backplanes each have a SFF-8087 connector.
What type of controller do I want/need?
(And how many)
Bonus question:
Any suggestions for the right motherboard? (Up to E-ATX)
Приведение в порядок моих файловых архивов несколько сократилось благодаря утилите backdown, обыскивающей указанные места на предмет дубликатов: https://github.com/Canop/backdown
Если у вас где-то образовались слабоорганизованные свалки из файлов с нескольких систем, которые вы свалили на другой носитель не разбираясь, это отличный префильтр перед тем, как осознанно сортировать то, что останется.
Долго, зато автоматически.
does someone have a file server hosted at home (or privately enough that it is YOURS) and wants to archive all Linus Tech Tips Floatplane Exclusives??? (in a way I could still download/access them if I want to)
I really need some storage space :'D and I want to archive this forever if possible
it's 154GB of 1080p30fps videos, all the Floatplane exclusives from when they started being a thing, until 20th February (I really could only afford one month so some recent vids are already missing)
(please boost for maximum reach 
)
https://mastodon.ml/@mintbug/114086408290299199
Написал скрипт, который принимает https-ссылку на пост, получает ссылки на вложения с помощью #toot, преобразует https-ссылку в fedi-ссылку и сохраняет вложения в директорию с именами вида `<fedi-ссылка> <номер вложения>.ext` (см. иллюстрацию). Слеши в fedi-ссылке заменены на знак `∫`, потому что ~~а почему бы и нет~~ я ещё не настолько сошёл с ума, чтобы использовать слеши в именах файлов, поэтому их нужно на что-то заменить.
Запуск `fedi-save (wl-paste)` повесил на хоткей в #sway; открытие ссылки, содержащейся в имени файла (`fedi-save --open <path>`) повесил на хоткей в #yazi.
![```fish
#!/usr/bin/env fish
argparse o/open -- $argv
or return
set dir ~/Downloads/fedi
function parse --argument-names uri
string match -rg -- 'https?://([^/]+)/@([^/@]+)@?([^/]*)/?([[:digit:]]*)' $uri
end
function convert --argument-names instance user user_instance toot
echo fedi:∫∫{$user}@(test $user_instance && echo $user_instance || echo $instance)∫$toot
end
function save --argument-names uri out
aria2c --dir $dir --out=$out --continue=true $uri
end
if set -q _flag_open
for path in $argv
string replace --all -- ∫ / (string match -rg -- '.*(fedi:∫∫[^∫]+∫[[:digit:]]+).*' $path)
end | fedi-hack --stdin
return
end
set parts (parse $argv[1])
or return
set status_id $parts[4]
set name (convert $parts)
set count 0
for attachment in (toot status --json $status_id | jq -r '.media_attachments[] | .url')
save $attachment $name(test $count -gt 0 && echo " $count" || echo)(path extension $attachment)
set count (math $count + 1)
end
```](https://mastodon.ml/system/media_attachments/files/114/111/150/682/295/646/original/0f2f04f4085de593.png "```fish
#!/usr/bin/env fish
argparse o/open -- $argv
or return
set dir ~/Downloads/fedi
function parse --argument-names uri
string match -rg -- 'https?://([^/]+)/@([^/@]+)@?([^/]*)/?([[:digit:]]*)' $uri
end
function convert --argument-names instance user user_instance toot
echo fedi:∫∫{$user}@(test $user_instance && echo $user_instance || echo $instance)∫$toot
end
function save --argument-names uri out
aria2c --dir $dir --out=$out --continue=true $uri
end
if set -q _flag_open
for path in $argv
string replace --all -- ∫ / (string match -rg -- '.*(fedi:∫∫[^∫]+∫[[:digit:]]+).*' $path)
end | fedi-hack --stdin
return
end
set parts (parse $argv[1])
or return
set status_id $parts[4]
set name (convert $parts)
set count 0
for attachment in (toot status --json $status_id | jq -r '.media_attachments[] | .url')
save $attachment $name(test $count -gt 0 && echo \" $count\" || echo)(path extension $attachment)
set count (math $count + 1)
end
```")
Just discovered ArchiveBox — FOSS, self-hosted internet archiving.
The way the web is going, with the US government redacting and outright erasing historic content, publishers segmenting content by region (and also sometimes redacting/censoring it), and CloudFlare shitting all over everything, I think it's time for me to start my #archiving and #DataHoarding journey.
- I've setup a VNET thick jail on my FreeBSD NAS.
- The jail has its own IP address on my LAN.
- I declared a devfs ruleset to unhide /dev/tun* for the VNET jail.
- I installed Wireguard in the jail.
- I enabled Wireguard with a ProtonVPN configuration.
- I installed qbittorrent-nox and configured it to use the Wireguard interface.
I now have a home ISP-proof qBittorrent setup with which to torrent Anna's Archive.
I probably spent half the morning reinventing the wheel, but I couldn't find a nice, tidy dataset containing the #J6 defendants so I spent some time cleaning and tidying the list on npr.org
Since I did this, maybe it helps someone else. It's just a spreadsheet with 1500 rows or so. It could be parsed further (especially the parts about the charges and follow-up).
https://drive.google.com/drive/folders/1dIhF8nI4c4lfMF-LumDyZWq5D_3YMGhF?usp=sharing
Syncing the last two good Wikimedia XML data dumps. It's been almost two days. It was at 96% 10 hours ago. The bar keeps raising because it hasn't accounted for all the files that are going to be transferred yet lol.
Amazon will remove the ability to download the ebooks for Kindle at the end of the month. So if you ever close your amazon account, you'll no longer be able to access the books you had bought.
Let's fix that
1. Bulk Exporter: https://github.com/treetrum/amazon-kindle-bulk-downloader
2. Calibre to manage books https://calibre-ebook.com/download
3. Calibre plugin to remove DRM: https://github.com/noDRM/DeDRM_tools/releases
Source: https://bsky.app/profile/remysharp.com/post/3lihtiq2rqc22
Linear Tape Open (LTO) AKA Ultrium is a fascinating tape technology. The drive itself seems stupidly simple but seems to have incorporated all the lessons learned from decades of tape drive design and usage.
Plus it sounds cool when running. :)
Data hoarding question from an ally: What things on the internet are in most need of saving?
I have backup tapes and hard drives and the capability of providing more. I want my resources to be available for the greater good.
I’m aware of people backing up YouTube channels that are at risk as well as archive.org and government data.
What else is at risk?
I’m also open to do tape backups if someone want to collab. I have an LTO-4 drive.
A learning experience for me, so I thought I'd share my knowledge.
Title: How I Host My Own #Kiwix Archive
Subtitle: My own little corner of the internet dedicated to keeping information free, open and accessible.
I had another brain dump, this time about data sovereignty and "backing up the internet".
A stack of LTO-4 tapes. Around 3.5-ish TB. One of those tapes has the Geocities archive on it. :)
Not shown is the 4TB hard drive that I will then dupe to another drive to be stored offsite, satisfying the 321 rule.
I intend to publish a catalog of the contents online along with hashes, minus anything personal.
Our PV Spec:
24.14kWDC tied to twin 10k SolarEdge inverters
4.8kWDC is DC coupled to the XW systems
91x Solar Modules, average 330w each
eGauge Circuit-Level Monitoring System
3x Schneider XW6848 Inverters - AC coupled together for power sharing
1x 20kWh DIY LFP battery
1x 5kWh Fortress LFP battery
1x 40kWh AGM Unigy II Battery
1x 60kWh battery in EV with 12v-48v DC converter for charging house batteries
Total 28.94kWDC PV Capacity
Ground mount installed 2017, roof mount 2019. System paid off Oct 2023. The utility pays me!
We bank approximately 5MWh each summer, and draw on it in winter via 1:1 net metering in Maryland.
Additional efficiencies include HP water heater, inverter mini-split, dual-stage central HVAC, and blown insulation.
We use 25MWh annually, which is a ton. But #datahoarding storage drives burn watts!!
Want my tool? You can have my tool.
