programs we use

System Util

ncdu - TUI command line disk usage analyzer

also lets you delete stuff.

# analyze current working dir, stay in the same filesystem
ncdu -x

Bscp - Secure and efficient copying of block devices

Bscp copies a single file or block device over an SSH connection, transferring only the parts that have changed.

In other words, it handles the edge case where rsync fails.

# The default BLOCKSIZE is 65536 (64 KiB).
# The default HASH algorithm is sha256 (SHA-256).
bscp SRC HOST:DEST [BLOCKSIZE] [HASH]

link: https://bscp.njh.eu/

Midnight Commander - TUI file manager

mc

Try using it over the network too, once you have it open:

cd sftp://user@host

or if that doesnt work

cd sh://user@host

selectdefaultapplication - Set MIME type handlers and finally have it work

https://github.com/magnus-ISU/selectdefaultapplication which is a fork of https://github.com/sandsmark/selectdefaultapplication.

This is the only tool we’ve found that actually lets us easily set all the things we need to to get programs to open the way we want.

grep & find - search stuff

these don’t really need introduction. here’s some recipes.

# recursive grep but better because it ignores dev nodes
find /path -print0 -type f | xargs -0 grep 'pattern'

# same thing but don't traverse mounts
find /path -print0 -xdev -type f | xargs -0 grep 'pattern'
# filter the contents of a single directory (better than `ls | grep`)
# -mindepth 1 prevents printing parent dir
# -maxdepth 1 prevents recursing deeper
find /path -mindepth 1 -maxdepth 1 [args...]
# Learn what changes to the filesystem a different process is making
# This detects files with modification dates newer than the checkpoint
touch /tmp/checkpoint
some_command_you_want_to_learn_about
find / -newer /tmp/checkpoint

# Same thing but for access time. Only works if you're using atime as a mount
# option, and not relatime or noatime.
touch /tmp/checkpoint
some_command_you_want_to_learn_about
find / -neweraa /tmp/checkpoint

ssh-keygen - ssh key management for OpenSSH

# change the passphrase on a private key (or remove it)
ssh-keygen -p -f path/to/key

sg_vpd - check hard drive features, even over usb.

This is in sys-apps/sg3_utils.

via https://superuser.com/questions/1740543/no-trim-discard-with-a-sata-ssd-connected-through-an-uasp-enabled-usb-adapter

Apparently this can lock up some USB drives. be careful. Works on my WD Passport.

# check features.
vi@localhost ~> sudo sg_vpd -p lbpv /dev/sdc
Logical block provisioning VPD page (SBC):
  Unmap command supported (LBPU): 1
  Write same (16) with unmap bit supported (LBPWS): 0
  Write same (10) with unmap bit supported (LBPWS10): 0
  Logical block provisioning read zeros (LBPRZ): 0
  Anchored LBAs supported (ANC_SUP): 1
  Threshold exponent: 0 [threshold sets not supported]
  Descriptor present (DP): 0
  Minimum percentage: 0 [not reported]
  Provisioning type: 1 (resource provisioned)
  Threshold percentage: 0 [percentages not supported]

# we have unmap supported. set it up so we can trim our SMR drive
echo unmap | sudo tee /sys/block/sdc/device/scsi_disk/*/provisioning_mode

Multimedia

cdw - TUI CD/DVD/Bluetooth reader/writer

cdw

mpv - CLI media player

mpv <file|url|whatever>

swiss army knife of playing media. if you install yt-dlp too it can even play youtube videos directly!

cmus - TUI music player

good tutorial in

man 7 cmus-tutorial

works well even with networked filesystems!

Data Processing

yq (tomlq, etc.) - jq wrapper for YAML/XML/TOML documents

https://github.com/kislyuk/yq

Basically, use a jq query, but run yq or tomlq instead of jq. It’ll parse the input, convert it to json, then feed it to jq. By default it will output whatever jq returns as-is, but you can ask it to convret it back to the original format too.

fq - like jq for binary data

https://github.com/wader/fq

vi@localhost ~/p/music> fq -d wav . softsaw.wav
     |00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11|0123456789abcdef01|.{}: softsaw.wav (wav)
0x000|52 49 46 46                                          |RIFF              |  id: "RIFF"
0x000|            0d 02 00 00                              |    ....          |  size: 525
0x000|                        57 41 56 45                  |        WAVE      |  format: "WAVE" (valid)
0x000|                                    66 6d 74 20 10 00|            fmt ..|  chunks[0:2]:
0x012|00 00 01 00 01 00 00 7d 00 00 00 7d 00 00 01 00 08 00|.......}...}......|
*    |until 0x214.7 (end) (521)                            |                  |

Debian

apt & aptitude tricks

apt, apt-get, aptitude are all frontends to the debian package database. Here’s some commands for dealing in them.

# Install a package from backports
apt install -t <release>-backports somepackage


# If you decide using backports was a mistake actually, List all packages
# installed from backports. Note dependencies are marked with "A" for automatic
aptitude search '?narrow(~i, ~Abackports)'

Gentoo

bump-go - sam’s script for vendoring go packages

https://github.com/thesamesam/sam-gentoo-scripts/blob/main/maint/bump-go

# run this on the thingy that builds binary packages to
# rebuild the metadata index
emaint binhost --fix

eclean - clean up source/binary package cache

# delete source for anything without an ebuild in any repos
eclean-dist

# delete source for anything that isn't installed
eclean-dist -d

# delete binary packages from /usr/portage/packages for anything
# without an ebuild
eclean-pkg

# delete binary packages from /usr/portage/packages for anything
# that isn't installed
eclean-pkg -d

### clear out old kernels
eclean-kernel

equery - query package database & tree

# verify integrity of files of all installed packages
equery check -o '*/*'

# list installed packages with no maintainer
qlist -I \
   | xargs equery -CN m -m 2>/dev/null \
   | sed -n '/^ \*/h; /None specified/{ x; p }'

qlist - list packages, or files owned by package

# list all installed packages
qlist -I

Git

Sublime Merge - GUI git frontend

in general, it’s a good program for dealing with merges/rebases. we use it regularly. Free version only has light mode :/.

rewrite author while keeping commit dates

git -c rebase.instructionFormat='%s%nexec GIT_COMMITTER_DATE="%cD" GIT_AUTHOR_DATE="%aD" git commit --amend --no-edit --reset-author' rebase -f <commit/branch before wrong author and email, or --root to rebase all>

sign and signoff all commits since some rev

for when i forget to signoff my stuff when doing gentoo package maintenance.

git rebase --exec 'git commit --amend --no-edit -n -S -s' origin/dev

via stackoverflow

git-backdate - helps you to change the date of one or multiple commits to a new date or a range of dates.

git-backdate

Code

icdiff - ergonomic side-by-side diff tool

This is an easier to use alternative to diff for the purposes of side-by-side diffing. It’s not interested in replacing diff for automation purposes.

https://www.jefftk.com/icdiff

icdiff -r /path/to/dir1 /path/to/dir2

tokei - lines-of-code counter

https://github.com/XAMPPRocky/tokei

vi@localhost ~/p/artemis.sh (mistress)> tokei
===============================================================================
 Language            Files        Lines         Code     Comments       Blanks
===============================================================================
 JavaScript              1           93           63           17           13
 JSON                    2           11           11            0            0
 Nix                     1           63           44            6           13
 Ruby                    1          119           90           11           18
 Sass                    4          162          118           14           30
 Shell                   3           86           73            4            9
 Plain Text              7          498            0          408           90
 YAML                    1           43           21           17            5
-------------------------------------------------------------------------------
 HTML                   11         2323         2209           97           17
 |- CSS                  5          126          114           12            0
 |- JavaScript           2          661          576           40           45
 (Total)                           3110         2899          149           62
-------------------------------------------------------------------------------
 Markdown               98        10262            0         7307         2955
 |- BASH                 2           22           21            0            1
 |- Python               1           19            6            9            4
 (Total)                          10303           27         7316         2960
===============================================================================
 Total                 129        13660         2629         7881         3150
===============================================================================

Irssi Scripts

hlscroll - scroll through highlighted messages

Not our script, but we modified it slightly to make it work with modern irssi. Put it in ~/.irssi/scripts/autorun/. Read the script for how to set up keybinds

hlscroll.pl