Latest 10 recent news (see index)

December 24, 2018

The Advent of Void: Day 24: 'Twas the Night Before Christmas

‘Twas the Night Before Christmas,

When all through the cluster, not a build was running, not even a revbump. The packages were all signed by the server with care, in hopes that an upgrade soon would be there.

The maintainers were nestled all snug at their desks, while visions of waterfalls danced in their heads; and Gottox in his home and I in mine, had just settled our brains for a long winter break; when out on the frontends there arose such a clatter, I sprang to my terminal to see what was the matter.

Away to my console I flew like a flash, called for my tmux and threw open the logstash. The status page on my screen lent a luster of midday to errors below, when what to my wondering eyes should appear, but a stuck qt5 build and 8 staged SONAMEs.

With a large webengine so slow and so massive, I knew in an instant it must be chromium. More rapid than runit my commands to the system: “Now Ansible! now Grafana! now buildbot and xbulk! On rindex! on snooze! on, ext4 and repo-cleaner! To the top of the graph to the top of the stack now build, build away, build away all!

As disk platters that before the online scrub fly, when they meet with an obstacle always retry, so up to the servers the tasks they all flew, with their mirror of toys and BDFL, too.

And then in a twinkling, I saw in reply the server starting and launching of each single build. As I drew in my buffers, and was closing up shop, down the pipe the json came with a stop. It was padded with null bytes from its start to its end, and its braces were all tarnished with codepages and parity bits; a slice of package it contained in its body, and it looked like a startup just opening its reply. The descriptors how they twinkled, the keys how merry!

A close of a brace and a retcode of 0 soon gave me to know I had nothing to dread. Buildbot said not a word but went straight to its work, and filled the mirrors with packages. Waiting for but a moment before confirming my intent, the BuildBot went on its way; indexes sprang to their clients, and to the fleet I did signal and away they all built like the sight seldom seen. But before I C-b d I exclaim to all users:

Happy Christmas to all, and to all a Good Night!

–maldridge

We hope you’ve enjoyed the Advent of Void. This is a fun project that we’ve done now for two years to try and shed some light on lesser known packages. We’ve got a few honorable mentions that may make their way to the feed over the next few days, but this post concludes the series. Void is a volunteer effort that we enjoy sharing with you, and hopefully you’ve learned some neat new tips and tricks these last 24 days. Our volunteers don’t just make packages and manage infrastructure, we also write fun posts like these (though I recognize I’m a quite poor poet). If you want to become involved and help us keep Void going, join us in #xbps on freenode, but maybe wait until after the holidays to do so.

Enjoy the winter season and the updates that just keep rolling!

December 24, 2018

The Advent of Void: Day 24: gromit-mpx

Have you ever wanted to doodle on a screenshot? Ever? Just a single underline, or an arrow, or something to help point out something about a screenshot? Or maybe during a presentation, on the fly, suddenly you want to point out a particular thing… wouldn’t it be nice to have a tool to help?

Turns out, there is one, a very simple tool to do this. I use it every month.

Gromit-mpx is simple. <F9> to start being able to draw, Shift-Ctrl-<F9> to clear everything you drew, and Ctrl-<F9> to stop being able to draw (but leave everything on the screen). Just make sure gromit-mpx is running in the background, and you are set for all your drawing and annotating needs. Even people like me, who just run dwm and not with any compositor, I can annotate my screen quickly.

But with just a few lines, I can clarify what I meant to share. I won’t be winning any art awards, but it works.

Do note that the annotations stick around, even after you scroll the window, or switch to a different screen entirely.

So keep gromit-mpx around, it might be useful someday.

For more information, there is always the gromit-mpx(1) manpage.

December 24, 2018

Void Linux at 35c3

2018 has been an intense year for Void. There were many ups and downs. To wrap up the year we’re meeting at the 35c3.

To meet other Voiders and chat about related and unrelated topics there’s the Void Linux Assembly

Also, on day 1 (2018-12-27) 19:00, @Gottox is doing his annual talk about The State Of the Void.

After the talk, around 20:00, Void is inviting you to a Void User Meetup. We’ll meet at the Open Infrastructure Orbit in front of the lecture area, find the nearest bar, grab a Beer or a Tschunk, and get into interesting conversations.

I’m looking forward to a great event with an awesome community!

–Gottox & the Void Linux Team

December 23, 2018

The Advent of Void: Day 23: extrace

Today’s tool is extrace(1), a small, simple and powerful tool to trace program executions. By default extrace(1) will look at system wide executions, alternatively it can start a new process or start looking for executions in child processes of a specified pid. The output is a tree like structure including the process id, the command and its arguments, additional flags can add the environment of the process, the executing user or even the exit status and runtime duration.

The following example is the truncated extrace(1) output of checking the dhcpcd(8) man page and then executing dhcpcd -k to release all interfaces followed by executions from dhcpcd hooks.

$ extrace
    911 man dhcpcd
      912 less -T /tmp/man.XXXXH357hT /tmp/man.XXXXEOs5Na
    930 doas dhcpcd -k
    930 dhcpcd -k
      932 /bin/sh /usr/libexec/dhcpcd-run-hooks
        933 rm -f /run/dhcpcd/resolv.conf/br0.ra
          936 sed -n 's/^domain //p' br0.dhcp
          938 sed -n 's/^search //p' br0.dhcp
          941 sed -n 's/^nameserver //p' br0.dhcp
        943 cat /etc/resolv.conf.head
        944 cmp -s /etc/resolv.conf /run/dhcpcd/resolv.conf.br0.ra
        945 cat /run/dhcpcd/resolv.conf.br0.ra
        946 rm -f /run/dhcpcd/resolv.conf.br0.ra
        947 chmod 644 /etc/resolv.conf
        948 rm -f /run/dhcpcd/resolv.conf.br0.ra
        950 sed '/^# Generated by dhcpcd/,/^# End of dhcpcd/d' /etc/ntpd.conf
        951 cmp -s /etc/ntpd.conf /run/dhcpcd/ntp.conf.br0.ra
        952 rm -f /run/dhcpcd/ntp.conf.br0.ra
		...

Here is another example where we trace the executions by make into a file, with the -t flag extrace(1) adds the exit status and the duration.

mblaze$ extrace -t -o make.extrace make
cc -Wall -Wno-switch -Wextra -g -O2 -fstack-protector-strong -D_FORTIFY_SOURCE=2   -c -o msed.o msed.c
cc   msed.o blaze822.o mymemmem.o mytimegm.o seq.o slurp.o  -lrt -o msed
mblaze$ cat make.extrace
1992+ make
  1993+ uname
  1993- uname exited status=0 time=0.000s
  1994+ cc -Wall -Wno-switch -Wextra -g -O2 -fstack-protector-strong '-D_FORTIFY_SOURCE=2' -c -o msed.o msed.c
    1995+ /usr/lib/gcc/x86_64-unknown-linux-gnu/8.2.0/cc1 -quiet -D '_FORTIFY_SOURCE=2' msed.c -quiet -dumpbase msed.c '-mtune=generic' '-march=x86-64' -auxbase-strip msed.o -g -O2 -Wall -Wno-switch -Wextra -fstack-protector-strong -o /tmp/ccDnNZ91.s
    1995- /usr/lib/gcc/x86_64-unknown-lin exited status=0 time=0.212s
    1996+ as --64 -o msed.o /tmp/ccDnNZ91.s
    1996- as exited status=0 time=0.019s
  1994- cc exited status=0 time=0.234s
  1997+ cc msed.o blaze822.o mymemmem.o mytimegm.o seq.o slurp.o -lrt -o msed
    1998+ /usr/lib/gcc/x86_64-unknown-linux-gnu/8.2.0/collect2 [...]
      1999- /usr/bin/ld exited status=0 time=0.027s
    1998- /usr/lib/gcc/x86_64-unknown-lin exited status=0 time=0.029s
  1997- cc exited status=0 time=0.031s
  2000+ /bin/sh -c 'test -n "$SOURCE_DATE_EPOCH" || BUILDDATE=$(date '\''+ (%Y-%m-%d)'\''); \'$'\n''    printf '\''#!/bin/sh\nprintf "User-Agent: mblaze/%s%s\\n"\n'\'' \'$'\n''        "$({ git describe --always --dirty 2>/dev/null || cat VERSION; } | sed '\''s/^v//'\'')" \'$'\n''       "$BUILDDATE" >museragent'
    2001+ date '+ (%Y-%m-%d)'
    2001- date exited status=0 time=0.000s
      2004+ sed 's/^v//'
        2005+ git describe --always --dirty
        2005- git exited status=0 time=0.008s
      2004- sed exited status=0 time=0.009s
  2000- /bin/sh exited status=0 time=0.012s
  2006+ chmod +x museragent
  2006- chmod exited status=0 time=0.001s
1992- make exited status=0 time=0.286s

December 22, 2018

The Advent of Void: Day 22: autossh

SSH. The very use of the acronym screams “Knows which end of the computer needs to be plugged in.” It can be used to confirm your computers are working, to transfer files (two ways!), as a SOCKS proxy (didn’t know this? Go read the man page, option -D, it’s brilliant), and even as a poor man’s VPN.

Poor man’s VPN?

Well, I have a computer that lives behind a firewall. I can’t adjust the firewall. But I want to use that computer. Legally, mark you, I have permission from the network owners, it’s just not practical to adjust the firewall.

What to do?

Well, set up an ssh tunnel of course! In this case, localhost is the target computer, and the other hostname is a place I can access more readily than localhost.

$ ssh -NR $portA:localhost:22 nbis.gov

But problems. My local network isn’t the best, and sometimes when there is a power outage my computer comes up first. I could just create a runit service to make sure the ssh connection is running…. and I did, for a while, but when it failed it failed very badly. Finally, I settled on autossh. What does autossh do? Automatically runs an SSH command. You can configure a lot about how it does this, how many times it tries, how long it waits between tries, what the backoffs are, everything like that. So, I combined the best of both worlds: autossh (which should never crash) wrapped in a user service with runit (which should always remember to start autossh)

#!/bin/sh
[ -r ./conf ] && . ./conf
: ${portA:=2222}
: ${targetHost:=localhost}
exec autossh -M 0 -NR ${portA}:localhost:22 $targetHost

And my problem is (mostly) solved! So, in summary: SSH is an amazing tool, and autossh helps you keep your ssh tunnels alive.

One point about configuration, from the manpage:

Other than the flag to set the connection monitoring port, autossh uses
environment variables to control features. ssh seems to be still
collecting letters for options, and this seems the easiest way to avoid
collisions.

and there are 13 environment variables right now, so plenty you can choose to configure.

For more information, examples, and to see the list of relevant environment variables, please read the autossh(1) manpage.

December 21, 2018

The Advent of Void: Day 21: fex

fex is a simple yet powerful field extraction tool for working with text.

If you spend enough time in a shell fiddling with code or data, you inevitably want to pull fields out of some text you have. For example, say you want the hex sha256sum of a file, minus the name:

$ sha256sum launch-codes
a0cd7db7343bed89416660b4f92c43fe7b556439daa2e26e9844ce82491191c6  launch-codes

You could write this output to a file and remove the filename from it with a text editor, or use cut or awk. However, fex can save you a bit of time by extracting the first field containing the checksum for you:

$ sha256sum launch-codes | fex 1
a0cd7db7343bed89416660b4f92c43fe7b556439daa2e26e9844ce82491191c6

Or, say you want a list of all home directories from passwd. To do this, we ask fex to split by colon and select the sixth field:

$ fex </etc/passwd :6
/root
/dev/null
/var/lib/colord
/var/empty
...

Or maybe we want gid:login pairs to work with:

$ fex </etc/passwd ':{4,1}'
0:root
99:nobody
999:colord
22:dbus
...

Here we can see that fex will keep the separator we split by, letting us reduce the data to only what we want while maintaining its format. And, it allows us to rearrange fields using {N,M,...} selectors.

You can alse use fex to pluck fields by narrowing the selection with different split characters. So let’s say we want to get an idea of what the most common pairs of ‘subject: verb’ are in commit messages for void-packages – maybe to see which packages receive the most updates.

To do this, we’ll use git to search for commit message subjects with a colon and pipe that to fex. Using fex, we’ll write our first selector (:1) to take the first field behind the colon. We’ll then use a second selector (:2 1) to take the text after the colon, split that by spaces, and select the first word from that. With this, we get a list of subjects and verbs we can sort and count:

$ git log --pretty=%s --grep ':' | fex ':1' ':2 1' | sort | uniq -c | sort -rh | head
    328 youtube-dl update
    206 xbps-git bump
    160 git update
    151 Adapta update
    149 ImageMagick update
    143 exiftool update
    142 firefox update
    136 kernel update
    133 rpi-kernel update
    129 python-setuptools update

From that, we can see that updates make up all but one of the top ten pairs, with xbps-git bumps being the second in line after youtube-dl updates. This only covers basic use of fex, too – you can also select field ranges, fields matching a regular expression, and combine all of these.

There are lots of ways to use fex in day to day data munging, programming, and writing your own tools. For many field extraction tasks, fex allows you to easily get the data you want without writing small awk programs or messing with cut. Plus, it’s just fun to use.

For more information and examples, please read the fex(1) manpage.

December 20, 2018

The Advent of Void: Day 20: xsnow

If you’re dreaming of a white christmas, dream no more with the quick and easy installation of xsnow!

This small utility is best suited to window manager environments that do not tile, and will cause snow to fall on your desktop. The snow will collect on top of your windows and you’ll even have periodic visits from Santa flying across your desktop.

Its a quick and fun utility, and worth a bit of Christmas cheer as we go into the final days of the Void Advent.

December 19, 2018

The Advent of Void: Day 19: upx

Code golf is trying to solve a problem in the fewest lines of code. I like to think there’s a similar thing for trying to build a complete working system in the fewest number of bytes. You can get pretty far using projects like busybox to replace large parts of the system with a single binary that provides shared functionality, but to get the last bit of the way, its often necessary to shrink the remaining binaries as well. That’s where upx comes in.

The Ultimate Packer for eXecutables does exactly what its name implies. It packs executables to make them smaller. This is done with compression which is undone on the fly when a binary is called.

Lets look at a quick example:

$ ls -lh
total 128K
-rwxr-xr-x 1 maldridge maldridge 127K Dec 19 00:03 ls

This is the standard ls that ships with Void and is already pretty small. Lets make it smaller!

$ upx ls
                       Ultimate Packer for eXecutables
                          Copyright (C) 1996 - 2018
UPX 3.95        Markus Oberhumer, Laszlo Molnar & John Reiser   Aug 26th 2018

        File size         Ratio      Format      Name
   --------------------   ------   -----------   -----------
    129760 ->     60624   46.72%   linux/amd64   ls

Packed 1 file.

As you can see, the file is now only 60k in size. A pretty good start!

Lets take a look now at a Go executable. Go is known for generating large binaries and upx can help shrink them down to size.

For this example we’ll use the netauthd binary which forms part of Void’s server authentication system.

$ ls -lh
total 17M
-rwxr-xr-x 1 maldridge maldridge 17M Dec 19 00:08 netauthd

We can get part of the way there by stripping symbols out of the binary, since Go builds those in by default:

$ strip netauthd
$ ls -lh
total 13M
-rwxr-xr-x 1 maldridge maldridge 13M Dec 19 00:10 netauthd

Now to see how much further upx can get us:

$ upx netauthd
                       Ultimate Packer for eXecutables
                          Copyright (C) 1996 - 2018
UPX 3.95        Markus Oberhumer, Laszlo Molnar & John Reiser   Aug 26th 2018

        File size         Ratio      Format      Name
   --------------------   ------   -----------   -----------
  12724112 ->   4714692   37.05%   linux/amd64   netauthd

Packed 1 file.

The netauthd file is now just 4.5M, quite a bit smaller than the 17M starting size.

UPX can be a great tool to keep in your embedded toolbox, just remember not to run it on files owned by the package manager!

December 18, 2018

The Advent of Void: Day 18: sc

We’ve all been there. We have important numbers to crunch, and don’t want to break out a program as user-friendly as bc. So you go for your favorite tool: spreadsheets. So you start installing libreoffice to get yourself LibreOffice Calc.

$ sudo xbps-install -S libreoffice
...
Size to download:              437MB
Size required on disk:       1934MB
Space available on disk:      1024MB

No. We need something lighter.

$ sudo xbps-install sc

Name         Action    Version           New version            Download size
ncurses-libs install   -                 6.1_2                  248KB
sc           install   -                 7.16_3                 194KB

Size to download:              443KB
Size required on disk:        1967KB
Space available on disk:      1024MB

Do you want to continue? [Y/n]

This is something I can get behind. Yes!

When you type sc at a prompt, you are greeted with a very unhelpful looking display.

sc 7.16:  Type '?' for help.

        A         B         C         D         E
  0
  1
  2

(not shown is the sweet color coding, showing A0 to be the active cell).

In this, sc is slightly more helpful than vim, because if I type q I quit, and if I type ? I get a friendly looking prompt (this command-line spreadsheet has it all!)

Which Screen? [a-p, q]

     Overview:

     A:   This overview
     B:   Toggle Options
     C:   Set Options
     D:   Cursor movement commands
     E:   Cell entry and editing commands
     F:   Line Editing
     G:   File commands
     H:   Row and column commands
     I:   Range commands
     J:   Miscellaneous commands
     K:   Variable names/Expressions
     L:   Range functions
     M:   Numeric functions
     N:   String functions
     O:   Financial functions
     P:   Time and date functions

     Q:   Return to main spreadsheet

Now, here’s a guilty admission. I don’t know a hundredth of the capabilities this tool has to offer. So because I want to learn, I pick “Financial Functions” at random, and am told:

Which Screen? [a-p, q]

     O: Financial functions:

         @pmt(e1,e2,e3)    @pmt(60000,.01,360) computes the monthly
                           payments for a $60000 mortgage at 12%
                           annual interest (.01 per month) for 30
                           years (360 months).

         @fv(e1,e2,e3)     @fv(100,.005,36) computes the future value
                           of 36 monthly payments of $100 at 6%
                           interest (.005 per month).  It answers the
                           question:  ``How much will I have in 36
                           months if I deposit $100 per month in a
                           savings account paying 6% interest com-
                           pounded monthly?''

         @pv(e1,e2,e3)     @pv(1000,.015,36) computes the present
                           value of an ordinary annuity of 36
                           monthly payments of $1000 at 18% annual
                           interest.  It answers the question: ``How
                           much can I borrow at 18% for 30 years if I
                           pay $1000 per month?''

Oooh, mortgages! Well, I don’t want to calculate that right now, it’s too depressing for the season, so let’s look at something else.

Well, I plugged in values to Drake’s equation, it was amazing. My text titles were very long though, so I ended up finding out that if I pressed f then held down l, the cell got wider! So I went from narrow cells to:

G0 (59 2 0) |"Fraction of civilizations that release detectable signals"

                                 G                                                      H
  0  Fraction of civilizations that release detectable signals Length of time they bother talking to the universe
  1                                                        0.80                                         1000000.00

Frankly, it doesn’t look too bad. At the end, my cell looks like:

A1 (10 2 0) [@prod(B1:H1)]

And I tell you, Drake’s equation is not particularly useful, but I can tinker with values, and I get instant recalculation! So if we get some favorable numbers, eventually we get lots of future civilizations we can talk to. Something to anticipate!

How does sc work for saving these wonderful spreadsheets? Phello.sc followed by a newline will Put the database into a file called hello.sc. Then I ZZ to save and exit (another throwback to the more usable cousin vim).

And if I look at hello.txt?

# This data file was generated by the Spreadsheet Calculator.
# You almost certainly shouldn't edit it.

format B 29 2 0
format C 34 2 0
format D 72 2 0
format E 68 2 0
format F 56 2 0
format G 59 2 0
format H 51 2 0
label A0 = "Num Civs"
label B0 = "Rate of Star Formation"
label C0 = "Fraction of stars with planets"
label D0 = "Average number of planets per star with planets that can support life"
label E0 = "Fraction of planets that could support life that eventually do"
label F0 = "Fraction of planets with life that go on to get life"
label G0 = "Fraction of civilizations that release detectable signals"
label H0 = "Length of time they bother talking to the universe"
let A1 = @prod(B1:H1)
let B1 = 1
let C1 = 0.2
let D1 = 2
let E1 = 0.1
let F1 = 0.1
let G1 = 0.8
let H1 = 100
goto B1 A0

Now, is sc user friendly? No. Not nearly. It’s a tool that takes looking up the man page to do basic things. How do you make a label (text)? \ or | or / depending on how you want to justify it. = will actually drop you into an editable line where x deletes and <DEL> backspaces. It takes some keyboard flailing to make it work.

But it works, on most terminals (here’s looking at you over there, with the actual teletype), and works on plain text documents. Why not!

For more information please read the sc(1) manpage.

December 17, 2018

The Advent of Void: Day 17: qrencode

qrencode is a command line QR-Code generator.

You can generate a QRCode from an arbitrary string:

$ qrencode -o qrcode.png "QRCode Message"

This will create a new png image containing the QR-Code. You can use feh to display it:

$ feh qrcode.png

If you’re a terminal guy, you might dislike the window popping up. qrencode has you covered there:

$ qrencode -t ansi "QRCode Message"

This will generate a scannable QR-Code directly on the terminal window using ANSI escape sequences. If you’d like to avoid escape codes qrencode also supports pure UTF-8 encoding using -t utf8.

$ qrencode -t utf8 "QRCode Message"
█████████████████████████████
█████████████████████████████
████ ▄▄▄▄▄ █▀▄█▀▄█ ▄▄▄▄▄ ████
████ █   █ █▀▄█▀██ █   █ ████
████ █▄▄▄█ █▀▀ █ █ █▄▄▄█ ████
████▄▄▄▄▄▄▄█▄█ ▀ █▄▄▄▄▄▄▄████
████▄▄ ▄▄▀▄ ▄ ▀▄▄ ▀ █▄▀ █████
████▀██▀▄ ▄▀ ██ ▄ █▀█▀▀ █████
█████▄█▄██▄█ ██▄▄▀▄ ▀█ ▀█████
████ ▄▄▄▄▄ █▄▀█   █▄█ ▀▄▀████
████ █   █ █ █▄█▄▄▄▀▀▀█▄▄████
████ █▄▄▄█ █ ▀  ▄ ▄▀█ ▀██████
████▄▄▄▄▄▄▄█▄▄▄█▄▄▄████▄█████
█████████████████████████████
█████████████████████████████

qrencode supports a rich set of output formats including PNG, SVG, EPS, XPM, and even pure ASCII.

More colors? No problem! You can define custom foreground- and background colors:

$ qrencode -o - "QRCode Message" --foreground=00ff00 --background=0000ff | feh -

For more information please read the qrcode(1) manpage.