Latest 10 recent news (see index)

February 12, 2019

VoidLinux.eu Is Not Ours

We would like to warn people of a domain name that is no longer under Void Linux control. voidlinux.eu lapsed in its original registration, and was purchased by an unknown 3rd party before Void Linux could regain ownership. At this time, please assume that anything involving voidlinux.eu is not related to Void Linux, and should be considered potentially malicious. Of course, if the person who owns the domain now would like to transfer it to our control, we’d be grateful, and will update voidlinux.org to indicate if this happens.

Our official domain is VoidLinux.org.

February 03, 2019

Cross compilation support for gobject typelibs

Thanks to the work of maxice8 we’re pleased to announce cross compiling support for gobject-introspection and packages that depend on this tool. This is a big step forward to support more packages through cross compiling and stay future proof for glib development.

maxice8 put together a comprehensive post about his work. This article will be higher level overview of our work in this field.

What are gobject and typelibs?

gobject is an abstract description of types defined in glib. Glib is used on a wide range of application and libraries, most notably gtk. Glib itself is written in C but it’s possible to provide dynamic language bindings to other languages such as python.

This mechanism is not builtin into glib, but instead is provided by the gobject-introspection tool. This tool scans both the source and the binary blob of a library and creates a metadata file that is used to provide type information for other language bindings. For a deeper dive into gobject-introspection please consider the docs at readthedocs.io

The problem

When it comes to Void, we have high demands on portability of applications. Our armv6l, armv7l and aarch64 platforms a completely and exclusively cross compiled, meaning they are built on an x86_64 host. This leads to an interesting problem with gobject-introspection which by design needs to run on the same platform as the produced binaries are targeted.

Unfortunately, as already stated this has been a design decision of this software and we can’t do anything to change this constraint itself.

Furthermore, during the last few years the Gnome community drifted towards extensively using gobject-introspection in their products. This led to the breakage of more and more applications in the gnome-ecosystem.

The solution

During the last few months we considered different solutions to this problem. This first proposal to not support gir through cross compilation and only allow it through native build became more and more unsatisfying as the number of packages using this mechanism increased over time.

The second approach which was the followup to the one described above was to add native builders to the system and build those packages on those native builders. The issue was here that this would require a lot of infrastructure work and added a lot of complexity.

The third solution and the one we chose was to use the userspace emulation features of Qemu to emulate the target system only for those calls that needed to be on the target platform. The yocto project is already using this approach to cross compile Gnome Applications to other platform. We reused many of their tooling for void-packages. Nontheless, we hit many porting issues - our tooling must run on musl, too - , new bugs were triggered in void-packages - see for example these two posts at maxice8’s blog, and many other small issues needed to be resolved.

Basic support for cross compilation has already landed in void-packages, the musl port is not yet finished, but we expect it to work soon(tm).

maxice8 also prepared a blog post about his porting work on libgusb.

February 03, 2019

The Life of a Pull Request & Where Commit Bits Come From

Sometimes people will join into #voidlinux and ask why changes are taking so long to get merged. This is an interesting question, and it boils down into a few parts. In this article we’ll take a look at the life of a pull request and how it gets from idea to package.

For any package to exist, someone first has to want it. This want is often expressed as someone in IRC saying “I want to use Void, but I need package foo” and someone looks at it and decides they will package and maintain foo. Sometimes this want will manifest as an issue filled to the void-packages repository on GitHub, and sometimes it manifests as a “blind” PR showing up for a new package. This last one is what we’ll follow as it’s where things get interesting.

A Call to Action

When a PR is created, this triggers GitHub to notify people according to its settings. For some members of the project this will trigger an email, for some people this will trigger other automation via API, and up until very recently, the #xbps channel on freenode was NOTICE’d (this last one is gone as GitHub has removed this functionality with no clear replacement path).

This notification will hopefully grab the eye of at least one reviewer, who will then open the link to view the PR on GitHub, or will otherwise begin reviewing by their preferred means. Many of our reviewers prefer to pull the PR locally and view it in their editor of choice, or build it locally on more powerful boxes than Travis CI provides.

Changes Requested

This is usually the slow part of getting something accepted to void-packages. This is the part that involves going back and forth on what changes need to be made or what improvements are requested and often involves discussion of potential changes to package policy.

For folks in IRC this often will be a rapid fire exchange of comments in channel or query, and often ends with something like this:

15:30 <bobertlo> i'm making the suggested changes [...]
15:31 <maldridge> bobertlo: ping me when you're ready for review+merge

At this point the ball is back in the author’s court, and the committer is waiting for changes to be made. At this point it’s also considered that a committer is tenuously “assigned” to a PR.

In a good flow, the next thing that happens is another message from the author:

15:53 <bobertlo> maldridge: i pushed those changes

The Reviewer Follows Up

When a reviewer is “assigned” to the PR, it doesn’t really mean that they own it, but there is some expectation that they will follow the work through to merge unless they explicitly state otherwise. Sometimes this looks like a comment on a PR, and other times it looks like a poke in IRC to see if the author is still working on the changes.

At the end of this phase, the reviewer will either approve or dismiss the PR. Usually dismissing a PR is only done if the package is completely unworkable and the author and reviewer have come to an agreement that it won’t be accepted for some concrete reason, but other reasons exist.

Merged

When a package is accepted the reviewer will merge it, and then our automated build infrastructure will kick into action. A webhook from GitHub will signal to our infrastructure that a new change is to be built, and the systems will pull down the changes via git. After it has been ascertained what needs to be rebuilt, the change is dispatched to our builders to begin compiling the new or updated package.

After compilation, the built package is copied back to the build master and is signed, then indexed into the repository for interested users to download. Past this point is a complicated web of rsync that is beyond the scope of this article.

Why does it take so long?

First of all, it doesn’t. When you compare Void’s process to other distributions, the process of getting a package included can be as short as an hour from initial request to installable globally.

The slightly longer answer is that there are bottlenecks. One of the easiest to understand is that there are only so many servers we can build on, and the time it takes to compile packages (especially large ones) has to be taken into account.

The single largest bottleneck though is reviewer loading. In the process that’s described above, reviewers are loosely connected to PRs, and follow them through. While our reviewers are very fast in many cases, they are still human and can only hold so many things at once.

So if the reviewers are the bottleneck, the question logically follows: where do reviewers come from and how do they get their special powers?

Special Powers

Within the Void vernacular, people who can merge changes are referred to generally as people with a commit bit. For those who are curious, this term comes from privileged users on a BSD system often having a “wheel bit”. At the time of this writing, there are 13 people with a commit bit. These people can merge to almost any repository in the organization, but there are a few exceptions. Having a commit bit can also come with other powers, like the ability to op up in IRC, but such additional powers are not automatic, and are beyond the scope of this article.

So how does one get these special powers?

The astute Void follower may already know this answer, as has been documented for some time now on InfraDocs.

The full documentation can be found in the onboarding documentation. But to save the effort of going to that page, we’ll briefly recap it here.

First a person has to be noticed within the project as a good fit for having the ability to merge changes on their own. Usually this means that they’ve been an active contributor continuously for some time and have a history of submitting high quality work.

After an existing committer thinks they’ve found a good candidate, they’ll propose the membership change to the project owners in private. This allows the project owners time to discuss the proposal with the proposer and exchange information without needing to talk publicly. Sometimes there are circumstances that a proposer might not be aware of that may affect if they wish to continue suggesting someone, and this initial conversation gives a time to discuss that.

After this discussion has happened a final decision is made to either accept or reject the proposal. In the event it has been accepted, the candidate is informed and has the final choice to accept.

I’ve Got a Bit, Now What?

Just getting the powers though isn’t enough. People who hold a commit bit have to justify that they have them. This means means either reviewing changes, commenting on PRs, or otherwise advancing the state of the Void. In the past when people have left the project either by explicitly stating they’ve left, or by just not being a part of it for a while, we revoke powers. This helps us keep Void reasonably secure by knowing exactly who all has the ability to push changes to our repositories.

Of course if any of these people were to come back, they’d be welcome to once again demonstrate their skill and again earn the special powers as they did before. Just having had the power at one point is not sufficient cause to be handed it again.

In summary the life of a PR is part of a much larger process of getting and maintaining a pool of contributors and qualified reviewers. This process is quite involved, and this post only just scratches the surface. Interested persons are always encouraged to reach out to us in #voidlinux on freenode. The author of this post can always be contacted in #voidlinux as maldridge.

January 31, 2019

The curious case of Python and global variables

Void isn’t all smooth sailing. Sometimes things break in obscure and difficult to troubleshoot ways. This was recently the case when one of our maintainers noticed that certain Python packages weren’t behaving as expected. If you’re curious about how this kind of troubleshooting happens, [maxice8]’s blog post is highly recommended as its a good intro into what its like to troubleshoot things from the perspective of a Void maintainer.

Read the troubleshooting process here.

January 23, 2019

dhcpcd>=7.1.0 update may require manual intervention

Starting with 7.1.0, the dhcpcd(8) package is dropping the 10-wpa_supplicant hook. Instead of the hook, upstream suggests to build wpa_supplicant(8) with CONFIG_MATCH_IFACE enabled, this is done in the wpa_supplicant package since version 2.6_14.

Users of the 10-wpa_supplicant hook will need to adjust their systems to guarantee that their wireless connection keeps working.

The following actions are required when dhcpcd(8) is updated to 7.1.0 for users of the 10-supplicant hook.

1. Make sure wpa_supplicant(8) is at least 2.6_14
2. Enable the wpa_supplicant(8) service.

Other users should be on the lookout as -M is now used to match all interfaces, using for configuration the CONF_FILE variable. Setting an specific WPA_INTERFACE is now optional.

December 24, 2018

The Advent of Void: Day 24: 'Twas the Night Before Christmas

‘Twas the Night Before Christmas,

When all through the cluster, not a build was running, not even a revbump. The packages were all signed by the server with care, in hopes that an upgrade soon would be there.

The maintainers were nestled all snug at their desks, while visions of waterfalls danced in their heads; and Gottox in his home and I in mine, had just settled our brains for a long winter break; when out on the frontends there arose such a clatter, I sprang to my terminal to see what was the matter.

Away to my console I flew like a flash, called for my tmux and threw open the logstash. The status page on my screen lent a luster of midday to errors below, when what to my wondering eyes should appear, but a stuck qt5 build and 8 staged SONAMEs.

With a large webengine so slow and so massive, I knew in an instant it must be chromium. More rapid than runit my commands to the system: “Now Ansible! now Grafana! now buildbot and xbulk! On rindex! on snooze! on, ext4 and repo-cleaner! To the top of the graph to the top of the stack now build, build away, build away all!

As disk platters that before the online scrub fly, when they meet with an obstacle always retry, so up to the servers the tasks they all flew, with their mirror of toys and BDFL, too.

And then in a twinkling, I saw in reply the server starting and launching of each single build. As I drew in my buffers, and was closing up shop, down the pipe the json came with a stop. It was padded with null bytes from its start to its end, and its braces were all tarnished with codepages and parity bits; a slice of package it contained in its body, and it looked like a startup just opening its reply. The descriptors how they twinkled, the keys how merry!

A close of a brace and a retcode of 0 soon gave me to know I had nothing to dread. Buildbot said not a word but went straight to its work, and filled the mirrors with packages. Waiting for but a moment before confirming my intent, the BuildBot went on its way; indexes sprang to their clients, and to the fleet I did signal and away they all built like the sight seldom seen. But before I C-b d I exclaim to all users:

Happy Christmas to all, and to all a Good Night!

–maldridge

We hope you’ve enjoyed the Advent of Void. This is a fun project that we’ve done now for two years to try and shed some light on lesser known packages. We’ve got a few honorable mentions that may make their way to the feed over the next few days, but this post concludes the series. Void is a volunteer effort that we enjoy sharing with you, and hopefully you’ve learned some neat new tips and tricks these last 24 days. Our volunteers don’t just make packages and manage infrastructure, we also write fun posts like these (though I recognize I’m a quite poor poet). If you want to become involved and help us keep Void going, join us in #xbps on freenode, but maybe wait until after the holidays to do so.

Enjoy the winter season and the updates that just keep rolling!

December 24, 2018

The Advent of Void: Day 24: gromit-mpx

Have you ever wanted to doodle on a screenshot? Ever? Just a single underline, or an arrow, or something to help point out something about a screenshot? Or maybe during a presentation, on the fly, suddenly you want to point out a particular thing… wouldn’t it be nice to have a tool to help?

Turns out, there is one, a very simple tool to do this. I use it every month.

Gromit-mpx is simple. <F9> to start being able to draw, Shift-Ctrl-<F9> to clear everything you drew, and Ctrl-<F9> to stop being able to draw (but leave everything on the screen). Just make sure gromit-mpx is running in the background, and you are set for all your drawing and annotating needs. Even people like me, who just run dwm and not with any compositor, I can annotate my screen quickly.

But with just a few lines, I can clarify what I meant to share. I won’t be winning any art awards, but it works.

Do note that the annotations stick around, even after you scroll the window, or switch to a different screen entirely.

So keep gromit-mpx around, it might be useful someday.

For more information, there is always the gromit-mpx(1) manpage.

December 24, 2018

Void Linux at 35c3

2018 has been an intense year for Void. There were many ups and downs. To wrap up the year we’re meeting at the 35c3.

To meet other Voiders and chat about related and unrelated topics there’s the Void Linux Assembly

Also, on day 1 (2018-12-27) 19:00, @Gottox is doing his annual talk about The State Of the Void.

After the talk, around 20:00, Void is inviting you to a Void User Meetup. We’ll meet at the Open Infrastructure Orbit in front of the lecture area, find the nearest bar, grab a Beer or a Tschunk, and get into interesting conversations.

I’m looking forward to a great event with an awesome community!

–Gottox & the Void Linux Team

December 23, 2018

The Advent of Void: Day 23: extrace

Today’s tool is extrace(1), a small, simple and powerful tool to trace program executions. By default extrace(1) will look at system wide executions, alternatively it can start a new process or start looking for executions in child processes of a specified pid. The output is a tree like structure including the process id, the command and its arguments, additional flags can add the environment of the process, the executing user or even the exit status and runtime duration.

The following example is the truncated extrace(1) output of checking the dhcpcd(8) man page and then executing dhcpcd -k to release all interfaces followed by executions from dhcpcd hooks.

$ extrace
    911 man dhcpcd
      912 less -T /tmp/man.XXXXH357hT /tmp/man.XXXXEOs5Na
    930 doas dhcpcd -k
    930 dhcpcd -k
      932 /bin/sh /usr/libexec/dhcpcd-run-hooks
        933 rm -f /run/dhcpcd/resolv.conf/br0.ra
          936 sed -n 's/^domain //p' br0.dhcp
          938 sed -n 's/^search //p' br0.dhcp
          941 sed -n 's/^nameserver //p' br0.dhcp
        943 cat /etc/resolv.conf.head
        944 cmp -s /etc/resolv.conf /run/dhcpcd/resolv.conf.br0.ra
        945 cat /run/dhcpcd/resolv.conf.br0.ra
        946 rm -f /run/dhcpcd/resolv.conf.br0.ra
        947 chmod 644 /etc/resolv.conf
        948 rm -f /run/dhcpcd/resolv.conf.br0.ra
        950 sed '/^# Generated by dhcpcd/,/^# End of dhcpcd/d' /etc/ntpd.conf
        951 cmp -s /etc/ntpd.conf /run/dhcpcd/ntp.conf.br0.ra
        952 rm -f /run/dhcpcd/ntp.conf.br0.ra
		...

Here is another example where we trace the executions by make into a file, with the -t flag extrace(1) adds the exit status and the duration.

mblaze$ extrace -t -o make.extrace make
cc -Wall -Wno-switch -Wextra -g -O2 -fstack-protector-strong -D_FORTIFY_SOURCE=2   -c -o msed.o msed.c
cc   msed.o blaze822.o mymemmem.o mytimegm.o seq.o slurp.o  -lrt -o msed
mblaze$ cat make.extrace
1992+ make
  1993+ uname
  1993- uname exited status=0 time=0.000s
  1994+ cc -Wall -Wno-switch -Wextra -g -O2 -fstack-protector-strong '-D_FORTIFY_SOURCE=2' -c -o msed.o msed.c
    1995+ /usr/lib/gcc/x86_64-unknown-linux-gnu/8.2.0/cc1 -quiet -D '_FORTIFY_SOURCE=2' msed.c -quiet -dumpbase msed.c '-mtune=generic' '-march=x86-64' -auxbase-strip msed.o -g -O2 -Wall -Wno-switch -Wextra -fstack-protector-strong -o /tmp/ccDnNZ91.s
    1995- /usr/lib/gcc/x86_64-unknown-lin exited status=0 time=0.212s
    1996+ as --64 -o msed.o /tmp/ccDnNZ91.s
    1996- as exited status=0 time=0.019s
  1994- cc exited status=0 time=0.234s
  1997+ cc msed.o blaze822.o mymemmem.o mytimegm.o seq.o slurp.o -lrt -o msed
    1998+ /usr/lib/gcc/x86_64-unknown-linux-gnu/8.2.0/collect2 [...]
      1999- /usr/bin/ld exited status=0 time=0.027s
    1998- /usr/lib/gcc/x86_64-unknown-lin exited status=0 time=0.029s
  1997- cc exited status=0 time=0.031s
  2000+ /bin/sh -c 'test -n "$SOURCE_DATE_EPOCH" || BUILDDATE=$(date '\''+ (%Y-%m-%d)'\''); \'$'\n''    printf '\''#!/bin/sh\nprintf "User-Agent: mblaze/%s%s\\n"\n'\'' \'$'\n''        "$({ git describe --always --dirty 2>/dev/null || cat VERSION; } | sed '\''s/^v//'\'')" \'$'\n''       "$BUILDDATE" >museragent'
    2001+ date '+ (%Y-%m-%d)'
    2001- date exited status=0 time=0.000s
      2004+ sed 's/^v//'
        2005+ git describe --always --dirty
        2005- git exited status=0 time=0.008s
      2004- sed exited status=0 time=0.009s
  2000- /bin/sh exited status=0 time=0.012s
  2006+ chmod +x museragent
  2006- chmod exited status=0 time=0.001s
1992- make exited status=0 time=0.286s

December 22, 2018

The Advent of Void: Day 22: autossh

SSH. The very use of the acronym screams “Knows which end of the computer needs to be plugged in.” It can be used to confirm your computers are working, to transfer files (two ways!), as a SOCKS proxy (didn’t know this? Go read the man page, option -D, it’s brilliant), and even as a poor man’s VPN.

Poor man’s VPN?

Well, I have a computer that lives behind a firewall. I can’t adjust the firewall. But I want to use that computer. Legally, mark you, I have permission from the network owners, it’s just not practical to adjust the firewall.

What to do?

Well, set up an ssh tunnel of course! In this case, localhost is the target computer, and the other hostname is a place I can access more readily than localhost.

$ ssh -NR $portA:localhost:22 nbis.gov

But problems. My local network isn’t the best, and sometimes when there is a power outage my computer comes up first. I could just create a runit service to make sure the ssh connection is running…. and I did, for a while, but when it failed it failed very badly. Finally, I settled on autossh. What does autossh do? Automatically runs an SSH command. You can configure a lot about how it does this, how many times it tries, how long it waits between tries, what the backoffs are, everything like that. So, I combined the best of both worlds: autossh (which should never crash) wrapped in a user service with runit (which should always remember to start autossh)

#!/bin/sh
[ -r ./conf ] && . ./conf
: ${portA:=2222}
: ${targetHost:=localhost}
exec autossh -M 0 -NR ${portA}:localhost:22 $targetHost

And my problem is (mostly) solved! So, in summary: SSH is an amazing tool, and autossh helps you keep your ssh tunnels alive.

One point about configuration, from the manpage:

Other than the flag to set the connection monitoring port, autossh uses
environment variables to control features. ssh seems to be still
collecting letters for options, and this seems the easiest way to avoid
collisions.

and there are 13 environment variables right now, so plenty you can choose to configure.

For more information, examples, and to see the list of relevant environment variables, please read the autossh(1) manpage.