Since ingesting disks that were in different computers with different operating systems, sometimes the encoding used for the filenames are different. The nifty convmv utility can change the encoding of filenames. It is used like so to convert from latin-1 Windows encoding to utf-8: convmv -f latin-1 -t utf-8 -r /root/of/path It prints information on the filename encoding and whatever changes it will do. It will not perform any renaming until you re-run the command by adding the --no-test flag.
TIL: dd over netcat
I found out that you can transfer raw bytes as generated by the ddcommand over the network using netcat, which improves throughput compared to transfer over SSH using scp. Thanks to NDCHost for a comprehensive writeup. On the server (serverB.example.net), start a process listing for raw data packets: nc -l 19000 | bzip2 -d | dd bs=16M of=/ptah/to/output/file And on the client, send data packets: dd bs=16M if=/path/to/input/file | bzip2 -c | nc serverB.
Locate Files in a Datalake
A personal Datalake is basically a large folder with many files without a clear structure. So how does one find anything? There are several answers to this question which we will explore in different articles (see here for a list). The answer changes depending on the type of files and if they have metadata associated or not. One thing any files has is a file name. And more often than not that name has a meaning and we remember what file we are looking for by that name (and sometimes by the path).
In search of a new programming language
I have been programming in R for the past 10 years, mostly as a scientist in academia. R is extremely useful to quickly validate hypotheses and generate plots to communicate results to collaborators. As I transition into industry I become increasingly frustrated with some of R idiosyncrasies that make it difficult to write error-free programs. Examples of such include recycling shorter vectors (and only warning if the size of the smaller vector is not a multiple of the longer one), and returning results when selecting using an NA value.
This project combines 3D printing and electronics to create large illuminated letters that are slowly fading through different colours. It is based on WS811 LED strips (30 LEDs per meter) and an arduino nano. The LEDs are turned on by a PIR motion sensor, saving energy. You can see the results below: Design I designed the letters in OpenSCAD. There are 2 parts to each letter, a base and a diffuser.
Ba3dp Part 2: Frame
In the first post of the series I looked at many available 3D printer designs. I also made some choices about the core electronic components. In this post I will decide on the frame build. First off, I should mention that my total budget for this build is 250 Euros excluding printed parts. So after purchasing electronics that leaves me with about 160 Euros for the frame and other mechanical parts.
Building a 3D Printer
My first printer was a cheap CTC i3 clone, that I bought around 100 Euros a few years back. It taught me a lot about 3D printing, but was also a constant source of frustration, especially regarding bed levelling and Z-banding. 18 months in, the heater cartridge of the hotend burned through the control board. For the replacement printer, I chose an Ender 3. It made quite a bit of difference: more reliable, less fidgeting.
TIL Newgrp Change Groups without Logout
Every instruction on the web to add/remove a linux user from a system group ends with the following: “Make sure to log out and back in for these changes to take effect”. Well I do not like to log out, especially when I am in the middle of a project and have all my windows set up the way I want. So today I found this gem: man newgrp SYNOPSIS newgrp [group] DESCRIPTION newgrp changes the group identification of its caller, analogously to login(1).
TIL: How to renew GPG subkeys on a smartcard
The subkeys on my OpenPGP smartcard expired. I had set them to expire around Christmas each year. I created all keys in 2017 and successfully renewed then in 2018. This time, I changed the expiry on my public key and thought that would be enough. Sure enough when I wanted to sign some messages soon after, the program complained. I still do not fully understand the relationship between the master key (to which the public key is associated) and the subkeys for signing, encryption and authentication that I have moved to the smartcard.