Upload A Disk Volume Directly to Openstack Image
When we upload an image using openstack image create --file, the image file will be transferred to image storage backend through glance api servers. If the image size is huge, the upload process might take time sometimes ends in failure. Glance API might become the bottleneck if multiple huge images are uploaded at the same time. To overcome glance api servers bottleneck, we could instead directly upload the image to ceph following the below procedure.
Verify Ceph Deb Package Build Flag
We can confirm what build flags are enabled at build time by checking the below places.
Default build flags in the source code’s Makefile or build tool’s declaration files (CMakeLists.txt). In the debian/rules file. The build system at build time (dh). As deb packages on different Ubuntu releases have different versions, it is necessary to verify the build flags in deb package and upstream package within the same version. For example, below are the process to verify the build flags of ceph package in Ubuntu 22.
Technical Note: Jq Array Transformation
Problem Given a JSON data below, output the data in CSV/TSV format.
[ { "id": 0, "data": [0, 1, 2] }, { "id": 1, "data": [1, 2, 3] }, { "id": 2, "data": [3, 4, 5] } ] Expected output.
"0\t0,1,2" "1\t1,2,3" "2\t3,4,5" How to JQ provides @tsv and @csv function that convert data to the corresponding format. Each row of the output CSV table should be formated to an array type before these functions can consume.
Technical Note: Convert an ova image to qcow2 format
From Wikipedia of Open Virtualization Format, I learned that an the ova file is just a tar file of ovf directory. So I went open a Windows 10 Developer Evaluation ova file to see what exist inside it.
$ tar -tf WinDev2001Eval.ova WinDev2001Eval.ovf WinDev2001Eval-disk001.vmdk WinDev2001Eval.mf So inside the ova file includes
A virtual machine specification definition .ovf file A hash checksum .mf file A disk image in .vmdk image As qemu-img supports vmdk as disk image file, we actually could convert vmdk file to any other format that we like.
Technical Note: Disk Encryption using Cryptsetup with Vault as Key Management Service
In Linux Operating system, full disk encryption could be achieved by various solutions: encryptfs, dm-crypt… While a step-by-step setup of disk/partition encryption can be referred from official documentation, integration disk encryption with Key Management solution like Hashicorp Vault is unobvious. Fortunately, there are a package named vaultlocker in Ubuntu Universe repository that ease this integration. I decided to spend my free time to make cryptsetup work with Vault.
Notes: A similar request has been made to Cryptsetup but it goes outside of cryptsetup so it was closed.
Technical Note: How to Change Juju User Password
Juju User JuJu has an internal user framework, which supports sharing of controllers and models. Using JuJu user feature, a sysadmin could separated users for controllers as in multiple clouds deployment or for users as in multiple systems deployment. Each juju user can have its own permissions at application, model, or controller layer. This separation of responsibilities allow multiple users to manage multiple infrastructures with a clear border of responsibilities.
Technical Note: DynamicUser in Systemd
Background Recently, I received a really interesting question from my customer. He found out in his file system two files (directories), which belong to an user that not in /etc/passwd. Specifically in Ubuntu Linux 18.04, these 2 files are
/var/lib/private/systemd /var/lib/private/systemd/timesync These two files belong to a user named systemd-timesync with UID/GID in 62583 and this user does not belong to /etc/passwd.
Traditionally, a Linux user does not always need to be in /etc/passwd because it can come from many remote sources, for example