...on the machine hosting this site:
vim
: 109sudo apt-get
: 42sudo -i
: 35sudo -u
: 34bash
: 29ls
: 23sudo dpkg
: 21cat
: 15sudo apt
: 14gpg
: 13grep
: 12cd
: 12...on my personal laptop:
vim: 82
cd: 76
python3: 46
ls: 40
mocp: 20
mvn: 19
java: 18
git: 18
cat: 18
mv: 15
man: 13
rm: 10
echo: 10
./partitions.py: 10
You'll notice, these are loosely sorted from most abstract to most practical.
As a consequence, the most impressive ideas are at the top, but each layer of abstraction is fascinating in its own right.
At some point in his or her life every working mathematician has to explain to someone, usually a relative, that mathematics is hardly a finished project. The mathematicians know, of course, that it is far too early to put the glorious achievements of their trade into a big museum and become happy curators. Our subject has, in certain respects, hardly begun.
— Barry Mazur, Foreword to Fearless Symmetry
This is a work in progress! It will showcase some of the best, most underrated theorems in maths. Stay tuned and wait for your mind to be blown — or if you're impatient, solve the math jester challenges and submit your solutions to see the deeper meaning behind each problem.
'Tis not too late to seek a newer world.
Push constants to the stack machines,
And sitting well in order smite
The sounding furrows; for my purpose holds
To sail beyond the VM stacks, and the baths
Of all the western stars, until I die.
It may be that the GOOG will wash us down;
It may be we shall touch the Hasty Likes,
And see the great Turing, whom we knew.
Though much is taken, much abides; and tho'
We are not now that strength which in old days
Moved earth and heaven, that which we are, we are,
One equal temper of heroic hearts,
Made weak by time and fate, but strong in will
To strive, to seek, to find,
and not to yield [control].
— Based on "Ulysses" by Alfred Lord Tennyson (1809-1892)
Modded by FX of Phenoelit; republished in Phrack Magazine
Every year, nay, every month, there are new security vulnerabilities being found and exploited. This is especially true viz-à-viz our phones. Here I intend to keep a consolidated list of security vulnerabilities related to people's phones — security holes that most people are blissfully unaware of.
Cellular data plans got more affordable during recent years and cellular network coverage increased. Disabling Wi-Fi by default and only enabling it when using trusted networks can be considered a good security practice, even if cumbersome.
The paper's mitigation recommendation includes "disabling Wi-Fi by default," which suggests that we should be using mobile network data plans by default instead. However, using these networks has drawbacks and security vulnerabilities of its own:
There are even modern exploits that use the phone's sensors, including the gyroscope, to collect data on the phone's user. Take a look at those articles, and Dr. Guri's site for more.
Stay secure, my friends.
Computer science is no more about computers than astronomy is about telescopes.
— Edsger Dijkstra
Corollary: The best computer science courses are the ones that don't ever require you to touch a computer.
— The Math Jester
This is a place to collect interesting algorithms, many of which I did not even learn in my university curricula. Join me on a tour of some eye-opening algorithms. These are genuinely impressive creative accomplishments.
Never trust a programmer who says he knows C++.
— Louis Brandy
This is a collection of dialogues, some real and others imaginary, which help to solidify knowledge of C++. If you're reading this, please treat these as approachable lessons and parables, inspired by those in The Number Devil, Fearless Symmetry, and other books. The best part is, the code provided is easily copiable — so you can verify the results on your machine!
Prerequisite: These parables are geared toward someone with knowledge of C.
Disclaimer: They are provided as a reference[1] with NO WARRANTY.
I intend to use my own Dewey-Decimal System for organizing these... It will look something like the following:
0xxx | These are supposed to be high-level, less technical questions that a C++ beginner might have. If you want to start with something less technical, then start with this section. |
1xxx | These are supposed to be basic technical questions about some of the perceived paradoxes of C++. The author posts them here as an opportunity to share what he wishes he had known when starting C++. |
2xxx | A section dedicated to imaginary dialogues about intermediate concepts. |
3xxx | These will include lessons deemed to be more advanced, e.g. copy elision and relocatable objects. |
4xxx | This is the Restricted Section of the library. (lol) Seriously though, this will contain some questions and comments about the Assembly code to which C++ compiles. All hail the almghty compilation pipeline; it is beautiful. |
5xxx | These dialogues are inspired by (or outright excerpted from!) conversations with the experts on IRC. Relative to the other dialogues, they are expected to be of the utmost quality and the highest difficulty. |
[1] no pun intended.
These are my (very visceral) feelings when working with certain specific dynamically typed programming langauges. Please do not take this section too seriously! I will gladly use whatever language and toolchain is most suited for the job. We all have our preferences for the languages that we use for personal projects.
sum
, min
, and list
, are built-in functions in CPython, so it is a bad practice to use them as identifiers. This may not sound like such a bad thing — after all, it incentivizes you to "just use more descriptive names" — but it also restricts the developer's ability to write code that (a) is good-quality (using distinct variable names) and (b) doesn't take too long to write, e.g. for Advent of Code challenges, etc. It also appears to be an inescapable feature (is it not?).File "/home/chozorho/ctf/advent/2021/11/./solve.py", line 77, in <module>
if (newBoardState[size-1][width-1] > 9):
TypeError: 'int' object is not subscriptable
This message is infuriatingly vague. It uses one and only one line to describe the error, and the description uses no more than six words. It does not state which expression corresponds to the 'int' object
(e.g., is it newBoardState
, or is it newBoardState[size-1]
? etc.). It also does not give any column index, so this information cannot be easily inferred. In fact, because python is dynamically typed, it is difficult enough to track down the error: whereas a compiler would have checked this ahead of time before blindly running the program, with errors like these, the programmer must waste time re-checking the code manually. In this case, the error was caused on an earlier line of code, so as it turns out, the line being printed isn't even that helpful.
Traceback (most recent call last):
File "/home/chozorho/git/redacted/censored_main.py", line 71, in <module>
censored_locations = inner_object.get_stuff_locations(param1, empty_list)
File "/home/chozorho/git/redacted/censored/inner_object.py", line 54, in get_stuff_locations
loc_div = selenium_driver.find_elements_by_class_name(censored_html_class_name)
TypeError: 'NoneType' object is not callable
How fun, another six-word message. The documentation suggests that find_elements_by_class_name
is a valid method on a Selenium webdriver. In practice, it does not exist; no problem, it is probably a simple matter of different versions in use. But then, why should the error mention a "NoneType object" if the selenium_driver object is not a NoneType?? If it is a method that is not found, then this should be made as clear and consistent as possible, without using any ambiguous or counter-intuitive language about "NoneType objects." Anything else runs a high risk of :
Traceback (most recent call last):
File "/home/chozorho/git/redacted/censored_main.py", line 71, in <module>
censored_locations = inner_object.get_stuffs_locations(param1, empty_list)
File "/home/chozorho/git/redacted/censored/inner_object.py", line 66, in get_stuffs_locations
startInd = loc.index(">") + 1
File "/usr/lib/python3/dist-packages/bs4/element.py", line 1374, in index
raise ValueError("Tag.index: element not in tag")
ValueError: Tag.index: element not in tag
Does python have a rule that their error messages cannot exceed six words??? Apparently, even though I am print
ing the value of loc and see it displayed as a string, I soon discover that the variable loc
is not even a string. This is yet another perfect example of how dynamic typing causes headaches and confusion at runtime.
The auto-casting rules in JavaScript always confused me. Let me reproduce a table to explain why.
Expression | Numerical Value | String Value | Boolean Value |
---|---|---|---|
false | 0 | "false" | false |
true | 1 | "true" | true |
0 | 0 | "0" | false |
1 | 1 | "1" | true |
"0" | 0 | "0" | true |
"000" | 0 | "000" | true |
"1" | 1 | "1" | true |
NaN | NaN | "NaN" | false |
Infinity | Infinity | "Infinity" | true |
[] | 0 | "" | true |
[20] | 20 | "20" | true |
null | 0 | "null" | false |
undefined | NaN | "undefined" | false |
"false" | NaN | "false" | true |
The reason why this is so confusing is partly because it is not self-consistent: if you pass a variable through multiple stages of conversion, its boolean value may change in unexpected ways. undefined
is false, but if you were to convert to a string, it becomes true. If you read a string from an input
box, and the user input reads "false", then casting that string to a boolean yields true, but casting it to a number and THEN casting to a boolean yields false. Casting [355, 113]
to a number and THEN to a boolean yields false
, but converting it directly to a boolean yeilds true
. In a sensible language like C, zero is false and false is zero, no matter how many "intermediate types" you cast it through. See what I mean?
But it gets even worse.
When you are working with an array, and you try to retrieve an element outside its bounds, you get an undefined
. Simple enough, right? But here's the problem: you can manually set a value in the array to undefined
(in which case the array length doesn't change!), and you can even set a value of the array at an out-of-bounds index! This is absurd because it violates the very concept of a fixed-length array.
But it gets even worse.
It turns out that, much like PyPI packages have security vuilnerabilities, so do the packages on NodeJS.
This reinforces a joke I have made in the past: that Python and JavaScript are in constant competition for the title "Least Enjoyable Programming Language."
Every time I have this discussion with a C# fanboy it always goes like this:
A: Yea, use C#!! It's faster and better-designed than crappy old Java!
B: But I run Linux. I refuse to accept spyware and keyloggers on my computer just to be able to use a goddamn programming language.
A: Aw, don't worry root! C# is cross-platform! It's great! You can just download this DotNET Core on Linux and voilà!
B: Huh, ok, maybe I'll try it out.
B: Oh, by the way, does your C# game project run on Linux?
A: What? Oh no of course not lol. It uses winforms, silly!
🤪B: Count me out.
Here are some obscure protocols & FOSS applications, if you're into that sort of thing:
This is something I'm still wrapping my head around. Apparently the vtools, which I finally found on another blog post, is an implementation of a "republican version control system."
This is a program that will allow you to access an Android phone from your laptop. After a lot of struggling and debugging, I have successfully tested this program from my Linux machine.
I also want to save this bookmark about restarting the ADB server — which addresses an error I frequently got when running scrcpy
.
To access my phone and send texts from my computer has been a dream of mine for years (I will make a blog post about this sooner or later). Now this dream is becoming a reality!
99 little bugs in the code.
99 little bugs in the code.
Take one down, patch it around,
127 little bug in the code...
— The Programmer's Drinking Song
This is a place to collect many technical problems I've faced, across my different Linux machines (laptop and Linux server). Treat it as a way to store some valuable life lessons, and share them with whoever stumbles upon this page. A common theme you'll notice is that getting help from people on IRC makes solving these problems so much easier.
apt sources
in order to include "chimaera," the more experimental repository. But when I did a reboot, I could no longer even decrypt my disk drive, let alone log in!!#!/usr/bin/env bash
set -o xtrace
# step 0: set non-root user settings
mkdir /home/devuan/misc /home/devuan/.ssh
echo "set nu hlsearch" | tee /home/devuan/.vimrc
# step 1: decrypt and mount the disk drive
sudo cryptsetup open --type luks /dev/sda3 crypto_LUKS
sudo mount /dev/quicksilver-vg/root /mnt
sudo mount /dev/quicksilver-vg/home /mnt/home
sudo swapon /dev/quicksilver-vg/swap_1
# step 2a: copy SSH key
cp -pR /mnt/home/chozorho/.ssh/id_tmj_cho_rsa /home/devuan/.ssh/id_rsa
# step 2b: download the firefox developer edition tarball itself
cd ~/misc
wget https://www.themathjester.com/common/firefox-93.0b8.tar.bz2
# step 3a: initiate firefox developer edition to populate the local profile
tar -xjvf firefox-93.0b8.tar.bz2
cd firefox/
pushd .
#local_ffdev_pid=`./firefox & echo $!`
#local_ffdev_pid=`sh -c "echo $$; exec ./firefox"` # TODO debug this launch!!
#sleep 6
# step 3b: auto-kill this browser instance...
#kill -9 $local_ffdev_pid
# This automatic "kill -9" feature is pending testing/optimization...
# For now, simply close the browser manually when it opens.
./firefox
# step 4: get the new, temporary (livecd local) profile name, e.g. do0o40b9.dev-edition-default
cd ~/.mozilla/firefox
local_profile=`ls -t | grep dev-edition | head -n 1`
# step 5: get the inner profile name and copy it. DO NOT MODIFY IT!!!
inner_profile=`ls -t /mnt/home/chozorho/.mozilla/firefox | grep dev-edition | head -n 1`
# step 6: actually run the replacement routine. Delete and copy the new into the old.
rm -rf $local_profile
cp -ipR /mnt/home/chozorho/.mozilla/firefox/$inner_profile $local_profile # do0o40b9.dev-edition-default/
# step 7: re-run firefox. The correct profile should be selected!
popd
./firefox &
# step 8: install preferred software. Sit back and relax while it gets installed!
sudo apt-get install moc newsboat build-essential git git-core texmaker \
valgrind fonts-arphic-ukai fonts-arphic-uming \
fonts-ipafont-mincho fonts-ipafont-gothic \
fonts-unfonts-core
As it turns out, this is caused by an upgrade to GCC 10 and libpthread.
The thread above claims that "the upload of gcc-10 moved libgcc_s.so to /lib
instead of /lib/x86_64-linux-gnu
." But, on my machine, the library is in /lib/x86_64-linux-gnu
and not in /lib
. I try copying the library to make sure it is in both locations, but that doesn't fix the issue. Am I missing something obvious? It seems I need to modify a find
command somewhere, but it's not clear where to add this.
In October, I finally find a time to ask the gods on IRC for some help. And sure enough, they are able to give me various recommendations for solving the root of the problem. I chroot
into the system, mount
the /boot
files, and then re-install the initramfs for kernel version 5.10.0-8
. I also may have re-installed grub2, but I cannot recall if that made a difference.
On October 15th, after doing another apt-get update/upgrade to be safe, I confirm that I can decrypt my drive and log in as usual. So: the issue is finally resolved!
For the longest time, I believed that the hardest part of backend webdev is setting up and connecting to a database in PHP.
2009 was the year I first tried to learn PHP, and I had no ideea what I was doing. Proceeding into the early 2020s, I experienced an astounding amount of stress and confusion whenever I tried to access a database from a PHP script.
Let me illustrate one issue I had in 2022 — and mind you, this is just one small taste of the PHP difficulties I've experienced.
I decide to actually try out a NoSQL database for a change. Therefore, I try to install mongodb on my Linux host (the same Debian droplet hosting this website! Say hello).
Thankfully, I am able to install this fairly easily; the instructions on the MongoDB website are very intuitive and to-the-point, and apt
installs version 5.0.5
.
I soon stumble upon some demos from tutorialspoint. Aha, I think to myself, this looks promising. I've used tutorialspoint before, in my efforts to learn perl and Rust (these deserve their own articles!) and when researching things like AJAX and Selenium. I know they won't let me down this time!
Alas, the very first line of their PHP script results in a failure. I dig deeper into the Apache logs (which is already a slightly painful endeavor) and find the following error:
[Thu Jan 13 05:46:44.933952 2022] [php7:error] [pid 23821] [client **.**.**.**:53632] PHP Fatal error: Uncaught Error: Class 'MongoDB' not found in /var/www/html/*****.php:31\nStack trace:\n#0 {main}\n thrown in /var/www/html/*****.php on line 31
For better or for worse, I am able to find a StackOverflow post that matches this error message (see also, my blog post criticizing of StackOverflow). This one at least cites a source that also suggests instantiating the MongoDB\Driver\Manager
.
But as soon as I go to run this recommended one-line adjustment, I see a awfully simiar-looking error:
[Thu Jan 13 08:23:15.791746 2022] [php7:error] [pid 23821] [client **.**.**.**:53632] PHP Fatal error: Uncaught Error: Class 'MongoDB\\Driver\\Manager' not found in /var/www/html/*****.php:31\nStack trace:\n#0 {main}\n thrown in /var/www/html/*****.php on line 31
How does this make any sense??? Have I really not installed the right PHP MongoDB driver? But I had already installed php7.4-mongodb
, which matches my active version of PHP (that's right, active version... more ranting about that is sure to come later) and thus should've done the job! After all, when I invoke ls /usr/lib/php/20190902/
as recommended in still other StackOverflow posts, I indeed see a mongo.so
in my installed extensions.
I proceed to look up this latest error on DuckDuckGo, yet the top search results are different errors (e.g., connection timeout, authentication failure, an error in the pthreads extension, etc.). No luck.
Do I seriously need to install this specific MongoDB PHP Driver using PECL? If so, why isn't it enough to do the php7.4-mongodb
installation above? Only time will tell; I am too exhausted to continue.
To this day, part of me wonders: does anyone else have this much trouble setting up databases and connecting with PHP? Surely I cannot be alone on this one. Perhaps it is comparable to the process of setting up a build environment when one is doing game development, i.e., a necessary prerequisite that is notoriously tedious but is always worth the effort. Or perhaps I started learning PHP in the "wrong order," before I even knew what Linux was and how to install a web server.
Sure enough, the following night, I try to install the aforementioned "mongo-php-driver." The command recommended on that GitHub page fails, with a simple "pecl: command not found" error. Ok, so I need to install PECL somehow.
I stumble through the top search results, which are vague websites stating that pecl is accessed through PEAR, which can be installed with a simple php invocation. But does this really help me? No. While it does give me access to pearl
and pecl
executables, once I try to actually use them to install mongodb (you know... like the GitHub page suggests), they exit with the following error:
WARNING: channel "pecl.php.net" has updated its protocols, use "pecl channel-update pecl.php.net" to update
downloading mongodb-1.12.0.tgz ...
Starting to download mongodb-1.12.0.tgz (1,392,375 bytes)
.........................................................
...done: 1,392,375 bytes
633 source files, building
running: phpize
sh: 1: phpize: not found
ERROR: `phpize' failed
This is becoming increasingly frustrating the further I go. I then attempt to do research on this new command phpize
.
Upon doing research on phpize
, I need to remember to disregard the search result titled "How to install phpize for PHP7+", which, I kid you not, literally redirects to the base domain site newbedev.com. From other search results, I discern that it is used in the build-process for PHP extensions, which at least fits into the knowledge I already have, and it is allegedly included in a package called php<version>-dev. I then try to install that package as root, which miraculously works (all hail the debian maintainers).
Then I can finally run pecl
to install the full, unmitigated mongodb
package, right? The culmination of all this work! To the victor go the spoils!
...
The build proceeds...
...
...and terminates with the following few lines:
Build process completed successfully
Installing '/usr/lib/php/20190902/mongodb.so'
install ok: channel://pecl.php.net/mongodb-1.12.0
configuration option "php_ini" is not set to php.ini location
You should add "extension=mongodb.so" to php.ini
Segmentation fault
This is extraodinarily frustrating. Running a package manager ends in a segfault?? Normally, I am happy to debug software using GDB to diagnose a problem, but this is a package manager! One would expect it to be well-maintained and robust.
The only saving grace is that the error message gives a clear prescription for how to avert the problem (that is, assuming that the "you should add" suggestion is even related to the segmentation fault, which is no guarantee). But here's the problem: I have been checking my PHP configuration files (/etc/php/7.4/cli/php.ini
and subdirectories) numerous times, and I have triple-checked the mongodb.so
is set there explicitly.
None of this makes any sense.
As always, asking for help on IRC reveals answers that you never would've thought possible (see also, A Word with the Gods, a C++ dialogue).
phpinfo();
.
The version of PHP being invoked at terminal (e.g., if I run php --version
and so on) is 7.4. By contrast, the version being invoked by the Apache2 web server is 7.3! They do not match! So, as a quick solution, I ended up enabling the PHP 7.4 module and disabling the PHP 7.3 module. I believe I could have stuck with the existing versions, but then I would have had to recompile the mongodb module for PHP 7.3, which could have been a hassle.
Special thanks to da_wunder
nim on libera chat for sharing this insight.
Unfortunately, even this solution caused something else to break (I'm detecting a pattern here). Stay tuned for later when I debug Nextcloud, which broke entirely after the update to PHP 7.4!
I don't know if it's worth telling the full story. Just know that I try running docker and encounter a very difficult bug. After asking the gods on IRC, I am finally able to resolve the error. The problem is very well-explained on another tech blog you'll find here: my-take-on.tech
I am experiencing a problem whenever I try to "push" commits, either on an existing branch or on a new branch, to a remote git repository on bitbucket. Note that it is a private repository, but this should not matter, since I am using SSH authentication, something I used to do all the time for previous repositories, public and private.
The error message reads:
Pushing to bitbucket.org:workspace/repo.git
Unauthorized
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
The obvious answer is that I have not added my SSH key to the ssh agent. But I have done so, using ssh-add and the infamous eval "$(ssh-agent)"
command and updating the known_hosts.
When I try ssh -T git@bitbucket.org
, the response is "authenticated via ssh key. You can use git to connect to bitbucket
."
I have been trying to follow the suggestions here at cyberciti. I have upgraded git (verison is now 2.30.2).
Even the git remote.origin.url
matches the "clone URL" that bitbucket tells me! So this StackOverflow answer does not apply either.
Any advice for me?
I am exploring a library to use PGP on PHP. However, I run ito a great many difficulties, as I describe below.
So far, I can:
In order to verify the message's authenticity, I use the following example script:
/* Parse public key */
$senderPublicKey = OpenPGP_Message::parse(file_get_contents('ale-public-noascii.key'));
/* Parse signed message from file */
$m = OpenPGP_Message::parse(file_get_contents('signed_message_binary'));
/* Create a verifier for the key */
$verify = new OpenPGP_Crypt_RSA($senderPublicKey);
/* Dump verification information to STDOUT */
var_dump($verify->verify($m));
If I run this basic example, using binary files, then the output is a rather complex JSON dump, but it appears to correctly identify the fingerprint of the public key, as well as the text in the file being verified.
That's a good start, I think to myself. But, for my eventual use-case, I really want to make this process more user-friendly, so I want to allow the user to submit ASCII (base-64-encoded) text as inputs to the function.
So I start to follow the same steps:
But if I simply go to modify the php script to de-armor the files, it fails completely!
I try modifying the example script to use the OpenPGP::unamor method (as in the example below):
/* Parse public key */
$senderPublicKey = OpenPGP_Message::parse(OpenPGP::unarmor(file_get_contents('ale-public.asc'), "PGP PUBLIC KEY BLOCK"));
/* Parse signed message from file named "t" */
$m = OpenPGP_Message::parse(OpenPGP::unarmor(file_get_contents('t'), "PGP SIGNED MESSAGE"));
/* Create a verifier for the key */
$verify = new OpenPGP_Crypt_RSA($senderPublicKey);
/* Dump verification information to STDOUT */
var_dump($verify->verify($m));
The server script does load the public key correctly. However, the rest of the script fails, posting an extraordinarily unclear output:
PHP Notice: Uninitialized string offset: 2050 in /var/www/html/singpolyma-openpgp-php-9651038/lib/openpgp.php on line 471
array(0) {
}
There must be some additional byte-processing calls that I am missing. If you know what these are, please contact me! In the meantime, I have decided to require binary signatures.
As of March 2023, I have finally resolved this issue. However, if you're interested in diving deep into technical nonsense, click to show the problem.
In an effort to write my DRM client, I have been trying to learn and apply the OpenPGP Message specification.
Consider the following example depicting the bytes of a signed message (a properly-generated "gold standard"):
00000000: a301 01da 0125 fe90 0d03 000a 0125 584d .....%.......%XM
00000010: dcd3 4e2b f701 ac13 6206 6568 2e74 7874 ..N+....b.eh.txt
00000020: 627b 73d5 3430 3633 3739 0a89 01b3 0400 b{s.406379......
00000030: 010a 001d 1621 0470 f409 6976 e55e 9d4f .....!.p..iv.^.O
00000040: bda7 b525 584d dcd3 4e2b f705 0262 7b73 ...%XM..N+...b{s
00000050: d500 0a09 1025 584d dcd3 4e2b f776 ff0b .....%XM..N+.v..
00000060: ff72 e481 4e07 b991 0ea8 dcba e6d3 67d8 .r..N.........g.
00000070: 1e5d dbed dd7f 1b8b 1207 459a 55c1 b1dc .]........E.U...
00000080: db8e c7ba 0067 e3c7 69e6 2b60 a781 f6aa .....g..i.+`....
00000090: 8787 a773 c533 d670 645c 9993 6537 c9c4 ...s.3.pd\..e7..
000000a0: f8db 08a6 849e 80c9 fcb2 991f 2ffc 9156 ............/..V
000000b0: abe0 d1a7 c400 8e67 5488 4795 8eaa 028d .......gT.G.....
000000c0: f529 c6be 2c46 5595 9e95 d1f8 2a86 6385 .)..,FU.....*.c.
000000d0: ceee b38b 3e65 8f23 9cdc 168b a8e1 442a ....>e.#......D*
000000e0: 524a 9a0b 3963 b8fd 2d15 b99b cc4b 0a61 RJ..9c..-....K.a
000000f0: b055 18d9 a638 ac5e 76e3 d4f2 ef9b de3f .U...8.^v......?
00000100: 6710 5409 c0c5 44bf 91d7 9e2f c6aa 966b g.T...D..../...k
00000110: 2a64 2233 705b 949a 87ae 883e 5423 8acb *d"3p[.....>T#..
00000120: 7b58 3549 6fcd 2dd1 5c69 bdba 0bdb db99 {X5Io.-.\i......
00000130: 8304 c4ac 2489 7674 920b 7d54 19d1 6f8a ....$.vt..}T..o.
00000140: 157e 138d 4a28 7def c2dd f28e 2209 af8d .~..J(}....."...
00000150: b54f e96d 3829 736a 0a4a 8010 37a5 bfd9 .O.m8)sj.J..7...
00000160: 4197 8856 fe7a c6fd 9e1f b44e 08a8 0eac A..V.z.....N....
00000170: 0853 8977 88c5 4c89 e787 620c 4e8c d77a .S.w..L...b.N..z
00000180: c705 d52a 8579 4b2a f165 5767 8b02 f0d1 ...*.yK*.eWg....
00000190: 0c22 94db 6d3a c1a2 8778 3ce5 6e90 7bd9 ."..m:...x<.n.{.
000001a0: 6739 8941 4bdf 9dd7 352d 90c8 ec53 5015 g9.AK...5-...SP.
000001b0: 130f 01a3 2172 142b 68ac b15a 1456 6c73 ....!r.+h..Z.Vls
000001c0: d28d f99a 5cee 4f63 8307 b493 6582 294b ....\.Oc....e.)K
000001d0: b935 975a 4536 5b4a ab29 cfa3 a492 14b0 .5.ZE6[J.)......
000001e0: 3c <
Converting from Hex to binary, this looks like:
00000000:
|
10100011000000010000000111011010000000010010010111111110100100000000110100000011000000000000101000000001001001010101100001001101
|
00000010 :
|
11011100110100110100111000101011111101110000000110101100000100110110001000000110011001010110100000101110011101000111100001110100
|
00000020 :
|
01100010011110110111001111010101001101000011000000110110001100110011011100111001000010101000100100000001101100110000010000000000
|
00000030 :
|
00000001000010100000000000011101000101100010000100000100011100001111010000001001011010010111011011100101010111101001110101001111
|
00000040 :
|
10111101101001111011010100100101010110000100110111011100110100110100111000101011111101110000010100000010011000100111101101110011
|
00000050 :
|
11010101000000000000101000001001000100000010010101011000010011011101110011010011010011100010101111110111011101101111111100001011
|
00000060 :
|
11111111011100101110010010000001010011100000011110111001100100010000111010101000110111001011101011100110110100110110011111011000
|
00000070 :
|
00011110010111011101101111101101110111010111111100011011100010110001001000000111010001011001101001010101110000011011000111011100
|
00000080 :
|
11011011100011101100011110111010000000000110011111100011110001110110100111100110001010110110000010100111100000011111011010101010
|
00000090 :
|
10000111100001111010011101110011110001010011001111010110011100000110010001011100100110011001001101100101001101111100100111000100
|
000000a0 :
|
11111000110110110000100010100110100001001001111010000000110010011111110010110010100110010001111100101111111111001001000101010110
|
000000b0 :
|
10101011111000001101000110100111110001000000000010001110011001110101010010001000010001111001010110001110101010100000001010001101
|
000000c0 :
|
11110101001010011100011010111110001011000100011001010101100101011001111010010101110100011111100000101010100001100110001110000101
|
000000d0 :
|
11001110111011101011001110001011001111100110010110001111001000111001110011011100000101101000101110101000111000010100010000101010
|
000000e0 :
|
01010010010010101001101000001011001110010110001110111000111111010010110100010101101110011001101111001100010010110000101001100001
|
000000f0 :
|
10110000010101010001100011011001101001100011100010101100010111100111011011100011110101001111001011101111100110111101111000111111
|
00000100 :
|
01100111000100000101010000001001110000001100010101000100101111111001000111010111100111100010111111000110101010101001011001101011
|
00000110 :
|
00101010011001000010001000110011011100000101101110010100100110101000011110101110100010000011111001010100001000111000101011001011
|
00000120 :
|
01111011010110000011010101001001011011111100110100101101110100010101110001101001101111011011101000001011110110111101101110011001
|
00000130 :
|
10000011000001001100010010101100001001001000100101110110011101001001001000001011011111010101010000011001110100010110111110001010
|
00000140 :
|
00010101011111100001001110001101010010100010100001111101111011111100001011011101111100101000111000100010000010011010111110001101
|
00000150 :
|
10110101010011111110100101101101001110000010100101110011011010100000101001001010100000000001000000110111101001011011111111011001
|
00000160 :
|
01000001100101111000100001010110111111100111101011000110111111011001111000011111101101000100111000001000101010000000111010101100
|
00000170 :
|
00001000010100111000100101110111100010001100010101001100100010011110011110000111011000100000110001001110100011001101011101111010
|
00000180 :
|
11000111000001011101010100101010100001010111100101001011001010101111000101100101010101110110011110001011000000101111000011010001
|
00000190 :
|
00001100001000101001010011011011011011010011101011000001101000101000011101111000001111001110010101101110100100000111101111011001
|
000001a0 :
|
01100111001110011000100101000001010010111101111110011101110101110011010100101101100100001100100011101100010100110101000000010101
|
000001b0 :
|
00010011000011110000000110100011001000010111001000010100001010110110100010101100101100010101101000010100010101100110110001110011
|
000001c0 :
|
11010010100011011111100110011010010111001110111001001111011000111000001100000111101101001001001101100101100000100010100101001011
|
000001d0 :
|
10111001001101011001011101011010010001010011011001011011010010101010101100101001110011111010001110100100100100100001010010110000
|
000001e0 : |
00111100
|
Notice that the signed message contains the ASCII plaintext of the signed message as a short segment in the middle!
This gold standard was generated via direct gpg
terminal call. Thus, the essence of my investigation (and my bounty) is, what is the best way to replicate this when running my Java program?
This implies that the preceding stream, the first thirty-six bytes of the file, constitute a One-Pass Signature Packet.
Assuming that Chapter 4 of RFC4880 isn't lying and that the message has a packet Header whose first byte includes the "Packet Tag." Well, in that case, We'd have to assume that a3
is our first packet tag. Using the bit interpretation specified in Section 4.2, we can conclude that:
0b1000
, or 8
in decimal;If my work so far is correct, then the conclusion of Chapter 4 tells us that this packet is a "Compressed Data Packet" (tag 8) rather than a "One-Pass Signature Packet" (tag 4). Does this make sense? It is possible that the One-Pass Signature (and perhaps other data packets) are wrapped inside a Compressed Packet, but that sure would increase the tedium of this analysis. We'll either have to contact an expert or press forward until we get a clear answer.
I would have expected the tag to be a One-Pass Signature packet; however, it is perhaps noteworthy that "a One-Pass Signature does not interoperate with PGP 2.6.x or earlier." Isn't this a bad sign? Earlier in chapter 4, it reads:
PGP 2.6.x only uses old format packets. Thus, software that
interoperates with those versions of PGP must only use old format
packets. If interoperability is not an issue, the new packet format
is RECOMMENDED. Note that old format packets have four bits of
packet tags, and new format packets have six; some features cannot be
used and still be backward-compatible.
But, look. If we did this correctly so far, then note that Section 5.6 directs us to the "Packet Composition" section to understand how this data is compressed. Section 11.3 is most relevant to us, since this is an OpenPGP message.
As it turns out, this StackExchange answer suggests that my work so far is correct — and that gpg
contains a feature to list the packets in a given PGP-related file! And sure enough, when run on our gold-standard file, the first packet is indeed a Compressed Packet! (I didn't bother reading the rest of the output yet, so as to not spoil it.)
To Be Continued...
One upshot of this entire investigation (reading the specification and experimenting) is: were the BouncyCastle experts wrong when they told me to use a LiteralData Packet??? A literal data packet uses tag 11 rather than tag 8, and in Chapter 11.3, Literal Data Packets are shown to comprise Literal Messages rather than Signed Messages and One-Pass Signed Messages. In fairness, in Chapter 5.6 does state that a Compressed Data Packet "contains a literal data packet," but why didn't they explain this missing link to me in their emails?? It doesn't make any sense!
This is a place to store some of the more obscure Linux commands that I often forget.
certbot certonly --standalone -d themathjester.com -d www.themathjester.com
sudo rm /var/lib/snapd/cache/*
sudo journalctl --vacuum-size=100M
sudo snap list --all
sudo snap remove "$packagename" --revision="$revision"
/usr/lib/libreoffice/program/soffice.bin --draw $filename
searchmonkey
— it contains a GUI that you can launch from terminal.scanimage -vp -d epson2:libusb:001:007 --resolution 1200dpi --format png --mode colorl > image_name.png
alternatives
). sudo update-alternatives --config java
:set noendofline binary
.bashrc
:
export HISTTIMEFORMAT="%F %T $ "
history | awk '{print $2}' | sort | uniq -c | sort -nr | head
awk
syntax is counter-intuitive: print $0
prints an entire line, and then each token in the line is indexed from $1
(akin to how python regexes work).
In this case, invoking index $1
prints a line index (simply the line number, starting at one!),
but I believe this is because it is part of the output from the history
command.$2
may need to change to a $5
if you apply the
"history timestamps" suggested above.if [ ! ${EUID} -eq 0 ]; then
...
fi
Finally, take a look at many utilities written by Zaiste in Rust.
If you've made it this far, congratulations! Here's a GIF I made, using the advice on this forum:
First, allow me to explain that I have a passion for understanding a system at a deep conceptual level and I also have a fascination with old-fashioned technologies (languages and hardware). These interests lend themselves well to mainframe development. But I have no hands-on experience with these things, because, how would I (how many people have a decades-old mainframe sitting in their basement?). Meanwhile, by pure coincidence, I hear rumors about how people can get paid large sums of money to do work with fortran
and COBOL
— allegedly, because there are so few programmers who know these languages. Soon after I start recording these narratives in writing, a tech journalist writes that, "if you can wrap your head around this relatively simple but verbose language, you are basically guaranteed a job."
However, as I soon discover, there are no entry-level mainframe positions, as I will attempt to illustrate below.
Ok, well, perhaps this is like the other good jobs in computer science: they demand a few years of experence even for their entry-level positions. Over the course of several years, as I gain more years of experience, the job openings explicitly require an even greater number of years of experience (a moving goalpost). This leads me to the conclusion that the most interesting jobs at the most interesting companies are being reserved for a static set of people: those born before 1997.
So, which is it? Are the jobs paying a high salary because of low Supply, i.e., they cannot find any programmers who know the languages (mere knowledge)? Or are they using that as a pretense, when in reality, they only want highly experienced engineers (several "years of experience")? Or is there an element of ageism, in which Gen Xers "pulled the ladder up behind them?" I have set out to investigate these narratives, by recording data about job vacancies and how their "minimum qualifications" change over time (month after month).
The following are actual excerpts from COBOL-related vacancies. I promise you, I did not skew the data by cherry-picking jobs wth unusually high requirements for "prior experience;" rather, these are very representative job openings. If you doubt this, then I challenge you to search it yourself and prove me wrong. In the long run, I intend to integrate a PHP script to automatically append vacancies to this table to drive the point home.
Date Posted | Demanded Years of Experience | Type of Experience Preferred | Position Title | Additional Notes |
---|---|---|---|---|
Feb. 20, 2022 | 4-5 Years | "ESB Integration experience" plus unspecified amnt. of experience with "COBOL, CICS, JCL/PROC, VSAM, DB2, Easytrieve." |
Mainframe Developer | |
Feb. 20, 2022 | 10 Years (z/OS) | "z/OS, TSO, SYSVIEW and/or ISPF, Basic IBM utilities IDCAMS" | Mainframe IMS Database Administrator | "Candidates that do not meet or exceed the minimum stated requirements (skills/experience) will be displayed to customers but may not be chosen for this opportunity." The grammar of this sentence is both incorrect andambiguous. I'm surprised they would dare to make it this unclear. |
Feb. 20, 2022 | 5-8 Years* | "ESB Integration experience" plus unspecified amnt. of experience with "COBOL, CICS, JCL/PROC, VSAM, DB2, Easytrieve." |
Senior Software Developer, Technical Lead |
*"Experience with COBOL and other programming languages such as [...]" i.e., the number of years of COBOL is technically unclear. |
Feb. 19, 2022 | 10 Years | "The candidate must have the below skills: [...] 10+ years' experience in basic z/OS utilities, TSO, SYSVIEW and/or ISPF, Basic IBM utilities IDCAMS, IEFBR14" | Mainframe Administrator | |
Feb. 19, 2022 | 7 Years | "7+ years of experience in COBOL/JCL/VSAM/CICS." | Mainframe Developer | |
Mar. 22, 2022 | 7 Years | "Must have seven (7) years of experience in the Information Technology field," including "At least five (5) years of experience as a Mainframe SME." "Proven experience on IBM Mainframe z/OS" |
Mainframe Developer | |
Apr. 19, 2022 | 10 Years | "Hands-on technical experience with mainframe non-x86 legacy systems and with technologies such as COBOL, JCL, CICS, DB2 for z/OS, Assembler, PL/I, Java, Rexx, flat/sequential files, GDGs, and VSAM — 5 years." | Mainframe Developer | |
June 8, 2022 | 7-12 Years | "Minimum of 3 years of experience working on mainframe, SQL and Relational Databases such as DB2, Oracle, and SQL Server." | Systems Analyst | The mainframe-specific year threshold is more attainable, for once, but they still insist on seven years of professional experience. |
August 26, 2022 | 20 Years | "opportunity to work on the same systems you used to as a kid: IBM Mainframe; IDMs; COBOL; [...]" | Software Engineer | |
September 23, 2022 | Bachelor's Degree + 3 Years | "Autosys: 3 years (Required)" "z/OS Mainframe: 3 years (Required)." "Cobol: 3 years (Required)" |
Infrastructure Governance Deployment Technician |
Notice that many positions give ambiguous descriptions that do not give an exact number of years for COBOL specifically, but nonetheless state that it is required. Examples include: "Experience with mainframe and Cobol is a plus" (Edward Jones); "Experience with COBOL and other programming languages such as [...]" (Zigabyte); "Experience with COBOL, CI/CD, DB2 is required" (Benedsoft).
© 2024
MIT License
Copyright (c) 2019 AKIRA-MIYAKE
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
Copyright (c) 2014, The Fira Code Project Authors (https://github.com/tonsky/FiraCode)
This Font Software is licensed under the SIL Open Font License, Version 1.1.
This license is copied below, and is also available with a FAQ at:
http://scripts.sil.org/OFL
SIL OPEN FONT LICENSE Version 1.1 - 26 February 2007
PREAMBLE
The goals of the Open Font License (OFL) are to stimulate worldwide
development of collaborative font projects, to support the font creation
efforts of academic and linguistic communities, and to provide a free and
open framework in which fonts may be shared and improved in partnership
with others.
The OFL allows the licensed fonts to be used, studied, modified and
redistributed freely as long as they are not sold by themselves. The
fonts, including any derivative works, can be bundled, embedded,
redistributed and/or sold with any software provided that any reserved
names are not used by derivative works. The fonts and derivatives,
however, cannot be released under any other type of license. The
requirement for fonts to remain under this license does not apply
to any document created using the fonts or their derivatives.
DEFINITIONS
"Font Software" refers to the set of files released by the Copyright
Holder(s) under this license and clearly marked as such. This may
include source files, build scripts and documentation.
"Reserved Font Name" refers to any names specified as such after the
copyright statement(s).
"Original Version" refers to the collection of Font Software components as
distributed by the Copyright Holder(s).
"Modified Version" refers to any derivative made by adding to, deleting,
or substituting -- in part or in whole -- any of the components of the
Original Version, by changing formats or by porting the Font Software to a
new environment.
"Author" refers to any designer, engineer, programmer, technical
writer or other person who contributed to the Font Software.
PERMISSION & CONDITIONS
Permission is hereby granted, free of charge, to any person obtaining
a copy of the Font Software, to use, study, copy, merge, embed, modify,
redistribute, and sell modified and unmodified copies of the Font
Software, subject to the following conditions:
1) Neither the Font Software nor any of its individual components,
in Original or Modified Versions, may be sold by itself.
2) Original or Modified Versions of the Font Software may be bundled,
redistributed and/or sold with any software, provided that each copy
contains the above copyright notice and this license. These can be
included either as stand-alone text files, human-readable headers or
in the appropriate machine-readable metadata fields within text or
binary files as long as those fields can be easily viewed by the user.
3) No Modified Version of the Font Software may use the Reserved Font
Name(s) unless explicit written permission is granted by the corresponding
Copyright Holder. This restriction only applies to the primary font name as
presented to the users.
4) The name(s) of the Copyright Holder(s) or the Author(s) of the Font
Software shall not be used to promote, endorse or advertise any
Modified Version, except to acknowledge the contribution(s) of the
Copyright Holder(s) and the Author(s) or with their explicit written
permission.
5) The Font Software, modified or unmodified, in part or in whole,
must be distributed entirely under this license, and must not be
distributed under any other license. The requirement for fonts to
remain under this license does not apply to any document created
using the Font Software.
TERMINATION
This license becomes null and void if any of the above conditions are
not met.
DISCLAIMER
THE FONT SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT
OF COPYRIGHT, PATENT, TRADEMARK, OR OTHER RIGHT. IN NO EVENT SHALL THE
COPYRIGHT HOLDER BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
INCLUDING ANY GENERAL, SPECIAL, INDIRECT, INCIDENTAL, OR CONSEQUENTIAL
DAMAGES, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
FROM, OUT OF THE USE OR INABILITY TO USE THE FONT SOFTWARE OR FROM
OTHER DEALINGS IN THE FONT SOFTWARE.