Posts

Posted on

On FreeBSD, git can’t find the certificate store

When I was playing with git checkout of modules I discovered that git doesn’t know how to set the certificate store for curl when it tries to retrieves a module via https. In general, I don’t recommend using git with https unless you have to. Using git+ssh obviates away a bucket of authentication issues. In this case, https is the better choice. To tell git where to look for certificates, to verify and https website, I had to add the following to my ~/.gitconfig:

[http]
sslCApath=/etc/ssl/certs

The command that does this is: git config --global http.sslCAPath "/etc/ssl/certs". If your operating system uses a CA file rather than a CA directory this is the setting: git config --global http.sslCAInfo "/etc/ssl/cert.pem". You can also make this work by setting an environment variable for curl in /etc/profile.

Mirroring in Gitlab

I normally strongly prefer git+ssh over git+https. If you are mirroring between two gitlab-ce instances over git+https, you can handle your mirroring with a single authentication token.

Posted on

Pip + git for development

I’m working on what should be a simple raspberry pi display project and I came up with the need for a set of ad-hoc python modules that were installable from my gitlab server. It was a bit of a journey. Here are the broad steps:

  • Create a gitlabs project for your python module. Since will probably have a few of these, it might be good to make a group for them right now.
  • I think that you can use git+ssh://git@gitlabs.example.com... for this but I chose to use a gitlabs impersonation token for this since ssh isn’t installed everywhere and sometimes the installation needs a bunch of hints in ~/.ssh/config.
  • A standard install will be done with pip as follows pip install git+https://{user}:{password}@git.example.com/example-group/example-project.git. If you created a gitlabs impersonation token about, you can substitute it for password here.

Sometimes I need to edit the installed package that I’m working on. The way to do this is to use the –editable flag to pip. To do that you need to specify some extra information to git when checking out the project. I found that this command line:

pip install --editable git+https://{user}:{token}@gitlabs.example.com/example-group/example-project.git#egg={module_name}

I think that the #egg={module_name} piece provides pip with name of the module as installed. I found the documentation that explains this here: “https://pip.pypa.io/en/stable/topics/vcs-support/”. Assuming that you are doing this in a venv, and it doesn’t make sense not to, you’ll get a new directory called venv/src/{module_name} which has a git checkout of your module so you can edit it to your needs for this particular project.

Posted on

Nuke and Pave

I recently reinstalled MacOS on my work and home laptops and then brough back my working state using Time Machine on both. I’m always impressed by how much faster and better a computer is after you do this. My friend Matt Zagaja: https://zagaja.com calls this a “Nuke and Pave” from here: https://www.macsparky.com/blog/2016/3/t0kcqkdxmkapwyo9eno0hv98ojd2kx and I love the term. In my opinion, one of the bad side effects of MacOS’ success is that you don’t have to *Nuke and Pave* very often. I think I’d been carrying my working environment forward for better than 10 years without a refresh and moving from High Sierra to Catalina added a bunch of unwanted quirkiness. This was probably because Apple is deprecating a bunch of the tools that I used in 2012 and while I don’t use them today, they were still installing kernel extensions and other stuff that was making my machine a little unstable. If you want to do your own *Nuke and Pave* on mac, you’ll need the following:

  1. The operating system you want to install. I used Big Sur 11.6. I find that for MacOS you want to download the OS and then use instructions like these: https://support.apple.com/en-us/HT201372 to create usb install media.
  2. If you use MacPorts see the notes at the end to save a list of the ports that your run. You’ll need it when you rebuild.
  3. Backup media: If it’s important you should have one or two backups of it . In this case you want a Time Machine backup. Disk Clone style backups would normally be quicker but don’t give you the granularity you need here. I use a USB-C to NVMe drive enclosure for speed here. My second backup is on rotating rust.

The operation is pretty simple. You want to:

  1. Boot your Mac from the USB installer by shutting down completely and then booting and pressing *Option* and holding it until your Mac presents you with a choice of boot media. It’s handy that newer Macs will boot on a keypress so you can start this process by simply pressing and holding *Option* If you are on Catalina or later you have to boot to _recovery mode_ first by shutting down your mac completely and using the utilities menu to enable booting from other media. If you have a firmware password on your Mac, you’ll need that to change this setting.
  2. Once you’ve booted from your install media, you need to erase and repartition the hard drive on your Mac. This is the point of no return so don’t take this step unless you trust your backups.
  3. Follow the install media instructions to reinstall MacOS on your computer. It will pause and ask you how to build users. What’s going on behind the scenes is the mac is using Migration Assistant to populate your home directory. Choose Time Machine backup and go into the menus and trim all of Applications, Settings, etc. You really only want to carry over data at this point. If you don’t migrate enough information, you can use Migration Assistant or Time Machine to catch anything that you missed.
  4. Reinstall your apps using the App Store, and whatever other sources you have. As a developer I have a bunch of software installed that requires me to Control-Click on the Application and then give permission to run one time.
  5. Restore security permissions as needed. App Store packages generally won’t have this problem. Other packages will. I use Emacs as my main editor because I’ve been doing this for a while. That requires me to go into the System Preferences -> Security & Privacy -> Privacy pane and grant Emacs permission to read files from my specified locations.

That’s most of what you need. I did the operation overnight. I handled steps 1 ~ 3 and then went to sleep. When I woke up I finished up 4 and 5.

A side note here for MacPorts or Homebrew users. You’ll want to restore your MacPorts/Homebrew environment also. For MacPorts this isn’t hard. Basically run sudo port list requested > ~/Desktop/ports-requested.txt This will leave a copy of the ports you installed by hand in a text file. When you are rebuilding your machine, you’ll need to perform the prerequisites needed to run MacPorts. Then you can use this output to install the packages that you used. I don’t use HomeBrew but I’ imagine that there must be something similar to this in HomeBrew.

Posted on

Git Sparse Checkout

Git Sparse Checkout

At work we had a very large monorepo. I’m tempted to quote Douglas Adams here but the reference is good enough. Checking out the whole thing runs the possibility of confusing git status messages as a result changes in the other part of the tree. These messages are a distraction. Dealing with them can consume large amounts of time. The best way to avoid them perform what’s called a sparse checkout. This is a checkout that only puts what you need into your working directory. In a normal checkout:

 $ git clone ...

You get the entire code base in your working directory. A sparse checkout is more complicated to perform:

 $ mkdir _target directory_
 $ cd _target directory_
 $ git init .
 $ git config core.sparsecheckout true
 $ echo "_your desired subdir_" >> .git/info/sparse-checkout
 $ ## Repeat the echo for each directory you need.
 $ git remote add origin https://git.neopost.com/PPT/IBMHSM.git
 $ git fetch
 $ git checkout master

It’s eight steps but if you do it this way, you gain complete control over what’s in your working directory.

Posted on

OpenBSD’s ksh adds configurable tab completion

I saw a configuration for bash tab completion a few years ago and I’ve always wanted it for the korn shell. I use either the “true” ksh from AT&T via David Korn or one of the variants that has sprung out of the pdksh project. OpenBSD’s ksh is a descendant of pdksh. In a recent release of OpenBSD someone patched it to kludge configurable tab completion via environmental arrays. The article is here: https://www.vincentdelft.be/post/post_20210102

This ksh is shells/oksh in FreeBSD.

Posted on

Everything needs a reset button

I don’t think I’m an Apple fan boy but I definitely like their stuff. That said, everyone could stand some improvement. One place Apple could improve is resets, and general controls.

The Story

I used to run a desktop machine 24/7 as an NFS server for backups. During the summer this heats up my office almost to the point where I can’t use it. I decided to change the Desktop to only be on when I need and to be suspended or off when idle. something has to be a target for the backups though. I chose a Mac Mini that I had. I brought the mini to my office and refreshed the machine with a new Operating System. Then I moved the NFS service to it. Finally, I turned off the desktop.

The Mini runs headless. At somepoint, Apple decided to be “helpful”. they changed OS X so that machines without any keyboard at all will plead for a bluetooth keyboard and mouse at periodically. The keyboard that connected to my work laptop obliged and connected and paired without a pin code exchange.

This morning I spent 20 minutes of my time trying to figure out why my keyboard didn’t work before I realized that pairing without a pin code was even possible.

I get that pairing in this manner reduces support calls and I understand that my use case of a Mac Mini running headless is a corner case by far but decisions that theOS X UI team have made lately really don’t help me. I disconnected the keyboard and turned off bluetooth on the mini but I’ll be surprised if that’s the end of things. Apple’s in the midst of a war on wires these days so if you connect your computer using ethernet and turn of wifi, your machine will pester you forever to turn the wifi back on. I’ll see if bluetooth is the same.

Posted on

OpenBSD on Raspberry Pi

 

I haven’t played with Raspberry Pi quite enough. I’ll do a write up on my garage door opener project at another time. But a really promising place for the Pi in my opinion is the role of a traveling router/access point. I don’t find the process of connecting to the WiFi in a hotel room particularly hard. It’s connecting back into my own network to access my my services that is difficult. The methods that I have at my disposal are:

  • IPSec VPN
  • SSH/SCP to selected service
  • Direct access where things are configured for it

Running OpenBSD on a Raspberry Pi gives me a solid place to put an IPSec connection for the whole hotel room network. Getting there involves installing OpenBSD on the Pi in the first place

Ingredients:

  • A Raspberry Pi 3B or 3B+ each model has plusses and minusses
  • An OpenBSD Raspberry pi snapshot release available at this url.
  • OpenBSD does not support the Pi video yet. The install console is serial. You need an Arduino/Raspberry Pi serial cable. The link points to a 4pin style. It connects as follows:
    1. Black <-> Pi GND
    2. White <-> Pi TX0
    3. Green <-> Pi RX0
  • A fast USB stick. OpenBSD can’t run from MicroSD card yet. This one works.
  • A WiFi adapter that you can live with this is going to be a compromise because WiFi has somewhat left the BSDs behind. These two CanaKit Wifi, and  TP-Link WiFi, work.

Continue reading “OpenBSD on Raspberry Pi”