welcome to cathode ray dude's blog. please be aware that i'm a lot cruder on here than on youtube.

you can subscribe via RSS. point your reader here: RSS Feed


silly shit i had to do with git submodules

it is not generally in my nature to spend any length of time talking about git or github or anything related to programming but i did find myself in a situation a couple days ago that was not easy to google, so here's what i figured out.

break

in case you end up in the specific situation i was in:

i was having some graphical issues with a game on linux and people suggested i use gamescope. i had never heard of this; it is apparently part of steamos, the linux distro for the Steam Deck, and is one of the elements of the special sauce that makes gaming more consistent compared to your typical desktop linux.

in short, it's a nested X server. you launch it with e.g. a 1280×800 resolution, it attaches to e.g. desktop :1, then you export :1 as your X server and run a game. the game connects to gamescope and sees what appears to be a 1280×800 desktop, but instead of taking over your real display, the output appears in a window you can drag around. i think you can also fullscreen it without the games knowledge or involvement, among many other things; it's neat, and it papers over some problems.

unfortunately it has had a bug for several months, starting in 3.14.2, which causes it to shit the bed on startup if you're running any GPU that doesn't support a thing called “modifiers.” i think this primarily affects older AMD and intel GPUs, which is what I was using, and unfortunately every OS repo is serving up a broken version at the moment because the patch took a long time to make it into production. such is life

i could not find a .deb of any version newer than like 2021, so i had to compile from an older revision. here's the forum thread i followed. be aware that the dependencies it asks you to install will not integrate with current versions of Mint (at least) and solving that is an exercise for the reader. i ended up having to purge every single i386 package related to wayland and probably broke a ton of stuff so… just wait for the repos to be updated, tbh, don't do this. but if you do anyway, i have some tips

googling this is nearly impossible because everyone assumes you know mostly what you're doing in git; i do not, so it was not obvious how to use the instructions i was being given. i had two problems:

  1. how to retrieve a copy of a Github project as it was at a specific point in time
  2. how to retrieve all the dependencies, or “submodules” (a concept i had never encountered before) in the form they had at that same point in time

the first one is easy: you just find the revision you want in Releases or Tags and download the zip. but if it uses submodules (a git technique for pulling in other code libraries without actually merging them together) then it's a pain in the ass to find all the associated modules. i guess in retrospect i probably could have used some of the steps i describe below to go back and get all the submodules as zips and extract them to appropriate places manually but, well, i didn't think to do that, and this might actually be easier, idk? there's also theoretically easier ways to do this but idk what they are so here's one way that worked for me

okay actually i have a better way now

as i expected, i immediately received a comment (thanks nic!!) explaining how to do this MUCH more concisely. simply perform:

git clone –recursive -b 3.14.1

this is what i WANTED to do in the first place, but i didn't know that you could use tag names as input here. some parts of git accept tags/releases (apparently they're the same thing), branches, and SHAs; this command apparently only accepts tags/releases and branches. sigh. so yeah, do it this way and you'll get the whole ball of wax in one go.

also the version number here is not actually a version number, it's just a freeform text field, so you'll have to find the revision you're interested in in the git tag or release section, then copy the exact name you see there. just look in my screenshot below; see where it says “3.14.1”? copy whatever you see there instead and it should work.

here's the way i did it that you don't need to do

Step 1: Find the revision you need

we assume you have been told that e.g. “version 3.14.1 does not have this problem.” go to the Releases page in github and find that version. if there are no releases, look under Tags; I don't really understand the difference but w/e.

see that string of gibberish under the 3.14.1 link, “f1cdbd1”? make a note of that, then click it.

many git tutorials refer to “the SHA” or “the hash.” this is a very long string of gibberish representing a “commit”, meaning a specific point in the history of the project. you can see it in your address bar now, after the last slash.

apparently you can refer to any commit by the first seven characters of its hash. i don't know how they avoid collisions but whatever: f1cdbd1 is the name for this version of the code. write that down.

Step 2: Find the revisions for each submodule

in the upper right there's a Browse files button. click that and you'll be taken to a view of the whole repository as it existed when this code was committed.

valve used a “subprojects” folder to store their submodule links. i'm sure not all projects use this convention; i think any directory can be a link to a submodule. you'll know them because they have folder-with-arrow icons, they're blue, and the links include SHA hashes:

find everything that looks like that and copy it to a notepad.

step 3: clone the project and submodules

we assume you have git installed and it's in your PATH because every single compiling tutorial on earth describes how to do that.

from a shell, type: git clone https://github.com/ValveSoftware/gamescope

you now have a copy of the main code repo in its current form. enter the dir: cd gamescope

to revert it to the commit you want: git checkout f1cdbd1

now grab all the submodules: git submodule update –init

you now have copies of all the submodules, but they aren't the right versions. you need to cd into each submodule directory and checkout the right version, e.g.

cd subprojects/openvr
git checkout 15f0838

repeat for each directory, using the SHAs you got from github for each module. when you're done, you should have the correct versions of everything. follow whatever forum post or wiki page you can find for instructions on compiling that specific project, gl;hf

conclusion

i am absolutely not a git expert, this is just one method that worked for me. there is probably an easier way. in particular, a lot of this would be unnecessary if we only wanted to get the current code and all current versions of submodules; for that you can supposedly just do git clone –recurse-submodules [url], but as far as I can tell this cannot be used along with a SHA, it will only grab the latest version of everything.

Discussion

nic 2024/11/10 12:03

random git facts to answer implicit questions, if you're curious (this is fully optional reading, but, like, i really like git and i want to talk about it,,,)

  • a release is a github concept. it's a tag plus some metadata, e.g. release notes. tags are a git concept; they're a way to name a specific commit in a way that promises the name won't change. (and a commit is, as you identified, a snapshot of the repo at a point in time.)
  • you don't need the hash – you can just git checkout 3.14.1.
  • hashes are unique by sheer random chance, but git automatically expands the length if there happens to be a collision. you can actually use any unique prefix of any length – git just by default doesn't emit one shorter than 6-7 bytes.
  • but you can skip that too: git clone –recursive -b 3.14.1 https://github.com/ValveSoftware/gamescope
  • that'd download the full repo and every submodule, which is slow. add `–depth 1 –shallow-submodules` to only download what you request: git clone –recursive –shallow-submodules –depth 1 -b 3.14.1 https://github.com/ValveSoftware/gamescope
  • you're 100% correct about submodules. they are indeed a repo referenced as though it's a subdirectory, and any directory can be one. they're listed in .gitmodules in the repo if you need it.
  • you're better at git than like 50% of professional developers if you managed to figure out submodule versions on your own.
rando 2024/11/10 12:06

…all those flags (recursive, shallow-submodules) should start with two hyphens, -, not one en dash. it looks like double hyphens got converted to en-dashes. mildly frustrating lol.

cathode ray dude 2024/11/10 12:10, 2024/11/10 12:11

womp womp. i'll have to see if i can turn that off in doku. ty for pointing it out, looks like it's happening to comments too.

oh i didn't realize this was in response to the comment lol. i think it's ONLY happening in comments. sigh. computers

nic 2024/11/10 12:54

lol i just noticed

  • it turned all the double-hyphens into single-en-dashes
  • it turned all the urls, even the ones in code, into links so it doesn't look like they're there
  • you can disable formatting for a bit with double-%

so the actual correct final command is this:

git clone --recursive --shallow-submodules --depth 1 -b 3.14.1 https://github.com/ValveSoftware/gamescope

cathode ray dude 2024/11/10 12:09

thank you for all of this. i had no idea you could use versions - is “3.14.1” here an actual semantic version that git understands as such, or is it just a freeform string that people generally populate with a version?

SHAs work in checkout, but not in clone, which is why i thought you couldn't do this all in one go; nobody told me i could use the version numbers, which apparently work in both.

also thank you for the compliment

nic 2024/11/10 12:24

3.14.1 is freeform text, the name of the tag. people usually use tags for versions because a tag is a name you promise will point to some commit forever. technically tags (and branches) are just “named refs”, i.e. human-friendly names for some commit somewhere; the only difference is semantic.

yep – -b only works for named refs. it works for both branches and tags despite the name, but it can't point to an arbitrary commit sha.

(if it sounds like git is layers and layers of abstractions: yep! it's a design that appeals a lot to me, but that also makes it hard to understand quickly, which makes it rather notoriously shitty for distributing software. usually projects use releases to distribute binaries; i guess with gamescope valve didn't feel a need?)

cathode ray dude 2024/11/10 12:31

excellent! thank you again!

for the record my opinion of git itself is “wow, what a neat concept” while my opinion of having to use git to get anything done is “wow, we should burn it all down and start over” so i can appreciate why you appreciate it, it's a really fascinating tool if it's not just something you're being forced to interact with six times a year max, haha

Enter your comment. Wiki syntax is allowed:
 

Last modified: 2024/11/10 12:40