Finger info for flibitijibibo@icculus.org...

dyld Bug!

It appears a new breakage has occurred in macOS! This one seems very unintentional, however, and in a shocking turn of events, very understandable!

macOS Monterey (Big Sur may have previewed this based on other reports we've gotten?) debuts a new implementation of dyld, which is supposed to dramatically improve performance when loading apps. This seems to be part of a series of changes macOS has made recently regarding shared objects, and they're not alone - a recent change was made in glibc to dramatically improve performance for Unreal Engine's use of a billion so files, so it seems like this has become a priority for multiple vendors. I like performance improvements, especially when I didn't have to do anything, so hooray for that.

But as with any major change there's a chance that a breakage will occur... and it turns out FNA found one. I'm not on the inside of the paywall so I can't submit this to something like Radar, but I want to try and help as best as I can, so this .plan update is my best attempt to do that while also explaining things to end users, because this affects a lot of apps right now and they will want to know why a bunch of apps suddenly stopped working after upgrading their systems:

From what we can tell, -rpath is no longer prioritized like it used to be. When we inspect the output from DYLD_PRINT_SEARCHING=1, it turns out rpath is actually last on the list! This basically means bundling is no longer possible with the new dyld. Consider an application that uses SDL and FNA3D via dlopen, and has LC_RPATH set to @executable_path/osx/. FNA3D depends on SDL, and the output from otool -L looks something like this:

[flibitijibibo@flibitTower MacOS]$ xcrun otool -L BreakDyld osx/*
BreakDyld:
    /System/Library/Frameworks/Foundation.framework/Versions/C/Foundation (compatibility version 300.0.0, current version 1673.126.0)
    /System/Library/Frameworks/GSS.framework/Versions/A/GSS (compatibility version 1.0.0, current version 1.0.0)
    /usr/lib/libiconv.2.dylib (compatibility version 7.0.0, current version 7.0.0)
    /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1281.0.0)
    /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 1673.126.0)
    /usr/lib/libobjc.A.dylib (compatibility version 1.0.0, current version 228.0.0)
osx/libFNA3D.0.dylib:
    @rpath/libFNA3D.0.dylib (compatibility version 0.0.0, current version 0.21.12)
    /usr/lib/libobjc.A.dylib (compatibility version 1.0.0, current version 228.0.0)
    @rpath/libSDL2-2.0.0.dylib (compatibility version 17.0.0, current version 17.0.0)
    /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.250.1)
osx/libSDL2-2.0.0.dylib:
    @rpath/libSDL2-2.0.0.dylib (compatibility version 19.0.0, current version 19.0.0)
    # Blah Blah Blah

By itself this is fine; it will know to look in ./osx/ when we call dlopen. So the problem isn't that it fails to look there, the problem is when it looks. Say you have a copy of SDL2 in your sysroot, likely /usr/local/lib/. This is something a lot of people with Homebrew installed have! This used to be a non-issue, because as you'd expect the loader would prioritize what's bundled first, so our copy would get picked over the root copy.

Now it's the opposite! The copy in the root gets picked first, then rpath comes later. This is a really big problem because it basically means bundling is completely impossible to do without doing something extremely invasive like hacking the DYLD variables in a launcher script, which is the kind of thing SIP tries to prevent... so we don't want to do that.

If you're lucky, there might be a workaround - for Streets of Rage 4 it turns out if we throw the dylibs into Contents/Resources/ it loads successfully, but this is of course extremely volatile since you're now depending on fallback paths. In fact, I reproduce why this isn't safe in this test app. By default it should run on something like Mojave, but if you move the dylibs to Resources it will load /usr/local/lib/libSDL2-2.0.0.dylib before the one that's bundled. Monterey seems to be the exact opposite, which makes Mac support really hard right now.

I totally get that this is pretty narrow; AFAIK this applies to apps that are not fully sandboxed, and most of that software is going to be in the root tree anyway (i.e. the stuff you would get via Homebrew), so everyone that helped find this bug pretty much agreed right away that this was probably an oversight and not intentional. But, being able to have consistent loader behavior (and a properly working rpath!) would really help users who may not be strictly in one case or the other (fully sandboxed retail apps and stuff that's make installed).

If this report makes it to any official channels I'll link to it here. Assume it hasn't been seen until that's been done!

Bill's Hat

Turn a minimal Fedora installation into a SteamOS box!

1. Write Fedora Workstation NetInstall ISO to a USB drive
2. Boot USB image, install Minimal configuration with standard partition layout matching SteamOS'
3. Set root password, create a user called 'steam', set a password for it
4. Reboot, log in as root
5. A whole bunch of commands:

dnf group install hardware-support
dnf install Xorg xorg-x11-drv-evdev libglvnd-egl vulkan-loader.x86_64 vulkan-loader.i686 lightdm flatpak NetworkManager-wifi kernel-modules-extra bluez
dnf config-manager --add-repo=https://negativo17.org/repos/fedora-steam.repo
dnf install steam steamos-compositor steamos-modeswitch-inhibitor.x86_64 steamos-modeswitch-inhibitor.i686
setsebool -P allow_execheap 1
systemctl enable sshd.service
systemctl enable lightdm.service
systemctl set-default graphical.target

6. Edit /etc/lightdm/lightdm.conf:

pam-service=lightdm-autologin
pam-autologin-service=lightdm-autologin
user-session=steamos
autologin-user=steam
autologin-session=steamos

7. Create /var/lib/AccountsService/users/steam:

[User]
Session=steamos
XSession=steamos
Icon=/home/steam/.face
SystemAccount=false

8. Reboot, should work now!

9. Additional steps for NVIDIA users:

dnf config-manager --add-repo=https://negativo17.org/repos/fedora-nvidia.repo
dnf install kernel-devel dkms-nvidia nvidia-driver-libs.x86_64 nvidia-driver-libs.i686
reboot # Should be using the NVIDIA driver now!

flibitBuild

Want to replicate my build machine? Grab CentOS 7 and have a look...

# Update base install before doing anything else
yum update
# Add EPEL
yum install https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
# Add SCL
yum install centos-release-scl
# All the base build tools!
yum install bzip2 bzip2-devel chrpath cmake cmake3 freetype-devel gcc-c++ \
    git gtk3-devel hg libcxx-devel libstdc++-static libusbx-devel \
    libuuid-devel libvorbis-devel libxml2-devel lzma-sdk-devel meson \
    ocl-icd-devel openal-soft-devel opencl-headers openssl-devel patch \
    perl-IPC-Cmd svn unix2dos yum-utils yum-plugin-copr zlib-static
# SDL2 dependencies
yum-builddep SDL2
# Coprs for MinGW as well as GCC 10, needed to build Clang/LLVM/osxcross
yum copr enable mlampe/devtoolset-10
yum copr enable alonid/mingw-epel7
yum install devtoolset-10 mingw32-* mingw64-*

# CMake3 by default
alternatives --install /usr/local/bin/cmake cmake /usr/bin/cmake 10 \
    --slave /usr/local/bin/ctest ctest /usr/bin/ctest \
    --slave /usr/local/bin/cpack cpack /usr/bin/cpack \
    --slave /usr/local/bin/ccmake ccmake /usr/bin/ccmake \
    --family cmake
alternatives --install /usr/local/bin/cmake cmake /usr/bin/cmake3 20 \
    --slave /usr/local/bin/ctest ctest /usr/bin/ctest3 \
    --slave /usr/local/bin/cpack cpack /usr/bin/cpack3 \
    --slave /usr/local/bin/ccmake ccmake /usr/bin/ccmake3 \
    --family cmake

# Fake libdecor, for building with Wayland client decorations
mkdir /usr/local/include/libdecor-0
curl -o /usr/local/include/libdecor-0/libdecor.h \
    https://gitlab.gnome.org/jadahl/libdecor/-/raw/0.1.0/src/libdecor.h
touch /usr/local/lib/libdecor-0.so.0
ln -s libdecor-0.so.0 /usr/local/lib/libdecor-0.so

# Fake libdecor pkgconfig
cat <<ENDOFFAKELIBDECOR > /usr/lib64/pkgconfig/libdecor-0.pc
prefix=/usr/local
libdir=${prefix}/lib
includedir=${prefix}/include
Name: libdecor-0
Description: library for Wayland client-side window decors
Version: 0.1.0
Libs: -L${libdir} -ldecor-0
Cflags: -I${includedir}/libdecor-0
ENDOFFAKELIBDECOR

# Also:
# Modify unistd.h to change __block to anything but __block
# The CMake config provided by SDL2-mingw is terrible, delete the block
#  that starts with "if(NOT TARGET SDL2::SDL2)".
# Be prepared to pass -std=gnu99 manually. A LOT.

# To enter Mac building mode (and to build osxcross itself):
# scl enable devtoolset-10 bash

Wait, a "plan"?

Well, this is a .plan file, so here's my TODO. Expect nothing from it, ever.

In Progress
    Codename DanishBootleg
    VVVVVV FAudio
    FNA Display Refresh

Waiting Room
    Codename Devolution Assessment
        - Waiting for a source drop
    Super Hexagon "Neo" Rewrite
        - Waiting on Android stuff
    Proteus "Neo" Rewrite
        - Not actually a thing, I just wish it was
        - Pending some replies
    Coding History
        - On call throughout development
    VVVVVV 2.4
        - Giving the localization project as much time as it wants
    ScoreRush PC
        - Need AppAdmin for publish, crosshair for mouse
    SDL_ActionSet
        - Needs a pretty big grant
    Anodyne 1
        - Retirement project

When this .plan was written: 2022-01-02 14:37:43
.plan archives for this user are here (RSS here).
Powered by IcculusFinger v2.1.27
You could be playing Pringles for Genesis but instead you're here?