OP here
I couldn't get it to work the easy way using SMLoginItem, and ended up resorting to the extremely tedious method of calling the command line tool launchctl via NSTask to tell launchd to treat my helper app as a startup item.
Basically, you'll need to create and maintain a helper.launchd.plist in an appropriate library folder (appropriate folders) which contains the path to the executable of your helper app and some other configurations most importantly KeepAlive which will tell launchd to automatically reopen your app should it ever crash or be closed by the user, and RunAtLoad which will lead to your app to be started at login. This is what my helper.launchd.plist looks like:
You can then register your helper app via the terminal with the commands:
launchctl bootstrap gui/$UID [path to your helper.launchd.plist]
to activate your helper as login item
and
launchctl bootout gui/$UID [path to your helper.launchd.plist]
to deactivate your helper as a login item.
You can use these terminal commands from within your code. I did it using NSTask, but I think there might be better ways of doing it.
Note that $UID is an internal variable, which the terminal automatically expands into the user id of the current user before it feeds the gui/$UID argument to the launchctl command line tool. My UID is 501, so launchctl actually gets the input "gui/501", not "gui/$UID". I use [NSString stringWithFormat:@"gui/%d", geteuid()] to create an appropriate user domain argument for launchctl with Objective-C.
Random tips:
Note that the my.prefpane can be in different locations depending on
whether the prefpane is installed for all or just for a specific
user.
I recommend you put a helper.launchd.plist template into your
project. Then you only have to edit the Program field depending on
the user and where the prefpane is installed and not create the whole
plist file programmatically.
I put my helper.launchd.plist into
/Users/[User]/Library/LaunchAgents and I have noticed, that that
folder does not exist on a fresh install, so you might have to check
whether the library folder supposed to contain your helper.launchd.plist actually exists and create it if necessary.
Apple rewrote launchctl in macOS 10.10 Yosemite, they added launchctl bootstrap and launchctl bootout as replacements for the now deprecated launchctl load and launchctl unload. If you want to be backwards compatible you might have to fallback to load and unload on older versions of macOS
Hope this helps and good luck with your project!