<basshelal[m]>
Someone's having a very strange macOS library loading problem, I don't know if this is indeed the JVM bug he referenced or just more macOS strangeness
<basshelal[m]>
Maybe you know more about this or know someone that can guide us in the right direction, I do hope it's just macOS just being weird again
<headius>
I will try to repro today. Feels like that error might be a generic "I couldn't load" covering up the real cause.
<basshelal[m]>
Also what time is it in Yankeeland?
<headius>
Early
<headius>
I have a primary school aged son to get going in 15 min
<headius>
If that is simply a dlopen error then I am back to wondering if this is an architecture mismatch
<basshelal[m]>
Ah ok, thought I woke you up cuz I know you start your day much later
<headius>
But I will have a look in an hour or so
<headius>
My day is fluid 😀
<basshelal[m]>
Sure no worries, you'll probably know better than me
<headius>
basshelal: were you able to build this uapki project?
<headius>
it fails to copy the first dylib it builds and then bails out
<headius>
for me
<basshelal[m]>
No, the sample repo the issuer gave already has the dylibs so I just used those
<headius>
oh I see
<basshelal[m]>
btw `nm` and `otool` show no errors with those libs on x86 Big Sur so I *assume* they're ok
<headius>
basshelal: I'm leaning toward this being a quirk of MacOS library pathing
<headius>
I can reproduce. Adding the resources dir to library.search gets past the first library and fails on the second
<headius>
it is not seeing that the image is loaded probably because it sees a different name for the loaded library than what it's looking for
<headius>
at the dlopen/dyld level
<headius>
when I copy the libs to . they load
<headius>
at the moment I would say this is not a jnr-ffi bug, since I can get the libraries to load from CWD, but I have no good explanation for why it cannot load the libraries simply by adding `library.search("src/main/resources")`
<headius>
boo, element needs to handle multiline code snippits better... still have to cmd+shift to hit enter without it sending
<headius>
```
subbu has joined #jruby
<headius>
deploy job is failing to auth for some reason
<headius>
looking into it
<basshelal[m]>
sorry was cooking lunch
<headius>
enebo: I think you forgot to update VERSION
<headius>
do I just update that one file?
<enebo[m]>
oh on master?
<enebo[m]>
where?
<headius>
9.3
<enebo[m]>
oh for next snapshot
<headius>
yes
<enebo[m]>
hahah I thought you meant for release
<enebo[m]>
ok yeah you just update release and run maven
<enebo[m]>
it will update 4 poms
<headius>
ok
<basshelal[m]>
so when the second library comes to load `dlopen` and friends are not finding the library it depends on?
<headius>
basshelal: I believe that's right
<basshelal[m]>
But when the libs are on the project root they load??
<headius>
the fact that it works fine from CWD tells me this is just a pathing issue
<headius>
yep, works for me
<basshelal[m]>
damn, ok I'll try that now
<headius>
I do not have a better explanation for why it fails but it does not seem to like the way they are being loaded from the subdir and does not recognize them as having been loaded
<headius>
enebo: pushed... I don't think that will fix the deploy builds but perhaps
<enebo[m]>
headius: There is something special about version not being the actual release.
<enebo[m]>
I think it being name SNAPSHOT fixes it
<headius>
that's what I was thinking also
<headius>
failed deploys correspond to the 17th which is when we released I think
<basshelal[m]>
Yeah I was pretty sure it's a macOS thing, I suspect it's very similar to another issue we were facing with macOS library paths where the environment isn't being preserved.
<basshelal[m]>
I'm not super sure but they feel related somehow
<basshelal[m]>
Yep works for me when they're in the project root!
<basshelal[m]>
That's... very strange and very annoying :/
<headius>
it is!
<headius>
🤷♂️
<headius>
would you agree this is probably not a jnr-ffi issue specifically though?
<basshelal[m]>
Oh yeah absolutely, I hate to say it, but it's not our problem 😂
<headius>
I will keep it open until we get confirmation that they have it working but I'm not sure how to help now
<basshelal[m]>
Still I was very curious and quite concerned at first but it became obvious it wasn't us
<basshelal[m]>
We can't help really, other than to tell him that it works when they are in the project dir
<basshelal[m]>
which is actually better than the proposed solutions I had for him so I think it's a win
<headius>
yeah it bothers me that . works and src/main/resources does not even though I add the latter to `library.search`
<headius>
I tried full path too, didn't help
<headius>
and clearly . is a relative path
<headius>
security? what security? 😀
<headius>
enebo: just occurred to me... deploy depends on other jobs passing so it won't ever deploy 9.4 snaps with current setup
<headius>
I'm going to kill master build from k77ch7 merge so I can see if it deploys right on 9.3
<headius>
we can manually trigger master snap deploys if we want
<headius>
we need to start deciding what to tag off too
<enebo[m]>
headius: it may not but I was hoping to not tag out 250 failures just so we get snapshots
<headius>
yeah agree on that for sure
<enebo[m]>
headius: I have tagged a few things already...most of what is left is doable
<headius>
just saying it will either not deploy until green or we need to soften that requirement
<enebo[m]>
and should be done but it is perhaps less than it looks too if you see probably 100+ are 3 things
<headius>
sure... I want to get back to it but we have had a steady trickle of user issues
<enebo[m]>
So I think we are not that far from tagging so I don't think we need to worry too much about this yet
<enebo[m]>
I think by next week we can have this tagged out
<enebo[m]>
with issues for what is missing
<headius>
ok
<headius>
basshelal: how did you run Main btw?
<headius>
I have a theory
<basshelal[m]>
headius: Do you think this is a JVM issue or a macOS thing
<basshelal[m]>
I used IntelliJ's run button like a plebian
<headius>
novel
<headius>
running this via maven may be a cause
<headius>
that security thing on mac, SIP or whatever, prevents library paths from being propagated into child processes, like some of these posts mention
<headius>
specifically it prevents passing them through sh/bash
<headius>
`mvn` is such a script
<basshelal[m]>
I don't think I used maven to run it though
<headius>
our search path in jnr-ffi works to find the dylib files but it can't negotiate DYLD_LIBRARY_PATH for us
<headius>
huzzah, I got it to work
<headius>
basshelal: I'm not sure how intellij starts that up but it may be another script, and in any case it does not twiddle lib paths
<basshelal[m]>
oh yeah that's what's been happening
<headius>
the default java executable for the system will just choose whatever is the newest JVM installed and run that
<headius>
but I think it probably qualifies as one of these protected system executables
<headius>
so that is another lovely wrinkle with this
<headius>
try one of the JavaVirtualMachines installs directly
<headius>
enebo: we could possibly detect this in JRuby or jnr-ffi and warn or something
<headius>
or include in errors when we fail to load libs
<headius>
"DYLD environment variables are unset; JRuby may have been launched via a protected system executable? See http blah"
<headius>
or something
<enebo[m]>
wot
<headius>
I wonder how macos JRuby users are doing this.. perhaps they are simply not loading a stack of interdependent dylibs
<basshelal[m]>
ugh nothing
<enebo[m]>
I just reread that twice...because the latest java is loaded through a protected launcher it is not allowing us to load from the actual java loaded?
<basshelal[m]>
Calling my vm directly doesn't change this
<headius>
enebo: Apple security protocols prevent passing LD/DYLD_LIBRARY_PATH like variables from parent to child process when the child is among a set of protected system executables
<enebo[m]>
hahah
<headius>
basshelal: you're killing me Smalls
<headius>
ok
<headius>
so this is on an M1 machine right?
<headius>
let me crack mine open and try there
<enebo[m]>
another path to take could be to have our launcher follow and launch the real java
<basshelal[m]>
I'm on x86
<headius>
if this is still broken there I want to know it
<enebo[m]>
but only on M1 Macos
<headius>
oh but I thought I saw something about arm in your comments or something
<enebo[m]>
wot!
<enebo[m]>
:)
<headius>
enebo: this applies to x86, or at least I was able to reproduce the same error on x86
<basshelal[m]>
nope, good ole intel
<headius>
oh right I just saw the universal binary platforms
<headius>
ok
<headius>
well then I'm stumped
<enebo[m]>
ok well same idea just on macos period I guess
<headius>
what Java version?
<headius>
enebo: yeah
<enebo[m]>
assuming we can know the right java executable somehow
<headius>
I don't know that this is a property of any other OS
<enebo[m]>
your which worked I guess
<headius>
yeah but regardless of executable we could add to error some text about how these DYLD properties don't seem to be present, maybe that's your problem
<headius>
if those aren't there and we are failing to load a dylib it's probably likely related
<enebo[m]>
I can see the logic in why they think this is a good idea but not for a generic runtime like java
<headius>
or e.g. if you specify a search path and we don't see that in any DYLD vars
<enebo[m]>
Apple is not putting /usr/bin/java though is it?
<headius>
so custom search path for the file but no dl env to support it
<headius>
well I'm not sure
<headius>
that may not actually be the bug here
<headius>
it was a good guess
<enebo[m]>
If it is not Apple making this special launcher app then openjdk or whoever will move this away from a special directory
<headius>
for whatever reason these env are not propagating into the program for basshelal
<enebo[m]>
ok whatever reason I guess is the issue atm
<headius>
basshelal: what is `which java` now?
<enebo[m]>
lunch
<headius>
or what path are you using
<basshelal[m]>
`/usr/bin/java`
<headius>
sorry I mean the direct path to the install
<headius>
docs here make me wonder if it is better or not
<headius>
beats me man
<basshelal[m]>
why?
<basshelal[m]>
This is annoying because when someone has to launch their application they have to pass in this environment before hand, I wonder if putting that whole command in a script will work?
<headius>
Should be ok I think
<headius>
Might be a way to set this from within JVM but we might have to bind the native setenv function to do it 😀
<basshelal[m]>
wow
<basshelal[m]>
Btw the issuer used the old version because our README had it hardcoded into the copy-paste textbox so I made it more flexible here without having to update it every time
<basshelal[m]>
headius: I do wonder if this issue can affect yall on the JRuby side of things, that could be a problem, can users add and load their own dylibs in JRuby?
<basshelal[m]>
Also, should I add some better error message with the env variables like you had mentioned at some point earlier?
<basshelal[m]>
Like, maybe checking if the error message was because of `dlopen` failing and if so, in the exception message add the DYLD environment variables? Might not be a bad idea, pretty easy addition
subbu has joined #jruby
<headius>
Absolutely...ruby FFI support is pretty widely used
<headius>
It is how we manage to run a few libs that have native bits