<byteit101[m]> Ok, pushed more changes. Last question is what types of exception should I be throwing (RawFieldVariableAccessor)
drbobbeaty has joined #jruby
katafrakt[m] has joined #jruby
<basshelal[m]> Someone's having a very strange macOS library loading problem, I don't know if this is indeed the JVM bug he referenced or just more macOS strangeness
<basshelal[m]> Maybe you know more about this or know someone that can guide us in the right direction, I do hope it's just macOS just being weird again
<headius> I will try to repro today. Feels like that error might be a generic "I couldn't load" covering up the real cause.
<basshelal[m]> Actually "`image not found`" is a native error message returned from `dlopen` not JNR-FFI because you can find others online who have had this exact error message from `dlopen` ie: https://stackoverflow.com/questions/19776571/error-dlopen-library-not-loaded-reason-image-not-found
<basshelal[m]> Also what time is it in Yankeeland?
<headius> Early
<headius> I have a primary school aged son to get going in 15 min
<headius> If that is simply a dlopen error then I am back to wondering if this is an architecture mismatch
<basshelal[m]> Ah ok, thought I woke you up cuz I know you start your day much later
<headius> But I will have a look in an hour or so
<headius> My day is fluid 😀
<basshelal[m]> Sure no worries, you'll probably know better than me
<headius> basshelal: were you able to build this uapki project?
<headius> it fails to copy the first dylib it builds and then bails out
<headius> for me
<basshelal[m]> No, the sample repo the issuer gave already has the dylibs so I just used those
<headius> oh I see
<basshelal[m]> btw `nm` and `otool` show no errors with those libs on x86 Big Sur so I *assume* they're ok
<headius> basshelal: I'm leaning toward this being a quirk of MacOS library pathing
<headius> I can reproduce. Adding the resources dir to library.search gets past the first library and fails on the second
<headius> it is not seeing that the image is loaded probably because it sees a different name for the loaded library than what it's looking for
<headius> at the dlopen/dyld level
<headius> when I copy the libs to . they load
<headius> at the moment I would say this is not a jnr-ffi bug, since I can get the libraries to load from CWD, but I have no good explanation for why it cannot load the libraries simply by adding `library.search("src/main/resources")`
<headius> boo, element needs to handle multiline code snippits better... still have to cmd+shift to hit enter without it sending
<headius> ```
subbu has joined #jruby
<headius> deploy job is failing to auth for some reason
<headius> looking into it
<basshelal[m]> sorry was cooking lunch
<headius> enebo: I think you forgot to update VERSION
<headius> do I just update that one file?
<enebo[m]> oh on master?
<enebo[m]> where?
<headius> 9.3
<enebo[m]> oh for next snapshot
<headius> yes
<enebo[m]> hahah I thought you meant for release
<enebo[m]> ok yeah you just update release and run maven
<enebo[m]> it will update 4 poms
<headius> ok
<basshelal[m]> so when the second library comes to load `dlopen` and friends are not finding the library it depends on?
<headius> basshelal: I believe that's right
<basshelal[m]> But when the libs are on the project root they load??
<headius> the fact that it works fine from CWD tells me this is just a pathing issue
<headius> yep, works for me
<basshelal[m]> damn, ok I'll try that now
<headius> I do not have a better explanation for why it fails but it does not seem to like the way they are being loaded from the subdir and does not recognize them as having been loaded
<headius> enebo: pushed... I don't think that will fix the deploy builds but perhaps
<enebo[m]> headius: There is something special about version not being the actual release.
<enebo[m]> I think it being name SNAPSHOT fixes it
<headius> that's what I was thinking also
<headius> failed deploys correspond to the 17th which is when we released I think
<basshelal[m]> Yeah I was pretty sure it's a macOS thing, I suspect it's very similar to another issue we were facing with macOS library paths where the environment isn't being preserved.
<basshelal[m]> See: https://en.wikipedia.org/wiki/System_Integrity_Protection#:~:text=System%20Integrity%20Protection%20will%20also%20sanitize%20certain%20environmental%20variables%20when%20calling%20system%20programs%20when%20SIP%20is%20in%20effect.
<basshelal[m]> I'm not super sure but they feel related somehow
<basshelal[m]> Yep works for me when they're in the project root!
<basshelal[m]> That's... very strange and very annoying :/
<headius> it is!
<headius> 🤷‍♂️
<headius> would you agree this is probably not a jnr-ffi issue specifically though?
<basshelal[m]> Oh yeah absolutely, I hate to say it, but it's not our problem 😂
<headius> I will keep it open until we get confirmation that they have it working but I'm not sure how to help now
<basshelal[m]> Still I was very curious and quite concerned at first but it became obvious it wasn't us
<basshelal[m]> We can't help really, other than to tell him that it works when they are in the project dir
<basshelal[m]> which is actually better than the proposed solutions I had for him so I think it's a win
<headius> yeah it bothers me that . works and src/main/resources does not even though I add the latter to `library.search`
<headius> I tried full path too, didn't help
<headius> and clearly . is a relative path
<headius> security? what security? 😀
<headius> enebo: just occurred to me... deploy depends on other jobs passing so it won't ever deploy 9.4 snaps with current setup
<headius> I'm going to kill master build from k77ch7 merge so I can see if it deploys right on 9.3
<headius> we can manually trigger master snap deploys if we want
<headius> we need to start deciding what to tag off too
<enebo[m]> headius: it may not but I was hoping to not tag out 250 failures just so we get snapshots
<headius> yeah agree on that for sure
<enebo[m]> headius: I have tagged a few things already...most of what is left is doable
<headius> just saying it will either not deploy until green or we need to soften that requirement
<enebo[m]> and should be done but it is perhaps less than it looks too if you see probably 100+ are 3 things
<headius> sure... I want to get back to it but we have had a steady trickle of user issues
<enebo[m]> So I think we are not that far from tagging so I don't think we need to worry too much about this yet
<enebo[m]> I think by next week we can have this tagged out
<enebo[m]> with issues for what is missing
<headius> ok
<headius> basshelal: how did you run Main btw?
<headius> I have a theory
<basshelal[m]> headius: Do you think this is a JVM issue or a macOS thing
<basshelal[m]> I used IntelliJ's run button like a plebian
<headius> novel
<headius> running this via maven may be a cause
<headius> that security thing on mac, SIP or whatever, prevents library paths from being propagated into child processes, like some of these posts mention
<headius> specifically it prevents passing them through sh/bash
<headius> `mvn` is such a script
<basshelal[m]> I don't think I used maven to run it though
<headius> our search path in jnr-ffi works to find the dylib files but it can't negotiate DYLD_LIBRARY_PATH for us
<headius> huzzah, I got it to work
<headius> basshelal: I'm not sure how intellij starts that up but it may be another script, and in any case it does not twiddle lib paths
<headius> yeah this is working fine now
<headius> DYLD_FALLBACK_LIBRARY_PATH=`pwd`/src/main/resources java -cp
<headius> rs/headius/.m2/repository/org/ow2/asm/asm-tree/9.2/asm-tree-9.2.jar:/Users/headius/.m2/repository/org/ow2/asm/asm-util/9.2/asm-util-9.2.jar:/Users/headius/.m2/repository/com/github/jnr/jnr-a64asm/1.0.0/jnr-a64asm-1.0.0.jar:/Users/headius/.m2/repository/com/github/jnr/jnr-x86asm/1.0.2/jnr-x86asm-1.0.2.jar:target/uapki-test-1.0-SNAPSHOT.jar ua.koval4uk.Main
<headius> /Users/headius/.m2/repository/com/github/jnr/jnr-ffi/2.2.11/jnr-ffi-2.2.11.jar:/Users/headius/.m2/repository/com/github/jnr/jffi/1.3.9/jffi-1.3.9.jar:/Users/headius/.m2/repository/com/github/jnr/jffi/1.3.9/jffi-1.3.9-native.jar:/Users/headius/.m2/repository/org/ow2/asm/asm/9.2/asm-9.2.jar:/Users/headius/.m2/repository/org/ow2/asm/asm-commons/9.2/asm-commons-9.2.jar:/Users/headius/.m2/repository/org/ow2/asm/asm-analysis/9.2/asm-analysis-9.2.jar:/Use
<headius> boom
<headius> DYLD vars are not getting passed through maven or whatever other exec launchers are being used so we can't specify the path necessary
<basshelal[m]> niiiice
<headius> calling the java binary directly it works ok
<basshelal[m]> I'll try it now
<headius> I will comment
<headius> funny that it is insecure to allow env propagation of DYLD lib paths, but totally ok to include `.` in the default path 🙄
<headius> we may want to add env LD/DYLD variables to the error output
<headius> that would have cleared this up pretty quickly
rdubya[m] has joined #jruby
<headius> basshelal: I closed the issue since I think we have a pretty clear explanation now
<headius> gonna take an early lunch and do a Hades run to reward myself
<basshelal[m]> I still couldn't get it to work
<basshelal[m]> Because I'm an idiot not because you're wrong
<headius> oh yeah?
<headius> same error?
<basshelal[m]> Can't figure out how to start java from a terminal on macos 😂
<headius> fyi I used `mvn dependency:build-classpath` to get that part
<headius> just add target/uapkiwhatever.jar and main class and DYLD var and it should be fine
<headius> JVM installs should be in /Library/Java/JavaVirtualMachines ... we use a script called "pickjdk" to switch between them
<basshelal[m]> Cool, go have lunch I'll keep trying it and let you know
<headius> bbiab
<basshelal[m]> Nope, no luck, I'm doing something wrong :/
<basshelal[m]> s/=`/=$(/, s/`//)//
<basshelal[m]> Does `DYLD_FALLBACK_LIBRARY_PATH` appear in your `env`?
<headius> No semicolon
<headius> The env should be part of the Java command line
<basshelal[m]> Yeah still
<basshelal[m]> This is so annoying but it's bothering me to leave it, I want to win
<headius> Show me your session
<headius> 9.3 hung in some Signal.trap spec but the manual builds are working so it seems ok again
<headius> what's the command line you are running now?
<basshelal[m]> default macOS terminal application, zshell
<basshelal[m]> System.out.println(System.getenv("DYLD_FALLBACK_LIBRARY_PATH"));`
<basshelal[m]> `System.out.println(System.getenv("DYLD_LIBRARY_PATH"));
<basshelal[m]> These are both `null`
<basshelal[m]> From Main.java
<headius> I mean the java command line that is still failing
<basshelal[m]> `DYLD_FALLBACK_LIBRARY_PATH=$(pwd)/src/main/resources java -cp
<basshelal[m]> /Users/user/.m2/repository/com/github/jnr/jnr-ffi/2.2.11/jnr-ffi-2.2.11.jar:/Users/user/.m2/repository/com/github/jnr/jffi/1.3.9/jffi-1.3.9.jar:/Users/user/.m2/repository/com/github/jnr/jffi/1.3.9/jffi-1.3.9-native.jar:/Users/user/.m2/repository/org/ow2/asm/asm/9.2/asm-9.2.jar:/Users/user/.m2/repository/org/ow2/asm/asm-commons/9.2/asm-commons-9.2.jar:/Users/user/.m2/repository/org/ow2/asm/asm-analysis/9.2/asm-analysis-9.2.jar:/Users/user/.m2
<basshelal[m]> /repository/org/ow2/asm/asm-tree/9.2/asm-tree-9.2.jar:/Users/user/.m2/repository/org/ow2/asm/asm-util/9.2/asm-util-9.2.jar:/Users/headius/.m2/repository/com/github/jnr/jnr-a64asm/1.0.0/jnr-a64asm-1.0.0.jar:/Users/user/.m2/repository/com/github/jnr/jnr-x86asm/1.0.2/jnr-x86asm-1.0.2.jar:target/uapki-test-1.0-SNAPSHOT.jar ua.koval4uk.Main`
<headius> hmmm it looks fine
<basshelal[m]> I know right ughhhhh
<basshelal[m]> Is there a previous step I missed?
<headius> run `file \`which java\``
<headius> bleh
<headius> check if your java command is a shell script
<basshelal[m]> Dont think so
<basshelal[m]> `file $(which java)
<basshelal[m]> /usr/bin/java: Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit executable x86_64] [arm64e:Mach-O 64-bit executable arm64e]
<basshelal[m]> /usr/bin/java (for architecture arm64e):Mach-O 64-bit executable arm64e`
<basshelal[m]> /usr/bin/java (for architecture x86_64):Mach-O 64-bit executable x86_64
<basshelal[m]> * `file $(which java)
<basshelal[m]> /usr/bin/java: Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit executable x86_64] [arm64e:Mach-O 64-bit executable arm64e] /usr/bin/java (for architecture x86_64):Mach-O 64-bit executable x86_64 /usr/bin/java (for architecture arm64e):Mach-O 64-bit executable arm64e`
<basshelal[m]> * `file $(which java) /usr/bin/java: Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit executable x86_64] [arm64e:Mach-O 64-bit executable arm64e] /usr/bin/java (for architecture x86_64):Mach-O 64-bit executable x86_64 /usr/bin/java (for architecture arm64e):Mach-O 64-bit executable arm64e`
<headius> Aha
<headius> My son looked into this a little as well and found a page that says all Apple system binaries have this property
<headius> I wonder if this Java command in /use/bin qualifies
<headius> usr
<basshelal[m]> wait which property?
<basshelal[m]> what does yours look like then lol
<headius> That those environment variables are not passed from parent to child
<headius> $ which java
<headius> /Library/Java/JavaVirtualMachines/adoptopenjdk-11.jdk/Contents/Home/bin/java
<basshelal[m]> oh yeah that's what's been happening
<headius> the default java executable for the system will just choose whatever is the newest JVM installed and run that
<headius> but I think it probably qualifies as one of these protected system executables
<headius> so that is another lovely wrinkle with this
<headius> try one of the JavaVirtualMachines installs directly
<headius> enebo: we could possibly detect this in JRuby or jnr-ffi and warn or something
<headius> or include in errors when we fail to load libs
<headius> "DYLD environment variables are unset; JRuby may have been launched via a protected system executable? See http blah"
<headius> or something
<enebo[m]> wot
<headius> I wonder how macos JRuby users are doing this.. perhaps they are simply not loading a stack of interdependent dylibs
<basshelal[m]> ugh nothing
<enebo[m]> I just reread that twice...because the latest java is loaded through a protected launcher it is not allowing us to load from the actual java loaded?
<basshelal[m]> Calling my vm directly doesn't change this
<headius> enebo: Apple security protocols prevent passing LD/DYLD_LIBRARY_PATH like variables from parent to child process when the child is among a set of protected system executables
<enebo[m]> hahah
<headius> basshelal: you're killing me Smalls
<headius> ok
<headius> so this is on an M1 machine right?
<headius> let me crack mine open and try there
<enebo[m]> another path to take could be to have our launcher follow and launch the real java
<basshelal[m]> I'm on x86
<headius> if this is still broken there I want to know it
<enebo[m]> but only on M1 Macos
<headius> oh but I thought I saw something about arm in your comments or something
<enebo[m]> wot!
<enebo[m]> :)
<headius> enebo: this applies to x86, or at least I was able to reproduce the same error on x86
<basshelal[m]> nope, good ole intel
<headius> oh right I just saw the universal binary platforms
<headius> ok
<headius> well then I'm stumped
<enebo[m]> ok well same idea just on macos period I guess
<headius> what Java version?
<headius> enebo: yeah
<enebo[m]> assuming we can know the right java executable somehow
<headius> I don't know that this is a property of any other OS
<enebo[m]> your which worked I guess
<headius> yeah but regardless of executable we could add to error some text about how these DYLD properties don't seem to be present, maybe that's your problem
<headius> if those aren't there and we are failing to load a dylib it's probably likely related
<enebo[m]> I can see the logic in why they think this is a good idea but not for a generic runtime like java
<headius> or e.g. if you specify a search path and we don't see that in any DYLD vars
<enebo[m]> Apple is not putting /usr/bin/java though is it?
<headius> so custom search path for the file but no dl env to support it
<headius> well I'm not sure
<headius> that may not actually be the bug here
<headius> it was a good guess
<enebo[m]> If it is not Apple making this special launcher app then openjdk or whoever will move this away from a special directory
<headius> for whatever reason these env are not propagating into the program for basshelal
<enebo[m]> ok whatever reason I guess is the issue atm
<headius> basshelal: what is `which java` now?
<enebo[m]> lunch
<headius> or what path are you using
<basshelal[m]> `/usr/bin/java`
<headius> sorry I mean the direct path to the install
<basshelal[m]> oh ok
<basshelal[m]> `/Users/user/Library/Java/JavaVirtualMachines/openjdk-17.0.2/Contents/Home/bin/java`
<headius> I'll try 17 also
<headius> fails
<headius> back to 11 and it works... so maybe this is java-related after all
<basshelal[m]> lemme try on an 11
<headius> reported was on 11 so if this is a 17 thing it wasn't the cause for them
<headius> it is indeed not picking up this env
<headius> they may have broken something
<basshelal[m]> 11 receives the `DYLD_LIBRARY_FALLBACK_PATH` in `System.getEnv()` but still fails to `dlopen`
<headius> latest openjdk 17 still seems to fail for me
<basshelal[m]> yep
<basshelal[m]> how did you get it work before then??
<headius> 11 works fine for me
<basshelal[m]> which jvm?
<headius> $ java -version
<headius> openjdk version "11.0.4" 2019-07-16
<headius> OpenJDK 64-Bit Server VM AdoptOpenJDK (build 11.0.4+11, mixed mode)
<headius> OpenJDK Runtime Environment AdoptOpenJDK (build 11.0.4+11)
<headius> 16 fails
<headius> 14 fails
<headius> 13 works
<headius> seems to have broken after 13
<headius> hah almost all my 11 installs worked but temurin failed
<headius> now I have to know if they are backporting this breakage
<basshelal[m]> !Thats the one Ive been using right now
<headius> latest zulu 11 also fails
<basshelal[m]> Trying to download openjdk but im so clumsy on macos and on this tiny macbook air
<headius> so they "broke" something and backported it all the way to 11
<basshelal[m]> honestly this just means there's inconsistencies
<headius> that issue I linked was just resolved last summer, I wonder if it is related after all
<basshelal[m]> i don't know what to do
<headius> AARGH
<headius> remove FALLBACK
<headius> that works on latest 17 for me as well
<headius> so this other variable is not propagating, seriously?
<headius> but you said it was in there?
<headius> what a frustrating mix of results
<basshelal[m]> ahhhhh
<basshelal[m]> yep without fallback 11 coretta works
<basshelal[m]> and temurin 11
<headius> I got that env from https://stackoverflow.com/a/3172515/152946
<headius> it sounded like a better choice for something like this
<basshelal[m]> And openjdk 17 as well
<basshelal[m]> damn
<basshelal[m]> thats good?
<headius> docs here make me wonder if it is better or not
<headius> beats me man
<basshelal[m]> why?
<basshelal[m]> This is annoying because when someone has to launch their application they have to pass in this environment before hand, I wonder if putting that whole command in a script will work?
<headius> Should be ok I think
<headius> Might be a way to set this from within JVM but we might have to bind the native setenv function to do it 😀
<basshelal[m]> wow
<basshelal[m]> Btw the issuer used the old version because our README had it hardcoded into the copy-paste textbox so I made it more flexible here without having to update it every time
<headius> Ah yes
subbu has quit [Ping timeout: 240 seconds]
<basshelal[m]> headius: I do wonder if this issue can affect yall on the JRuby side of things, that could be a problem, can users add and load their own dylibs in JRuby?
<basshelal[m]> Also, should I add some better error message with the env variables like you had mentioned at some point earlier?
<basshelal[m]> Like, maybe checking if the error message was because of `dlopen` failing and if so, in the exception message add the DYLD environment variables? Might not be a bad idea, pretty easy addition
subbu has joined #jruby
<headius> Absolutely...ruby FFI support is pretty widely used
<headius> It is how we manage to run a few libs that have native bits