<headius>
yeah at this point there's no issue here but we'll see what they come back with
<nirvdrum[m]>
This is probably a dumb question, but JRuby is failing to launch if I don't have `$JAVA_HOME` set. `java` is available on the `$PATH`. If I run `jruby.bash`, everything starts up fine. Is this expected?
<headius>
not sure what you mean... it works but it doesn't work?
<headius>
we use the location of `java` and try to infer a JAVA_HOME from that... if we fail then we fall back on Java 8 behavior that doesn't depend on knowing if modules are present
<headius>
jruby.bash and the native jruby exe should both work the same way
<headius>
so I'm not sure which way you are trying to run that is failing
<headius>
there's no exec in either launcher if you run with --environment so this is something broken with rbenv or your setup
<nirvdrum[m]>
I just grabbed the tarball and extracted that locally. Now I do get the --environment output.
<headius>
seems like it is not setting the binary to be launchable by you or something?
<nirvdrum[m]>
Maybe ruby-build did something weird.
<headius>
I dunno how rbenv works
<headius>
or at least I don't know what it does that could cause this
<nirvdrum[m]>
I've been using it for years. This is the first time I've had a problem with JRuby.
<headius>
you could try to reproduce with tarball + gem install jruby-launcher and maybe it is a problem with the native launcher
<headius>
otherwise I don't know what to tell ya... need more input
<nirvdrum[m]>
Weird. The md5sum is different from the tarball and the ruby-build install.
<nirvdrum[m]>
I guess I'll dig in on that side. Thanks for confirming that this shouldn't be problematic.
<headius>
ok
<headius>
looks like it does install the native launcher, so perhaps start by confirming the native launcher works for you with a clean tarball
<headius>
I think I see some possible problem
<headius>
oh maybe not
<headius>
it does some monkeying around with the other scripts in bin/ but that doesn't seem to affect the main ruby/jruby command
<headius>
bleh why are they rewriting these shebangs
<headius>
🤷♂️
<headius>
what are you benchmarking? I could give it a shot here
<headius>
enebo: how much do you hate the idea of adding an optimize flag that turns on indy?
<enebo[m]>
you mean --optimize?
<headius>
yeah
<enebo[m]>
I don't know. It is obviously easier to remember
<headius>
we have talked about it for years and we are no closer to enabling heavy optz all the time
<headius>
it just adds too much to startup
<enebo[m]>
As a name it means nothing
<enebo[m]>
but as far as other names I don't know java had --server
<headius>
it would at least be an obvious flag we could put in --help rather than having this buried in --properties
<enebo[m]>
--optimize > --fast so I think that was an outcome of older discussion
<headius>
yeah
<enebo[m]>
--prod is a dual of --dev
<headius>
I think people understand that "optimize" might mean it takes longer to get going, or other trade-offs
<enebo[m]>
although --prod implies you should use it for prod
<headius>
--prod isn't terrible
<enebo[m]>
and it is not a duality since no options is between them
<headius>
better than --fast and worse than --optimize to me
<headius>
I think we talked about --indy too but that means nothing to anyone but us
<enebo[m]>
well I am ok with --optimize. It just means something will happen and I am guessing 100% of people who think performance is always good will just use it and then complain about warmup
<enebo[m]>
but that is on them a bit
<headius>
--prod might make people think we will also set rails to production or something
<headius>
maybe
<enebo[m]>
We have just seen it quite a bit with --fast
<headius>
--andale
<enebo[m]>
people look at these terms as positive ones
<headius>
yeah maybe it is just me but optimize has less of that implied positivity
<enebo[m]>
but I think --compile.invokedynamic is too tough to remember
<headius>
more like "we will make more of an effort"
<enebo[m]>
I don't think I agree but I do think something simpler to remember is a good thing
<enebo[m]>
--indy is simple too but not something anyone will remember or think of doing
<headius>
maybe I will poll twitter
<enebo[m]>
HAHA
<enebo[m]>
you can try
<enebo[m]>
twitter is a good way of getting more loud and fringe opinions but I have no other suggestion
<enebo[m]>
headius: I will say I don't dislike --optimize as much as I think people will optimistically use it
<headius>
tweeted
<headius>
it seems like the best option so far
<enebo[m]>
The only downside will be some reports of confusion why it is not optimized because it runs slower
<headius>
best single-word flag so far
<enebo[m]>
slower == slower warmup
<enebo[m]>
or times without taking warmup into consideration
<nirvdrum[m]>
headius Looking into this more, ruby-build installs the jruby-launcher and that's what's failing to start. If I install that gem in my local JRuby download, I run into the same problem.
<headius>
ok perhaps it is not building the binary correctly?
<headius>
this is the first we have heard of this particular error so we will need more info about your env
<nirvdrum[m]>
Possibly. I don't really know how that works. If I set `$JAVA_HOME` to a vaild Java home directory, everything works.
<nirvdrum[m]>
Generally, it's just blank.
<headius>
check what the `java` command is... executable? symlink to where?
<headius>
we do need to be able to find JAVA_HOME but it should be inferred on most platforms from the `java` command
<headius>
I don't understand why it gives the execve error even with jruby --environment
<headius>
hmm
<headius>
tracing system calls might show what it actually fails to launch... there are a few system utils we might be using to figure out JAVA_HOME
<headius>
something non-posix that might not be in a minimal container, for example
<nirvdrum[m]>
It's a symlink. In this case, it's set up by Homebrew for Linux. But, Ubuntu's update-java-alternatives would also have set a symlink.
<nirvdrum[m]>
I did set JAVA_HOME to the resolved parent directory in the Homebrew installation and that allowed the launcher to work.
<nirvdrum[m]>
I can file an issue. I didn't mean to derail your day.
<headius>
that's what I'm here for
<headius>
sure, open an issue and dump some env info about how that symlink is set up
<headius>
we do a best effort to find the real JAVA_HOME from even deeply-nested symlinks, but we can't prepare for all possible rigs
<headius>
nirvdrum: that is really more of a meta-issue
<headius>
the general challenge is inferring JAVA_HOME on all environments when they each chop up JDK installs a different way
<headius>
it would be best if you filed a more specific issue for this, because I have never seen it before
<headius>
you're the first to be doing this with a homebrew linux JDK install, perhaps
<nirvdrum[m]>
Will do.
<headius>
worst case it should still be able to launch but not use JDK module flags we need for full compat
<headius>
exec failing is weird
<headius>
long lunch, bbl
<headius>
enebo: despite conversations on twitter --optimize is still my favorite
<enebo[m]>
yeah I am ok with it
<enebo[m]>
it will have similar problems to --fast but it is easier for people to remember
<headius>
I like it better than -O* precisely because the -O flags already have meaning to people
<headius>
it won't mean the same on JRuby so I think that is a messy way to go
<enebo[m]>
I also think the people suggesting that HIGHLY OVERESTIMATE how many people know anything about a C compiler and its flags
<enebo[m]>
So it won't really help too many people to remember it. A strong yet simple name is much easier
<enebo[m]>
The unfortunate thing is any simple name will be interrpeted simply
<enebo[m]>
a confusing opaque option will likely not get implied meaning but will probably be hard to remember
<enebo[m]>
So I am ok with optimize because people will imply some meaning but it will solve the issue of knowing what to type to try out more indy
<enebo[m]>
More or less my opinion is that a good name is hard to find
<headius>
I agree
<enebo[m]>
I just realized a good one!
<enebo[m]>
-Oi
<headius>
Oi!
<enebo[m]>
exclamation point can be even more indy
<enebo[m]>
-Oi and -Oi!
<headius>
I did think about having a parameterized flag, like with +indy
<headius>
But it's unclear if it's any better than what we have now, other than being shorter
<enebo[m]>
well oi as a term is memorable in the sense it is funny...at least to english speakers how have heard oi
<enebo[m]>
but I am not really saying i think we should have oi as much as it is pretty funny and memorable (to me)
<headius>
It also ends up being fine grained again, and I wanted an optimized flag that gives users the "best" settings for straight line perf
<enebo[m]>
I actually like the notion of something being more opaque just because it removes implicit misunderstanding
<headius>
Yeah it would be fun
<headius>
The other problem is that the best optimized mode might be different from runtime to runtime. Like on graal we might want to disable the fixnum cache but should that be in this opaque flag or a separate one?
<enebo[m]>
headius: I was also thinking about recipes of optimizations
<enebo[m]>
if we itemized what things did then they could be aggregated by something which encompasses them but then they should also be individually done (like disabling fixnum cache)
<enebo[m]>
I don't really want to get bogged down on this though. A common simple to remember name for a realistic workload makes sense. It just needs to carry some implied notion that it has a warmup tradeoff
<headius>
Well perhaps this could play off of the .java_opts files we have
<headius>
--dev tweaks are all in a config file in the bin directory named appropriately
<headius>
If we added a flag that essentially picked a specifically named optimization config file then people could mix and match as they like but the default file would just turn on invoke dynamic
<headius>
--config:optimized to load bin/.optimized.java_opts
<headius>
Just spitballing here but your use of the term recipe makes a lot of sense
<enebo[m]>
yeah I think the general idea of this makes more sense if we have multiple options which only make sense in particular environments
<enebo[m]>
we definitely have some so maybe this is mostly something to consider if we decide to add more
<enebo[m]>
I don't like the idea of custom -- options if when people report we cannot see that trivially in the bug repiort
<enebo[m]>
but we have a template so people will get a command which will perhaps print out those configs
<headius>
--environment already includes anything in those opts files so we have that
<headius>
Wow where is it getting that weird path to exec
<headius>
nirvdrum: you see that path it tries to exec? It looks like a dir
<nirvdrum[m]>
Yeah. I need to look into that more. That directory does exist. It's OpenJDK 15 installed by Ubuntu.
<nirvdrum[m]>
But the only reference I can find to it as an env var I have named CUSTOM_JAVA_HOME.
<nirvdrum[m]>
Maybe jruby-launcher is doing a suffix match?
<nirvdrum[m]>
It looks like my javac comes from that installation.
<nirvdrum[m]>
That seems a more likely candidate.
<headius>
We don't do any suffix matching but that is very interesting
<headius>
We also don't look for javac so that wouldn't be the vector
<headius>
And --environment with no other flags does the same thing? I am really confused by that because it shouldn't try to exec anything in that mode
<headius>
The info it spits out would be a big clue
<headius>
You could try -Xhelp to the native launcher and there's a debug log we could try
<nirvdrum[m]>
Same issue.
<nirvdrum[m]>
I'll dig into the launcher source and see if I can work out what's going on.
<nirvdrum[m]>
Okay. That could be something. /usr/bin/java resolves to that directory.
<nirvdrum[m]>
As far a I know, Ubuntu manages all of that.
<headius>
That's seems wrong
<headius>
I'll have a closer look when I'm off mobile
<nirvdrum[m]>
So, /usr/bin appears earlier in the PATH than /home/linuxbrew/.linuxbrew/bin. It looks like `findOnPath` is picking the entry out of there and just assuming if it's named "java" that it's an executable.
<nirvdrum[m]>
The shell resolution logic is evidently different.
<headius>
Yeah strange
<nirvdrum[m]>
I'll see if I can reproduce in Docker.
<headius>
That would be a big help
<headius>
We need to be checking executable and directory flags
<headius>
That's a clear miss
<headius>
If we saw it is a dir we would move on to the later path entries and get the right one
<headius>
Really weird that this is resolving to a dir though
<headius>
The bash logic may be doing [ -x ... ]
<headius>
All our problems would be solved if there was a way to compile a bash script into a binary
<nirvdrum[m]>
Okay, so I think I've found the problem, but I need to dig into Ansible to be sure. I have some Ansible config that runs `update-alteratives` and sets the value for java. While `update-java-alteratives` takes a directory, `update-alternatives` will change only `/etc/alternatives/java` and setting it to a directory is incorrect. If I try running `update-alternatives` manually, it won't let me set the `java` value to a directory. But, my
<nirvdrum[m]>
Ansible config has never failed, so I'm worried that module is bypassing the Ubuntu supplied binary and just manually reimplemented it in Python.
<nirvdrum[m]>
If that's the case, this has been broken for me for years.
<nirvdrum[m]>
So, my analysis may be incorrect. Or, I broke my Ansible config along the line. Or Ansible changed its implementation. Or jruby-launcher changed its resolution mechanism. Hard to say.
<headius>
It does seem like the Java command provided through alternatives is broken on your system somehow
<nirvdrum[m]>
Deleting /etc/alternatives/java was enough for the PATH search to find the Homebrew one.
<headius>
Does /usr/bin/java even work? I have never seen that resolve to a directory
<nirvdrum[m]>
No. But I never fully qualify invocations anyway. It just got picked up on the PATH later on.
<nirvdrum[m]>
I guess I don't run a ton of Java applications directly from the command line.
<headius>
So it seems like your alternatives got screwed up somehow, but we also are not doing the search for an executable properly and happily tried to execute a directory
<nirvdrum[m]>
Yeah. A comedy of errors.
<nirvdrum[m]>
Thanks for helping track it down.
<nirvdrum[m]>
Does anyone know if there's a reason that fast_blank has source for a JRuby extension but doesn't have one pushed to rubygems.org?
<nirvdrum[m]>
Answering my own question, the gem ships with a precompiled JAR. But, `gem install fast_blank` still tries to compile the C native extension, causing it to blow up.