marcheiligers[m] has quit [Ping timeout: 268 seconds]
kares[m] has quit [Ping timeout: 268 seconds]
marcheiligers[m] has joined #jruby
kares[m] has joined #jruby
drbobbeaty has quit [Ping timeout: 240 seconds]
drbobbeaty has joined #jruby
<headius>
good morning!
<headius>
ahorek: ok we can sort that out today
<headius>
byteit101: I will review today and look at that failure
<kares[m]>
<ahorek> "headius: any thoughts about..." <- this is a great idea but the gotcha is you can not test against Java 8 as javac in that release does not support --release ... but otherwise much better than doing manual casts
<headius>
kares: I believe that plugin knows to apply the Java 8 flags on 8
<headius>
it gets around having a build that requires either 8 or 9 but not both (or you have to do custom config)
<headius>
I will give it a try in some of our other 8-supporting projects... almost all the dependencies we control have basic module support and need release flag now
<headius>
I disagree with those saying you can't have the compiler plugin do both
<headius>
if you are doing --release 8 and you build on 8 omitting the release flag should be just fine
<headius>
kares: I did not get around to patching the RubyArray#collect issue this weekend
<kares[m]>
NP - do not worry about that
<headius>
I am confused why it did not break for jossl releasees earlier last year though
<kares[m]>
silly me not using .dup
<headius>
this has been in 9.3 since .0
<kares[m]>
I added the collect hack in the release I did on Friday
<headius>
ok
<headius>
I did not notice release, thank you for that
<headius>
so you just patched that and built with 8
<headius>
the snippit from that bug does appear to work in jnr-ffi and it
<kares[m]>
fixed both regressions - doing a final round of TLS 1.3 testing -> seems some real-world scenarios when the protocol is forced have issues.
<kares[m]>
this is kind of expected (the porting on the OpenSSL logic is incomplete) but I still would like to try figure smt out as a small work-around ...
<headius>
it's the smallest example I've seen
<kares[m]>
after that I plan on pushing 0.12.1
<headius>
ok
<kares[m]>
we'll also have CI back and 🟢 so that will be nice
<headius>
oh yes that will be great
<headius>
hey can you point me to release process for jossl these days? I would have done a quick rebuild and release on Friday but I don't know the process
<headius>
I would also like to contrib but failing tests and my own unfamiliarity with build processes have stopped me from messing with it too much
<headius>
I never know whether I should be running rake targets or maven targets for anything
<headius>
ick, jffi has conditional compilation for a class that accesses Unsafe...
<headius>
I forgot about that
<kares[m]>
yeah - let me document all of that better. locally I mostly use rake targets which shell-out to mvnw
<kares[m]>
rake test
<kares[m]>
rake jar
<kares[m]>
we still rely on the maven plugin to do the magic of building the gem (mvn package) but need to skip tests as the ruby-maven-plugin's way of running tests is outdated
<headius>
ok
<kares[m]>
after that it's just a gem push - I'll review the BUILDING.md with up-to-date instructions (it's outdated due we no longer push to sonatype just rubygems)
<headius>
ok thank you
<headius>
I suppose varhandles are the answer to heap memory access
<headius>
jnr-posix also has conditional building for sun.misc Signal stuff
<headius>
I'm going to spike a pom change for JRuby
<headius>
if we can start safely building with 9 it would be more convenient and less likely to cause a problem
<headius>
enebo: mrnoname1000 said we could go ahead and merge the jruby.sh script now and he will continue to improve it
<headius>
I say we go for it and start having it in the test cycle on master
enebo[m] has joined #jruby
<enebo[m]>
ok
<headius>
good thing about this is if it doesn't work I can throw something at him and make him fix it
<enebo[m]>
headius: sure and it is on our main devline so it is the best place to try it
<headius>
yeah we have at least a few weeks of bake time
<headius>
stack overflow issue in ostruct from rsim also overflows on CRuby
<headius>
ostruct aliasing all these methods is a bad pattern imgo
<headius>
in my grand opinion?
<headius>
enebo: 3.1 stdlib + tests merged to master
<headius>
shell script also (retargeted the PR from 9.3 to master without issues)
<headius>
I will do some cleanup to remove jruby.bash (or have it invoke jruby.sh) and modify build to copy jruby.sh for bin/jruby
<enebo[m]>
headius: I don't know if we have any ruby builders assuming that name. I would at least softlink
<headius>
not a fan of softlinks since they would be borked on Windows... I am thinking relaunch jruby.sh with a deprecation warning
<enebo[m]>
headius: yeah it can be a copy also but I am not sure we care if a .bash link is borked on windows. I don't know if we run on cygwin anymore or care...long live WSL
<enebo[m]>
I guess WSL maybe will get confused if you try and run windows install from WSL
<headius>
right that's a fair point
<headius>
I will push a PR for you and mrnoname to look at
<headius>
I think I fixed all relevant references to jruby.bash but I'm sure mrnoname will have some opinions on my jruby.bash stub
<headius>
enebo: hey I am doing a call tomorrow with some folks from Google and folks that are interested in gRPC/protobuf support on JRuby to discuss how to make it happen
<headius>
dunno if you want to be on that or not
<enebo[m]>
headius: I don't think so. I am heads down on getting our 3.x stuff smashed down. I would really like to tag off stuff soon but we need more things fixed first
<headius>
ok I'll let you know how it goes
<enebo[m]>
For some reason I thought protobuf stuff already had some bindings for JRuby
<enebo[m]>
or perhaps someone just made a library of sorts and it was not google itself
<headius>
yeah I think it was a third-party that did some protobuf work, but it is not up to latest gRPC (which uses newer tech) and the hope is that gRPC would support JRuby like it supports Ruby right now (via ext)
<headius>
of course you can just call the Java gRPC stuff from JRuby but full support would allow defining protobuf types from Ruby and reducing the gap
<enebo[m]>
ah I see
sagax has joined #jruby
<headius>
this might have relevance to the "true fields" PR that byteit101 just put together
<byteit101[m]>
The new Advancedâ„¢ Java Integrationâ„¢
<byteit101[m]>
good timing I guess
<headius>
yeah given that feature with normal Ruby types, we can present a normal set of Java fields to something like protobuf
<headius>
not sure if that is strictly necessary since this usually works with opaque byte[] blob and views, but it seems relevant
<byteit101[m]>
Oh, did you ever inspect the JVM bytecode generated from the fields?
<headius>
I did not but I'm circling around to PR review now
<byteit101[m]>
Oh, I should push the jrubyfx changes I have staged for this so you can see the integration maybe?
<byteit101[m]>
Cool
<byteit101[m]>
Thanks!
<headius>
that's up to you but it would be nice to see
<byteit101[m]>
I'm actually quite pleased with it with all the new changes in the past years
<byteit101[m]>
I guess I was waiting until I nailed down the config to commit, and since it all works now I think I can commit that
<headius>
how would you feel about doing a blog post to describe this work and the concrete subclass work for JRuby users?
<headius>
mostly just showing examples and how it integrates better with stuff like JavaFX (which would also show how we have been improving JRubyFX)
<headius>
we introduce these great features and nobody knows about them
<headius>
enebo: we need to discuss how to deal with this changed RubyArray#collect signature
<byteit101[m]>
Hmm... I could be persuaded to do that, though maybe in a few weeks. Been rather busy recently, I think I should have more time for that after the 16th of Feb
<headius>
it returns RubyArray in 9.2 and IRubyObject in 9.3 due to an oversight when I switched which method body gets bound to #collect
<headius>
I'm not sure what is best now, leaving it as is or restoring it to the 9.2 return value
<headius>
byteit101: yeah there's no rush... just doco on wiki that we could tweet or blog-link works too
<headius>
and I know there's some there already
<byteit101[m]>
Oh drat, I haven't cleaned up jrubyfx commits yet, still peppered with debug statements. I'll try to clean that up, but I can gist you the gist :-P
<headius>
no problem, I get the PR and can still review
<byteit101[m]>
Then I don't call copy_fxml_instances
<byteit101[m]>
Also similar thing addeed to the last line of module/fxml_raw_accessor: + java_field "@javafx.fxml.FXML #{type.java_class.name} #{symbol_name}", bind_variable: true
<byteit101[m]>
I would say it's straightforward, but that's part of jrubyfx's heavy reliance on generated shenanagains
<byteit101[m]>
fxml_helper is where most of the recent magic is going
<headius>
JavaFX's use of reflection and fields definitely stresses the more exotic JVM languages
<enebo[m]>
hmmm
<enebo[m]>
headius: Wasn't this because collect could return something not an Array?
<enebo[m]>
headius: collect with no block returns an enumerator
<headius>
Yes
<headius>
This is not about what the Ruby bound collect returns as much as restoring the old signature in Java
<headius>
To fix it I would flip the return value back in Java and add a new bound method collect2 or something with the less awful name
<enebo[m]>
Ah so you could possibly restore the old signature and name the jrubymethod yeah
<enebo[m]>
Did I miss this being an issue for something?
<headius>
We can't fix the existing 9.3 releases with this incompatibility though
<headius>
Yeah I'm not sure if you tracked the jossl issues on Friday but it was built against 9.3 and used the new signature which made it blow up on 9.2
<enebo[m]>
oh
<enebo[m]>
I didn't see that
<headius>
We got a few reports within minutes of that gem being pushed so I had to yank it and kares is working on a fix
<headius>
The gem was also built on Java 9 so we discovered the build needs to be tweaked but that is trivial to fix
<enebo[m]>
I am not sure how common it is to call collect from a native ext
<enebo[m]>
I saw the classic bytebuffer mismatch
<enebo[m]>
I just didn't see this collect thing
<enebo[m]>
So we have 3 cases to consider I guess
<headius>
Users on 9.2 might see either issue depending on how they load openssl. Users on 9.3 would only see the bytebuffer issue on Java 8
<enebo[m]>
9.2 which still send RubyArray which works with older and current jossl
<enebo[m]>
9.3.{0-3} which returns IRubyObject
<enebo[m]>
jruby-9.3 which can be either
<headius>
Yeah
<enebo[m]>
So how did 9.3.3 even work with an older jossl?
<headius>
It's unfortunate that we can't have overloads based on return type but this ain't no Scala
<enebo[m]>
Did it just assume IRubyObject in older jossl
<headius>
I am unclear about that myself. The code that calls collect might be newer and so this is the first time it was built against 9.3 using that method
<headius>
Java won't let you fudge a method call to a super type so it had to have been explicit
<enebo[m]>
oh if so then it is worth considering how likely this is to happening again
<enebo[m]>
"if so" == it was never in a jossl release before
<enebo[m]>
I feel like calling collect() from Java is going to be pretty uncommon
<headius>
That is my concern
<headius>
Going forward anyone else that decides to write an extension that calls this collect method will end up producing a binary that doesn't work on 9.2
<enebo[m]>
The safest is to leave it as RubyArray
<enebo[m]>
we should not narrow return values from bound @JRubyMethods
<headius>
And without replacing the signature there's no option that's compatible with both release branches
<headius>
Since there's only the one collect
<enebo[m]>
but with that said a public method is the only way to access this logic and we probably do not tend to make XXXCommon methods as public
<headius>
I don't know if I agree about narrowing return types but it does raise a question about the difference between our public Java API versus methods bound into Ruby
<enebo[m]>
I think if we restore original signature and just make @JRubyMethod(name="collect") public IRubyObject _collect()....
<headius>
Having narrower types on the Java API is clearly preferable but what happens when we need to widen that type as in this case
<enebo[m]>
well the problem is any Ruby API change in the future can break it
<headius>
And not just for users but for our internal use of the Java API
<enebo[m]>
So we future proof by just returning IRubyObject
<enebo[m]>
yeah for internal is different
<enebo[m]>
I am just talking about @JRubyMethod
<headius>
Using IRubyObject everywhere means we have to constantly cast to the expected type even when we know what the expected type will be
<headius>
It's not a big deal but it's messier
<enebo[m]>
My position is that we should not actually be using @JRubyMethod java APIs internally or externally by native extensions
<headius>
That's an interesting thought. The method binding logic does currently warn if a method throws exceptions and it could certainly check if a return type is narrower than IRubyObject
<enebo[m]>
but we never made things in such a way where we have a real API for calling stuff internally/native-extension wise
<headius>
That seems reasonable but maybe we should start naming those methods in a less java-friendly way
<headius>
ruby_collect()
<enebo[m]>
So if we use collect as an example I almost feel like we should have a @API public RubyArray collect(...)
<headius>
I have lived the idea of the API annotation if we actually use it and publish something based on it
<enebo[m]>
This is an older idea that we can generate an embedding API java doc by just annotating the methods we endorse
<headius>
Liked
<headius>
It would not be difficult to add such auditing into rake-compiler as well
<enebo[m]>
The second half was the idea of just having a lot of static import methods in an .embed/.api package
<headius>
I wish there was something like the deprecated annotation but only for warning people about internal API usage
<enebo[m]>
so most entry points are through static helpers but you have to call methods on objects and forcing static methods for that is pretty icky ergonomically
<enebo[m]>
RubyArray =- new_ary(5);
<enebo[m]>
Err
<headius>
The static method scheme might be easier for porting C extensions but I would not expect a typical job a user to go that route
<enebo[m]>
RubyArray elt = new_ary(5);
<headius>
Java user
<enebo[m]>
elt.collect(...)
<enebo[m]>
just to get the objects
<enebo[m]>
no one will new RubyArray() in our APIs
<headius>
If we really wanted to do this right we build up a set of interfaces for all the core types and have a parallel API on our core classes that implements those interfaces
<enebo[m]>
we have a bunch of convenience methods already for making reasonable instances of Ruby Objects so I think we just push those into static helpers
<headius>
So only the methods we want to expose make it to the interfaces and we have an explicit external API for core classes
<headius>
The API annotation is basically that but without any real way of enforcing it
<enebo[m]>
but we don't want to enforce it I don't think
<enebo[m]>
we will not be able to know all the things people will need so public is their escape hatch and then our clue later we missed something
<enebo[m]>
but I think we could make something to lint their calls into jruby modules that will flag out unauthorized calls
<enebo[m]>
this though is all a bunch of work :)
<headius>
jruby-lint could be expanded to inspect Java code and see whether it calls internal APIs
<headius>
I'm just saying the API annotation alone does nothing if we have no other mechanism for encouraging people to stick to the API
<headius>
Maybe that's just publishing docs
<enebo[m]>
yeah but going back to static methods I think for construction this is very simple and just a single laundry list of types
<enebo[m]>
the annotation of allowable calls is just reflecting the reailty we are OO
<headius>
I just see people using a Java IDE to develop JRuby extensions and then every public method is going to be visible
<headius>
Probably without an easy way to see the annotations on those methods
<enebo[m]>
yeah that is true
<headius>
Nobody wants to have to go back to a document every time they call a method to know if they're calling the right one
<enebo[m]>
but I think it is also an uncommon activity
<enebo[m]>
if we had an extension guide which had a command and a few small documents it is probably ok
<headius>
These are varying levels of both effort on our part and reduced effort on the user's part
<enebo[m]>
I think intellij goto def on classes will still show annotations
<enebo[m]>
yeah
<enebo[m]>
unlimited resources and we would put out an idea plugin for this
<headius>
A linter is a middle ground where they would have to go back and edit their code and an explicit set of API interfaces is the highest amount of effort for us but guarantees they'll never call an internal method
<enebo[m]>
It also could provide an ability to open an issue when a method they want is not endorsed
<headius>
In the short term though I think we are agreed that we should restore the old signature and add a separate method for the new one
<enebo[m]>
yeah
<headius>
Possibly deprecating the old signature since we know it doesn't work with early 9.3 releases and breaks 9.2 compatibility?
<enebo[m]>
it is least risk and I wonder how big our project would be if we removed all deprecated methods :)
<enebo[m]>
We may win a duke award for project with the most deprecated methods
<headius>
I think we should also split into two collect methods, one that returns RubyArray and requires the block, and one which returns any numerator and does not take any block at all
<headius>
An enumerator
<enebo[m]>
we can remove them ALL for JRubyWarHammer 40k release
<headius>
Damn voice typing
<enebo[m]>
I am pretty impressed with your voice typing
<enebo[m]>
I like the idea it might have just read out JRubyWarHammer 40k as individual characters
<headius>
I have to take care not to use jargon it won't recognize, but sometimes it still falls down on unusual terms like enumerator
<headius>
I can't say I've ever heard someone not in IT use the term "enumerator"
<enebo[m]>
you obviously never worked for the government during census collection
<headius>
enumerate, sure, but enumerator?
<headius>
"one who enumerates" but I have never heard anyone outside programming use it
<headius>
in investigating this behavior I figured out why define_method visibility seems so weird and unpredicatable: they only use the frame visibility if the frame cref matches the target module
<enebo[m]>
yeah this is pretty weird
<enebo[m]>
Makes me think this will get fixed in some fashion perhaps
<headius>
nearly all cases of def obj.foo or obj.define_singleton_method will not match cref class and use public, so it seems better to me to make it explicit
<headius>
I have obvious reservations about merging my PR without some buy-in from ruby-core here
<enebo[m]>
yeah I think seeing what they say makes sense but our behavior is different already right
<enebo[m]>
or can be different
<headius>
no we match right now
<headius>
because of that check I linked above (which is mostly right but missing some nuance from CRuby)
<enebo[m]>
ah so that ostruct change affects MRI too?
<headius>
it doesn't because they don't statically flag methods as needing a frame... they always frame
<headius>
we warn about this alias because define_singleton_method needs a frame in a tiny percentage of use cases and we won't know that define_singleton_method! needs a frame
<headius>
I wanted to chat about that also
<headius>
if we see someone aliasing a method to a new name that never existed before, perhaps it's safe to allow the alias and just add the new name to our frame-aware method tables?
<headius>
so the warning would be reduced to "define_singleton_method! already compiled into existing code; alias may not work as expected"
<headius>
or something like that
<headius>
we can also throw out compiled code that depends on frame-aware method list but that doesn't help code on stack
<headius>
which is an even smaller percentage of cases
<enebo[m]>
if we added aliases to frame-aware it would have removed seeing this issue with define_singleton_method but it may affect performance too
<headius>
that was another possible fix on our end for ostruct, just add those core methods as frame-aware
<headius>
the bang versions
<enebo[m]>
I don't know how many aliases end up being called a lot. It feels like a huge class is to inject a method which aggregates through the original method
<headius>
obviously I didn't like cluttering up that table unnecessarily
<headius>
right
<headius>
just aliasing without wrapping is rare
<headius>
and wrapping breaks on CRuby too
<headius>
we warn in part because usually you alias AND wrap and the wrapper will be broken
<enebo[m]>
So we alias a method which already exists and it is frame aware and we know that at that point
<enebo[m]>
so we put out a warning right now saying it might be broken?
<headius>
yes
<enebo[m]>
Believe it or not I have been looking at comments on this but I needed to restate that :)
<headius>
we warn that this method is known to access caller's frame and alias (with or without wrapping) may not work right
<enebo[m]>
Adding that alias in that case will not likely fix many issues if we need that frame
<enebo[m]>
that == the original method
<enebo[m]>
like block_given? or caller
<headius>
if we add it at runtime and no code that calls the new alias has already been optimized, there's no harm
<headius>
we will eventually see code that calls the alias and know at that point that it needs a frame
<headius>
we could also just not warn but I like that we are forced to deal with these issues as they come up, rather than have someone's frame-aware alias just start failing
<enebo[m]>
I wonder if we could invent "needs_frame_for_parent"
<enebo[m]>
blah ignore that
<headius>
the frame-aware list would need to be made runtime-specific but that is not a big change
<headius>
(it is static right now and should not normally be modified after boot)
<enebo[m]>
Just thinking if we knew we were calling a method just to redispatch to original we could say it needs the frame but that frame should be kept for child calls
<enebo[m]>
alias :my_caller :caller
<enebo[m]>
my_caller says it needs a frame but then any call in there which requires a frame would inherit current methods frame
<headius>
It wouldn't be the right frame though
<enebo[m]>
which I would guess would be my_callers frame and no frame would get pushed for actual caller
<headius>
We would not push a frame for code that calls my_caller
<headius>
Yeah
<enebo[m]>
but that for the sake of caller would be the right result
<headius>
I'm just trying to think how we can narrow the cases that deserve a warning to such an extent that no one ever sees the warning
<enebo[m]>
name would be wrong I guess
<headius>
In this case none of these bang methods existed before loading ostruct so they're really isn't any harm in allowing the aliases as long as we track those names from then on
<enebo[m]>
yeah
<headius>
s/they're/there/
<enebo[m]>
It doesn't sound like a bad solution to me
<headius>
I can mock it up
<enebo[m]>
it is constrained to already knowing it is aliasing something needing frame
<enebo[m]>
you are already in the weeds a bit at this point
<headius>
I did merge a PR into 9.3.4 that allows aliasing from one frame aware name to another frame aware name but that's an even rarer case
<headius>
In other words if the frame requirements of both names match then there's no warning
<headius>
So this sort of extends that to situations where the new unframed name has never been seen in code so no harm no foul
<enebo[m]>
yeah
<headius>
This might want to be a mapping from the RubySymbol, weakly, as a safe in-memory list of every method name we've seen
<headius>
enebo: mrnoname gave the .sh move a thumbs up so I will merge it
<headius>
jruby.bash will echo a warning to stderr and relaunch jruby.sh
<headius>
you know it would sure be nice if Ruby just eliminated frame visibility altogether and required you to make methods private
<enebo[m]>
yep
<enebo[m]>
I have always thought the section notion is gross
<headius>
nobody would notice until they expect an error calling a private method
<enebo[m]>
you have a larger class and you just need to scan to see which section you are in
<headius>
yeah seems like the trend is toward defining the method and then explicitly calling `private :foo` anyway
<headius>
since the section form is really gross and error-prone
<enebo[m]>
private def foo(); end
<enebo[m]>
I know it is explicit and people want to golf this
<headius>
I like it, which means normal Rubyists must hate it
<enebo[m]>
I just like not having to misunderstand code
<enebo[m]>
Anything unadorned is public and if it is private/protected it deserves the ceremony of being right next to the def name
<enebo[m]>
but I also dislike class << self; def a; end .... ;end
<enebo[m]>
I would be happier is only def self.a; end was allowed
<headius>
mostly the latter form is used if you need to use things like alias_method, which could just be made public on Module
<headius>
if all the metaprogramming methods on Module were public you would not need to use class << self except on non-module objects
<enebo[m]>
alias is an interesting example of metaclass syntax
<enebo[m]>
yeah
<headius>
byteit101 enebo: update on google search indexing of wiki...
<headius>
it does now appear to be indexing some of the wiki but the majority of pages are still excluded
<enebo[m]>
headius: do you know why?
<headius>
"Excluded by ‘noindex’ tag"
<headius>
this is after closing edits from non-committers, which at least allowed it to start indexing some of it
<headius>
I'm not sure what is causing most pages to be "noindex"
<headius>
the migration path for 9.3.0 through 9.3.3 is probably to use #collect19 instead, since that changed behavior in the same way but did not change signature
<headius>
it was deprecated in 9.3 so this would be transitional
<headius>
my error was in blindly replacing #collect with #collect19 without noticing the return types change
<headius>
there's not a lot of good options here with those bad revs
<headius>
added a comment spelling that out
<headius>
enebo: I noticed today the Ruby GSoC folks are starting to talk about 2022 so I think we should talk about it for this year