<enebo[m]>
It is not because of JRuby but because I have had my machine and browser running for months
<headius>
hah
<headius>
oh I see
<enebo[m]>
but once I hit that limit I never dip below
<enebo[m]>
That does make me wonder how reclamation works
<enebo[m]>
I think Idea also uses tons
<headius>
oh yeah
<headius>
probably for every dir in every open project
<headius>
at least
<enebo[m]>
headius: yes to your question. %a,A was one part of the reason
<headius>
that has not landed yet correct?
<enebo[m]>
no it is a branch still
<headius>
ok I linked them
<enebo[m]>
it is missing effectively a dtoa impl (which I am working on outside of that)
<enebo[m]>
Another fascinating element to dtoa and sprintf is not a single implementation combines the two
<enebo[m]>
So you make a string representing the number then post-process it
<headius>
oh can I turn that branch into a PR?
<headius>
I can't link to it easily otherwise
<enebo[m]>
I get it. Combining them is not simple but if you think about how much effort is put into optimizing things I am surprised to not see that happen
<enebo[m]>
OMG webpacker 5.4.4 cannot find some JS for 5.4.4 and I look and it was release jan 31
<headius>
hahaha
<enebo[m]>
I think the JS perhaps has not been pushed
<enebo[m]>
So I believe the issue too will be that the spec will say 5.4 so it will pick up 5.4.4 as a gem
<enebo[m]>
I think I am not going to debug this anymore we generate everything and this is just blowing up trying to compile assets
<enebo[m]>
I could try and tweak this manually but I don't think anything is actually broken
<headius>
here's hoping 😀
<enebo[m]>
I know it will serve me up that error very nicely
<enebo[m]>
If only 9.3 could run rails 7 since it replaced webpacker with hotwire
<enebo[m]>
ok paranoia got the better of me. I put explicit version to 5.4.3 and it works fine
<enebo[m]>
webpacker install obviously is hitting some js repo and it does not have 5.4.4 yet
<enebo[m]>
I am going to create an issue about the warns but I don't think it should hold up release
<enebo[m]>
tried java 17 too now. we do have something with extensions which gets confused when switching
<enebo[m]>
bindex and sassc will both require a bundle install to effectively replace themselves so it is some oddity in how native extensions consider java version?
<enebo[m]>
I am going to push maven and gem since maven can take time
<headius>
Yeah is that the warning about extensions not being built?
<headius>
I filed an issue about it with rubygems but since I couldn't reproduce easily nothing's been done about it
<headius>
It seems to be something specific to extension libraries that have a dummy make file or that build something that isn't actually a native extension, like libsass
<enebo[m]>
system ~/work/release/release/jruby-9.3.10.0/frogger master # 2204% ../bin/jruby -S rails s
<enebo[m]>
Could not find bindex-0.8.1, sassc-2.4.0 in any of the sources
<enebo[m]>
Run `bundle install` to install missing gems.
<enebo[m]>
So however is spec matches something about the bindex installed on java 8 is not visible for java 17 (or similar version differences)
<enebo[m]>
So I half wonder if spec matching with native extensions embeds the Java version into it
<headius>
Oh that's something different. I'm not sure
<headius>
Does the lock file list things like Java version? I would have expected Ruby implementation version maybe
<headius>
This isn't because you switched gem homes right?
<enebo[m]>
it is only changing version of Java and nothing else
<enebo[m]>
I deleted it already but I see it all the time when I switch
<headius>
I wonder if that's the cause of my warning as well
<enebo[m]>
the pristine thing is the same issue if that is the warn
<headius>
Tracking the Java version is probably a good idea but the warning should be more specific
<headius>
Yeah that is the one I'm talking about
<enebo[m]>
but I mean we have like one incompat I really have ever seen but we design around the 8 -> higher gap
<headius>
If it's failing to match due to the Java version it should say it failed to match due to the Java version
<enebo[m]>
so we should not need to be that picky
<enebo[m]>
I can see the need for checking but I am not sure how you specify that
<enebo[m]>
we have generally specified X of higher
<headius>
The sass one is particularly odd, since it doesn't build any Java
<headius>
So this must be a blanket check for all libraries that say they have an extension
<headius>
Regardless of whether that extension is built with Java
<enebo[m]>
yeah that's what I think since we do not actually compile anything
<headius>
What is in bindex? I feel like that's one of the dummy make files
<enebo[m]>
could be. I am re-running runner to get a .lock
<enebo[m]>
I could see it be something in the spec saved stuff too
<enebo[m]>
Another random bundler observation...they should not require same Ruby to be used but they could at least put it in the lock file for people to know
<headius>
Yeah it's oddly specific about some things and oddly non-specific about others
<enebo[m]>
bindex (0.8.1)
<enebo[m]>
So nothing in Lockfile
<enebo[m]>
PLATFORMS
<enebo[m]>
universal-java-17
<enebo[m]>
hmm
<enebo[m]>
This could be the issue
<enebo[m]>
It is nothing specific to any gem but if you have a native ext and the platform check fails it will fail?
<headius>
ah there it is
<headius>
so it is like if you moved this to a different OS version it would complain too
<headius>
like x86_64-linux-5.4 or something
<enebo[m]>
ok 11 now doesn't work. I will manually edit the file
<enebo[m]>
hmm still didn't work but perhaps my 11 will display PLATFORM differently
<enebo[m]>
doing a bundle install to check
<enebo[m]>
ok so fun times...it is universal-java-11
<enebo[m]>
which means this is not just the .lock file
<enebo[m]>
This has to be only things which end up with cext make stuff which does nothing right?
<enebo[m]>
we have other native extensions (Java) and those are fine
<headius>
yeah
<headius>
they don't build anything so it doesn't register them as extension gems
<enebo[m]>
but we really don't build anything with sassc
<enebo[m]>
at least I thought that was their big change to have native backends
<enebo[m]>
as separate gems
<headius>
yeah all it is looking for is whether the gem has an ext specified
<enebo[m]>
So rather than java it uses the entire platform string
<enebo[m]>
and for us all this is for those two gems is make output saying nothing has been done
<enebo[m]>
boo
<enebo[m]>
interesting too that you can have n directories here but the gems do not prefix with platform so all the other gem subdirs compete and force you to reinstall
<enebo[m]>
I honestly do not see the point of specifying platform here for anyone
<enebo[m]>
if all the other files overwrite why make this dir different
<headius>
ah yeah
<headius>
I think they added the platform version later on
<enebo[m]>
Having some lunch (or making it) but I have the bits uploaded. We largely just need notes and the finishing stuff now
<headius>
got sidetracked with something else
<headius>
ok I will look into notes
<headius>
oh yeah labels and that led me to fixing some simple test_rubyoptions failures
<headius>
and I discovered there's a Ruby discord server
<enebo[m]>
headius: I added a section for that and corrected those bottom links
<headius>
ok
<headius>
looks good
<enebo[m]>
alright. I will push site and email and GH release
<enebo[m]>
should I make it "latest release" on GH release?
<enebo[m]>
I guess this is a very short time frame question if we assume we will be getting out 9.4.1.0 tomorrow?
<enebo[m]>
wow github multiple file upload got extra weird...it is animating all files in the same progress line so it is just blinking around to various states of complete. Almost like an engineer did not realize people were uploading more than one thing
<enebo[m]>
GH release out
<headius>
sorry had some repair stuff happening here
<headius>
I'm not sure how to mark latest release
<headius>
we'll have 9.4.1.0 out so it doesn't matter yeah
<enebo[m]>
never mind I just did it. I think it just puts it at the top
<enebo[m]>
which is fine
<headius>
ok
<enebo[m]>
9.4.1.0 will be at the top soon anyways
<enebo[m]>
I think we just need the stuff you have done the last several releases
<headius>
I'll get the remaining items done in a little bit. Parenting duties
<headius>
I started looking at implementing the new buffer class, I think it's a ruby 3.2 feature or maybe experimental in 3.1, and it's basically just ByteBuffer
<headius>
It's almost close enough to just add some decorator methods to the Java class and call it a day, but there's some mutability that white buffer would not support
<headius>
Oops ByteBuffer
<headius>
There's also some memory mapping flags that don't seem to have a Java equivalent
<enebo[m]>
white buffer
<enebo[m]>
I did not mention this but I doubled perf of split(/,/) and made split(',') 20-25% faster
<enebo[m]>
I need to refactor a lot to make it cleaner but nothing in it will get slower in the refactored clean version
<enebo[m]>
for /,/ we are up to 6x faster than MRI
<headius>
Very nice! I wonder which benchmarks this will affect. I think parser used a lot of regular expressions
<headius>
The single character regular expressions would be such an amazing candidate for jit though
<enebo[m]>
I opened an issue about csv processing
<enebo[m]>
we run the older fastcsv quite a bit faster than the current one
<enebo[m]>
looking at profiling strscan is significant
<enebo[m]>
what is strange is MRI runs both versions pretty close to the same speed
<headius>
I saw some of that discussion
<headius>
regexp fix doesn't help?
<enebo[m]>
it helps the old fastercsv but that is unrelated to rovz since most of his impact is after he has read it in
<enebo[m]>
I just happened to try csv out after I updated split to see if it got faster
<enebo[m]>
old one uses it and does new one doesn't
<enebo[m]>
I also noticed we do not fast path != on fixnum which ironically only helps out the older fastercsv
<enebo[m]>
I will make a PR for that soon. Fixed both with indy and not
<headius>
ok all done
<headius>
yeah cool
<headius>
strscan has not had much work other than compat either
<headius>
it probably can be a lot better
<enebo[m]>
I would like to possibly see it defer regexp processing into some common utility method in JRuby too so that any improvements on making fast paths could just get picked up without having to replicate that code
<enebo[m]>
like my new code makes isSimpleString() or some name like that on regexp which means I can just use source as a string and do string splitting (or anything else knowing the regexp is just a common string)
<headius>
yeah makes sense to move more into JRuby in general... just a little extra complicated with the gem
<headius>
I mean move more utility code into JRuby
<enebo[m]>
/,/ just becomes ',' as a split and clean 7bit at that
<enebo[m]>
If joni opts to same speed as a tight split loop (I made n byte and single byte versions)
<headius>
very unlikely
<enebo[m]>
but the code is as small as it gets and no notion of morphism exists in those methods
<enebo[m]>
the only challenge is rewriting it so scan and split can use them AND it can deal with bytelists directly when that is more practical
<enebo[m]>
I shoehorned the bytelist part
<enebo[m]>
the ultimate here would be to pair up the right processor back to the site and remove the decision process on successive calls
<enebo[m]>
(which would still require some fallback since the string being processed could make something less optimal needed)
<headius>
Yeah I see that happening for sure once we can jit regex but there's no reason it couldn't happen now
<enebo[m]>
I have really wanted a anno like "callSiteData = true" which would pass in Object data
<headius>
At least for match operators there could easily be a special call site that binds directly
<headius>
Split and scan could do something similar, but it would be easier if they were written in Ruby
<enebo[m]>
It may need a little more since it needs to alloc once but then we can just cast and use
<enebo[m]>
but it could be generic
<headius>
Looking at my small optimization's recently I realized so many places where moving Java logic out to Ruby would open up a bunch more call site optimization
<headius>
Like all those type coercions
<enebo[m]>
hell I think annos can allow generics so it could literally be Java generic (although I don't personally care)
<enebo[m]>
sprintf is a big one for me though. I want to just store the compiled result into the site and just grab it again
<headius>
Yeah I don't know if generic data is stored but you can have class literals in annotations
<headius>
Or you annotate the parameter and then we can see the type anyway
<enebo[m]>
oh it is probably erased but you could use it perhaps in the JRubyMethod signature
<headius>
I will probably start introducing parameter annotations to let us expand the list of arguments we can receive without building custom logic every time
<enebo[m]>
but I don't think there is much value in it
<headius>
At the very least you'll be able to specify which parameter should be rest arguments and which one should be keyword rest and which ones should map to which keyword names
<enebo[m]>
but I really really really just want a parameter which will get instantiated once at the site and passed into the method. The method impl itself can use and change state on it
<headius>
@Keyword IRubyObject exception
<headius>
Optional name for the keyword
<enebo[m]>
yeah nothing wrong with that
<headius>
Oh yeah, callee manipulating the call site
<enebo[m]>
null values means optional value with otherwise failing
<headius>
That would be pretty slick
<enebo[m]>
It would allow us to kill off a bunch of global caches
<headius>
Yeah for sure
<enebo[m]>
at the cost of maybe compiling the same strptime string n times but eliminate all lookup costs and contention
<enebo[m]>
but there are a lot of strategies which could just be in the callee like memoziing a result with some simple identity check
<enebo[m]>
when that actually makes sense
<enebo[m]>
It would be easy to put too much logic in the callee where it could perhaps be better in indy but it also provides an easy way to play
<headius>
We could also annotate methods with a pointer to specialized indie logic
<headius>
Like instead of just binding this one of the generic ways, call this method that knows how to memoize or inline type coercions or whatever
<enebo[m]>
for something like memoization I can see that more than saving off a compiled format in something like strptime but yeah
<headius>
Then we just start building up a set of little utilities that use method handles for all these key language features and it should inline and specialize like mad
<enebo[m]>
We both are looking at this from a slightly different lens too
<enebo[m]>
you are thinking more about generic processing and I am thinking a bit more on being able to exploit specific methods which have recurring costs
<enebo[m]>
I think we both are seeing both but I mean interest wise
<enebo[m]>
It is not lost on me that you could make this into a PE engine
<headius>
Eventually we'll just compile Ruby straight into method handles and let the JVM figure it out
<headius>
We'll call it PortobelloRuby
<enebo[m]>
but near term all I really desire is ability to retrieve a specified writable object I can consume from the callee
<enebo[m]>
invariant processing in the implementing methods can just store that stuff away
<headius>
A first pass that would serve both our goals would probably be to make the native method args more generally configurable
<enebo[m]>
in the case of split probably it can save away the "processor" of the split
<enebo[m]>
I don't know
<headius>
I'm also picturing that leading us out of a max of three direct arguments
<enebo[m]>
I just want something which adds a single parameter with an allocated holder for an object
<headius>
You just put whatever arguments you want to receive, including caller data like frames and call sites, and then the annotations and code gen stitch it all together
<enebo[m]>
the rest of the parameters seems pretty orthogonal
<enebo[m]>
but yeah this would be one argument I care about on the method
<enebo[m]>
It could be as simple as ..., CallSiteHolder data) {
<enebo[m]>
data.set/get
<headius>
Yeah
<enebo[m]>
in non-indy world it would probably literally be the callsite instance
<enebo[m]>
in indy it would be something I guess
<headius>
Well that is a sticky wicket
<headius>
Making the Java side configurable makes it a lot harder to avoid using indy
<enebo[m]>
because it would mean adding an if statement to every invoker
<headius>
That's the main reason we haven't added any new parameters types or arity... I end up having to permute all possible combinations so plain old Java can also call
<enebo[m]>
oh wait the invoker does not even have the callsite at all
<headius>
Yeah this is fully new arguments on that call path
<enebo[m]>
invoker instanceof CallSiteAware :)
<headius>
Which will have to happen anyway for keywords but at least keywords have a simple fallback to just dump things in a hash
<enebo[m]>
our current native still doesn't care about non-kwargs as kwargs mostly
<headius>
I guess it's not impossible to wire up without Indy but the alternative is reflection and intermediate boxes
<enebo[m]>
so we do not error in most cases where calling convention is wrong (e.g. we still act like 2.6)
<headius>
Right
<headius>
All we really can encode into the signature right now are required arguments and rest arguments
<enebo[m]>
I have been marking keywords for things which consume keywords as a better marker
<headius>
There's no real intelligence for optional or any of the keyword forms
<headius>
Optional is just a restricted case of rest arguments
<enebo[m]>
and I need to try and land the new kwargs which can pass **{} anywhere and not error (which alst removes the bifurcation of calls which have explicit kwargs)
<headius>
I'm hoping to get some decent seats on the way to Stockholm but either way I'll be working during my down time next week
<enebo[m]>
We can definitely mark all required and we could mark all optional but not process the optional values
<enebo[m]>
I am talking about kwargs
<enebo[m]>
optionals would just be null if nothing was passed
<enebo[m]>
There are actuall kwrest in natives too where it ignores extra keys but that one is fairly simple
<headius>
I'm super keen to get some of the common keywords wired straight through, like the :exception keyword that's all over IO methods
<headius>
The ideal case would be all signatures are just a long string of single arguments and the method called plumbing puts the values in the right spots
<headius>
Anyway, this all exists in my head, just need to find some time to do it
<enebo[m]>
lots of keyword use growing too
<headius>
Yeah it is almost embarrassing how many things we don't optimize right now and we're still fastest at most meaningful benchmarks
<headius>
If we could get things like dynamic scopes plumbed through the call path they might even start to escape analyze away
<headius>
My goal is to get everything to inline and then start passing around more contacts
<headius>
Context
<headius>
If we can pass the scope through, we don't need a scope stack anymore
<headius>
If we can pass the frame through we don't need a frame stack