<headius>
there seems to be some intermittent failures we'll deal with as we see them
<headius>
enebo: I'm going to merge this to master so we can get updated results there
<MattWelke[m]>
<headius> "that 2000 might only apply to..." <- The limit only applies to private repos. Public repos within the org get unlimited minutes.
<MattWelke[m]>
Something that I didn't know till now though that stood out to me: "Jobs that run on Windows and macOS runners that GitHub hosts consume minutes at 2 and 10 times the rate that jobs on Linux runners consume."
<headius>
Matt Welke: yeah I saw something about that... more expensive to license I guess
<headius>
enebo: failure in sequel job on master seems to be due to improper handling of kwargs
<headius>
only two failures but there may be more if these are fixed... both the same error in the same place
<headius>
similar failures in concurrent-ruby
<headius>
hmmm and the JI specs too... the rake/ant integration is breaking while passing kwargs
<headius>
I guess we should prioritize that
<headius>
drb needs a patch to avoid _id2ref, guess I should finally do that
<headius>
weakmap supporting immediates is going to be problematic
<headius>
the semantics are identity keys but we cannot guarantee identity for e.g. fixnum-ranged Integers
<headius>
which is exactly the case needed to switch this logic to using WeakMap
<headius>
bother
<headius>
omg enebo you implemented Module#const_source_location already
<headius>
I figured that would be gross and we might punt
<headius>
master CI has improved a bit with some small fixes
<headius>
lots of specs and MRI tests left failing though
<headius>
the mri:stdlib hangs seem to be the same place that DRb patch hangs, so something is insufficient about my weak map version
<headius>
oh boy drb is some ancient Ruby code
<mattpatt[m]>
headius: hurrah, saw all the specs on 9.3 passing and only the snapshot failing
<mattpatt[m]>
That definitely needs the sonatype username and password setting as secrets in Github
<mattpatt[m]>
but now you've merged down to master, provided the SHA's are still legit you can test that manually now
<enebo[m]>
heh I don't remember doing a bunch of things I did
<enebo[m]>
these master runs on test:mri and spec:ruby:fast are pretty much what I was seeing locally
<headius>
fever dream
<headius>
you got at least half of the checkboxes, however it happened
<enebo[m]>
about 100 on each will likely fall out just from fixing kwargs argument passing (in methods and blocks)
<headius>
yes
<enebo[m]>
It is probably not totally apparent but like that dup change is an example of how unobvious it can be
<headius>
it is a hard break on kwargs handling and we are still doing the squishy version
<enebo[m]>
ceil and floor on Time I think is another which fails a lot
<headius>
not there yet
<headius>
probably easy
<enebo[m]>
I don't feel anything involving time or date is easy
<enebo[m]>
though this is likely a straightish port
<headius>
yeah, I don't enjoy working on temporal data types
<headius>
it must ceil or floor to something, maybe configurable, but it's just rounding
<headius>
so yeah kwargs
<enebo[m]>
I have ruby2_keywords implemented but not properly used
<headius>
so we will just flip some kwargs logic depending on which mode
<enebo[m]>
but I took a break from reading semantics on kwargs passing
<enebo[m]>
yeah ruby2_keywords is to fall back to what I think is passing kwargs purely as just another argument
<enebo[m]>
which is how we do things now
<headius>
right, with some allowances for strings and munging together positional and kwargs and all that
<enebo[m]>
I need to re-read this all over again even though I read it about a week and a half ago
<enebo[m]>
I made mistake of reading the entire multiple year discussion
<headius>
probably for the best I never tried to do an optimized version of kwargs because with the old logic it was pretty gross
<headius>
should be easier now
<headius>
it's part of signature for real
<enebo[m]>
yeah I mean it will be gross no matter what
<enebo[m]>
Ruby I think to be clean at this point requires a total break from existing code and that will never happen
<enebo[m]>
but I guess kwargs are probably better than they were
<enebo[m]>
and I think we could use an enhancement for JRubyMethod
<headius>
yes, splitting up kwargs targets would be simpler now
<enebo[m]>
Not even a boil the ocean enhancement
<enebo[m]>
Just provide a hash arg
<enebo[m]>
later improvements can specify required or whatnot
<headius>
oh a dedicated arg
<enebo[m]>
Think about all that shit in RubyIO
<enebo[m]>
We have some other kwarg complicated methods bu IO sticks out
<headius>
yeah sure, anything you can specify in a Ruby signature could be represented in a JRubyMethod signature, it's just a matter of complexity in the generator
<enebo[m]>
yep
<headius>
but like now it could just fall back on hash versions
<headius>
most core methods only take a couple simple kwargs
<enebo[m]>
and in 3.0 we have some code path which needs to know if we are passing kwargs or not and whether we need to figure out if there is a kwargs but those two paths should merge back into whatever the populator will receive
<headius>
one or two or three kwarg could be generated easily enough, mapping kwarg names to positional on Java stack, or whatevs
<enebo[m]>
So I think it may be as simple as RubyHash kwargs as a first swipe
<enebo[m]>
but this is why I am not trying to boil the ocean
<headius>
yeah
<headius>
just separating them 3.0 way vs 2.6 way would knock most of this out, we just have opportunities to improve later
<enebo[m]>
the populator could be a month-long project but I think it would be cool if we just could get rid of all this is that last arg we got a RubyHash
<enebo[m]>
And nothing prevents adding the specific keys later (optional = ["limit", "encoding"], required = ["foo"], etc...
<headius>
@KeyArg("name") IRubyObject name
<enebo[m]>
I don't know if you saw but Java 18 plans on all reflection to be indy based
<enebo[m]>
ah yeah that is a nice syntax
<enebo[m]>
it may need more params or just more anno names
<headius>
it just gets dense beyond a couple kwargs
<enebo[m]>
thankfully I think most core functions only have a couple
<headius>
and more complicated fallback path to some hash based version
<headius>
but yeah it's doable
<headius>
right, just separating out exception: would cut a ton of alloc in basic IO now
<enebo[m]>
but at this point since we know the degenerate case is a hash that seems like a good bang for the buck addition
<headius>
if they don't use it we don't allocate, but anyone passing any options at all to IO are paying a big alloc
<enebo[m]>
well not as degenerate as pre-3 handling
<enebo[m]>
yeah it seems to only happen on this job but it is something not lining up I am guessing startup mixed with full
<headius>
at this point I run the tests verbose and note which one goes kaput
<headius>
usually it is some messed up signal that nukes the test process, or something like that
<enebo[m]>
This is not an issue with strftime so I will merge that
<enebo[m]>
this was the only failure
<headius>
ok
<headius>
there may be some new timing issues on this env
<headius>
opportunities
<headius>
ahem
<enebo[m]>
It is a big change to the impl of strftime for 9.3 but since we have hardly started I figure the risk is worth the reward
<headius>
tis the season
<enebo[m]>
there are also a lot of tests so if I broke something it will just expose a corner missing
<enebo[m]>
I thought this morning about boiling the logger ocean and just detecting the standard format string and having a method which literally does it with no conditionals at all :)
<enebo[m]>
Then I thought that is crazy
<headius>
well there have to be gobs of single format strings out there
<enebo[m]>
oh yeah there is that too
<enebo[m]>
I did not look at strftime but I was looking a sprintf/format for the rewrite of that
<enebo[m]>
we do almost 100 per request
<headius>
yeah logging is great
<enebo[m]>
so not super hot but it is destined to be IO of some kind
<enebo[m]>
9.4 also needs that tasty logger native impl
<headius>
I suppose that is the argument though, this is the thing Rubyists use so it should just be good
<enebo[m]>
MRI went from 200kish to 10Mish
<headius>
I could whip out a logger
<headius>
all loggers are the same
<enebo[m]>
stdlib/logger is a surprising amount of code in Ruby
<enebo[m]>
let me bring up this idea
<enebo[m]>
We ship gems to remain compatible with MRI
<headius>
where is this native logger?
<headius>
ruby/logger seems to be pure ruby still
<enebo[m]>
I don't know if logger is one or not (and I suppose if it is this idea is just academic atm) but if we made our own logger gem we could ship and then this could go back to 9.3 as well as 9.4
<enebo[m]>
hmm
<enebo[m]>
really?
<enebo[m]>
Ruby 3 perf is off the charts
<headius>
just blindly cloning it I only see bin and lib
<enebo[m]>
They for 2 orders of magnitude
<headius>
but 3 might have shipped with an unreleased logger
<enebo[m]>
s/for/got
<enebo[m]>
never notices fr and gt are just off by one on the keyboard
<headius>
2 orders of magnitude on what
<headius>
logger?
<enebo[m]>
yeah
<enebo[m]>
if I run Ruby 3 on the bench I attached (bench_strftime) it is waaaaay faster
<headius>
beats me, master has no logger ext either
<enebo[m]>
3.0.1 source looks to have same add method we do on 9.3
<enebo[m]>
Get this...the strftime I am running is what logger calls at some point
<enebo[m]>
so it should not be faster than the strftime call
<enebo[m]>
OR it is not calling that at all for some reason
<headius>
memoizing?
<enebo[m]>
the time?
<enebo[m]>
I guess maybe once a minute :)
<headius>
well not that
<enebo[m]>
but the time is only 1M/s
<enebo[m]>
err for MRI it is only half that
<enebo[m]>
so for them to get 10M/s they cannot be doing that strftime per call
<enebo[m]>
but if they don't then they are not getting time as much
<enebo[m]>
I wonder if they optimize away calling it if it is not really logging
<enebo[m]>
logger = Logger.new(File::NULL)
<headius>
if they improved argument handling to avoid all alloc along this path that could explain it
<enebo[m]>
if logdev && logdev != File::NULL
<headius>
it's a mess of varargs and kwargs and optional args and all of it
<headius>
oh wat
<headius>
too easy, can't be that
<enebo[m]>
I mean it has to be slower than the strftime
<enebo[m]>
or it is not outputting a time
<headius>
ok yeah
<enebo[m]>
so yeah @logdev is nil on 3.0
<headius>
so if it is actually using the strftime to do add it can't be faster than strftime
<headius>
mystery ext please stand up
<enebo[m]>
I am betting money it knows this and therefore is doing nothing at all
<enebo[m]>
if @logdev.nil? or severity < level
<enebo[m]>
return true
<enebo[m]>
end
<enebo[m]>
boo
<enebo[m]>
Ok mystery solved
<enebo[m]>
3.0 realizes people put log calls into their code but will disable the logger
<enebo[m]>
by passing in a null logger
<headius>
so you have to run this with some log level
<enebo[m]>
so they opt for this
<enebo[m]>
well you do and you also need a real log source
<enebo[m]>
if you pass in null log source by bother to call strftime
<headius>
I remembered why I don't like working with loggin libraries
<enebo[m]>
I guess this makes sense. Some people use logger but only turn it on when they have issues
<enebo[m]>
severity is supposed to be the right mechanism for not logging but I guess enough people swap out the log source itself to make this an opt
<enebo[m]>
This morning was first time running strftime on 3.0 but I should have realized the problem with the native ext theory
<enebo[m]>
So good news...one less thing to do for 3.0
<headius>
we could do it anyway
<enebo[m]>
oh for sure
<headius>
🧙
<headius>
hottest piece of code in every app
<enebo[m]>
we are like 260k i/s or something like that while strftime is 950k i/s
<enebo[m]>
So we are spending a lot of work in logger itself
<enebo[m]>
Time.now is being called over and over too
<enebo[m]>
I looked at that and we have some tiny overhead in getting TZ
<enebo[m]>
someone did opt that a bit by caching TZ RubyString but we store ENV as a caseinsenitive ruby hash
<enebo[m]>
This decision maybe is worth thinking about more
<enebo[m]>
I audited uses of it and we seem to locally look up TZ (we want a String) or process invoke we want a HashMap of Strings
<headius>
this is quagmire code
<enebo[m]>
in random ruby code someone may be constantly looking up an env in a hot loop but I question whether that is what we should opt for
<headius>
I have dipped my toe a few times but it really needs a rework
<enebo[m]>
This is another place where ENV#[] could be a special site
<headius>
every time I touch anything in there something else breaks so I am gunshy
<enebo[m]>
It wouldn't matter either way but of course env gets aliased to lvars and stuff by frameworks so they can test
<headius>
we can do special call sites for anything
<enebo[m]>
I did think we could leave it the way we have it and just acknowledge TZ can be a special field we write to whenever it changes
<enebo[m]>
yeah I just mean env = ENV so people can pass around env makes doing a site less useful
<headius>
just a matter of identifying a type or method quickly and making it a fast path or having a reasonable fallback, which could just be another specialized site
<headius>
it all comes out in the wash
<headius>
well, yeah
<enebo[m]>
libraries tend to do this so they can test without needing to actively change ENV
<enebo[m]>
so I callsite cache is only sometimes useful
<headius>
env has its own methods so it could be specialized, but that gets specific per type
<headius>
worth it for some I'm sure
<headius>
it isn't really just a Hash
<enebo[m]>
TZ though could eliminate a hash lookup by making it a writable field whenever ENV changes
<enebo[m]>
In the end this was my main takeaway since it is an obviously heavily used field
<headius>
hmm if indy binding logic could easily tree off different types and base methods, we could specialize a bunch of calls based on what they really do
<enebo[m]>
and it does not force a big rethink although I think the second huge use of ENV is process execution
<headius>
maybe nashorn does this
<headius>
yeah there have been a few passes on this TZ code to cache things
<enebo[m]>
ENV is mostly a read structure and when it is written it is either a) at beginning of program b) used in testing
<enebo[m]>
yeah but TZ right now assumes grabbing it out of the RubyHash with a cached TZ Ruby String
<enebo[m]>
The next opt is to just make TZ a String/ByteList on Ruby (somewhere) with no lookup at all
<enebo[m]>
on ENV write we duplcaite
<headius>
yeah and if it doesn't change that is it
<headius>
alloc is the root of all evil
<enebo[m]>
for real programs it won't
<enebo[m]>
for tests it will but we don't care
<enebo[m]>
for multiple threads if they are live mutating ENV[TZ] and expect it to work they have bigger issues
<enebo[m]>
fwiw I doubt we are talking about a major gain here either
<enebo[m]>
It is just some work that can be eliminated
<headius>
make an IR instruction for TZ
<enebo[m]>
HAHAH
<enebo[m]>
strftime(LOGGER)
<enebo[m]>
funny though...Logger uses 6N but we get 9N worth of values so in theory we could call Time.now less in a super heavy logging situation
<headius>
gross
<enebo[m]>
Current impl of strftime will write out 9 digits then adjust the length
<enebo[m]>
This can be improved but my first simple attempt did not really change anything
<enebo[m]>
once we site cache the parsed format string I will change this all anyways
<headius>
are you profiling allocs with something?
<headius>
it is annoying they keep taking away the built-in profilers
<enebo[m]>
I did profile with allocs but it was also all really obvious
<enebo[m]>
I saw the TZ stuff from allocs since we are making .bytes() on the result over and over
<headius>
oh yeah
<enebo[m]>
but I eliminated allocating token list, the tokens themselves, Strings which became bytelists, then the bytelists themselves
<enebo[m]>
We still create duplicated bytelists for padding when we cannot know the length before hand
<enebo[m]>
Even in that case I got rid of the most common one by making a longLength method
<headius>
9.4 numbers are faster for me with latest merges
<enebo[m]>
so strftime is same speed and the null log opt looks to be much better with indy on
<enebo[m]>
same speed == same speed after opt
<enebo[m]>
It is a little noisy since I have other stuff running so it floats around a little bit 940-960ish
<enebo[m]>
running with indy to see but it should be 1Mish
<headius>
the rich get richer
<headius>
did you put indy results somewhere>
<enebo[m]>
running now
<enebo[m]>
I will update
<headius>
oh this is on 9.3
<headius>
yeah?
<enebo[m]>
no master
<headius>
ok
<enebo[m]>
updated
<enebo[m]>
So I was getting closer to 1M earlier so I am not sure if that is a little interference with having other stuff up
<headius>
decent
<enebo[m]>
The logger methods themselves no longer do anything so I guess that is only measuring bench/ips overhead
<enebo[m]>
well they check to null logger and severity
<enebo[m]>
It is funny how many things in MRI have slowed down since 3.0 came out
<headius>
maybe 3.1
<enebo[m]>
I should be seeing how yjit is doing ehre
<enebo[m]>
3.1 should show some gains in at least a few places
<enebo[m]>
when it works out it will be noticeable
<headius>
meh, JIT is irrelevant if they are allocating too much, so maybe they just figure that out
<enebo[m]>
but I think the gains on something like rails was still pretty modest
<enebo[m]>
yeah for something like strftime they are done
<enebo[m]>
it is what it is since they are calling into C with a series of pointers and a buffer
<enebo[m]>
and more generally it seems like numbers and things like that are already not subject to memory issues but math is just not a major perf point for most Ruby apps
<enebo[m]>
specializing array access I think they do get gains
<enebo[m]>
but I am not as interested in them until they start working on inlining
lopex[m] has quit [K-Lined]
boc_tothefuture[ has quit [K-Lined]
MattWelke[m] has quit [K-Lined]
andrea[m] has quit [K-Lined]
CharlesOliverNut has quit [K-Lined]
XavierNoriaGitte has quit [K-Lined]
068AAB7IX has quit [K-Lined]
JulesIvanicGitte has quit [K-Lined]
BlaneDabneyGitte has quit [K-Lined]
annabackiyam[m] has quit [K-Lined]
UweKuboschGitter has quit [K-Lined]
MarcinMielyskiGi has quit [K-Lined]
RomainManni-Buca has quit [K-Lined]
enebo[m] has quit [K-Lined]
anewbhav[m] has quit [K-Lined]
MattPattersonGit has quit [K-Lined]
jswenson[m] has quit [K-Lined]
Leonardomejiabus has quit [K-Lined]
klobuczek[m] has quit [K-Lined]
kares[m] has quit [K-Lined]
ChrisSeatonGitte has quit [K-Lined]
MateuszFryc[m] has quit [K-Lined]
JasonvanZyl[m] has quit [K-Lined]
edipofederle[m] has quit [K-Lined]
danieljrubyquest has quit [K-Lined]
nirvdrum[m] has quit [K-Lined]
basshelal[m] has quit [K-Lined]
kai[m] has quit [K-Lined]
rcrews[m] has quit [K-Lined]
KarolBucekGitter has quit [K-Lined]
byteit101[m] has quit [K-Lined]
mattpatt[m] has quit [K-Lined]
OlleJonssonGitte has quit [K-Lined]
akshsingh[m] has quit [K-Lined]
rebelwarrior[m] has quit [K-Lined]
puritylake[m] has quit [K-Lined]
liamwhiteGitter[ has quit [K-Lined]
JesseChavezGitte has quit [K-Lined]
FlorianDoubletGi has quit [K-Lined]
074AACU6W has quit [K-Lined]
AndyMaleh[m] has quit [K-Lined]
headius has quit [K-Lined]
ahorek[m] has joined #jruby
subbu has joined #jruby
fidothe has quit [Read error: Connection reset by peer]
siasmj has quit [Read error: Connection reset by peer]
siasmj has joined #jruby
fidothe has joined #jruby
enebo[m] has joined #jruby
kai[m] has joined #jruby
MatrixTravelerbo has joined #jruby
lopex[m] has joined #jruby
subbu[m] has joined #jruby
nilsding has joined #jruby
JasonvanZyl[m] has joined #jruby
andrea[m] has joined #jruby
basshelal[m] has joined #jruby
byteit101[m] has joined #jruby
Leonardomejiabus has joined #jruby
CharlesOliverNut has joined #jruby
ChrisSeatonGitte has joined #jruby
UweKuboschGitter has joined #jruby
JesseChavezGitte has joined #jruby
puritylake[m] has joined #jruby
FlorianDoubletGi has joined #jruby
rebelwarrior[m] has joined #jruby
MattPattersonGit has joined #jruby
XavierNoriaGitte has joined #jruby
JulesIvanicGitte has joined #jruby
klobuczek[m] has joined #jruby
MattWelke[m] has joined #jruby
anewbhav[m] has joined #jruby
OlleJonssonGitte has joined #jruby
TimGitter[m]1 has joined #jruby
kares[m] has joined #jruby
BlaneDabneyGitte has joined #jruby
akshsingh[m] has joined #jruby
TimGitter[m] has joined #jruby
RomainManni-Buca has joined #jruby
liamwhiteGitter[ has joined #jruby
boc_tothefuture[ has joined #jruby
rcrews[m] has joined #jruby
MarcinMielyskiGi has joined #jruby
KarolBucekGitter has joined #jruby
jswenson[m] has joined #jruby
edipofederle[m] has joined #jruby
danieljrubyquest has joined #jruby
MateuszFryc[m] has joined #jruby
nirvdrum[m] has joined #jruby
AndyMaleh[m] has joined #jruby
annabackiyam[m] has joined #jruby
mattpatt[m] has joined #jruby
headius has joined #jruby
MatrixTravelerbo has quit [Quit: Client limit exceeded: 20000]
subbu[m] has quit [Quit: Client limit exceeded: 20000]
ahorek[m] has quit [Quit: Client limit exceeded: 20000]
nilsding has quit [Quit: Client limit exceeded: 20000]
subbu has quit [Ping timeout: 246 seconds]
subbu has joined #jruby
subbu has quit [Quit: Leaving]
subbu has joined #jruby
<edipofederle[m]>
Bassicaly the `rb_arithmetic_sequence_extract` looks returns the same values (for `aseq.begin` and `aseq.end`.
<edipofederle[m]>
Could somone give some help here?