subbu has joined #jruby
subbu has quit [Ping timeout: 240 seconds]
fidothe has quit [Ping timeout: 260 seconds]
fidothe has joined #jruby
subbu has joined #jruby
subbu has quit [Ping timeout: 240 seconds]
subbu has joined #jruby
subbu has quit [Ping timeout: 256 seconds]
<TimUckun[m]> Hello again. I have finally gotten around to fixing the broken rack benchmarks on the techempower benchmark suite. I am now trying to run the benchmarks on jruby but am having problems building the docker file.
<TimUckun[m]> The docker build fails at various gems because it can't build native extensions
<TimUckun[m]> I tried limiting them to ruby and windows platforms but it also fails at building jdbc-postgres
<donv[m]> <headius> "donv: Good news! At the end of..." <- Great! Glad to hear the presentation went well. :) I have now fixed all known problems with using JRuby 9.4.2.0 in the POC and updated the tooling. So now we can generate new applications using JRuby 9.4.2.0 with the tooling (ruboto/ruboto).
<TimUckun[m]> Once I get past this point I would also like to make sure the proper optimizations are applied in terms of java and jruby opts, max database connections etc.
<headius> donv: All problems except the Android bug I assume?
<headius> we may need to work around that since it's going to be in released 13 until fixed and people can update
<headius> Tim Uckun: It shouldn't be trying to build anything for jdbc-postgres
<headius> gem 'oj', '~> 3.14'
<headius> gem 'pg', '~>1.5' # , group: :postgresql
<headius> those should only be on CRuby
<headius> gem 'falcon', '~>0.42
<headius> that too
<headius> I think the others are guarded correctly
<donv[m]> <headius> "donv: All problems except the..." <- I have added a patched Dir.java and some other patches to work around the problem :)
<TimUckun[m]> The error is
<headius> Tim Uckun: I'm trying to clone now but in-flight internet is a little slow
<TimUckun[m]> No worries. Not in a huge hurry
<headius> rack-jruby: current directory: /usr/local/bundle/gems/io-event-1.0.9/ext
<headius> that's from falcon
<headius> falcon depends in async-io depends on io-event
<headius> I am working on support but it has a hard dependency on a native ext right now
<headius> we won't be able to run the falcon benchmarks (which I did not know existed!)
<TimUckun[m]> OK, I'll guard that too.
<TimUckun[m]> I Iam working on the falcon benchmarks now
<TimUckun[m]> but there is a bug in falcon too
<headius> 62% cloned
<TimUckun[m]> I have reported it
<headius> nice
<TimUckun[m]> So far only unicorn and puma are working
<headius> once I'm settled back in at home I'll be collaborating with ioquatix to get it working on JRuby
<headius> now that we have real fibers
<TimUckun[m]> The command to test it is ./tfb --mode=verify --test=rack-jruby
<TimUckun[m]> from the root of the project
<headius> ok
<headius> Tim Uckun: also need to guard pg and oj as I mentioned, and possibly sequel_pg if that depends on pg rather than jdbc
<TimUckun[m]> OK. If I am going to leave out OJ then I'll have to modify the code a bit
<TimUckun[m]> so leave it for now and I'll work on it some more tomorrow
<headius> enebo has a nearly-complete port of oj to JRuby but it has never been released
<TimUckun[m]> It's almost one AM in New Zealand now
<TimUckun[m]> OJ has a compatibility mode
<headius> a compatibility mode?
<TimUckun[m]> It mimics JSON gem
<headius> ah well we support json gem
subbu has joined #jruby
<headius> we just felt we needed to port oj because a lot of sites eventually move to it for performance
<headius> his changes are all on master I guess
<headius> woot done cloning
<headius> I probably could have run this on a server somewhere
<TimUckun[m]> OK the dockerfile built
<TimUckun[m]> but launching puma is failing
<TimUckun[m]> ERROR: worker mode not supported on jruby on this platform
<TimUckun[m]> so I'll have to put a guard on the puma config to only specify threads
<headius> yeah configure for threads
<headius> I am unfamiliar with puma autotune
<headius> yeah this connection is too slow to pull the docker bits
<headius> something like that
<headius> I don't know what autotune does exactly but 2 * CPU count is probably good for JRuby
<headius> huh I was going to try to submit a patch for puma_auto_tune but it hasn't been modified in 6 years
<TimUckun[m]> I see this in the sinatra torquebox dockerfile
<TimUckun[m]> export MAX_CONCURRENCY=$(( 2 * $(nproc) ))
<headius> yeah there ya go
<TimUckun[m]> so that makes sense
<TimUckun[m]> so is that the number of threads?
<TimUckun[m]> seems kind of low to me
<TimUckun[m]> why not hundreds?
<headius> it depends how many are likely to be blocking on external IO
<headius> if they don't block much then it will be CPU-bound and based on number of cores
<TimUckun[m]> all of them given there are database calls
<headius> yes but 2n may be enough to keep CPU busy while DB requests are being handled
<TimUckun[m]> except for the plain text and json tests which are simple and no database calls
<headius> and this runs everything on the same system so even if it's blocking on DB then a CPU will be consmed doing DB work
<headius> if it doesn't seem to max all CPUs then we can tune it upwards
<TimUckun[m]> BTW the PG gem has been updated to try and do everything Async and now also supports fibers
<headius> I have generally had good results with 2n with local docker PG
<TimUckun[m]> sequel also has an async way to running queries but I ran into a bug. Jeremy knows about the problem
<headius> yeah PG gem is probably not going to be supported for us any time soon but JVM fibers work ok with jdbc postgres in theory
<headius> we'll cross that bridge once we can run falcon
<TimUckun[m]> For now I'll do the two times procs
<headius> golden path right now is puma + sequel + jdbc until we can expand support
<TimUckun[m]> Good news. The bechmark suite verified
<TimUckun[m]> just pushed my changes
<TimUckun[m]> Tomorrow I'll run the benchmarks proper and report back. I am sure lots of fine tuning can be done
<headius> that's great, good progress
<TimUckun[m]> Should I also look into doing something with warbler and jetty? Would that be significantly faster?
<headius> once we have the benchmarks running I can help profile and see where we can improve
<TimUckun[m]> or maybe just invoking config.ru without going through puma?
<headius> warbler/war/jar-based deployments for JRuby generally were not faster, and that line of code is a bit out of date anyway right now
<headius> it's a deployment choice, not a performance choice
<headius> we're going to get warbler and jruby-rack updated to run well again and run fast but it's not in the immediately future
<TimUckun[m]> ok
<TimUckun[m]> It will be interesting to see how the updated ruby code runs on the benchmarks
<TimUckun[m]> I just got done updating the roda benchmarks and on my machine the updated code (ruby 3.2 yjit) ran about 30% faster than the older version
<headius> nice
<TimUckun[m]> anyway I really need to get to bed. Good night (or day or morning wherever you are)
<headius> yjit is definitely helping
<headius> yeah I'll be in the office this week on US time and monitoring Matrix a lot
<headius> thanks for looking into this stuff!
<donv[m]> headius: I'm getting test failures on a pull request, and they look unrelated. Anything I need to do? https://github.com/jruby/jruby/actions/runs/4981319906
<headius> I broke them before the conf and haven't fixed them
<headius> it's not you
genpaku has quit [Ping timeout: 268 seconds]
genpaku has joined #jruby
<donv[m]> headius: @enebo Anyone here got access to merge to the `release` branch at https://github.com/jruby-gradle/jruby-gradle-plugin ?
subbu has quit [Quit: Leaving]
<headius> donv: I changed the team role to maintain so I think you can merge now
<headius> GitHub ACLs have gotten more complicated since I last messed with teams
_whitelogger has joined #jruby