sagax has quit [Remote host closed the connection]
sagax has joined #jruby
<basshelal[m]>
headius
<basshelal[m]>
Hopefully I didn't say anything too stupid or incorrect, I haven't seen the codebase for some time and don't have access to it now so I had to do it from memory
<headius>
Doesn't look like the other openjdk folks have done anything with the issue I filed
<headius>
enebo: so let's have a quick status update when you are back from lunch
<basshelal[m]>
headius: That's a shame, well do let me know if you hear back from them. Otherwise we'll have to make our own fixes around it, I have some ideas in mind using options/flags that keep "pure" behavior and also allow for forcing the expected paths. I know this isn't _critical_ but I like to remove as many issues as possible, I get a different kind of high when I get issues fixed on JNR vs my own crap 😄
<headius>
yeah I think just keeping the default fallback on macOS would be fine, since we aren't going to have a situation like Nix where there's a bunch of custom lib paths
<headius>
the linux setups are much wilder and more varied and linux JDK seems to do the right thing with that path var
<basshelal[m]>
As expected GNU/Linux is superior 😏
<headius>
I am looking at issue list and will work on the extra backtrace frames
<enebo[m]1>
cool
<headius>
a bit of musing on Ractor I want to dump here...
<headius>
I think it could probably be implemented as an actor with restrictions on what objects can be copied in and out, and some IR work to take the block given and isolate it from the captured state
<headius>
the purpose of this is to allow CRuby to run a parallel execution machine safely, which we already can do, and to ensure objects within that machine are either entirely local (copied) or immutable and shared
<headius>
there may be little difference between a Ractor and a parallel-executing Thread in practice, other than the working set of objects the Ractor can access
<headius>
I don't think it would be appropriate or necessary to fully isolate the class hierarch and namespaces. That would be multi-VM and although it would be useful for CRuby and easy for us to implement, Ractor is not that
<headius>
other than safety checks, we could almost `class Ractor < Thread` and add some actor input/output logic and it work
<enebo[m]1>
I think the isolation of how sharable and non-sharable work is the only value of a ractor
<enebo[m]1>
This does not appear to actually even help perf for MRI
<headius>
from what I have gathered they do not even get isolated heaps so the objects are still just out there floating in the same sea
<enebo[m]1>
but it does make it difficult to access quite a bit of stuff
<headius>
there are series of commits of them trying to make libraries Ractor-friendly, mostly using more immutable state
<enebo[m]1>
yeah it is literally only enforcement that you did not explicitly make something sharable when you accidentally try to access it in a sharable way
<headius>
right
<headius>
so if we implement the right safeguards at the Ractor boundary it is mostly just a thread
<headius>
there might need to be some trickery to hide Ractor threads from the main threads but that is speculation on my part
<enebo[m]1>
sure so long as anything within a ractor has these extra access checks and things outside do not it seems like it could just be a thread
<enebo[m]1>
I don't think we need to be worried about ractor being aliased so we can probably make a limitation and detect it at parse time and then for things obbviously enforcable we emit that stuff directly and for more indirect (like a call to a callwhich mutates state) a flag for the thread to know it should be checking
<enebo[m]1>
That is probably the ickiest part of this
<headius>
we keep the IR around even in JIT mode so it should be possible to tweak the block at any time
<enebo[m]1>
we could make an entire bifurcated set of calls but it would not be worth it to me :)
<headius>
it needs to fast fail when surrounding scope is accessed but that is easy to detect
<headius>
personally I don't know why ko1 made it use a block when it is clearly not doing any of the things blocks enable
<enebo[m]1>
but if I do a.foo() and foo does self.class.set_instancevar("foo", 1)
<enebo[m]1>
We have to see below IR
<enebo[m]1>
we can detect some of it at the IR level
<headius>
yeah there will need to be checks added to know if we are in the "main ractor" or a child ractor and fail anything you are not supposed to do
<headius>
so thread/context will gain a ractor reference similar to fiber
<enebo[m]1>
yeah. so ractor threads need state but we then may have that state check for all code paths
<enebo[m]1>
unless we decide to have two parallel code paths
<headius>
perhaps later if it becomes a problem
<headius>
there are not a ton of things that need to be guarded
<enebo[m]1>
well if we call anything which mutates state all of those would need this logic
<headius>
instance var example would not work because either it has to be immutable or only live in the ractor's world
<headius>
class vars will require and extra check but boo class vars anywa
<headius>
I mean it would not work because either you are creating a new A in the ractor or the A passed in has to have been prepped for sharing
<enebo[m]1>
It should not work but I don't see how us calling a method of a shared class definition abort without knowing if it is ractor or not-ractor thread
<headius>
it can't be shared unless immutable
<enebo[m]1>
or marked sharable
<headius>
which I believe requires immutability?
<enebo[m]1>
no I don't think so
<enebo[m]1>
Maybe I misunderstood that
<enebo[m]1>
Perhaps make_sharable is only read-only sharing but not frozen?
<headius>
To make shareable objects, Ractor.make_shareable(obj) method is provided. In this case, try to make sharaeble by freezing obj and recursively travasible objects. This method accepts copy: keyword (default value is false).Ractor.make_shareable(obj, copy: true) tries to make a deep copy of obj and make the copied object shareable.
<headius>
deep freeze or deep copy
<enebo[m]1>
yeah perhaps make_sharable is just a way of reading the values (although master ractor can probably change the values)
<headius>
yeah unsure
<headius>
"Only the main Ractor can access instance variables of shareable objects."
<enebo[m]1>
So if we share class definitions I cannot then A.new on a shared A?
<headius>
so that is weird too
<enebo[m]1>
I guess it would be an instance in the child Ractor
<headius>
how do you use a custom class if you can't access ivars
<enebo[m]1>
so it would not matter how it was modified
<headius>
the example actually shows it erroring on a read
<enebo[m]1>
I think it is modifying something from parent ractor in child right?
<headius>
ugh the example says "Instance variables of shareable objects" but then shows class instance vars
<headius>
it is definitely a mess of checks going into and out of the ractor
<enebo[m]1>
yeah I even think A.my_fun then needs to make sure my_fun cannot change @a
<enebo[m]1>
A.my_fun is fine to call so long as it does not violate the rules
<headius>
I don't see how this is supposed to work if the class is still mutable
<enebo[m]1>
Or at least this is why I am talking about thread state
<headius>
perhaps the assumption is that guarding access to still-mutable state on those classes keeps them safe
<headius>
but most classes will need to access that state to work right
<enebo[m]1>
At this point without reading more than this document it seems parent ractor can do anything but shared things by default are all frozen
<headius>
I hate that this landed in 3.0 because regardless of how experimental it is, it will become standard now
<headius>
yeah that is my impression too
<enebo[m]1>
the class/module is the weird bit since those (T_CLASS etc) definitely seem like they can get modified
<enebo[m]1>
unless they just raise if they are being used in a ractor...which is then another state check
<headius>
yeah
<headius>
I could see these checks being the reason this is not actually faster than a thread or forked process
<enebo[m]1>
so I feel there is at least one piece of state we need to track in whether the thread is ractor or notmal
<headius>
constantly checking that we are allowed to access something or mutate something in this ractor
<headius>
right, so that is the ractor field on thread or thread context
<headius>
or some bit we check to know that extra checks are in force for this thread's accesses
<enebo[m]1>
and if we duplicated all call/consts code paths we could eliminate the cost
<headius>
could be possible to emit parallel code paths for ractor vs normal within IR/JIT but hand-written code will have to do the checks or be manually split
<enebo[m]1>
direct ivar/cvar within a ractor can be protected in IR
<headius>
call/const/var accesses would mostly be splittable in IR
<headius>
heh yes
<headius>
which might cover most accesses we care about
<enebo[m]1>
yeah seems like a lot of work to not pay a small penalty for ractors
<headius>
something safepointing elsewhere that can be optimized away when ractors are not in use
<headius>
hell, could be a CLI flag initially that just turns ractors on
<headius>
--enable:ractor
<enebo[m]1>
yeah switchpoint it and no one will use it so we won't pay
<enebo[m]1>
yeah that is actually a possibility I guess
<enebo[m]1>
as I said I don't think this is worth our time now
<enebo[m]1>
I don't think it is bad for us to figure out what it needed but I just doubt very many people are using this
<headius>
my hope is that this will push users to do more immutable stuff and that will help us in general
<headius>
it makes immutable objects and persistent data structures the easy way
<enebo[m]1>
yeah true immutability would be really helpful
<headius>
might be worth trying to push for some immutability-friendly data structures in core, like quick ways to create a frozen hash or array
<headius>
was there another refinement+prepend issue you fixed?
<headius>
oh six days ago
<headius>
so this needs to merge to 3.0 work
<enebo[m]1>
well it was respond_to? but I think I read that second one and misremembered what it was
<headius>
yeah just two symptoms of the same problem probably
<headius>
hmm maybe
<enebo[m]1>
in the case of respond_to? we need staticscope to [properly look for refinements
<headius>
my bug prevents using a prepended module as a refinement
<headius>
fixed in CRuby by looking in the right place for refinement
<headius>
so this is probably different
<enebo[m]1>
I tried to magic this by just asking context when we see it is refined but it messes up super with refinements since it wants one static scope up
<headius>
right so this is just more extending refinements to reflective methods
<enebo[m]1>
well probably related since you need the proper static scope
<headius>
there are other cases we don't handle right
<headius>
nah mine is not related to scoping
<enebo[m]1>
super needs one down. most things need current
<enebo[m]1>
ok
<headius>
if you refine a module and then prepend something into it it doesn't know that it is a refinement anymore
<headius>
because all the guts moving around
<headius>
something like that
<enebo[m]1>
yeah I am not surprised
<headius>
but that is a 3.0 item anyway
<enebo[m]1>
3.0 big change for prepend will be tracking where they happened so we can update them
<enebo[m]1>
yay we get to add another field :)
<headius>
wonderful
<headius>
and people wonder why DCI was so expensive
<enebo[m]1>
hah...funny. I wonder how DCI will live in 3.0 world
<enebo[m]1>
I guess not with prepend
<headius>
I still know how to implement DCI cheaply (cache similar transient class structures with modules) but it never picked up
<enebo[m]1>
or you just need to make sure you do not have the same module prepended and living past a new prepend ofthat mofule
<headius>
it's still a lot of work for every object you want to DCI though
<enebo[m]1>
Semanitcally this breaks DCI if it really uses prepend
<enebo[m]1>
I have only seen people use it with includes
<headius>
yeah my stuff predates prepend so unsure what I would have to change
<enebo[m]1>
oh wait include probably does do the same thing
<enebo[m]1>
why wouldn't it
<enebo[m]1>
I just think about it as a prepend feature because I did some work with prepend recently
<headius>
it should be similar though... I just cache "FooDeltaXYZ" that does runtime extends (or prepends, presumably) of X Y and Z
<headius>
it just keeps those temporary bottom classes handy if we do the same sequence of module includes
<headius>
only works if you don't proceed to add one-off methods... but it can be zero-alloc for any number of simple extends or prepends
<enebo[m]1>
but if you include Delta and someone later decides to include something into Delta the original thing will suddenly have it
<enebo[m]1>
seems like it may make DCI confusing
<headius>
it becomes a new meta
<headius>
for my logic
<headius>
but yeah that is a possibility
<enebo[m]1>
I am just talking about 3.0 semantics
<enebo[m]1>
not opt at all
<headius>
ah yeah
<enebo[m]1>
in 3.0 all includes/precludes go everywhere
<headius>
the 3.0 changes would have to be evaluated for my opt
<enebo[m]1>
which is weird for DCI if you use a module in different ways
<enebo[m]1>
yeah ancient and I wonder if they still use it
<headius>
how long have we been working on this thing
<enebo[m]1>
one company loved it and wanted the world to see how nice it made things but didn't care about how well it ran
<enebo[m]1>
which maybe is fine :)
<enebo[m]1>
I guess it depends on how much you are doing it andwhether the slowdown matters
<headius>
large coarse-grained DCI might be fine but I think people started trying to use it for everything that had modules include
<headius>
and it would probably still be better to create the amalgamated class once as a wrapper anyway
<enebo[m]1>
they also started doing 'include MyModule.new'
<headius>
torturing the class hierarchy serves nobody
<enebo[m]1>
or dup...
<enebo[m]1>
I guess we don't instantiate modules :)
<enebo[m]1>
we had/have an issue around this where people would expect different cvars
<headius>
yeah
<headius>
and CRuby dupes the methods too
<headius>
pretty sure we still do not
<enebo[m]1>
hmm
<headius>
(or at least, it used to dup the methods too)
<enebo[m]1>
originalModule.cloneMethods(this);
<enebo[m]1>
hmm this might be fixed
<enebo[m]1>
we call variables and constants sync but it looks like we putAll which perhaps means new table but same value
<headius>
woot
<headius>
yeah I dunno
<headius>
it has been years since I looked at it
johnphillips3141 has joined #jruby
<johnphillips3141>
Is there any update on jruby issue 6872 (DST root CA expired)? Trying to find a workaround
<headius>
johnphillips31416: hey I just got back to current issues today
<headius>
I am not sure about that one yet. kares has been doing most of the maintenance of jruby-openssl but he has been occupied with real job more lately
<headius>
if it is blocking you I can try to help look into it tomorrow
<headius>
it probably broke with the most recent jruby-openssl update
<johnphillips3141>
Thank you, will watch the ticket for updates