razetime has joined #jruby
_whitelogger has joined #jruby
razetime has joined #jruby
razetime has quit [Ping timeout: 245 seconds]
razetime has joined #jruby
subbu has joined #jruby
<enebo[m]> headius: ok. another big thing done will be great
razetime has quit [Ping timeout: 246 seconds]
razetime has joined #jruby
legolas[m] has quit [Remote host closed the connection]
razetime has quit [Remote host closed the connection]
<JasonLunn[m]> Is there any reason to expect Object:WeakMap to be broken for numeric keys?
<headius> Jason Lunn: I believe CRuby update it to work with numeric keys but we may not be able to do it the same way (our fixnum/float values are objects and not idempotent)
<JasonLunn[m]> I would have thought swapping out the class that is used as the backing store would be straightforward. If instead of IdentityHashMap the implementation was moved to use a map that compared using hashCode() - it should "just work", right?
<JasonLunn[m]> Or IRubyObject.id() would work, too
<headius> it would change the behavior of other objects that are ===
<headius> or it could
<headius> what is the use case here?
<headius> even if we used a hashtable with non-identity semantics it wouldn't work to keep objects alive; you'd have to maintain a reference to the RubyFixnum or RubyFloat object
<headius> or is it weak values... now I am trying to remember how weakmap is set up
<JasonLunn[m]> Weak values
<headius> right, our WeakValuedIdentityMap
<JasonLunn[m]> The use case is mapping memory addresses (used for native calls to low level protobuf c library calls) to their wrapper objects
<headius> we wouldn't want to touch that class but we could add a WeakValuedMap that works the same but with a non-identity map
<headius> hmm
<headius> ok
<JasonLunn[m]> We don't have an issue with keeping track of the wrapper objects... when they're no longer referenced by anything and GC'd, that's fine
<headius> I'm not sure changing the hash function here would help... if it still does an equality check for the key it would still fail
<JasonLunn[m]> But lookups for valid addresses with actual mappings in the map are returning nil when there is an entry observable in the weakmap
<headius> hmmm hashMap uses compareTo so it might be ok
<headius> we probably used identity so it would not call back into Ruby for hash values etc
<JasonLunn[m]> Basically any map that doesn't break the map contract the way IdentityHashMap does would be ok
<headius> so it's a concern
<headius> does MRI weakmap call #hash?
<headius> if it does then I guess it's not a concern and we can just align with it
<headius> Jason Lunn: your proposed fix may be fine if MRI also actually calls #hash and separate fixnum objects start working with it
<JasonLunn[m]> I could try my hand at a fix
<headius> that is hooked up to an identity map but I'l checking how they use it
<headius> it does not appear to call #hash
<headius> it uses rb_ident_hash and rb_ident_cmp
<headius> rb_ident_cmp does a direct pointer comparison and rb_ident_hash calculates the hash from the pointer, it looks like
<headius> so it is an identity map in the same way as ours
<JasonLunn[m]> So an identity map based on IRubyObject.id() should work
<headius> I'm not sure I follow
<headius> IRubyObject.id is not going to be the same for two fixnum objects
<headius> I think it might be better for this case to just create a numeric weak map implementation
<headius> or use something equivalent from Java
<headius> I'm going to add some comments to the issue about this
<JasonLunn[m]> headius: I do see that in practice
<JasonLunn[m]> I added a bunch of debugging output to my local build of jruby
<JasonLunn[m]> I would see stuff like this:
<JasonLunn[m]> Two fixnums
<JasonLunn[m]> They look the same (have the same to_s, object_id, Java hashCode() value), but not the same system identity hash
<headius> for the value 42?
<headius> we cache and reuse RubyFixnum objects in the range -256..255 so byte-range operations will not have to create new objects
<JasonLunn[m]> No, for the value of the key (105553124374448), rather than the value
<headius> hmmm
<headius> ok so I think this is because when we calculate id() for fixnum it has to follow the same rules as CRuby's tagged pointers... so the number will be the same every time
<headius> however we are still using an identity map against Object, and that id will end up turning into many different Long objects
<headius> id() for an object will be the same numeric value but when it goes into a map it may be a different wrapper
<headius> they end up getting the same value, calculated from the actual fixnum value, but the ID objects are still different wrapper instances
<headius> JRuby.ref here just makes sure we pass the object_id object directly rather than as a number (which will get reboxed for the identityHashCode call)
<JasonLunn[m]> Yeah, even if I take JRuby out of the equation, Java doesn't have identityHashCode stability for longs:... (full message at <https://libera.ems.host/_matrix/media/v3/download/libera.chat/84b5a043606f1061d88392a4b6cb2ac2b6824d6d>)
<headius> right
<headius> it's only useful for object references as keys
<headius> this might advise how to make a fixnum to weakref map using my weakling library: https://github.com/headius/weakling/blob/master/lib/weakling/collections.rb
<headius> this one just uses the id of the incoming object because it was a proof-of-concept for replasing __id2ref
<headius> but a modification to accept a numeric key would work and still function properly because it's a normal Hash
<headius> could be done with normal weakref too, you just don't get the reference queue feature to clean them up
<headius> $ jruby -rweakref -e 'class FixnumWeakMap; def initialize; @hash = {}; end; def put(a, b); @hash[a] = WeakRef.new(b); end; def get(a); w = @hash[a]; if w; v = w.__getobj__; if v.nil?; @hash.delete(a); end; end; v; end; end; fwm = FixnumWeakMap.new; x = 9001 - 1; y = 8999 + 1; str = "foo"; fwm.put(x, str); p fwm.get(y)'
<headius> "foo"
<headius> there's a trivial Hash-based weak map with poor cleaning capability
<headius> it will leak references to WeakRef if they don't get accessed again and purged
<headius> the smarter version would use weakling and reference queue to clean evacuated references periodically (like on every insert or so)
<JasonLunn[m]> For versions of Ruby before 2.7, protobuf uses a two tier cache to achieve the cleaning and purging...
<JasonLunn[m]> I found this issue when I was trying to see if JRuby 9.4 could use the single tier cache that is backed directly by WeakMap and it failed in really interesting ways
<JasonLunn[m]> I'll see if I can write up a PR to add FixNum key support for weak map without breaking existing tests
sagax has joined #jruby
subbu has quit [Ping timeout: 250 seconds]
<headius> ahh I see
<headius> the only way this could work would be to have the fixnum objects normalized to known instances, like a second map from fixnum to fixnum and always use the value there as the weakmap key
<headius> Jason Lunn: I updated issue based on this discussion and included my trivial numeric weak map impl
<headius> enebo: I went against my usual rule and did a mass reformatting of the mavengem repo
<headius> it had tabs some places, two spaces some places, really a mess to try to maintain so it just needed a clean break
<headius> master there now has reformatting + Java 8 update and using mavengem for its own dependencies... I have one commit on my branch starting to add support for the V2 API and I'm starting to understand where the pieces go
<headius> it's a complicated project but basically you are building a pseudo filesystem for Maven that's backed by rubygems.org API data... so you request a pom for a given gem, it assembles the info for that gem out of the API and produces a PomFile object in the end