15:05 <rjarry> Hi all, suddenly all our workers get rejected after a master restart

15:08 <rjarry> I get a load of twisted.spread.pb.RemoteError: Maximum PB reference count exceeded.

15:09 <rjarry> followed by: twisted.cred.error.UnauthorizedLogin

15:09 <rjarry> does that ring a bell to anyone ?

15:09 <rjarry> I tried stopping everything, and start everything again, same problem

15:09 <rjarry> I did not change anything in the config

15:46 <rjarry> ok, after inspection, it looks like we went above the 1024 builders count

15:47 <rjarry> <rjarry> I did not change anything in the config <-- that wasn't truc

15:47 <rjarry> true

15:47 <rjarry> I see that this is a twisted PB hard limit: https://github.com/twisted/twisted/blob/trunk/src/twisted/spread/pb.py#L782-L787

16:05 <vilhalmer> was that real builders, or can this be exceeded with virtual builders?

16:05 <rjarry> real builders

16:06 <vilhalmer> ah good, otherwise I'd be on the path to hitting that limit very soon

16:06 <rjarry> :)

16:07 <rjarry> it looks like we should switch to virtual builders

16:07 <rjarry> shame

19:36 <aakashjain> Is there a way to delete old builders information for buildbot database (maybe along-with corresponding builds)? (I have couple hundred old builders in one of buildbot instance)

19:36 <aakashjain> Also, can there be performance degrade due to having lot of old builders/builds?