<adam12>
Worse, the only thing I remember was they were using it to stream audio for the example. But I did end up finding the example, and it's Pushpin. Arrgh.
<adam12>
Grr
<weaksauce>
at least you have your answer i guess
<adam12>
I guess, tho I remember there being another one
<adam12>
PushPin is a reverse proxy. There was definitely a sidecar type one where you'd redirect them to it and then use RPC to talk to the sidecar.
<adam12>
So if your Ruby(Python/whatver) app restarted, the sidecar still had all the websockets open.
jenrzzz has joined #ruby
marlasinger has quit [Remote host closed the connection]
jenrzzz has quit [Ping timeout: 256 seconds]
<[0x1eef_]>
I switch between test-unit and rspec. It depends on the project. There's certain tests where I prefer rspec, usually if they're complex-ish.
<weaksauce>
i'll have to look at maxitest
<weaksauce>
there's some qol stuff that the rspec cl has
reset has quit [Quit: reset]
jenrzzz has joined #ruby
Pixi` has quit [Quit: Leaving]
Pixi has joined #ruby
jenrzzz has quit [Ping timeout: 252 seconds]
jenrzzz has joined #ruby
jenrzzz has quit [Ping timeout: 255 seconds]
brokkoli_origina has quit [Remote host closed the connection]
brokkoli_origina has joined #ruby
jenrzzz has joined #ruby
jenrzzz has quit [Remote host closed the connection]
jenrzzz has joined #ruby
jenrzzz has quit [Ping timeout: 264 seconds]
johnjaye has joined #ruby
jenrzzz has joined #ruby
jenrzzz has quit [Ping timeout: 260 seconds]
rvalue has joined #ruby
rvalue- has joined #ruby
rvalue has quit [Ping timeout: 272 seconds]
rvalue- is now known as rvalue
hightower3 has joined #ruby
hightower4 has quit [Ping timeout: 256 seconds]
ionface has quit []
jenrzzz has joined #ruby
_ht has joined #ruby
jenrzzz has quit [Ping timeout: 268 seconds]
mange has quit [Quit: Quittin' time.]
jenrzzz has joined #ruby
jenrzzz has quit [Ping timeout: 252 seconds]
otisolsen70 has joined #ruby
grenierm has joined #ruby
<rapha>
"Each worker gets its own database to avoid conflicts." lol
jenrzzz has joined #ruby
jenrzzz has quit [Ping timeout: 246 seconds]
jenrzzz has joined #ruby
jenrzzz has quit [Ping timeout: 260 seconds]
<weaksauce>
is there another way to do that
jenrzzz has joined #ruby
kerunaru has joined #ruby
jenrzzz has quit [Ping timeout: 260 seconds]
hightower3 has quit [Remote host closed the connection]
hightower3 has joined #ruby
hightower3 has quit [Ping timeout: 256 seconds]
jenrzzz_ has quit [Ping timeout: 264 seconds]
jenrzzz has joined #ruby
dixan has quit [Quit: leaving]
jenrzzz has quit [Ping timeout: 246 seconds]
hightower2 has joined #ruby
jenrzzz has joined #ruby
reset has joined #ruby
infinityfye has quit [Quit: Leaving]
jenrzzz_ has joined #ruby
szkl has quit [Quit: Connection closed for inactivity]
BSaboia has quit [Read error: Connection reset by peer]
dostoyevsky2 has quit [Ping timeout: 276 seconds]
mark22k has quit [Ping timeout: 276 seconds]
mark22k5 is now known as mark22k
Arsen has quit [Ping timeout: 264 seconds]
Arsen has joined #ruby
dostoyevsky2 has joined #ruby
sarna has quit [Ping timeout: 276 seconds]
<henk>
I have trouble with threads and I don’t understand how it happens. I have two variants of a function, one using threads, the other does not: http://pastie.org/p/1X1iI0AQKb1TCn3o9nRhZs the variant using threads "reliably" fails with an error indicating that values in the data structures in `attrs['auth']` get confused between the threads. is there anything wrong with these functions per se? I’m
<henk>
not yet sure how to minimize this or provide further context without showing the complete code. (please let me know if I’m not making sense or if it’s just not enough context. for now I really only want to know if something with how I’m using threads here seems problematic)
Guest16 has joined #ruby
Guest16 has quit [Client Quit]
<[0x1eef_]>
You probably have a race condition but there's not enough code to see why.
grenierm has quit [Ping timeout: 250 seconds]
<[0x1eef_]>
Look for shared state. Try to find where you mutate it. LOCK = Mutex.new; LOCK.synchronize { ... mutating code }. That's the best I can say without seeing more code.
<henk>
[0x1eef_]: https://git.netwichtig.de/cgit/user/henk/code/ruby/macir.git/tree/macir.rb?h=feature/challenges_per_apex#n597 is the full code and where the pasted section starts. I don’t think there is shared state in my code, but I can’t talk about the acme-client library … I’m almost assuming the way I use it doesn’t play well with it … but I’m too much of a noob to be certain. I tried to
<henk>
keep thread usage as simple as possible in my code.
<[0x1eef_]>
Why spawn threads inside threads?
<[0x1eef_]>
Ah sorry. I'm looking at _propagation, not _validation
jas-maelstrom has quit [Quit: Leaving]
<[0x1eef_]>
Is 'challenge' unique per iteration ? As a rule of thumb, you can usually be thread safe in ruby if you create an instance on that thread, it's not always true but I think at least some libs try to live up to that.
<[0x1eef_]>
It's not. From what I see, 'challenge' is the same object across all threads.
hightower2 has quit [Ping timeout: 252 seconds]
<henk>
[0x1eef_]: still, for context: I have multiple certs which can each have multiple domains. each domain requires one challenge to be deployed but for DNS challenges, this challenge needs to be propagated to all nameservers for that domain. i.e. I make one thread per domain/challenge, then for each domain/challenge I make one thread per nameserver. does that make sense?
<[0x1eef_]>
Yeah kind of. I think if 'challenge' will be the same object across threads, and looks like it could be, you might have a problem. You'd essentially call .request_validation twice, in the same object.
<[0x1eef_]>
Then .status might be off as well, there's definitely potential for things to go wrong without a unique instance of 'challenge' per thread.
<henk>
yeah, that would match the behaviour I’m seeing. but `challenge` is the name the function wait_for_challenge_validation uses for its first argument. the function is called from all threads. and the value for the first argument comes from the hash passed to the threat, from the .each_pair. that seems "clean" to me and should not be the same object across all threads. what am I missing? AFAIU it
<henk>
should be unique per thread. is something wrong with the logic in my code or with my understanding of when things are unique?
<[0x1eef_]>
try p challenge.object_id, look for duplicates.
wyclif has quit [Ping timeout: 268 seconds]
sarna has joined #ruby
wyclif has joined #ruby
marlasinger has joined #ruby
<henk>
I added "p challenge.object_id" more or less at the top of the function wait_for_challenge_validation and got http://pastie.org/p/1ZBmxajONYw03jEial2nCi lines 66 and 68 say "3040" and "3060".
donofrio has quit [Ping timeout: 256 seconds]
jas-maelstrom has joined #ruby
<[0x1eef_]>
Try to find what's shared, is the domain the same ? ACME could have its own shared state too.
<henk>
the "client" is the same. also the "certs" key.
<[0x1eef_]>
Same client could be problematic.
<henk>
I don’t think so (but again: I’m a noob), because in my previous approach I also shared the client between various threads each handling a given cert, see 'main' branch https://git.netwichtig.de/cgit/user/henk/code/ruby/macir.git/tree/macir.rb and that worked fine. I’m guessing more it’s about `order` of which there is one per cert, or `auth` of which there is one per domain in a cert.
<henk>
does that reasoning sound plausible?
<[0x1eef_]>
Order could be an issue too, the iteration order isn't gauranteed.
<[0x1eef_]>
Item 2 could go before Item 1
<henk>
oh, `order` in this case means "I’m ordering/requesting a certificate with these attributes"
<[0x1eef_]>
I'm honestly not sure. I don't have deep understanding of that code. A naive suggestion would be to try isolate 'client' per thread (unique instance), or LOCK.synchronize { ... } within the client itself. I would follow any ends where there is shared state.
<henk>
not sure if that was a confusion (: yes, I’m aware that both with threads and with things like .each_pair there is no guarantee for the order in which things are processed but that shouldn’t matter with how I wrote the code. I tried to put steps in the order required and only use threads where things "should" be allowed to be done in parallel …
<[0x1eef_]>
Makes sense. The best leads are looking for what's shared. Right now that appears to be client + certs. You could also use fork instead. It's not free of race conditions but these types of situations can usually be avoided with a subprocess.
otisolsen70 has quit [Quit: Leaving]
<[0x1eef_]>
Well, nevermind fork. Looking at your code that would appear to be a big rewrite.
<[0x1eef_]>
But Order.new itself is not shared, so I'm not sure.
<henk>
hm, yes, I do, not on the `order` object, but on the `challenge` object, though …
marlasinger has quit [Quit: Leaving]
<[0x1eef_]>
From what I can tell, that library makes an effort to not use shared state, it is often passing around locals but not always.
marlasinger has joined #ruby
wyclif has quit [Ping timeout: 260 seconds]
osc4rpt has quit [Ping timeout: 260 seconds]
osc4rpt has joined #ruby
infinityfye has joined #ruby
<henk>
I guess with my current understanding of ruby, there is not much I can do about it. I’ll probably try to see whether I can do the threading "per cert order" instead of "per domain challenge" and see if that works better. otherwise I’ll just not do any threading at that particular point in the code …
<henk>
[0x1eef_]: thank you very much for your effort and sharing your insight!
<[0x1eef_]>
You could also create a client per thread, and initialize it with shared state.
<[0x1eef_]>
No worries :)
infinityfye has quit [Client Quit]
infinityfye has joined #ruby
<henk>
[0x1eef_]: errr … I don’t get that. a client instance is "one object used for communicating with a given endpoint using a given identity". I’m not sure I should have multiple of those using the same endpoint and the same identity. from a ACME point of view.
<henk>
whoa, even without threading in this code section, my client is a lot faster with this approach than the one in the main branch. main branch took 1.5 minutes at least (for ca 15 certs with ca 70 domain names). this one just took less than 20s …
wyclif has joined #ruby
wyclif has quit [Ping timeout: 276 seconds]
TomyWork has joined #ruby
osc4rpt has quit [Ping timeout: 256 seconds]
osc4rpt has joined #ruby
jenrzzz_ has joined #ruby
jenrzzz_ has quit [Ping timeout: 256 seconds]
<rapha>
weaksauce: no, of course not. i was having to laugh because my client works with prod db dumps on their development laptops. both for development as well as testing. and a dump at the moment is 3GB+.
wyclif has joined #ruby
<rapha>
we've been trying to ween them off of it, but the habit sits deep and there's a lot of resistance against adopting a proper way of working with seeds and factories.
osc4rpt has quit [Quit: Lost terminal]
osc4rpt has joined #ruby
yaw has joined #ruby
<yaw>
i thought i'd seen a ruby feature request to allow `return` statements in a block to exit just the block, rather than the surrounding function context. any chance anyone might have a link to the issue? thanks :)
<yaw>
henk: at the moment that's how it works, i think. but i thought i'd seen a proposal to change the behavior of `return` in blocks.
<henk>
oh ok, sorry, I misread or misunderstood
<yaw>
no worries :)
<yaw>
i could be misremembering as well.
jenrzzz_ has quit [Ping timeout: 276 seconds]
jenrzzz_ has joined #ruby
Vonter has quit [Ping timeout: 240 seconds]
jenrzzz_ has quit [Ping timeout: 255 seconds]
yaw_ has joined #ruby
yaw_ has quit [Client Quit]
yaw has quit [Ping timeout: 252 seconds]
jenrzzz_ has joined #ruby
<[0x1eef_]>
henk: Don't know much about acme-client. I think I'm using Python for that. It's a letsencrypt client or more general in this case ?
jenrzzz_ has quit [Ping timeout: 260 seconds]
<henk>
[0x1eef_]: letsencrypt implements the ACME protocol (with only minor divergences AFAIU) and acme-client is an implementation of that protocol. main use case for me is use with LE, yes.
<[0x1eef_]>
Cool.
Vonter has joined #ruby
wyclif has quit [Quit: leaving]
donofrio has joined #ruby
jenrzzz_ has joined #ruby
osc4rpt has quit [Ping timeout: 268 seconds]
Vonter has quit [Quit: WeeChat 4.2.1]
osc4rpt has joined #ruby
Vonter has joined #ruby
<marlasinger>
Why not use the language as a standard and not a constantly updated interpreter?
otisolsen70 has joined #ruby
donofrio has quit [Read error: Connection reset by peer]
<rapha>
Because the earth revolves around the sun and the sun revolves around the center of the galaxy and the universe keeps expanding.
donofrio has joined #ruby
pgib has quit [Ping timeout: 252 seconds]
reset has quit [Quit: reset]
jenrzzz_ has quit [Ping timeout: 272 seconds]
meimei has quit [Remote host closed the connection]
pounce has quit [Remote host closed the connection]
pounce has joined #ruby
meimei has joined #ruby
jenrzzz_ has joined #ruby
szkl has joined #ruby
jenrzzz_ has quit [Ping timeout: 252 seconds]
jenrzzz_ has joined #ruby
bambanxx has joined #ruby
jenrzzz_ has quit [Ping timeout: 264 seconds]
jenrzzz_ has joined #ruby
jenrzzz_ has quit [Remote host closed the connection]
jenrzzz_ has joined #ruby
Vonter has quit [Ping timeout: 276 seconds]
Vonter has joined #ruby
jenrzzz_ has quit [Ping timeout: 264 seconds]
meimei has quit [Remote host closed the connection]
pounce has quit [Remote host closed the connection]
pgib has joined #ruby
meimei has joined #ruby
pounce has joined #ruby
osc4rpt has quit [Ping timeout: 264 seconds]
osc4rpt has joined #ruby
kerunaru has quit [Quit: Iridium IRC Client]
<[0x1eef_]>
mindblown.gif
Munto has joined #ruby
bambanxx has quit [Quit: My MacBook Air has gone to sleep. ZZZzzz…]
jenrzzz_ has joined #ruby
jenrzzz_ has quit [Ping timeout: 268 seconds]
jenrzzz_ has joined #ruby
jenrzzz_ has quit [Ping timeout: 252 seconds]
otisolsen70 has quit [Quit: Leaving]
pgib has quit [Quit: 00 PC LOAD LETTER]
jenrzzz_ has joined #ruby
osc4rpt has quit [Ping timeout: 246 seconds]
osc4rpt has joined #ruby
polishdub has quit [Quit: leaving]
osc4rpt has quit [Ping timeout: 256 seconds]
osc4rpt has joined #ruby
rvalue has quit [Ping timeout: 272 seconds]
rvalue has joined #ruby
<weaksauce>
rapha that's kinda crazy
<weaksauce>
3gb test db wat
TomyWork has quit [Remote host closed the connection]
TomyWork has joined #ruby
jenrzzz_ has quit [Ping timeout: 256 seconds]
<[0x1eef_]>
The wild wild west never went away it just morphed into tech companies.
SunClonus has joined #ruby
jenrzzz_ has joined #ruby
user71 has quit [Quit: Leaving]
<adam12>
lol
<rapha>
weaksauce: they use the prod db dump for tests and development lol
<rapha>
lol [0x1eef_]
<adam12>
I worked on a project once that did that, back in ~ 2013 or so.
<adam12>
Then someone added real SMTP credentials to the Rails config.
<adam12>
Customer forwards an email to the PM, with a customer asking why they were receiving multiple new invoice emails every hour.
<adam12>
Client forwards email to the PM rather, with _their_ customer asking
<[0x1eef_]>
xD
<adam12>
The database ended up in the dev's test environment. Hehe.
<[0x1eef_]>
It happens. I remember when defunkt DROP'ed the GitHub database by mistake.
<adam12>
Yep. I've moved all my maintenance to 8AM because it's less likely for me to make a mistake. I used to do it at 3-4AM.
<adam12>
But brain isn't functioning at that time. And frequently I'd run into issues where my ISP was doing their maintenance at that time too.
SunClonus has quit [Quit: Leaving]
SunClonus has joined #ruby
polishdub has joined #ruby
jenrzzz_ has quit [Ping timeout: 255 seconds]
TomyWork has quit [Remote host closed the connection]
_ht has quit [Remote host closed the connection]
jenrzzz_ has joined #ruby
pgib has joined #ruby
jenrzzz_ has quit [Ping timeout: 272 seconds]
jenrzzz_ has joined #ruby
johnjaye has quit [Ping timeout: 272 seconds]
pgib has quit [Quit: 00 PC LOAD LETTER]
johnjaye has joined #ruby
hightower2 has quit [Remote host closed the connection]
hightower2 has joined #ruby
jenrzzz_ has quit [Ping timeout: 272 seconds]
hightower2 has quit [Remote host closed the connection]
hightower2 has joined #ruby
Retr0Punk has quit [Quit: cya]
jenrzzz_ has joined #ruby
SunClonus has quit [Quit: Leaving]
hightower2 has quit [Remote host closed the connection]
hightower2 has joined #ruby
gramps has quit [Ping timeout: 252 seconds]
hightower2 has quit [Remote host closed the connection]
hightower2 has joined #ruby
ruby[bot] has quit [Remote host closed the connection]
ruby[bot] has joined #ruby
caleb_ has quit [Ping timeout: 256 seconds]
g_sg has quit [Ping timeout: 256 seconds]
graemefawcett has quit [Ping timeout: 256 seconds]
pjaspers has quit [Ping timeout: 264 seconds]
fourstepper has quit [Ping timeout: 264 seconds]
KOTP has quit [Ping timeout: 264 seconds]
caleb_ has joined #ruby
whereiseveryone has quit [Ping timeout: 256 seconds]