02:42 <FromGitter> <skinnyjames> > @skinnyjames hard to say w/o a reproducible scenario ⏎ ⏎ @Blacksmoke16 , just a follow up on shards install failing in docker, I am able to somewhat reproduce this with shards install on a docker build. ⏎ It seems to generally happen on cloning `https://github.com/mamantoha/http_proxy.git` as a dependency of `https://github.com/mamantoha/crest.git` ⏎ ...

02:42 <FromGitter> ... [https://gitter.im/crystal-lang/crystal?at=63a3c40eff1fd121b30bdb53]

02:49 <FromGitter> <skinnyjames> Itd oes seem that `git clone` is wrapped in retry (https://github.com/skinnyjames-forks/shards/blob/master/src/resolvers/git.cr#L319). I'd be surpised if it was netowrk related though.

03:44 <FromGitter> <Blacksmoke16> hmm

03:45 <FromGitter> <Blacksmoke16> what happens if you try to manually run that git command a few times? does it reproduce that way?

03:47 <FromGitter> <moe:busyloop.net> also what `shards --version` is that? 🤔 ⏎ my output of `shards --verbose` is completely different, doesn't use clone at all, only `git fetch`

03:48 <FromGitter> <moe:busyloop.net> your output almost looks like it's trying to clone the same shards multiple times in concurrently

03:49 <FromGitter> <Blacksmoke16> its in a retry so its failing to clone a few times until the retry is exhausted id guess

03:50 <FromGitter> <moe:busyloop.net> ah, it's serial, ok

03:50 <FromGitter> <Blacksmoke16> is it possible you already have them installed so it just needs to look for updates? afaik `git fetch` doesnt really allow for pulling a remote repo to your machine?

03:51 <FromGitter> <moe:busyloop.net> hm, `rm -rf lib ; shards install --verbose` doesn't show me any clones

03:52 <FromGitter> <Blacksmoke16> sec

03:52 <FromGitter> <moe:busyloop.net> which is a bit strange indeed

03:52 <FromGitter> <Blacksmoke16> no it deff clones

03:53 <FromGitter> <Blacksmoke16> `rm -rf ~/.cache/shards` then try again

03:53 <FromGitter> <moe:busyloop.net> ah, that was the trick yep

03:53 <FromGitter> <Blacksmoke16> 👍

03:55 <FromGitter> <moe:busyloop.net> @skinnyjames: fwiw, you could try `--jobs=1` to make sure it's not something weird with the parallel fetching

03:56 <FromGitter> <moe:busyloop.net> but that has been released quite a while ago and i haven't heard complaints, yet, so seems unlikely to me

03:56 <FromGitter> <moe:busyloop.net> if you really want to go deep i suppose you could replace your `git` with a wrapper script that injects some verbose logging parameters

03:57 <FromGitter> <Blacksmoke16> or just see if you can run the command manually w/o `--quiet` and see if it reproduceds

03:57 <FromGitter> <moe:busyloop.net> yup that too

03:59 <FromGitter> <moe:busyloop.net> it's a bit annoying that shards doesn't pass --verbose through to the git commands

03:59 <FromGitter> <Blacksmoke16> Maybe it should if you pass it to shards

03:59 <FromGitter> <moe:busyloop.net> haven't problems with it myself but i can see how this is a more cumbersome to debug than it should be :/

03:59 <FromGitter> <Blacksmoke16> Could check the issues

03:59 <FromGitter> <moe:busyloop.net> yea

04:01 <FromGitter> <moe:busyloop.net> yea nah i don't fix it anyway. i'm generally trying to postpone all issues until chatgpt supports "fix issue #124"

04:02 <FromGitter> <moe:busyloop.net> in fact, wouldn't even be surprised if it could add that if you paste it the code, but too lazy to try

08:12 <FromGitter> <skinnyjames> Ah, when running it manually it works the first time and then fails due to the file / directory already existing.

08:15 <FromGitter> <skinnyjames> Oh making a git wrapper is probably a good idea. I think it'd also be cool to log the full error (https://github.com/skinnyjames-forks/shards/blob/master/src/resolvers/git.cr#L437) as a side effect instead of only swallowing it in the retry. (Happy to open a PR)

08:37 <FromGitter> <skinnyjames> also: ⏎ ⏎ > Ah, when running it manually it works the first time and then fails due to the file / directory already existing.

09:07 <FromGitter> <skinnyjames> based on the output it does seem to be running `git clone` for different shard dependencies concurrently, and i notice that both of the deps for `crest` live in `github.com/mamantoha` so i do wonder if there is some sort of race condition at play? Thanks all, I'm going to keep digging into this angle.

10:41 <FromGitter> <moe:busyloop.net> @skinnyjames: mind sharing the `dependencies` part of your `shard.yml`? i couldn't reproduce it here

10:43 <FromGitter> <skinnyjames> absolutely! ⏎ ⏎ ```code paste, see link``` [https://gitter.im/crystal-lang/crystal?at=63a434b5378d512c1827c418]

10:50 <FromGitter> <skinnyjames> Hm.. I just got a (perhaps related) error when doing a docker build and installing apt packages: ⏎ ⏎ ```code paste, see link``` [https://gitter.im/crystal-lang/crystal?at=63a4368f8f8df4262ab95bd9]

10:51 <FromGitter> <skinnyjames> I wonder if something is up with my..uh..ISP

10:55 <FromGitter> <skinnyjames> Or my docker configuration? I'm not sure this is an issue with shards. Big apologies for the misdirection - I was confused because it always failed on the same dependency

13:06 <FromGitter> <moe:busyloop.net> np at all. DNS errors are iffy. ⏎ ⏎ fwiw, can confirm that i don't see any duplicate cloning when running `rm -rf ~/.cache/shards lib shard.lock ; shards install --verbose` with your `shard.yml`. so i think blacksmoke is right, the duplicates you've seen are most likely retries - due to the dns lookup errors.

13:09 <FromGitter> <moe:busyloop.net> if you have access to the resolver config on that machine then i'd try switching to the cloudflare or google resolvers: ⏎ ⏎ ```1.1.1.1 ⏎ 1.0.0.1 ⏎ 8.8.8.8 ⏎ 8.8.4.4``` ⏎ ⏎ those are usually more reliable than what ISPs and hosters provide by default. [https://gitter.im/crystal-lang/crystal?at=63a4570c378d512c18280952]

15:40 <FromGitter> <skinnyjames> Ah nice! thanks!