15:48 <Guest4> hi there all! very beginner/silly question

15:49 <Guest4> looking to see if there's any chance there might creative way to cache package dependencies btw builds

15:50 <devo-devo-devo> looking to implement something like this to speed up or internal build workflow https://docs.github.com/en/actions/using-workflows/caching-dependencies-to-speed-up-workflows

16:25 <glogan> I use the docker containers, and either SCP/S3 the artifacts to a central source, or write them to the worker via a volume

16:26 <glogan> depending on what's needed

16:26 <glogan> For maven artifacts I just cache on the local workers. For our built tarballs I copy them to a central location, and then pull them back down when building packages

19:24 <devo-devo-devo> thanks glogan!

19:31 <devo-devo-devo> what route might you go for python and npm dependencies? would you right those to a volume as well?

19:38 <glogan> If you're going with docker containers then yeah. In my case I actually bind-mount a local directory on the worker rather than storing things in a docker volume, but it's the same idea

19:39 <glogan> I'd probably do the same without the docker volumes tbh. Just move/copy/rsync the stuff you want somewhere safe

21:49 <devo-devo-devo> glogan thanks again. you wouldn't happen to have any sample or pseudo code I could take a glance at?

21:49 <glogan> sure, sec

21:50 <glogan> So, this is all ansible-ized, but it should be fairly clear to figure out

21:51 <glogan> https://github.com/opencast/opencast-project-infrastructure/blob/master/ansible-buildbot-cluster/roles/buildbot-config/templates/master.cfg#L84 is setting the latent worker(s) shared volumes (bind mounts) to the underlying storage medium/disk

21:51 <glogan> and then I use https://github.com/opencast/opencast-project-infrastructure/blob/master/ansible-buildbot-cluster/roles/buildbot-config/templates/common.py#L169 to sync tarballs/builds back and forth as needed

21:52 <glogan> This is set of playbooks is what underpins http://ci.opencast.org/

21:53 <glogan> Our builds call https://github.com/opencast/opencast-project-infrastructure/blob/master/ansible-buildbot-cluster/roles/buildbot-config/templates/common.py#L87, which sets the relevant environment variables so that the maven dependencies end up in the right places

21:53 <glogan> among other things