<DarkKnight>
Hello. I'm sharing sstate between multiple users with 777 permission. But for non owner users bitbake raises lots of "operation not permitted" errors. What is the correct way of sharing sstate between multiple users?
dmoseley has joined #yocto
mvlad has joined #yocto
<brabander>
do you mean they also have write access and could update the sstate simultaneously ?
<DarkKnight>
brabander Yes, but in my case they are not doing it simultaneously.
mckoan|away is now known as mckoan
<mckoan>
good morning
<mckoan>
DarkKnight: AFAIK sharing sstate between multiple users is unsupported
<JaMa>
brabander: writable SSTATE_DIR (e.g. shared over NFS between multiple builders) is also fine (and more efficient as sstate gets available as soon as first builder finishes building given recipe, instead of waiting for some of the builds to finish completely and rsync to the SSTATE_MIRRORS location)
florian_kc has joined #yocto
florian_kc has quit [Ping timeout: 252 seconds]
dmoseley has quit [Excess Flood]
seninha has joined #yocto
dmoseley has joined #yocto
<mckoan>
brabander: those examples are not covering a multi-user access rights. AFAIK because that is not supported
<RP>
JPEW: I suspect it is may be the use of callables as keys. I have a patch on -next adding locking
<mckoan>
only a signe user can access in Write mode the SSTATE_DIR
<mckoan>
s/signe/single
<mckoan>
JaMa: do you have any detail on that?
<DarkKnight>
brabander I don't share the SSTATE_DIR. I have a separate path for SSTATE_MIRRORS and rsync SSTATE_DIR on it. Other users can use the mirror path by setting SSTATE_MIRRORS. Yet, I got errors from bitbake.
<DarkKnight>
read only SSTATE_MIRRORS does not work for me. no setscene task will run.
<JaMa>
mckoan: shared DL_DIR is protected by locks, shared SSTATE_DIR uses atomic moves, both work fine in big build farms like YP autobuilder and I'm using it in LG for 10+ years with sometimes 30 parallel builds as well
<JaMa>
the only downside is that SSTATE_DIR grows quickly when every developer can trigger a build which uses completely different sstate signatures, but disk space is relatively cheap and sstate easy to prune - better than rebuilding the same set of new signatures many times until some "official" build updates the R-O SSTATE_MIRROS with the new update
<JaMa>
maybe it's time to update that wiki page as this question in quite common here
<mckoan>
JaMa: thx
dmoseley has quit [Excess Flood]
dmoseley has joined #yocto
dmoseley has quit [Excess Flood]
dmoseley has joined #yocto
dmoseley has quit [Excess Flood]
dmoseley has joined #yocto
amitk_ has joined #yocto
amitk_ has quit [Ping timeout: 246 seconds]
DarkKnight has quit [Quit: Client closed]
goliath has joined #yocto
gsalazar has quit [Remote host closed the connection]
florian_kc has joined #yocto
dmoseley has quit [Excess Flood]
PhoenixMage has joined #yocto
dmoseley has joined #yocto
<LetoThe2nd>
what is the scope that a PACKAGECONFIG affects in a recipe?
<PhoenixMage>
I am trying to compile yq for an aarch64 board and getting a bunch of linker errors similar to "runtime/cgo(.text.fatalf): unknown symbol fputs in callarm64" dont suppose anyone has come across this before? Everything is kirkstone
gsalazar has joined #yocto
alessioigor has quit [Quit: alessioigor]
alessioigor has joined #yocto
<mihai>
LetoThe2nd: what? :)
<LetoThe2nd>
mihai: yeah. a coworker seems to have trouble in the packageconfig not kicking in for additional PACKAGES
<JaMa>
I don't see how PACKAGECONFIG relates to additional PACKAGES
sakoman has joined #yocto
* LetoThe2nd
will need to dig deeper with him probably
gsalazar has quit [Remote host closed the connection]
GNUmoon has quit [Remote host closed the connection]
GNUmoon has joined #yocto
dmoseley has quit [Excess Flood]
dmoseley has joined #yocto
invalidopcode has quit [Remote host closed the connection]
invalidopcode has joined #yocto
dmoseley has quit [Excess Flood]
dmoseley has joined #yocto
dmoseley has quit [Ping timeout: 246 seconds]
alessioigor has quit [Quit: alessioigor]
alessioigor has joined #yocto
dmoseley has joined #yocto
alessioigor has quit [Quit: alessioigor]
alessioigor has joined #yocto
<JPEW>
RP: Ok, let me take a look...
<JPEW>
RP: You still kinda have a race there because you use the keys without the lock locked
<RP>
JPEW: except that externally there is only addition, never deletion
<RP>
JPEW: only that code can ever delete an entry so I think that is ok. I did also assume the len() calls were atomic enough not to matter :/
<JPEW>
RP: Which len()?
<RP>
JPEW: line 425
<JPEW>
Hmm, ya.... have to think about that one
<JPEW>
You can still use items, since you are locking anyway:
<JPEW>
items = list(self._idlefuncs.items())
<JPEW>
with self._idlefuncsLock:
<JPEW>
with function, data in items:
<JPEW>
I think you need a condition variable instead of an event
<RP>
JPEW: For the idle state?
<JPEW>
Ya
<JPEW>
I can write up a patch to show what it looks like
<RP>
perhaps. The requirements of that changed a bit as I discovered all the races
<RP>
JPEW: there is some other signature issue going on too. I've merged the receiver counting one as I think I'd prefer to see if that fixes things and debug from there
<JPEW>
Events should really only be used when the only state is the event itself. If you rely on any other state, they race
* RP
is probably going in too many directions at once :(
<JPEW>
(and a condition variable is better)
<RP>
JPEW: the concept there was that the system was at true "idle" which is an event
<JPEW>
You can get into a case where you have to unnecessarily wait the full 30 seconds for the timeout with bad timing
<JPEW>
e.g. if the event is set, right before it gets cleared
<JPEW>
Events just can't be used in this way without racing
<RP>
A condition would indeed be better for that
<RP>
I'm still surprised I needed the locking around the dict, I can only think the callable keys were an issue
<JPEW>
No, it's state your accessing from multiple threads
<JPEW>
The only reason it doens't explode entirely is because of the GIL, but you shouldn't rely on that
<RP>
JPEW: I'm just surprised the GIL didn't make things work :/
<JPEW>
They GIL doens't exactly exist to save developers from correct locking behavior :)
<JPEW>
The can (and do) change when it is locked and unlocked from release to release
<RP>
JPEW: I thought that dict was documented as a thread safe type though
<RP>
anyway, we can have the locks :)
<JPEW>
Thread safe doens't mean all operations are atomic
<JPEW>
It just means if multiple threads access it, its not going to explode