<moose>
Quick online shows a few threads suggesting to add do_configure_append() cluase in my receipt but none of that is mentioned in yocto official documentation for how to modify kernel config. Any inputs? Thank you.
RobertBerger has joined #yocto
rber|res has quit [Ping timeout: 272 seconds]
seninha has quit [Remote host closed the connection]
seninha has joined #yocto
money_ has quit [Ping timeout: 256 seconds]
money_ has joined #yocto
starblue1 has quit [Ping timeout: 272 seconds]
starblue1 has joined #yocto
money__ has joined #yocto
seninha has quit [Ping timeout: 272 seconds]
money__ has quit [Read error: Connection reset by peer]
money__ has joined #yocto
money_ has quit [Ping timeout: 256 seconds]
jclsn has quit [Ping timeout: 265 seconds]
jclsn has joined #yocto
Vonter has quit [Quit: WeeChat 3.7.1]
goliath has quit [Quit: SIGSEGV]
seninha has joined #yocto
Vonter has joined #yocto
money__ has quit [Read error: Connection reset by peer]
money_ has joined #yocto
amitk_ has quit [Ping timeout: 246 seconds]
seninha has quit [Quit: Leaving]
amitk has joined #yocto
money_ has quit [Read error: Connection reset by peer]
money__ has joined #yocto
nemik_ has quit [Ping timeout: 260 seconds]
nemik_ has joined #yocto
mckoan_ has joined #yocto
nemik_ has quit [Ping timeout: 260 seconds]
nemik_ has joined #yocto
mckoan|away has quit [Ping timeout: 256 seconds]
money__ has quit [Ping timeout: 256 seconds]
money_ has joined #yocto
money_ has quit [Read error: Connection reset by peer]
money_ has joined #yocto
money__ has joined #yocto
money_ has quit [Ping timeout: 256 seconds]
<Haxxa>
I have a PLC running yocto linux which I cannot rebuilt the OS for. Is there anyway to add something for monitoring filesystem changes? (i.e.inotifywait)
invalidopcode7 has quit [Remote host closed the connection]
invalidopcode7 has joined #yocto
moose has quit [Quit: Client closed]
thomasd13 has joined #yocto
nemik_ has quit [Ping timeout: 268 seconds]
nemik_ has joined #yocto
nemik_ has quit [Ping timeout: 252 seconds]
nemik_ has joined #yocto
money_ has joined #yocto
money__ has quit [Ping timeout: 256 seconds]
pml has joined #yocto
money__ has joined #yocto
money_ has quit [Ping timeout: 256 seconds]
tomzy_0 has joined #yocto
tomzy_0 has quit [Client Quit]
tomzy_0 has joined #yocto
<tomzy_0>
morining
money__ has quit [Quit: late]
alessioigor has joined #yocto
money_ has joined #yocto
olani has quit [Ping timeout: 268 seconds]
schtobia has quit [Quit: Bye!]
schtobia has joined #yocto
manuel1985 has joined #yocto
nemik_ has quit [Ping timeout: 252 seconds]
nemik_ has joined #yocto
mihai has joined #yocto
nemik_ has quit [Ping timeout: 268 seconds]
nemik_ has joined #yocto
rfuentess has joined #yocto
manuel1985 has quit [Ping timeout: 260 seconds]
rob_w has joined #yocto
<LetoThe2nd>
yo dudX
<LetoThe2nd>
Haxxa: if it is something that only is a userspace binary, it might be possible - but yocto is not going to help you until you have source. if you need bootloader or kernel modifications, probably no.
<Haxxa>
LetoThe2nd Yeah no source. How hard would it be to compile a new distro for the hardware? I have root access on yocto linux
<Haxxa>
I tried a couple of userland binaries but a lot expect libs that aren't present that are heavily integrated with kernel
<LetoThe2nd>
Haxxa: without source and schematics, depends on your expertise. as you have to ask, i would say "very hard"
<Haxxa>
I'd imagine
<LetoThe2nd>
Haxxa: why not ask the vendor for source?
<Haxxa>
LetoThe2nd lol, there is no way. This is a PLC manufacturer, these guys never release source code, they don't even stipulate it runs linux. My role is generally to make the box do things it's not designed to
<LetoThe2nd>
Haxxa: they have to, bring legal into play then.
<LetoThe2nd>
Haxxa: and it definitely is not true that PLC manufacturers don't do this.
<Haxxa>
The device is a brodersen rtu32m, I don't have resources to bring legal. Also anytime I ask a company to release source code because licensing, it never works.
<LetoThe2nd>
Haxxa: contact FSF[-E] then. its exactly their cup of tea.
<Haxxa>
Out of interest, what implies they have to release source code, I thought it was only for specific parts of the source code which are under GPL
<LetoThe2nd>
Haxxa: yep. but then you've got the kernel source, and from there you can do mostly everything.
<LetoThe2nd>
still its not trivial then, but hey...
<Haxxa>
I think I will take the trivial route and avoid this process :)
* LetoThe2nd
shrug
<LetoThe2nd>
in the end you want to create some statically linked binary then. but its quite different from what yocto does. look at the toolchains that bootlin provides, for example.
kroon has joined #yocto
<kroon>
Hi. Am I the only one for whom bitbake always reparses the recipes on each invocation, even though no recipes are changing ? (using master branches of oe-core/bitbake)
Payam has joined #yocto
mckoan_ is now known as mckoan
<mckoan>
good morning
Algotech75 has joined #yocto
gho has joined #yocto
invalidopcode7 has quit [Remote host closed the connection]
invalidopcode7 has joined #yocto
money_ has quit [Quit: late]
mvlad has joined #yocto
leon-anavi has joined #yocto
kanavin has quit [Remote host closed the connection]
azcraft has joined #yocto
frieder has joined #yocto
<mcfrisk>
is linux-yocto sstate cache is wiped, and recipe rebuilt, but the hashes match the previous version, can this cause rootfs'es to not be re-generated on kirkstone?
<mcfrisk>
/is/if/
frieder has quit [Remote host closed the connection]
Payam has quit [Ping timeout: 268 seconds]
gsalazar has joined #yocto
xmn has quit [Ping timeout: 252 seconds]
money_ has joined #yocto
money_ has quit [Client Quit]
money_ has joined #yocto
money_ has quit [Quit: late]
money_ has joined #yocto
invalidopcode7 has quit [Remote host closed the connection]
invalidopcode7 has joined #yocto
money_ has quit [Client Quit]
money_ has joined #yocto
money_ has quit [Read error: Connection reset by peer]
thomasd13 has quit [Ping timeout: 268 seconds]
malsyned has joined #yocto
florian has joined #yocto
malsyned has quit [Ping timeout: 255 seconds]
florian has quit [Ping timeout: 272 seconds]
<michaelo>
Greetings. Did any of you who spoke at YPS2022.11 get a speaker gift this time? I saw the line in the conf budget, but apparently haven't received anything.
<LetoThe2nd>
ndec: your call ;-)
<ndec>
heh... yes, it is planned, i am almost done with getting the gift cards and sending them away!
<LetoThe2nd>
physical ones?
* LetoThe2nd
blows mind
tomzy_0 has quit [Quit: Client closed]
florian has joined #yocto
<mcfrisk>
if I wipe sstate cache of linux-yocto kernel and then rebuild target image, then linux-yocto will be recompiled, but the image will not be recompiled. I don't get this. Why doesn't the image get recompiled if kernel was recompiled. the task hashes of kernel did not change, but the binaries did.
<ndec>
LetoThe2nd: no, sending via email :)
<mcfrisk>
if signing is enabled, linux-yocto do_compile generates a key pair and do_install signs modules. Thus every rebuild is unique even if tash hashes did not change, and full dependency tree should be recompiled. How to enforce this?
<qschulz>
RP: wondering if scripts added by addpylib are watched by bitbake for changes? I remember that wasn't the case in Thud and we had to manually trigger a rebuild of everything when we did change some layer scripts
nemik_ has quit [Ping timeout: 260 seconds]
<qschulz>
(just reading the 4.2 migration notes :)
nemik_ has joined #yocto
nemik_ has quit [Ping timeout: 268 seconds]
nemik_ has joined #yocto
seninha has joined #yocto
seninha has quit [Remote host closed the connection]
<michaelo>
ndec: ah, great. Right in time for Xmas then ;) Thanks! I was asking because last time I almost missed the e-mail.
seninha has joined #yocto
<RP>
qschulz: I opted not to make them depend on the code although variable dependencies would change iot
<qschulz>
RP: ack, thx for the info
prabhakarlad has joined #yocto
<mcfrisk>
I guess there is no defined way to generate build specific secrets during a bitbake/yocto build? things like the kernel module signing keys, ssh host keys. ssh keys are delayed to first boot. but kernel mod signing really should use build time generated random key. static keys increase the risk of leaks and key re-use
<mcfrisk>
for kernel module signing, the do_compile task would need to invalidate/increment/change the task hash to trigger all dependencies to rebuild
<RP>
mcfrisk: people have static key recipes but there isn't any common infra for it
<RP>
mcfrisk: having a recipe/task generate the kernel module signing keys then control where they're used would seem sensible, then it's hash can control when it is reused or isn't
<mcfrisk>
that's different from nostamp task which should always be executed, and also makes the build of task not reproducible. but that's per design, on purpose. build machine has more entroby than target
<RP>
right, nostamp always runs
<mcfrisk>
is there some magic to change task hash within task execution?
ptsneves has joined #yocto
<RP>
mcfrisk: we have to compute hashes in advance :(
<RP>
mcfrisk: else you can't query sstate for example
starblue1 has quit [Ping timeout: 252 seconds]
<RP>
mcfrisk: the only way it can change course is locked sigs or the "taint" that bitbake -f adds
florian_kc has joined #yocto
starblue1 has joined #yocto
<mcfrisk>
RP: taint would do the job, I'll try
<RP>
mcfrisk: you can't do that when tasks are executing though
<mcfrisk>
can another task call invalidate_task? or is it only available at parsing time
<RP>
mcfrisk: it is only available before the taskgraph is constructed
<RP>
mcfrisk: I guess these days we do "rehash" after hashequiv was added. The code won't know how to handle a rehash for a changed taint on a task though
<RP>
mcfrisk: I'm just trying to help by warning you of this up front since I know the code!
Algotech75 has quit [Remote host closed the connection]
<mcfrisk>
RP: any help or guidance is highly appreciated!
Algotech75 has joined #yocto
<mcfrisk>
but I think there could be uses for a stage to generate secrets, which are not reproducible by design, and then trigger full dependency graph executions if something is recompiled
<RP>
mcfrisk: perhaps, but it does go against a lot of existing design so it isn't going to be straight forward
<RP>
mcfrisk: I'm just trying to make it clear where the design is going to make this hard and where there are possibilities that might allow it to work
<mcfrisk>
yea, using static keys would be easy way out...
<mcfrisk>
or 'static' really means generated outside of bitbake build with some manual or automatic steps
<ptsneves>
for secrets couldnt you just vardepexclude?
<RP>
mcfrisk: right, if you generate the keys into a recipe as files, bitbake knows what to do with those
seninha has quit [Remote host closed the connection]
<RP>
mcfrisk: we could add in some event handler hooks for things like this, the trouble is they'd be a bit open to abuse :/
<mcfrisk>
yea, I see the abuse point of view
<RP>
mcfrisk: I'm not against the idea, we'd just have to be careful about the implementation and docs
<mcfrisk>
"how to generate secrets at build time" would be nice to have in docs with some examples..
<mcfrisk>
for "leaf" recipes and tasks, they can generate stuff. Build is not reproducible but for the use case it doesn't matter. for any inter recipe and task dependencies this causes problems.
seninha has joined #yocto
<RP>
mcfrisk: right :/
<RP>
mcfrisk: even documenting the challenges now would be helpful even without solutions
<RP>
abelloni: I've worked out the reasons the hash tests were failing, trying new builds
<abelloni>
RP: ack, I'll let you work that out
<LetoThe2nd>
RP: no breakage before christmas, please!
mckoan is now known as mckoan|away
prabhakarlad has quit [Quit: Client closed]
prabhakarlad has joined #yocto
<qschulz>
LetoThe2nd: which Christmas :) ?
<abelloni>
LetoThe2nd: I feel like this is too late :)
seninha has quit [Quit: Leaving]
<hsv>
Are there any active irc channels more appropriate to discussion of embedded linux, stuff like device tree, device drivers, on ARM ?
<LetoThe2nd>
hsv: #armlinux?
<hsv>
LetoThe2nd: thank you :)
<mcfrisk>
recipe and task tainting don't actually trigger any recompilations, at least when the task hashes remain the same. I think the consumers of other recipes output should take into account both the task hash and the binaries from build output. Now only first part if used, the metadata. This is fine as long as SW component builds are reproducible, but not when they are not. I recompiled kernel after wiping
<mcfrisk>
sstate cache and initramfs and rootfs are not re-generated with the newly rebuilt kernel.
<RP>
LetoThe2nd: I have some patches which definitely cause some interesting issues!
<RP>
mcfrisk: we *cannot* take output into account. If we did, how would you know if a given sstate object were valid or not, or even know how to query for it?
<RP>
by definition, the taint will change the hash
rfuentess has quit [Remote host closed the connection]
<mcfrisk>
RP: taining linux yocto doesn't trigger re-generation of initramfs or rootfs, not on kirkstone in my config at least
<mcfrisk>
RP: for indexing, yes, metadata and task has is fine, but once you have the data file too, and decision is dont to rebuild or not, if the data files don't match, a recompilation is needed. now the assumption is that sstate hashes produce equal builds. that's maybe ok for poky but not valid elsewhere. a lot of things don't compile reproducibly.
<RP>
mcfrisk: how are you tainting it?
<mcfrisk>
bitbake -f linux-yocto
<RP>
mcfrisk: that won't work as it taints do_build (the default task)
<mcfrisk>
ok, from task dependencies I see that package task is dependency of do_rootfs for all packages in rootfs
<RP>
right
<RP>
so if you taint install or compile or package, it should have the effect you want
<mcfrisk>
but I still think relying only on task hashes is dangerous. sstate objects for a recipe are produced by the metadata for the task, metadata for dependent tasks and the content of the sstate objects (tar ball). if sstate is corrupt or wiped on purpose, the recipe with matching task hashes will be recompiled, but the dependency tree not because. then at some point in the future when the rootfs do get
<mcfrisk>
regenerated all the recompiled things magially appear there.
* mcfrisk
blames Friday for all typos..
<mcfrisk>
kernel module signing with build time generated keys only shows this problem because user sees freshly compiled kernel used with kernel modules on stale initramfs. without signing I would be harder to see that a new kernel is being used with older modules. with reproducible and compatible builds this works but outside of poky this assumption may not hold
<RP>
mcfrisk: By definition, the task only depends on the metadata for the task and the dependent task contents. This is how we defined and built the system
<RP>
We do assume builds are reproducible. Yes, you can design a task which isn't but we don't support that
<RP>
sstate is built on these assumptions. If we were to change them, we'd have to redesign everything
<RP>
mcfrisk: the alternative is always build everything. You can do that easily by not using/sharing sstate
<mcfrisk>
task hash is one input, second really should be the task output. if output changes, dependency tree needs to be recompiled. hash input can be the key to find data to compare.
<RP>
mcfrisk: I've tried to explain why that isn't practical and can't really work :/
<RP>
You could I guess make it hard error if the output differed but you cannot recompute the dependency tree
<RP>
some tasks don't binary reproduce either, in particular, native task output
<RP>
for reproducibility we only look at the target output at present, native is something we should look at but nobody has stepped up
seninha has joined #yocto
invalidopcode7 has quit [Read error: Connection reset by peer]
invalidopcode7 has joined #yocto
<mcfrisk>
after compiling a sstate cache hash, and the actual output, you can calculate the blob output hash too. and if the blob output now differs compared what user recipes previously saw with the task hash, the task has can't be trusted and recompilation is needed. if blob hashes do match, then binaries are reproducible.
goliath has joined #yocto
<RP>
mcfrisk: so to ensure a build is correct, we need to download each object, rebuild it and check the output checksum? Otherwise how do we know which objects we can use from sstate and which we have to rebuild?
<mcfrisk>
check output checksum only after recompilation. recompilaton can be forced or sstate entry can be missing.
<RP>
mcfrisk: so where does this output checksum to compare against come from if the sstate entry is missing? I guess you might have one if it is being forced, but if it is being forced, you may as well taint with -f
<RP>
then, how do you share this "taint" back into the wider ecosystem for another build to use?
<mcfrisk>
record the sstate hash and blob hashes which were used in every compilation. these are the inputs to the build. this is how I saw builds in sumo times, AFAIK..
<RP>
mcfrisk: record where? We have never worked on output hashes until we added hashequiv. Sumo did not do that
<mcfrisk>
sumo did recompile a lot more with sstate, the effect was similar. full dependency tree recompiled every time.
<mcfrisk>
I really thought hashequiv did take the recipe output into account too
<RP>
mcfrisk: the behaviour is unchanged from sumo until now apart from hash equivalence. hash equvalence does take the output into account
<mcfrisk>
and rebuilds dependency tree also if output changed? maybe I should use that then
<RP>
mcfrisk: it will, however it is still built on top of the assumptions in the system that tasks are idempotent
<RP>
mcfrisk: you can't just change that assumption as it doesn't work for your use case :(
<mcfrisk>
I know that assumtion to be false in so many :( you've made poky reproducible, that is great! many other layers don't match that, proprietary things are even worse
leon-anavi has quit [Quit: Leaving]
<RP>
mcfrisk: well, what can we do about that? They're not using it as designed (which is probably why they complain sstate doesn't do what they want too)
<RP>
mcfrisk: changing the design isn't an option, it is inbuilt into too many places (even without sstate, that just builds on earlier pieces). We probably can detect it using hash equiv in limited cases but to work it out you really have to compare two builds and we do already have tests for that people can run
<mcfrisk>
I'll think about this. at least I know now that reproducible recipe builds are required for sstate cache to work reliably. and if there are issues with build output, wiping full sstate cache is a must. wiping a single recipe sstate doesn't rebuild the dependencies and fix all data in sstate cache.
manuel1985 has quit [Ping timeout: 246 seconds]
sakoman has joined #yocto
kscherer has joined #yocto
xmn has joined #yocto
malsyned has joined #yocto
kroon has quit [Quit: Leaving]
<mcfrisk>
RP: how about emulating build specific keys in do_fetch()? generating them with custom openssl call in do_fetch() and storing results in download dir. this would be outside of sstate hashes and would regenerate keys when ever do_fetch() gets executed. or should the cached files be bound to PV and PR regenerate only when those change
<RP>
mcfrisk: do_fetch is supposed to get deterministic inputs but you probably could make some custom approach work. Not sure it would be much different to a custom recipe extracting keys from DL_DIR though
<RP>
mcfrisk: the hard part would be having the "hash" for the fetch task ready at parse time
<mcfrisk>
yea, that could be some nice set of variables to refer in the custom task or do_fetch(), but hacking this into do_fetch() could be ok but then please don't publich download cache with private keys... :)
<mcfrisk>
oh problem wasn't generation but invalidating task hashes when parsing or executing recipe tasks
invalidopcode7 has quit [Read error: Connection reset by peer]
invalidopcode7 has joined #yocto
<JPEW>
RP: I pushed up a few more "interesting" filters that might be really useful
<JPEW>
One converts a variable to a python object by treating it as JSON and the other does the same but as a python eval() string
<JPEW>
(this would make it very easy for people to encode structured data in a variable and access it from a python task)
seninha has quit [Ping timeout: 268 seconds]
Algotech75 has quit [Remote host closed the connection]
<JPEW>
And, I was thinking the bitbake parsing code that calls filter can inject "d" into the extra_globals, so you could do e.g.:
<rburton>
khem: perf blows up still but i think it helped.
<rburton>
khem: i've lost the perf fix you had
<rburton>
khem: perf fails with clang-14: error: no such file or directory: 'arm-poky-linux-gnueabi'
<rburton>
khem: but the strip/objcopy overrides work, yes
seninha has joined #yocto
<rburton>
RP: so zstd has native threading now, so we can drop pzstd (assuming that doesn't mean using a really new version). Will investigate.
mvlad has quit [Remote host closed the connection]
demirok has joined #yocto
malsyned has quit [Ping timeout: 252 seconds]
malsyned has joined #yocto
<malsyned>
I have a custom linux bb recipe based on NXP's linux-imx recipe which fetches the source using git, and every time I change the SRCREV bitbake fetches the entire repo from scratch rather than just pulling the new commits into an existing repo. Anybody know why or if there's a way to fix it? It's
<malsyned>
uldn't be necessary.
<malsyned>
like a half-hour every time, and it seems like it sho
florian_kc has joined #yocto
swedishhat[m] has joined #yocto
manuel1985 has quit [Ping timeout: 265 seconds]
kscherer has quit [Quit: Konversation terminated!]
azcraft has quit [Remote host closed the connection]
<RP>
rburton: in both the compressor and decompressor? I think one was the issue last time