<landgraf>
Hi there. What the best way to redeploy artifacts in do_deploy() if they're changed? I have a recipe which produces boot.scr in do_compile and deploys it. However sometimes wrong (old) version is deployed. For example I've changed the source and proper artifact has been produced but the one which was redeployed (after -c clean)was outadated. cleanall solved the issue but costed me 8 hours of not so
<RP>
__ad: someone could create a mixin layer for that...
prabhakarlad has joined #yocto
seninha has joined #yocto
starblue has quit [Ping timeout: 265 seconds]
starblue has joined #yocto
<RP>
JPEW: I have a patch which helps the spdx failures FWIW
<landgraf>
I'm looking for a way of sharing patches between my layer and dynamic-layer (for linux-yocto and linux-raspberrypi recipes to be precise). Is it possible or it's better to just copy them over ?
<RP>
landgraf: I'd have thought it might prove tricky to keep in sync since you don't control either of them :/
<landgraf>
FILESEXTRAPATHS:append = "LAYERDIR/..." I guess?
<landgraf>
RP: I have control over patches and apply them in bbappends..
<landgraf>
so far so good except the fact I don't want to have two copies of them
<landgraf>
and I cannot put linux-raspberrypi.bbappend into main layer because it breaks other machines which don't need rpi layer
hrberg has joined #yocto
<RP>
landgraf: you should be able to put a common path in both recipes
<landgraf>
RP: yes. Adding FILESEXTRAPATHS:prepend := "${LAYERDIR}/recipes-kernel/linux/files/xen_shared_memory/: into dynamic-layer recipe did the trick. Thank you.
* landgraf
spent few days of debugging of the issues caused by cheap ttl-usb adapter and burnt out :-/
hrberg has quit [Read error: Connection reset by peer]
<RP>
landgraf: doesn't sound good :/
hrberg has joined #yocto
yannd has joined #yocto
nemik has quit [Ping timeout: 240 seconds]
* mcfrisk
also had test HW failing in the past weeks: at least one board and a bunch of SD cards triggered really odd userspace and tmpfs data corruption, turning all CI test runs red...
nemik has joined #yocto
alessioigor has joined #yocto
<landgraf>
mcfrisk: in my case it was combined with few software issues and strict deadline, I started to think it's some sort of mystery (and blamed devicetree as usual). :-/
nemik has quit [Ping timeout: 250 seconds]
nemik has joined #yocto
<mcfrisk>
yea, we also had SW issues to deal with at the same time, regressions in firmware and higher up kernel and userspace, flaky tests. Luckily no hard deadlines though. CI makes it impossible to hide these details, which is both good and bad
rob_w has quit [Remote host closed the connection]
<mcfrisk>
some people want to avoid target HW and BSP SW stack issues completely and push for virtual validation. I think it's important to validate the real product, to not hide HW and BSP SW stack issues. I think both are needed when target is a realy physical product
<RP>
JPEW: I can fix the sdk pkgdata not found and hack around the world build issue, it looks like there is also some kind of race:https://autobuilder.yoctoproject.org/typhoon/#/builders/45/builds/7208/steps/12/logs/stdio https://autobuilder.yoctoproject.org/typhoon/#/builders/122/builds/2803
otavio_ has quit [Ping timeout: 240 seconds]
alessioigor has quit [Quit: alessioigor]
alessioigor has joined #yocto
<tct>
Hey guys, I'm using an existing recipe for SDL2. However, I need to apply a patch to the SDL2 source. How would one go about that properly?
yannd has quit [Ping timeout: 240 seconds]
otavio has joined #yocto
yannd has joined #yocto
prabhakarlad has quit [Quit: Client closed]
florian_kc has quit [Ping timeout: 256 seconds]
florian has quit [Ping timeout: 250 seconds]
<__ad>
what recipe includes useradd command ?
<__ad>
ok, looks shadow-utils
jtoomey has quit [Ping timeout: 240 seconds]
<__ad>
right recipe seems shadow, but it's only native
florian has joined #yocto
florian_kc has joined #yocto
<RP>
tct: add a patch to the recipe's SRC_URI ?
<tct>
RP, but then I'd have to modify the meta-oe recipe/repo :(
<RP>
tct: you can do it from a bbappend
<tct>
RP, oh... reasonable!
<tct>
thanks :)
<sudip>
__ad: you can use oe-pkgdata-util to find out, in my image it shows "shadow: /usr/sbin/useradd"
<RP>
JPEW: it is some kind of missing dependency, "bitbake python3-bcrypt -g" shows no dependency on python3-pycparser-native from bcrypt, only from cffi-native
<RP>
JPEW: master-next
<JPEW>
sdk pkgdata fixup?
<RP>
JPEW: yes, one second as I have another
<JPEW>
Why is that necessary? Does the SDK not have pkgdata?
<RP>
JPEW: there are two locations, target and SDK
<RP>
JPEW: master-next now has "meta-world-pkgdata-Fix-for-create-spdx" (will be syncing to the mirrors)
<RP>
JPEW: it is a bit hacky but was the best solution I could find right now. I was curious what builds looked like with that "fixed"
<RP>
the meta-world-pkgdata fix makes "bitbake world" work again when multilibs are enabled
<RP>
so then we're left with this weird sstate dependency issue
<JPEW>
OK. I'm running the bcrypt test now.... rust is building so... it will be a bit :)
<RP>
oh, and oe-selftest sstate sigs test failures
<RP>
JPEW: you could probably find a simpler example, that was just the one the autobuilder showed me :/
sakoman has joined #yocto
Minvera has quit [Remote host closed the connection]
<RP>
JPEW: changing do_create_spdx[deptask] = "do_create_spdx" to do_create_spdx[recrdeptask] = "do_create_spdx" "fixes" it
Xagen has joined #yocto
<RP>
whether that is the right thing or not, less sure
Minvera has joined #yocto
<JPEW>
RP: Why would it be in BB_TASKDEPDATA, but not run?
xmn has joined #yocto
<JPEW>
Hmm, `python3-pycparser-native` is a transitive dependency from `python3-cffi-native`; but I thought transtive dependencies didn't show up in BB_TASKDEPDATA?
* JPEW
is confused
Xagen has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
<RP>
JPEW: if an sstate task runs, it is assumed that "covers" everything needed
<RP>
i.e. sstate task dependencies won't run if the sstate task runs
<JPEW>
hmmm
<JPEW>
So is the assumption that python3-cffi-native:do_create_spdx "covers" python3-pycparser-native:do_create_spdx ?
Xagen has joined #yocto
<RP>
JPEW: yes. If you really want indirect dependencies, that is where recrdeptask comes into play
amitk_ has quit [Ping timeout: 250 seconds]
<RP>
JPEW: saying X uses Y-native to build. You don't want Y-native everytime you use X if it is just a build tool
<JPEW>
Is there a way to know python3-pycparser-native is transitive? I can just ignore it if I can know that because the SPDX relationship chain will still take it back there via python3-cffi-native
<JPEW>
(I didn't realize those transitive dependencies were in BB_TASKDEPDATA in the first place)
<RP>
JPEW: BB_TASKDEPDATA shows the complete dependency chain. It is up to the user of it to decide which dependencies make sense to them. You probably want to know if the task in question is an sstate one or not?
<JPEW>
RP: Hmm.... sort of I guess. I wouldn't want to filter on if sstate was ran because then the SPDX would be different in different circumstances
<RP>
JPEW: right, you can't do that
<RP>
JPEW: can you assume the other create_spdx task would have the links needed in that document?
<JPEW>
Philosphically speaking, I only _really_ need the direct dependencies because the relationship chains will eventually lead back to all of the transitive dependencies
<RP>
JPEW: that is what I was wondering
<JPEW>
But if python3-pycparser-native didn't run, there will be a broken document chain, even if we only report the direct dependencies
alessioigor has quit [Quit: alessioigor]
<JPEW>
So... may we need both a change to only report the direct dependencies and recrdeptask?
<RP>
JPEW: right, it would point at a document which isn't there. Which is why recrdeptask might be the correct thing to do?
<RP>
JPEW: taskdepdata is calculated twice as there is the "setscene" context and the real task context iirc
<RP>
one is quite different as it is setscene task dependencies rather than task dependencies
<JPEW>
Ah, but only relevent when restoring from sstate? So do_create_spdx never "sees" the setsecene one because if it's looking at it, it means the task is actually running
<JPEW>
?
<RP>
JPEW: do_create_spdx_setscene would see the setscene one but that is just straight into the sstate code
<JPEW>
Ya, that one should also be changed to recrdeptask I tink
<JPEW>
Does that mean all the runtime dependencies will be in BB_TASKDEPDATA for do_create_spdx also?
<JPEW>
(in addition to build time)
<RP>
JPEW: yes :/
<JPEW>
Well.... I guess no one will be able to claim our SPDX isn't through at least
<JPEW>
I'm sure there is a reason that recdeptask doesn't exist?
Xagen has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
Xagen has joined #yocto
yannd has joined #yocto
PobodysNerfect has quit [Ping timeout: 240 seconds]
<tct>
is there a way to clean stuff from the build directory but ONLY stuff that is not used in the currently built image? I removed a distrofeature (x11) and would like to free up some disk space but don't feel like building everything from scratch again
<JPEW>
tct: If you keep sstate, things that don't need to rebuild will restore from that instead, which should be reasonbly fast
<tct>
JPEW, could you be a bit more specific? what does "keep sstate" mean?
<JPEW>
tct: The sstate directory where bitbake caches build output; I think it defaults to ${BUILDDIR}/sstate ?
<tct>
JPEW, so you'd recommend me to just delete everything in the build dir except for conf/ and sstate and then bitbake again?
<JPEW>
tct: I think if you only delete $BUILDDIR/tmp it will do what you want
<JPEW>
leave everything else
<RP>
JPEW: recdeptask doesn't make any sense when you look at the code
<tct>
JPEW, thanks!
<RP>
it did exist once, kind of
<tct>
JPEW, actually, I am not sure why mine is called /build/tmp-glibc ever since I moved to a custom distro
<RP>
JPEW: meta-aws was green this time
<JPEW>
RP: I can't quite see why it doesn't make sense.... similar to recrdeptask handling, but doesn't call add_runtime_dependencies()?
PobodysNerfect has joined #yocto
<RP>
JPEW: at every other level beyond the top level, the runtime pieces end up pulled in regardless
yannd has quit [Ping timeout: 240 seconds]
<RP>
JPEW: I'm fuzzy on the exact details now but it ended up not making much sense :/
<JPEW>
Ya, the processing after that definitely is confusing so I can believe it does some stuff like that
goliath has joined #yocto
florian has quit [Ping timeout: 240 seconds]
<JPEW>
RP: Well.... I was hoping I could use the deps for the current task in BB_TASKDEPDATA to only transverse one level of task dependency data, but neither `do_create_spdx[deptask]` nor `do_create_spdx[recrdeptask]` actually cause the task depenencies in BB_TASKDEPDATA to change
<RP>
JPEW: they should!
<JPEW>
I though so too
<JPEW>
Let me see if it's user error
<RP>
JPEW: I just checked and you're right. It does add python3-pycparser-native to python3-bcrypt.do_create_runtime_spdx but not create_spdx
<JPEW>
Weird
<JPEW>
do_create_spdx[depends] is the culprit maybe?
<JPEW>
Nope
yannd has joined #yocto
zpfvo has quit [Remote host closed the connection]
frieder has quit [Remote host closed the connection]
<RP>
JPEW: the other dependency is "hiding" the race
<RP>
JPEW: I see what you mean about recursivetaskselfref :/
<RP>
JPEW: the fundamental issue is task using something can't also recursively depend upon it
<MartinX>
I'm pretty new to yocto and am building an image for some IoT devices. We got a yocto project from our SoC supplier. It's yocto 2.4.3 "rocko" and everything is really out of date. I've had some luck updating individual recipes, but it seems like there should be a better way to update the whole project?
<RP>
JPEW: disabling that cross linkage code results in ERROR: Task /media/build/poky/meta/recipes-devtools/python/python3_3.11.2.bb:do_create_spdx has circular dependency on /media/build/poky/meta/recipes-support/bash-completion/bash-completion_2.11.bb:do_create_spdx
<RP>
MartinX: using a new release would get you all the newer versions. Your supplier should really give you something more recent. rocko is old
<MartinX>
@RP Yeah.. We've talked about it with them, but I guess this SoC is mainly used for Android devices, so they don't want to spend engineering update to upgrade it for us. Any chance I could do it myself?
starblue has quit [Ping timeout: 240 seconds]
<RP>
MartinX: it certainly can be done. Depends what you need from the SoC specific pieces
<RP>
Personally I'd start from master or our last LTS kirkstone and add in the pieces you need, see if you can get the SoC to work
<RP>
JPEW: I'm going to change my mind. I think the deptask was right before and we need to limit how far the tar recurses
<RP>
JPEW: It is fine for someone to run all the tasks if they want all the spdx documents pulled from sstate
<RP>
s/tar/task/
<JPEW>
Limiting the recursion would be fairly easy if the recursive tasks were in BB_TASKDEPDATA
<MartinX>
RP Where would you start trying to add pieces in? It seems like they've got a lot of SoC specific pieces. To use any of the peripherals, there's some API or custom code to get it all working, so it's not always clear how it works under the covers.
<MartinX>
Alternatively, the minimum we need is python 3.10. Maybe I should just migrate a recipe from that to rocko?
<JPEW>
err, if the _circular_ tasks were in BB_TASKDEPDATA
<RP>
JPEW: that would have everything backwards though. You'd never get to the point where you could generate it!
<RP>
JPEW: I think create_spdx needs something like what extend_recipe_sysroot() has, where it has "# Create collapsed do_populate_sysroot -> do_populate_sysroot tree"
<RP>
but in this case, using the spdx tasks as markers
* JPEW
reads the code
<RP>
JPEW: so prune the tree to the next set of spdx recipes
<JPEW>
Right, I wrote that; AFAICT, it works for extend_recipe_sysroot because it's not a circular task dependency (`do_prepare_recipe_sysroot[deptask] = "do_populate_sysroot"`, not `do_prepare_recipe_sysroot[deptask] = "do_prepare_recipe_sysroot"`)
<RP>
JPEW: if we do this, we can drop the recrdeptask for do_create_spdx though ?
<JPEW>
Maybe? It might just fail when collecting all the document for image creation instead
yannd has quit [Remote host closed the connection]
seninha has quit [Quit: Leaving]
seninha has joined #yocto
sakoman1 has joined #yocto
sakoman1 has left #yocto [#yocto]
* paulg
fines sakoman $2 for not writing a one sentence json long log
dacav has quit [Quit: leaving]
leon-anavi has quit [Quit: Leaving]
starblue has joined #yocto
<fray>
I can push a master-next or something like that for testing
<fray>
oops
<RP>
fray: New TSC member in OE-Core takeover? :)
* RP
does know the real context but this is more fun
alessioigor has quit [Quit: alessioigor]
<fray>
lol much more fun to take things out of context. lol
alessioigor has joined #yocto
<fray>
I don't even have write/push access to the repository in question afterall..
<JPEW>
RP: I updated my contrib branch
<JPEW>
should be good for testing now
zwelch has quit [Remote host closed the connection]
zwelch has joined #yocto
<RP>
JPEW: thanks, I triggered a build with CVE and kernel stuff but I'll work something out after that
<JPEW>
Cool
<RP>
JPEW: I sent a patch for the world build issue, the main worry left is the selftest failures
<RP>
e.g. oe-selftest -r sstatetests.SStateHashSameSigs2.test_sstate_allarch_samesigs -j 1
<JPEW>
I'll try that locally
<RP>
JPEW: there are several failing but that is a place to start...
amitk_ has quit [Ping timeout: 240 seconds]
Haxxa has quit [Quit: Haxxa flies away.]
Haxxa has joined #yocto
nemik has quit [Ping timeout: 250 seconds]
nemik has joined #yocto
<JPEW>
do_create_spdx depends on STAGING_KERNEL_DIR, since it can peek into the kernel sources
<JPEW>
(which in turn depends on MACHINE)
<JPEW>
I think this variable can just be ignored with vardepsexclude
<JPEW>
Can I convince oe-selftest to leave the directories around for a closer look?
olani- has quit [Ping timeout: 246 seconds]
<JPEW>
Ah, nm. I found it
sakoman has quit [Quit: Leaving.]
sakoman has joined #yocto
<RP>
JPEW: we should really make that configurable somehow
<JPEW>
I just commented out the lines that delete them
<JPEW>
But ya
starblue has quit [Ping timeout: 256 seconds]
<RP>
JPEW: that is what I do too :/
<JPEW>
For the allarch hashes.... I don't know what to do there. The hash changed because a dependency changed, but isn't it supposed to do that?
amitk has quit [Ping timeout: 265 seconds]
amitk has joined #yocto
<JPEW>
do_prepare_recipe_sysroot presumably solves this I guess... I'll look there
amitk has quit [Ping timeout: 248 seconds]
alessioigor has quit [Quit: alessioigor]
alessioigor has joined #yocto
Xagen has quit [Ping timeout: 240 seconds]
Thorn has quit [Ping timeout: 250 seconds]
Minvera has quit [Ping timeout: 250 seconds]
<tct>
I've probably read the SDK vs eSDK documentation like three times now and I am still not sure which one to pick :D
bps has joined #yocto
bps has quit [Changing host]
bps has joined #yocto
paulg has quit [Ping timeout: 268 seconds]
<JPEW>
tct: SDK is a traditional SDK; you download, you install, you have a toolchain you can compile with
<JPEW>
eSDK is like taking a snapshot of your Yocto setup and users can download that and compile things against it
<JPEW>
I don't use the eSDK, so that's probably not a great explination
<tct>
now that sounds like an explanation worth putting into the documentation. Obviously it's not telling the full story but this helps a lot.
<tct>
JPEW, thanks a lot :)
tct is now known as jbo
<jbo>
JPEW, so assuming my fellow delevopers are all building the yocto images themselves anyway there's nothing wrong with going the traditional SDK way?
<JPEW>
We use the SDK this way:
bps has quit [Ping timeout: 240 seconds]
<JPEW>
Yocto devs publish SDKs for our products. App developers download the SDK to compile applications (never touching Yocto) and we have mechanisms to sideload the applications they build for testing. When they are happy with their app, they update the recipe for that application to point to the new SHA-1
<JPEW>
So they don't do app development "in-yocto"
<jbo>
JPEW, based on your previous explanation of SDK vs. eSDK that sounds more like eSDK rather than SDK, no?
<JPEW>
No... it's traditional SDK
<jbo>
so what you publish is basically the setup script that `bitbake <image> -c populate_sdk` creates?
<JPEW>
traditional SDK lets you compile applications without having to do any yocto things, you just have a toolchain
<JPEW>
jbo: Correct
<jbo>
cool
alessioigor has quit [Remote host closed the connection]
<jbo>
that's what I want/need :)
<jbo>
I'm actually interested in the sideloading you mentioned. Currently I just rsync the resulting binary/package to the target. However, I'm not particularly happy with that.
<JPEW>
Ya, it's complicated
<jbo>
is there a way to include what was built with the SDK into the image without rebuilding the entire image?
<jbo>
one of the devices I'm working with doesn't have any networking or USB :D
<jbo>
SDcard is literally my only interface.
<JPEW>
jbo: Ah ya, we do that with a sdk-packagegroup that feeds into both our "developer" image and the SDK, but not our "release" image
<JPEW>
By doing that, anything that was built against the SDK should be able to run on the target if it's running the "developer" image (e.g. shared libraries are all there)
<JPEW>
But in the "release" image, it's the minimal set of what you actually need
<JPEW>
jbo: Ah, I'm not sure about that
<JPEW>
Serial port?
<jbo>
that I have
<jbo>
but just the regular serial port to have a shell
<JPEW>
There are programs that can transfer files across the serial terminal
<JPEW>
old tech, but useful :)
<jbo>
I'm just using picocom on that serial port so far
<jbo>
might be worth looking into - could safe me a lot of trouble
<jbo>
how does that work / what should I be looking for?
<JPEW>
zmodem I think?
<JPEW>
I think theres another one too
<jbo>
I guess that will be pretty manual in use as in that I have to log out of the shell, use that particular tech to transfer files and then bring the shell back up?
<JPEW>
It's possible some terminal programs have support integrated
<JPEW>
I've not use it much, we usually have ethernet on our devices
<jbo>
same here.
<jbo>
I'll look into that - thanks for the tipp!
florian__ has quit [Ping timeout: 240 seconds]
Thorn has joined #yocto
starblue has joined #yocto
<RP>
JPEW: allarch recipes shouldn't have dependencies that change?
<fray>
allarch can change from distro to distro, but not within a single distro (unless something critical like init_manager) changes.. or well it shouldn't change
<RP>
fray: the context here is that a given allarch recipe wouldn't change in a given TMPDIR for different MACHINEs or SDKMACHINEs
<fray>
I have seen cases where someone will set a distro feature or INIT_MANAGER in a machine.. it's wrong, but it can do that
<JPEW>
RP: I traced it back to binutils-cross:do_unpacl
<fray>
but I agree, it shouldn't
<JPEW>
*do_unpack
<JPEW>
do_create_spdx directly depends on do_unpack so it can get the source code, but I'm not sure why do_unpack for binutils-cross causes problems here but not normally