<mborzecki>
does devtool work with multiconfig? the commands seem to interpret recipe name literally
wkawka has joined #yocto
<LetoThe2nd>
mborzecki: good question, i wouldn't bet on it.
zpfvo has joined #yocto
<mborzecki>
LetoThe2nd: yeah, i was originally using dunfell which is quite old but already has multiconfigs, but ti's the same on kirkstone, i'm guessing master will be identical in that regard
goliath has joined #yocto
<LetoThe2nd>
mborzecki: yes, i haven't noticed any substantial patches to devtool lately.
<qschulz>
but yeah, we should autogenerate doc from the docstring of libs
<mborzecki>
ok, maybe something I can look into
<LetoThe2nd>
mborzecki: would be much appreciated!
murych has joined #yocto
zpfvo has quit [Ping timeout: 240 seconds]
<Guest53>
is devtool part of Yocto effort, not OE?
<rburton>
There’s very little meaningful difference. It’s in the oe-core repository.
td31 has joined #yocto
<td31>
Hi,
<td31>
why is "${includedir}/python${PYTHON_MAJMIN}/pyconfig*.h" included in the python3-core package? Do we need the header for anything on target?
zpfvo has joined #yocto
goliath has quit [Ping timeout: 268 seconds]
goliath has joined #yocto
rsalveti has quit [Quit: Connection closed for inactivity]
jclsn has joined #yocto
vladest has joined #yocto
zpfvo has quit [Ping timeout: 264 seconds]
zpfvo has joined #yocto
<kayterina[m]>
hello, is archiver.bbclass used by default somewhere in poky or it has to be included from a recipe? I grep'ed and I did not find any "include archiver" ot in the local.conf of the project
<LetoThe2nd>
kayterina[m]: you need to include it.
<kayterina[m]>
A, LetoThe2nd I have homemade tsipouro from Epirus. Too bad you cant taste it from here.
xenador77 has quit [Remote host closed the connection]
xenador77 has joined #yocto
xenador77 has quit [Remote host closed the connection]
xenador77 has joined #yocto
<LetoThe2nd>
kayterina[m]: how comes you think of Epirus right now?
<LetoThe2nd>
;-)
<kayterina[m]>
A friend is from there and her grandmother made tsipouro, she gave my a bottle and I left it in my bag, I have it here at the office.
<LetoThe2nd>
kayterina[m]: hehe. I've actually been to Epirus a few times.
<kayterina[m]>
which village?
<LetoThe2nd>
sivota
prabhakarlad has quit [Quit: Client closed]
<kayterina[m]>
very popular. It has beach, it does not count!
<LetoThe2nd>
does it at least count a little if pointing out that we were at a small family hotel a few km outside? ;-)
<kayterina[m]>
yes, I will give you that. If you visit again go to metsovo, they have exceptional cheese.(ok for greek standards, we are not Gouda)
<LetoThe2nd>
:-)
<LetoThe2nd>
good keyword though, food. heading out for lunch!
nemik has quit [Ping timeout: 240 seconds]
nemik has joined #yocto
prabhakarlad has joined #yocto
jclsn has quit [Quit: WeeChat 3.5]
jclsn has joined #yocto
xenador77 has quit [Remote host closed the connection]
wkawka has quit [Quit: Client closed]
zpfvo has quit [Ping timeout: 268 seconds]
zpfvo has joined #yocto
Guest53 has quit [Quit: Client closed]
ptsneves has quit [Ping timeout: 240 seconds]
seninha has joined #yocto
leon-anavi has joined #yocto
prabhakarlad has quit [Quit: Client closed]
td31 has quit [Quit: Client closed]
Guest14 has joined #yocto
Guest14 is now known as brazuca
brazuca has quit [Quit: Client closed]
Guest14 has joined #yocto
Guest14 is now known as brazuca
ptsneves has joined #yocto
nemik has quit [Ping timeout: 252 seconds]
nemik has joined #yocto
nemik has quit [Ping timeout: 268 seconds]
nemik has joined #yocto
prabhakarlad has joined #yocto
<JPEW>
RP: Is there a better way to do the own-mirrors thing that I'm missing?
<JPEW>
(or, I can move it to our own internal class if you don't want the overhead in OE-core)
<RP>
JPEW: I couldn't think of a better way. I just worry every time we add more anon python and wish we had a better way :/
<RP>
particularly anon python that has to run for every recipe
<JPEW>
RP: Ya, I was a little worried about that. Are there ideas on ways that could be done to do it better?
<RP>
JPEW: I wondered if we should add a config level anon python
<JPEW>
Like in .conf files and part of the basehash?
<JPEW>
Is that the right term?
<RP>
JPEW: we "pre parse" the base configuration and then parse the recipes on top of a copy
<JPEW>
Ya; is that the same as the basehash or is there a basehash per-recipe?
<Saur[m]>
Isn't there a risk that will be confusing if people expect the anon python from the config files to actually be executed per recipe?
<RP>
we could run anon code at that point, with the caveat that changes made by the recipe specific bits would have no effect
<RP>
Saur[m]: it would have to be marked differently and it would risk confusing people
<RP>
JPEW: basehash is quite different
<JPEW>
Perhaps we could "taint" the variables and warn if they are modified in the recipe
<JPEW>
Got it
hcg has quit [Quit: Client closed]
<RP>
JPEW: that is a hash at the end of parsing of the recipe before the other taskgraph is accounted for
<JPEW>
RP: Right, got it
<JPEW>
RP: I though maybe moving it to a library function that gets called, but it's so short that didn't seem like it would help much
hcg has joined #yocto
<LetoThe2nd>
whats the build-ability on RHEL these days? specifically RHEL 8... if I read correctly that should be Fedora 28. Anybody experience with dunfell and/or kirkstone?
sakoman has joined #yocto
nemik has quit [Ping timeout: 268 seconds]
nemik has joined #yocto
jclsn has quit [Ping timeout: 268 seconds]
jclsn has joined #yocto
nemik has quit [Ping timeout: 268 seconds]
nemik has joined #yocto
<RP>
JPEW: right, I don't think a library function would buy much here
<RP>
LetoThe2nd: alma8 works
<RP>
LetoThe2nd: centos8 also did work
jclsn has quit [Client Quit]
<qschulz>
kanavin: aren't go programs statically compiled? in that case, I think there could be an alternative to the go version question on the ML in actually using multiconfig for that?
<qschulz>
create the binary with a different conf file for different go versions and then include this statically linked binary into the final image, which uses a different go compiler?
nemik has quit [Ping timeout: 252 seconds]
nemik has joined #yocto
agupta1_ has joined #yocto
nemik has quit [Ping timeout: 240 seconds]
nemik has joined #yocto
vladest1 has joined #yocto
vladest has quit [Ping timeout: 240 seconds]
vladest1 is now known as vladest
zeddiii is now known as zeddii
nemik has quit [Ping timeout: 268 seconds]
nemik has joined #yocto
rsalveti has joined #yocto
tomzy_0 has joined #yocto
Estrella_ has joined #yocto
Estrella____ has joined #yocto
manuel_ has joined #yocto
Estrella___ has quit [Ping timeout: 245 seconds]
Estrella__ has quit [Ping timeout: 272 seconds]
manuel1985 has quit [Ping timeout: 240 seconds]
goliath has quit [Quit: SIGSEGV]
gsalazar has quit [Quit: Leaving]
gsalazar has joined #yocto
manuel_ has quit [Ping timeout: 268 seconds]
berton[m] has joined #yocto
<LetoThe2nd>
RP: sorry, didn't notice the reply. so building dunfell should be fine?
pgowda_ has quit [Quit: Connection closed for inactivity]
<RP>
LetoThe2nd: I'd think so
<LetoThe2nd>
RP: kirkstone?
nemik has quit [Ping timeout: 252 seconds]
nemik has joined #yocto
nemik has quit [Ping timeout: 245 seconds]
nemik has joined #yocto
jsandman has quit [Quit: Ping timeout (120 seconds)]
jsandman has joined #yocto
<RP>
LetoThe2nd: should be fine
<LetoThe2nd>
RP: kaythanks!
gsalazar has quit [Ping timeout: 240 seconds]
denisoft81 has joined #yocto
<kanavin>
qschulz, I think that's more complicated than just figuring out how to use a single go compiler
yannholo has joined #yocto
nemik has quit [Ping timeout: 268 seconds]
nemik has joined #yocto
nemik has quit [Ping timeout: 252 seconds]
nemik has joined #yocto
yannholo_ has joined #yocto
lexano has quit [Remote host closed the connection]
yannholo has quit [Ping timeout: 268 seconds]
<yannholo_>
Hello, I have a file in my image "/lib/firmware/.../something.conf" I want to change it, when searching the recipe it came from with "oe-pkgdata-util find-path /lib/firmware/.../something.conf" I get an error "Unable to find any package producing path..." Is there another way to find the source recipe ?
<yannholo_>
(I know the sources it comes from and even have my patch ready, but am not finding the recipe)
gsalazar has joined #yocto
zpfvo has quit [Ping timeout: 268 seconds]
gsalazar has quit [Ping timeout: 240 seconds]
denisoft81 has quit [Quit: Leaving]
denisoft81 has joined #yocto
agupta1_ has quit [Ping timeout: 240 seconds]
dev1990 has quit [Quit: Konversation terminated!]
denisoft81 has quit [Quit: Leaving]
gsalazar has joined #yocto
agupta1_ has joined #yocto
xmn has quit [Ping timeout: 268 seconds]
florian has quit [Ping timeout: 268 seconds]
denisoft81 has joined #yocto
<vmeson>
JaMa: Can you explain what happened when you were testing the /proc/pressure regulation either here, in a email or in a defect ?
<vmeson>
My take so far is that you had two chromium builds going before and they always worked then you added /proc/pressure regulation and the same workflow resulted in an OOM, right?
denisoft81 has quit [Quit: Leaving]
<vmeson>
If so what pressure regulation values did you pick and why did you pick them?
denisoft81 has joined #yocto
denisoft81 has quit [Client Quit]
<vmeson>
Finally, I don't see how bitbake could avoid having two chromium builds running at once since it will eventually need to start at least one task given the current scheduler design that says that at least one task must be running.
florian has joined #yocto
lexano has joined #yocto
<yannholo_>
@yannholo_ I managed to make it work, the SRC_URI of the recipe used the destsuffix and name parameter, maybe that breaks oe-pkgdata-util...
yannholo_ has quit [Remote host closed the connection]
<JaMa>
vmeson: no, OOMK wasn't caused by /proc/pressure regulation, it was already happening, I was just asking if there is a technical reason not to track memory pressure (to avoid OOMK or at least lower the probability), if it was just omitted because it didn't look interesting on your builds, then I might add it to buildstats to see what it records in my builds
<JaMa>
vmeson: in https://github.com/shr-project/test-oe-build-time I have bunch of results where bitbake tasks were running when the builder was already using a lot of swap, if that's "easy" to detect from memory pressure before it gets really bad, than additional regulation on memory pressure might help
<vmeson>
JaMa: ok, agupta1_ is going to add memory regulation as well so maybe wait a day or two since it should be simple enough to add.
<vmeson>
JaMa: I think I told agupta1_ that no one would ever push their build machine so a hard that there would be lots of swapping! There goes that assertion!
<JaMa>
vmeson: I haven't pushed the bootcharts with pressure collection enabled in buildstats yet, because my last build failed (when building rust-llvm-native from dunfell), but I can push the partial results if you want to see my cpu/io pressure
<JaMa>
hehe
<vmeson>
JaMa: sure, we'll take a look.
<JaMa>
with nodejs/chromium/qtwebengine/qtwebkit/webkit-gtk in the mix it's difficult to find balance between PARALLEL_MAKE/BB_NUMBER_THREADS so even with just 8 BB_NUMBER_THREADS you can easily get to swap when the BIG do_compile align badly and run at the same time
<JaMa>
with pressure regulation it might be easier to tell bitbake to go ham and when it starts to go badly to backoff a bit
<vmeson>
JaMa: Yep, the plan is to teach make/ninja about pressure regulation and other metrics to be able to walk the line between good use of the system and overloading the box.
<JaMa>
and the RAM might be a bit unpredictable if someone happens to use the workstation for builds as well as regular work (e.g. 30G RAM in chrome tabs)
<JaMa>
OOMK was sometimes kind enough to kill chrome before g++, but sometimes it kills systemd or networking which is terrible :)
<JaMa>
PARALLEL_MAKE:pn-qtwebengine = "-j 40" + PARALLEL_MAKE:pn-chromium = "-j 40" + PARALLEL_MAKE:pn-webruntime = "-j 40" (instead of my default -j 70) helps a lot with OOMK, but also looses a bit when chromium is the only recipe I've changed and is rebuilt without any browser eating host's ram
murych has quit [Quit: Konversation terminated!]
<vmeson>
JaMa: have you played with PARALLEL_MAKE:pn-qtwebengine = "-j 40 -l 40" -- That way if you only have one build, you get 40 compile job but when there are more things going on, you keep the system load averaging at 40.
<kergoth>
richard had an old patch to make bitbake act as a make jobserver. not sure it ever got past prototype though
<vmeson>
I'm not sure how we'd share one job server between multiple builds (different users?) but I guess it's possible.
<vmeson>
kergoth: richard has an old patch for just about everything it seems! Let's see if he wanders by and digs it up before I ask him about it later this week.
seninha has quit [Ping timeout: 252 seconds]
<JaMa>
vmeson: not yet, but looks like I really should try it (and not only for qtwebengine, but in default PARALLEL_MAKE I guess)
<vmeson>
JaMa: yeah, let me look at the buildchart again...
<JaMa>
vmeson: with 8 modules I most likely couldn't use 3200Mhz ram and QVL for 8 modules is _very_ short
<JaMa>
and even with just 4 32G modules @2666 I'm now seeing some single bit errors in memtest86 :/
<JaMa>
upgrading from 3970X to 3995WX would be nice, but I'm not paid that well :)
<JaMa>
vmeson: you can try to run it on your builder as well to see what breaks, just checkout langdale branch and run test.sh
<vmeson>
JaMa: agupta1_ has the memory pressure regulation done but not tested...
<vmeson>
We should probably try your build... what a monster config!!
<JaMa>
vmeson: I've refreshed the bootcharts in repo to show mem pressure and sent the pybootchart patch to oe-core
<vmeson>
JaMa: kk. reloaded: the bust of swap pressure is rather short....
<vmeson>
JaMa: it would be interesting a buildchart if you set the max cpu pressure to say 500-2000 depend on the numbers in your logs.
<JaMa>
yes and in the example I was looking at it was already when chromium-x11, webkit-gtk, firefox do_compile were running (so too late not to start them)
florian_kc has joined #yocto
nemik has quit [Ping timeout: 252 seconds]
nemik has joined #yocto
<JaMa>
it could ease a bit by not starting anything more, but still these 3 will be able to trigger OOMK even when they continue to run only these 3
<vmeson>
JaMa: yeah with -j 40, you'd still have three big jobs start, maybe a bit later than usual, then once you start seeing cpu pressure, it's sustained so there shouldn't be another job started for a while.
<vmeson>
JaMa: hmmm, I guess you have to try it out so see if it helps. When the big jobs are linking, there's going to be more memory needed... so we'll have to think about how to handle that.
florian has quit [Ping timeout: 252 seconds]
<JaMa>
yes, I plan to run it few more times with various settings, it will take couple days to finish
<JaMa>
most of my bigger projects use ninja, so maybe the -l you mentioned will do most of the work
<vmeson>
our 192 core system has 256 GB RAM so maybe better balanced. Occasionally someone messes up and it OOMs or the scheduler hangs but it's so rare that we haven't figure out how to prevent it let along who to blame! ;-)
<JaMa>
if not will wait for agherzan to submit his cmake changes and test those as well
nemik has quit [Ping timeout: 252 seconds]
<JaMa>
it has less ram per core than mine, you're just not hammering it enough :)
<kergoth>
I ended up adding naive heuristic-based scaling down of jobs/threads based on mem-per-core for our distro, not ideal, but better than nothing
<kergoth>
yawn
Ram-Z has quit [Ping timeout: 252 seconds]
<JaMa>
vmeson: like bitbake cooker log? I can also find some graphs in grafana from that time, but not sure what else it would show
Ram-Z has joined #yocto
<vmeson>
JaMa: I'm wondering if the log.do_compile files would show when the big apps were being linked.
<vmeson>
JaMa: it's hard to say what's causing the spike, based on the Graphana data. Can you try a run with some value of cpu regulation, say 1000, and leave bitbake's max tasks at 64?
seninha has joined #yocto
<JaMa>
vmeson: yes, will do in couple days (tonight want to re-run dunfell build), then will go back to langdale testing
<vmeson>
JaMa: ok, Thursday morning is when we usually go over build regulation things. 9 AM ET - does that work for you?
<JaMa>
will try to attend and get the data before that
<JaMa>
tomorrow might receive new laptop, maybe I'll be able to let my workstation do the benchmark when setting the laptop
davidinux has quit [Ping timeout: 245 seconds]
<vmeson>
JaMa: super, thanks.
<JaMa>
vmeson: is it the same zoom meeting as Weekly Triage just 1,5 hour earlier?
nemik has quit [Ping timeout: 268 seconds]
goliath has quit [Quit: SIGSEGV]
nemik has joined #yocto
voyqsv has joined #yocto
florian_kc has quit [Quit: Ex-Chat]
florian has joined #yocto
voyqsv has quit [Remote host closed the connection]
<vmeson>
JaMa: yep
<vmeson>
JaMa: err, no... I'll send you an invite.
xenador77 has joined #yocto
manuel1985 has joined #yocto
seninha has quit [Remote host closed the connection]
nemik has quit [Ping timeout: 252 seconds]
nemik has joined #yocto
leon-anavi has quit [Remote host closed the connection]
goliath has joined #yocto
nemik has quit [Ping timeout: 240 seconds]
nemik has joined #yocto
xenador77 has quit [Remote host closed the connection]
xenador77 has joined #yocto
alimon has quit [Ping timeout: 272 seconds]
mvlad has quit [Remote host closed the connection]
Estrella_ has quit [Remote host closed the connection]
Estrella_ has joined #yocto
BWhitten has quit [Remote host closed the connection]
erbo has quit [Ping timeout: 240 seconds]
tomzy_0 has quit [Ping timeout: 252 seconds]
alimon has joined #yocto
willo has quit [*.net *.split]
willo has joined #yocto
manuel1985 has quit [Ping timeout: 268 seconds]
xenador77 has quit [Ping timeout: 252 seconds]
<RP>
vmeson: I['m sure we discussed my jobserver patch in the past?