michaelni changed the topic of #ffmpeg-devel to: Welcome to the FFmpeg development channel | Questions about using FFmpeg or developing with libav* libs should be asked in #ffmpeg | This channel is publicly logged | FFmpeg 7.1 has been released! | Please read ffmpeg.org/developer.html#Code-of-conduct
<compn>
its from the xz backdoor autoconf build script, curious if anyone else has ever used sed to cat like that.
System_Error has quit [Ping timeout: 260 seconds]
kasper93 has joined #ffmpeg-devel
<ePirat>
BtbN, is it expected that patchwork is down completely now?
<BtbN>
It should be back up now
<BtbN>
Just for some reason slow
<BtbN>
Completely upgraded it and made sense of it
<ePirat>
getting a gateway timeout
<ePirat>
so probably too slow for the reverse proxy even
<BtbN>
*** uWSGI listen queue of socket "0.0.0.0:8000" (fd: 6) full !!! (101/100) ***
<BtbN>
Yeah, it's working on 100/100 requests, and just hyper busy
<ePirat>
"fun"
<BtbN>
I guess a bunch of tools hammer it right now, cause it was down for an hour
<BtbN>
It is slowly working through requests, so I hope it settles
<ePirat>
how can this thing be so slow
<ePirat>
it doesnt even do much
<BtbN>
Well, if the DB is trying to process 100 requests in parallel, it gets quite slow
<BtbN>
The VM its on isn't a fast one
<BtbN>
I wonder if the DB is missing some index. It wasn't this slow before
<ePirat>
is it postgres or mysql/mariadb?
<BtbN>
mariadb
<BtbN>
It was a really old mysql version before. Now it's latest mariadb
<BtbN>
and it got dog slow
<ePirat>
weird
<BtbN>
"GET /project/ffmpeg/list/?state=*&archive=both¶m=-date&page=336 => generated 105812 bytes in 14074 msecs (HTTP/1.1 200) 5 headers in 264 bytes"
<BtbN>
14 seconds to generate that page. Why?
<ePirat>
that is incredibly slow…
<BtbN>
I'm gonna turn on slow query logging and watch
<ePirat>
BtbN, thanks a lot for looking into this btw
<BtbN>
The way it's doing mail submission now is postfix ssh'ing into the patchwork box, and submitting all mail to the ffmpeg-devel
<BtbN>
list
<ePirat>
does it do that periodically or per mail?
<BtbN>
every mail
<BtbN>
It's just me subscribing a magic internal address to the list, and adding an my-secret-address: |/my/script.sh
<BtbN>
to the aliases file I mean
<BtbN>
And it then just calls that script for every mail and feeds it the mail via stdin
<BtbN>
It's how patchwork documents to set it up with postfix. Just that there's no ssh in between
<JEEB>
so actual bool types are utilized in those, too
<JEEB>
so if that used to be the problem, then those modules would already fail compilation
System_Error has joined #ffmpeg-devel
rvalue has quit [Read error: Connection reset by peer]
rvalue has joined #ffmpeg-devel
Krowl has quit [Read error: Connection reset by peer]
MisterMinister has quit [Ping timeout: 244 seconds]
MisterMinister has joined #ffmpeg-devel
System_Error has quit [Remote host closed the connection]
Krowl has joined #ffmpeg-devel
System_Error has joined #ffmpeg-devel
<Lynne>
haasn: we don't usually use bools because setting them may involve cmovs or bitwise ops
<haasn>
on what platforms?
<Lynne>
x86, and others too
<kasper93>
could you show an example?
<Lynne>
afaik ABI-wise they're specified to always be 0 or exactly 1
<Lynne>
at least that's how BBB explained it to me more years ago than he remembers
<Lynne>
(not old, pretty sure he must've been in middle school back then or something)
<Daemon404>
i wonder if this really is measurable in 202r
<Daemon404>
2025 even
<Daemon404>
this sounds very gcc 2 era voodoo
<wbs>
most such microoptimizations aren't usually measurable, but if it can be showed in less nice code output from the compiler, that's at least an observable thing
<kasper93>
0/1 is true, but in practice it matters if you convert int to bool for example. But `int a = x == 0;` would compare too, similar to `bool a = x;`
<kasper93>
wbs: again true, but I'm just curious for my own informatio, in what situation the generated code would be worse
IndecisiveTurtle has quit [Ping timeout: 265 seconds]
deus0ww has quit [Ping timeout: 252 seconds]
deus0ww has joined #ffmpeg-devel
Lynne has quit [Read error: Connection reset by peer]
<Lynne>
compn: I too want more raw support in ffmpeg, particularly blackmagic raw
<compn>
yes blackmagic and red for sure.
<Lynne>
not sure why anyone would buy a red these days though
<compn>
old cameras
<Lynne>
ah
<compn>
i mean, supporting old cameras
<compn>
no idea if people buy red now
<Lynne>
the low cost cinema camera these days is ruled by the bmcc
<Lynne>
eveh though its got an L-mount which precisely no one cares about, blackmagic raw is a much better and modern codec, its full frame (!) and its got 2 xlr ports
<Lynne>
red raw converts to yuv internally afaik, blackmagic use a reversible transformation
<compn>
yuv is good enough /s
<Lynne>
(not even going to say anything about panasonic or sony cinema cameras, which use xavc)
<compn>
i think we're going to see a lot more new 8k cameras
<compn>
or maybe not
<Lynne>
oh they've been around for years now, they're just in the 15k price bracket, not the 3k
<Lynne>
what I'd really like is a camera which can shoot in raw or losslessly compressed codec
<Lynne>
kinefinity promised they'd implement cinemadng in their 6k camera years ago via firmware, but never did as far as I can tell
<Lynne>
yup, red are bullies that will go after you if you do something lossless, hence cinemadng
<Lynne>
which is just one xml file, plus one dng file... for each frame
<Lynne>
there is an open source cinema camera, which while well designed and everything, is pretty fully outdated today, and they've got no plans to bring it up in the 6k full frame era we're at now
<gnafu>
I wonder if anything will change with RED being owned by Nikon now.
<gnafu>
Nikon have been making some exciting cameras in recent years. Very impressive video capabilities, but not "cinema" bodies yet.
<gnafu>
I still shoot with an 8-year-old Panasonic that just does 100 Mbps H.264 for its UHD, but I would love to have FOSS support if I ever upgrade to something that can do raw video.
<gnafu>
First step for me is just moving to something that does 10-bit H.265...
<JEEB>
gnafu: 100mbps H.264 is quite nice, and with cameras you're always afraid of whether new video format settings are just dumb (I've seen this oh so many times, newer format meaning hilariously low bit rate or something)
aljazmc has joined #ffmpeg-devel
jarthur has joined #ffmpeg-devel
Krowl has joined #ffmpeg-devel
mateo` has quit [Quit: WeeChat 4.4.2]
mateo` has joined #ffmpeg-devel
aljazmc has quit [Remote host closed the connection]
ngaullier has quit [Ping timeout: 255 seconds]
aljazmc has joined #ffmpeg-devel
<fflogger>
[editedticket] Balling: Ticket #8385 ([avformat] libavformat/aviobuf: A part of conditional expression is always true: whence != 2) updated https://trac.ffmpeg.org/ticket/8385#comment:3
Krowl has quit [Read error: Connection reset by peer]
iive has joined #ffmpeg-devel
Sean_McG has joined #ffmpeg-devel
* Sean_McG
waves
<Sean_McG>
so I finally took some time to look at the FATE failure on PPC for checkasm-sw_yuv2rgb and I verified even on a little endian POWER9 that it does not work. Can I go ahead and send a patch to just remove the yuv2rgb Altivec pieces? I'm out of my league on fixing it and I doubt anyone else will step up to the plate.
<Sean_McG>
Unless should I try contacting Chip Kerchner @ IBM to see if he minds taking a crack at it?
<Sean_McG>
(he's the last to modify that file -- back in 2020)
<JEEB>
Sean_McG: if there's no clear revert available then it doesn't sound too bad of an idea to remove non-working SIMD
<Sean_McG>
OK.
<kurosu>
Sean_McG: why not ask some guidance to Luca? But maybe he isn't listed in git log for that file
<Sean_McG>
kurosu: I thought he didn't want to do that sort of thing anymore
Traneptora has quit [Quit: Quit]
<kurosu>
I think he did a bit for dav1d, but you probably know better than I, so sure
<BBB>
Sean_McG: is it possible for you to see whether the output differs in off-by-one ways, or if it's completely wrong?
<BBB>
int he former case, maybe it can be set to be assigned when the bitexact flag is not set
<BBB>
in the latter case, for sure remove it
<Sean_McG>
BBB: originally I thought it might just be due to UB on systems that can't do unaligned loads for AltiVec because it was also triggering on the UBsan FATE nodes, but now that I confirmed it is not working on little-endian POWER9 I threw away that notion.
<Sean_McG>
how do you confirm off-by-one with SIMD?
<BBB>
print c vs simd output and see how they're different
<BBB>
if the differences are one-off in a few places, it's off-by-one
<BBB>
if it's massively different and/or everywhere, it's something else
<Sean_McG>
OK, I'll try that when I have some cycle
<BBB>
ty
<BBB>
if it's not obvious, it's probably ok to just remove it
<BBB>
but someone will make the comment that we should try to keep it, so I'm just trying to help that end of the conversation move along in a practical way
<BBB>
dav1d's checkasm uses checkasm_check_pixel (etc) instead of plain memcmp
<BBB>
the advantage of that is that -v will print pixel differences automatically
<BBB>
so you don't need to write manual printf when something fails
<BBB>
which is kind of nice
<BBB>
we should eventually backport that to ffmpeg/x264/etc. and use it everywhere
mkver has quit [Ping timeout: 246 seconds]
<BBB>
there's a whole bunch of them: checkasm_check_{pixel,(u)int{8,16,32,64},coef,...}()
<BBB>
where pixel is a dav1d-specific concept which is slightly different from how ffmpeg does multi-bitdepth coding
<BBB>
coef is the same but for signed coefficients in residual coding
<BtbN>
hmm, I see why patchwork is so slow. Cause somehow, crawlers or something are clicking all the links all the time. Including the ones to the very last page of patches
<Sean_McG>
do we not have a robots.txt?
<BtbN>
It's modern to just ignore it
<Sean_McG>
ouch.
<BtbN>
All the LLMs need all the data in the world
<BtbN>
and they don't need no rules
<BtbN>
And that spawns SQL queries with stuff like OFFSET 36300
<BtbN>
And that's just slow in general, no matter how well the indices fit
<BtbN>
I'm tempted to just remove the links to the last pages
<kurosu>
BBB: "but someone will make the comment" <- too late, I've already done it :p
<Sean_McG>
I've had this argument at work and I have no problem with someone re-adding what is removed if they have a fix.
<BtbN>
When would someone ever need to go to the last page of patches anyway?
<BtbN>
Other out of courious interest
<BtbN>
which is less important than server stability
<Sean_McG>
^ probably that's the reason
<BBB>
Sean_McG: that's fine also. it was merely a suggestion, I'm ok with just removing it if checking it is not worth the effort
<Sean_McG>
BBB: I think I will just do that and if someone fusses we can handle it then & there.
<Sean_McG>
my gut feeling is that it is not "off-by-one"
<Sean_McG>
I'm crossing my fingers it won't cause too bad a perf regression on modern POWER.
<BBB>
Sean_McG: power has bigger fish to fry than a random swscale function
<Sean_McG>
TRU.DAT
<BBB>
:)
beastd|3 has joined #ffmpeg-devel
beastd|3 is now known as beastd
_whitelogger_ has joined #ffmpeg-devel
Traneptora has joined #ffmpeg-devel
<nevcairiel>
I had some issues with crawlers indexing my gitea and causing lots of load, but luckily putting in a robots seems to have stopped it, so even if some might ignore it, having it does no harm
<BtbN>
[pid: 111|app: 0|req: 493/2272] 157.55.xx.xx () {38 vars in 710 bytes} [Fri Nov 15 07:09:12 2024] GET /project/ffmpeg/list/?state=*&archive=both¶m=223&page=508 => generated 80543 bytes in 21229 msecs (HTTP/1.1 200) 5 headers in 263 bytes (1 switches on core 2)
<BtbN>
Who even goes to page 508?? There's only 216 pages.
jarthur has quit [Read error: Connection reset by peer]
<another|>
Every page twice means more training data, right?
* Sean_McG
sighs
<compn>
compile patchwork to C , then http cache all pages instead of serving/query directly /s
<BtbN>
I added a "leave me alone" robots.txt now, and removed the links to the deep pages
<Marth64>
I used to block a ton of user agents via regex (via an nginx sitting in front) when I was running captive portals in my ISP days. While not fool or malicious intent proof it can help filter a good amount of unintentional junk traffic.
<Marth64>
It might be effective for those stubborn bots that still identify with a honest user agent but ignore robots
<Marth64>
It can also cache static assets like images in memory. if desired or needed I can assist with nginx configs
<BtbN>
patchwork also still can't mail. And I don't know why
_whitelogger_ has quit [Remote host closed the connection]
<Marth64>
Application failure or smtp issues?
<Sean_McG>
oh I thought that was only me -- I can't get access to it because it won't email me from the password reset. I've emailed Andriy twice with no reply
_whitelogger_ has joined #ffmpeg-devel
cone-878 has joined #ffmpeg-devel
<cone-878>
ffmpeg Michael Niedermayer master:07904231cb97: doc/infra: Document gitolite
<BtbN>
I don't know what logging into it even does, so I don't particularily care all that much
<Sean_McG>
I was just going to mark some of my old patches as 'expired' or whatnot
<BtbN>
It tries to send the mail. I see a connection in postfix log. But it fails miserably and it doesn't log anything beyond connect and disconnect
<Sean_McG>
I still hope we'll get something similar to Gitlab sometime in the future -- I've been following the ML messages quite closely
<Marth64>
BtbN: IIRC there is a postfix setting to enable debug logs for a specified server
<BtbN>
There's too many incompatible "must"-demands by too many people
<BtbN>
Some demand that "Not Gitlab, or I'm gone", others demand "Must be able to do PRs via mail", which means "Only Gitlab"
<Sean_McG>
I don't understand the folks who think email is a tenable way to do things in 2024.
<Marth64>
debug_peer_list
<BtbN>
I'd still prefer Gitlab, given Forgejo is a bit weird
<Sean_McG>
like especially when some members push like a 10th revision of a very large patchset.
<BtbN>
the more I read their docs, the more doubts I have
<Marth64>
I imagine and empathize hating JavaScript as I also have it off but yeah mail gets to be a mess
<Marth64>
JavaScript being a factor*
<BtbN>
it feels like on every single page of the docs, vital information is missing, and replaced with "We're so FOSS and Libre compared to Gitea"
<Sean_McG>
I'm particular to Gitlab just because it is what I am used to (and I even stood up a Gitlab server at home), but we use Gitea at work
<BtbN>
I don't know "I hate JS" is just some weird scapegoat argument to block change I feel like
<BtbN>
UI wise, Forgejo/Gitea seems nicer than Gitlab
<Sean_McG>
and really just anything that wasn't a email-centric process would be better IMHO, but I get that there are plenty of reasons people are resistant
<BtbN>
But feature and future support wise, Gitlab is much better
<Marth64>
I am for either one of those. I can assist with ETL jobs and scripting to migrate data if one is chosen
<JEEB>
has anyone brought up the gitlab cli tools so far? since in theory having the required features there for those who just don't want webui. and missing features could be developed or sponsored, I guess.
<JEEB>
not sure if anyone has given out actual required use cases
<Sean_McG>
I didn't even know there were CLI tools... I gather it does something like XMLRPC?
<BtbN>
Yeah, the gitlab cli tools are nice. I never felt the need, but they exist
<BtbN>
I don't think Gitea and derivates have any
<Marth64>
GitLab has json api
<Marth64>
there is flexibility
<JEEB>
I think gitea had something since it was tested during FOSDEM
<Marth64>
the painful part will be CI/CD testing (which can probably be later phase) and migrating old discussion board if ML dies with it (unless archive is good enough)
<Marth64>
gitlab api was pretty straightforward
<Marth64>
in 2019 last I touched it when migrating from github for $prev_job
<Marth64>
"our license expires in 48 hours, we just realized it and won't renew, hurry we must migrate"
<Sean_McG>
ouch.
<Marth64>
I don't particularly hate ML its just iterative improvement of patches/feedback gets too verbose
<Sean_McG>
^ THIS. 110%
<BtbN>
I see no reason to have both a ML and a PR driven setup for however long it's needed
<BtbN>
*not to have
<Sean_McG>
duplication of effort?
<Marth64>
it could be phased gradual transition too. eg q1- MR model, q2- issue ticketing/trac, q3- so on
<Marth64>
one fear I sense from the emails is that this is a all or nothing but it doesnt have to be
<Sean_McG>
true, you could start with just the CI jobs even
<Sean_McG>
one of the biggest issues for ML patches is email corruption of the patch itself.
<BtbN>
initially, issues would be turned off
<BtbN>
until trac can be migrated
<BtbN>
And ML and PRs can coexist however long it takes
<Sean_McG>
OK, I'm off to get some food. Take care everybody.
_whitelogger_ has quit [Remote host closed the connection]
_whitelogger_ has joined #ffmpeg-devel
_whitelogger_ has quit [Remote host closed the connection]
<Marth64>
I'm overcommited for now but if I weren't I would volunteer to help organize ML thoughts (eg pros and cons of each approach) / phases in a powerpoint or something
<Marth64>
if anything just to be a discussion guide to facilitate an outcome
<Marth64>
I have no major problem against ML but this topic keeps coming up lol
<Marth64>
I just see it as a project that needs consensus and a plan
<BtbN>
I fear proper consensus is impossible. There's too many incompatible strong demands.
<BtbN>
No matter what's decided, someone will be upset.
<Marth64>
yes sadly, it is change, not all requirements can be met. with some facilitation and good communication maybe a middle ground can be found
<BtbN>
That's quite hard, given there's a very absolute and discrete set of options
<Marth64>
definitely won't be easy, wearing a project hat my first instinct would be to organize the thoughts in a table
Everything has joined #ffmpeg-devel
IndecisiveTurtle has joined #ffmpeg-devel
witchymary has quit [Remote host closed the connection]
witchymary has joined #ffmpeg-devel
Everything has quit [Ping timeout: 252 seconds]
ccawley2011_ has joined #ffmpeg-devel
Everything has joined #ffmpeg-devel
ccawley2011__ has joined #ffmpeg-devel
ccawley2011 has quit [Ping timeout: 265 seconds]
ccawley2011_ has quit [Ping timeout: 252 seconds]
ccawley2011 has joined #ffmpeg-devel
ccawley2011__ has quit [Ping timeout: 252 seconds]
ccawley2011 has quit [Quit: Leaving]
rvalue- has joined #ffmpeg-devel
rvalue has quit [Ping timeout: 260 seconds]
johnny__ has quit [Quit: johnny__]
johnny__ has joined #ffmpeg-devel
rvalue- is now known as rvalue
\\Mr_C\\ has joined #ffmpeg-devel
aljazmc has quit [Remote host closed the connection]