<fijal>
so a) how do we do this in cffi? and b) how do express it neatly in ll2ctypes?
<arigato>
that's a ctypes-specific question, I'm not even sure if it works in all cases
<arigato>
ctypes guesses it's variadic if you call it with more arguments than declared?
<fijal>
yes
<arigato>
but a function declared in C with "void foo(int a, ...);" might be called with no extra arguments
<arigato>
but is still variadic
<fijal>
no
<fijal>
then you can use normal prep_cif
<arigato>
are you sure?
<arigato>
I mean, possibly, but I'm not sure about that
<fijal>
I'm sure that's what the comment in cpython says
<arigato>
OK
<fijal>
/* Even on Apple-arm64 the calling convention for variadic functions conincides
<fijal>
* only with its fixed arguments. Thus, we do not need a special flag to be
<fijal>
* with the standard calling convention in the case that the function called
<fijal>
* set on variadic functions. We treat a function as variadic if it is called
<fijal>
* with a nonzero number of variadic arguments */
<arigato>
right, so someone looked it up and it happens (by chance) to work for M1
<fijal>
I'm not sure it's "by chance"
<fijal>
maybe it has to work like this
<arigato>
maybe
<fijal>
I don't know
<fijal>
let's assume it does, I think otherwise it's scary
<fijal>
do we have something like that on RPython though?
<arigato>
I don't so
<arigato>
I don't think so
<fijal>
ok, seems like we need to, in order to run tests
<arigato>
so that's the ll2ctypes question
<fijal>
yes and no, you still need to know "did I call the function with the right amount of parameters"?
<arigato>
can you give me an example?
<arigato>
of test
<fijal>
I think that's also a bug in cffi ABI mode FYI
<fijal>
like, open(2) is a variadic function on OS X
<fijal>
as is fcntl
<arigato>
no, in cffi you can (and thus must on M1) declare open with a "..." as argument list
<fijal>
ok
<arigato>
then cffi will use the variadic version of ffi_prep
<fijal>
even on pypy?
<arigato>
and only afterwards it looks as the number of arguments provided etc.
<arigato>
pretty sure, yes
<fijal>
I can't find ffi_prep_cif_var
<fijal>
(in pypy source code)
<mattip>
fijal: about ssl. I think the link is broken, so you don't actually have a tar.gz file, you have either empty or some http error file
<mattip>
that was changed in commit c131d87d7802 from 5 days ago
<fijal>
mattip: so I just hg merge default to m1?
<arigato>
hum
<mattip>
... or cherry-pick the commit, whatever is easier
<arigato>
yes, you're right
<fijal>
arigato: ok, so those are all a bit different questions (we need to fix ctypes, cffi, ll2ctypes and maybe rpython and maybe cppyy)
<fijal>
I *think* I would like to fix ctypes and ll2ctypes first, so I can run the tests at all (it's currently not really that possible)
<arigato>
OK
<fijal>
so maybe ctypes first? but I think I know how to do that, what I don't know how to do is how do I express the right things in RPython/ll2ctypes
<arigato>
do you have a test example?
<fijal>
what do you mean?
<arigato>
can you point me to a test that fails because of RPython/ll2ctypes?
<fijal>
anything that uses open/fcntl? I have skipped a whole lot of tests on OS X
<fijal>
not anything, but a lot of those things
<fijal>
look on m1 branch and commits there skipping tests
<fijal>
test_fdopen_buffering_line in test_rfile, for example
<arigato>
and it fails, because..?
<arigato>
it should be calling the C fopen(), not the C open(), unless I'm messing up
<fijal>
I don't know why it exactly fails
<fijal>
but that's the kind of reason why I want to fix it in the first place - I know the variadic function call is wrong, so I want to make sure that it's called correctly before digging into deeper platform issues
<fijal>
it's hard to debug things when you know there are 3 different unfixed reasons why it's not working
<arigato>
yes, maybe you should look at or write a test specifically for ll2ctypes, first
<fijal>
ok, fine, but I know it's broken
<fijal>
but no, it makes no sense
<arigato>
but then you can fix it
<fijal>
I can't write a test if I don't have a working python
<fijal>
I need working python to *run* tests - that requires fixing ctypes in pypy *first(
<arigato>
I guess you already gave up on obtaining a working cpython 2.7
<fijal>
yes
<arigato>
OK
<fijal>
it seems scarier and it seems like having a working pypy is a necessity anyway
<arigato>
then yes, maybe the first step is to go to a non-M1 machine, and change the code in _ctypes and _ffi to call ffi_prep_cif_var
<arigato>
it's hard to write a test for that, but not so important, the goal would be to make sure that the tests *still* work after you call ffi_prep_cif_var in the right cases
<arigato>
on a non-M1
<fijal>
I can do that, that seems quite easy
<arigato>
then the hope is that this gives you a working ctypes on M1 too
<fijal>
but what I want help with is how we express "this is a variadic function" in rffi
<arigato>
right, so can you give me an example of a variadic function expression :-)
<fijal>
open with 3 arguments
<fijal>
printg
<fijal>
printf
<arigato>
where is that?
<arigato>
I don't find with grep an rpython test that calls printf
<fijal>
arigato: does it really matter what specific RPython function is this?
<arigato>
I want to look at the rffi.llexternal() and think
<fijal>
ok, so open
<arigato>
and where is that?
<fijal>
rlib/rposix.py
<fijal>
it's just defined with 3 arguments there, which is wrong
mjacob has quit [Read error: Connection reset by peer]
<arigato>
ah, c_open, got it
<arigato>
yes
<fijal>
I think fcntl is in the same boat
<arigato>
then maybe rffi.llexternal needs an extra flag, "its_variadic_even_though_I_declare_it_with_a_specific_signature_here"
<fijal>
and then we just call it with *_var version?
<fijal>
it's a bit of a mess because you need to pass this information through ll2ctypes
<arigato>
yes
<fijal>
so more like "natural number of arguments"
<arigato>
ah, yes
<fijal>
there is one more complication which is that prep_cif_var has the number of arguments passed
<fijal>
(does ctypes really call prep for every call?)
<arigato>
yes, cffi too
<arigato>
(for variadic functions)
<fijal>
ok
<fijal>
so that requires some refactoring in clibffi maybe?
<arigato>
I'm looking at the implementation of rffi.llexternal
<arigato>
seems to call directly ll2ctypes.LL2CtypesCallable
<arigato>
so maybe just pass the flag there?
mjacob has joined #pypy
<fijal>
ok
<fijal>
so we need to call the variadic prep function every time we call the ctypes function (as a start)
<arigato>
I may be confusing levels
<fijal>
you definitely need *ctypes* to work, right?
<arigato>
I think there are two issues that need to be separated
<arigato>
1st you fix _ctypes to call the _var, and you can do that completely on a non-M1 machine
<fijal>
yes
<fijal>
or more likely fix it on an m1 machine and then run tests on both
<arigato>
no, because you can't run tests on M1 so far, or am I very confused
<fijal>
I can run the above ctypes example (because I can compile pypy -O2)
<fijal>
I can't run the tests through layers
<fijal>
but a *compiled* pypy runs
<arigato>
OK
<fijal>
how is open implemented on linux?
<fijal>
int open(const char *pathname, int flags);
<fijal>
int open(const char *pathname, int flags, mode_t mode);
<fijal>
what does that mean?
<arigato>
just saying, maybe a better approach would be to take a non-M1 machine where you know all tests run, and try to do changes there
<arigato>
even if tests don't fail before, you can still do changes that you know will be needed on M1, and test them
<fijal>
yes, maybe, but this is entirely not what I'm asking about
<fijal>
I would like to be able to declare it with llexternal (natural number of arguments sounds about right), how do I pass it to ctypes?
<fijal>
by using the convention of "pass the rest of arguments as undeclared"?
<arigato>
yes
<fijal>
ok
<fijal>
arigato: do you have any clue how is this implemented on linux?
<arigato>
it's the man page that is strange
<arigato>
it's done with a "..." too
<fijal>
ok
<fijal>
but it somehow does not matter to call the _var version
<arigato>
no, libffi didn't even have a _var version for a long time because traditionally you can call f(int, int) or f(int, ...) with two integers and that's the same code
<arigato>
only new architectures make different code because of modern optimizations
<fijal>
ok
<fijal>
right, so it's a bit of mess
<arigato>
I think even calling f(int) or f(int, ...) with one integer argument starts to require different code, but so far it probably works because if you call f(int) as if it were f(int, ...) then the right thing occurs anyway
<fijal>
what's CDECL
<fijal>
?
<arigato>
ignore that, that's the calling convention and only win32 has got more than one I think
<arigato>
(even win64 no longer does)
<arigato>
(I think)
<fijal>
ok
<fijal>
then I don't understand line 307 in _ctypes/function.py
<fijal>
does it not *always* explode if you pass the wrong number of arguments?
<arigato>
no, only if self._flags_ says the calling convention is not CDECL
<arigato>
but CDECL is the default
<arigato>
I think the alternative, STDCALL on win32, does not support variadic functions at all
<fijal>
ok, so I think I really misread "what's CDECL" "ignore that"
<arigato>
please note that when you asked that question, I had no way to guess you were refering to that code :-)
<fijal>
yes sure, but the answer could have been "the default calling convention" for example
<arigato>
(and of course there is __vectorcall that is an alternate calling convention on win64)
<fijal>
I still don't get that line
<fijal>
do we just cut the arguments and not pass them? feels wrong....
<arigato>
ah
<arigato>
yes, I seem to remember that?
<arigato>
now I'm confused
<fijal>
ok, so in that line it has not be "no, but call the variadic version of that/" right?
<arigato>
previously, you had to call variadic functions with no argtypes at all
<arigato>
not sure why that changed, btw
<fijal>
because you have the natural number of args
<arigato>
ah
<arigato>
yes
<fijal>
great...
<fijal>
ok, can I change symantics in ctypes or is it not working at all?
<arigato>
then on ctypes on M1, you *cannot* call a function if you didn't specify argtypes? or you can but it assumes it's not variadic and you just gave it the correct arguments?
<fijal>
apparently?
<arigato>
OK
<fijal>
I mean calling functions without giving argtypes is asking for trouble
<arigato>
yes
<arigato>
OK, so in theory, the cpython 2.7 behavior says that if you call a function with too many arguments, the rest is dropped to the floor
<fijal>
yes, in theory
<arigato>
but I *think* it's OK to change that into "it becomes a variadic call"
<arigato>
most of the time, a call done with too many arguments will still work as expected
<fijal>
whatever "as expected" really means...
<arigato>
anyway, isn't that the code around function.py line 356 that we should be looked at instead
<arigato>
I don't find any truncation of the argument list, in that case
<arigato>
but it's quite a lot of code so I'm not 100% sure
<arigato>
or maybe pypy/module/_rawffi/alt/interp_funcptr.py:96
<arigato>
which checks that exactly the right amount of arguments are passed?
slav0nic has joined #pypy
slav0nic has quit [Ping timeout: 248 seconds]
lritter has joined #pypy
jacob22 has quit [Ping timeout: 276 seconds]
reneeontheweb has joined #pypy
stkrdknmibalz has quit [Ping timeout: 276 seconds]
stkrdknmibalz has joined #pypy
reneeontheweb has quit [Quit: Client closed]
jacob22 has joined #pypy
greedom has joined #pypy
otisolsen70 has quit [Quit: Leaving]
lritter has quit [Ping timeout: 248 seconds]
greedom has quit [Remote host closed the connection]
[Arfrever] has quit [Read error: Connection reset by peer]
[Arfrever] has joined #pypy
[Arfrever] has quit [Read error: Connection reset by peer]