<ixelp>
cffi - Interpret foreign memory as lisp memory (or vice versa) without copying data - Stack Overflow
ym has joined #commonlisp
<younder>
Indeed copying the whole array in one is better that element for element
<younder>
memcpy
<bike>
this is sort of the other way around, no? with-pointer-to-vector-data lets you get a C pointer from a lisp array. but you have a C pointer and want to make a lisp array out of it.
yitzi has joined #commonlisp
yitzi has quit [Ping timeout: 246 seconds]
treflip has quit [Read error: Connection reset by peer]
<ixelp>
GitHub - gunnarmorling/1brc: 1️⃣🐝🏎️ The One Billion Row Challenge -- A fun exploration of how quickly 1B rows from a tex [...]
<Bubblegumdrop>
Originally I had aimed to do it with common lisp but perf issues with i/o. So I had finished it up in C/C++ and my current solution is pretty fast without using SIMD
<Bubblegumdrop>
anyway yeah basically at this stage since like bike describes I have a pointer I could just not use it in lisp at all
spdegabrielle has quit [Quit: Connection closed for inactivity]
<Bubblegumdrop>
I was just having a look at the common lisp solution recently and that's where I left off
<bike>
i suppose that what you can do is allocate a lisp vector, get a C pointer, memcpy from your file into that pointer, and then keep the vector
tfeb has joined #commonlisp
kevingal has joined #commonlisp
szkl has quit [Quit: Connection closed for inactivity]
<shawnw>
I wrote up a CL approach but it was pretty slow.
<Bubblegumdrop>
mine does okay once you start hitting the GC it chokes
tfeb has quit [Quit: died]
troojg has joined #commonlisp
istewart has joined #commonlisp
sveit has joined #commonlisp
treflip has quit [Ping timeout: 265 seconds]
treflip has joined #commonlisp
<younder>
I have a (setq inferior-lisp-program "/usr/local/bin/sbcl --dynamic-space-size 32768")
<younder>
Big heap less troubles :)
spdegabrielle has joined #commonlisp
treflip has quit [Read error: Connection reset by peer]
<younder>
Also sbcl built with SBCL_MAKE_JOBS="-j" SBCL_MAKE_PARALLEL=42069 ./make.sh –with-mark-region-gc
<younder>
the mark-region collector tries to minimize copying.
<younder>
That might help..
treflip has joined #commonlisp
treflip has quit [Read error: Connection reset by peer]
treflip has joined #commonlisp
<Bubblegumdrop>
I'm using 12GB lol
<Bubblegumdrop>
the 1 billion row file is about 13 GB
<Bubblegumdrop>
at this stage i'm just writing C with ffi
<Bubblegumdrop>
it's a silly exercise
treflip has quit [Read error: Connection reset by peer]
treflip has joined #commonlisp
varjag has joined #commonlisp
supercode has quit [Quit: Client closed]
treflip has quit [Quit: Quit]
<nil>
i have done something like this in lisp with real weather data
troojg has quit [Ping timeout: 248 seconds]
ym has quit [Ping timeout: 265 seconds]
molson has quit [Ping timeout: 246 seconds]
chomwitt has joined #commonlisp
cage has quit [Quit: rcirc on GNU Emacs 29.4]
jadzi has quit [Ping timeout: 252 seconds]
molson has joined #commonlisp
varjag has quit [Ping timeout: 276 seconds]
JuanDaugherty has joined #commonlisp
molson_ has joined #commonlisp
molson_ has quit [Remote host closed the connection]
molson has quit [Ping timeout: 272 seconds]
JuanDaugherty has quit [Quit: JuanDaugherty]
molson has joined #commonlisp
wacki has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
X-Scale has joined #commonlisp
attila_lendvai has quit [Ping timeout: 246 seconds]
lucasta has joined #commonlisp
skeemer has quit [Ping timeout: 248 seconds]
alternateved has quit [Remote host closed the connection]
amb007 has quit [Ping timeout: 276 seconds]
troojg has joined #commonlisp
yitzi has joined #commonlisp
donleo has quit [Ping timeout: 272 seconds]
<qhong>
is there a Just Work^TM somewhat portable code walker out there?
<qhong>
I was using hu.dwim.walker but the hu.dwim.* systems seem unmaintained and fail to build
gioyik_ has quit [Ping timeout: 260 seconds]
chomwitt has quit [Ping timeout: 265 seconds]
troojg has quit [Ping timeout: 260 seconds]
yitzi has quit [Ping timeout: 248 seconds]
yitzi has joined #commonlisp
yitzi has quit [Client Quit]
amb007 has joined #commonlisp
kevingal has quit [Ping timeout: 244 seconds]
alcor has quit [Remote host closed the connection]
alcor has joined #commonlisp
amb007 has quit [Ping timeout: 265 seconds]
alcor has quit [Ping timeout: 252 seconds]
jonatack has quit [Read error: Connection reset by peer]