memset has quit [Remote host closed the connection]
memset has joined #osdev
gcoakes has joined #osdev
xenos1984 has quit [Ping timeout: 258 seconds]
xenos1984 has joined #osdev
gcoakes has quit [Ping timeout: 258 seconds]
min0911 has quit [Quit: Client closed]
bauen1 has quit [Ping timeout: 252 seconds]
heat_ has joined #osdev
heat has quit [Read error: Connection reset by peer]
gog has joined #osdev
Starfoxxes has quit [Remote host closed the connection]
goliath has joined #osdev
bauen1 has joined #osdev
<immibis>
Griwes: iirc they did manage to actually exploit it
<immibis>
it's not just theoretical
<immibis>
so also keep that in mind the next time you hear about a vulnerability that seems only theoretical
<immibis>
and if your only barrier between normal and total pwnage is having 14 extra bits of ASLR, maybe you want to strengthen your system somewhat...
<immibis>
Mutabah: part of the login process used alarm(2) to implement a timeout, and SIGALRM called the logging function
* Mutabah
is away (Sleep)
xenos1984 has quit [Ping timeout: 248 seconds]
<immibis>
to exploit it you had to complete the login at the same moment it timed out (normally 5 minutes?) within not that many cycles of accuracy, that's why it took 3 weeks to hit that race condition window. The attack uses low bandwidth, so you could execute it with high concurrency - maybe against the whole ipv4 internet at once.
<zid>
but it was however, impossible to hit in practice, because they landed it on *32bit*
<zid>
because it was significantly easier
<zid>
like, OoMs easier
<zid>
it'll be patched before anybody could possibly land it
xenos1984 has joined #osdev
netbsduser has joined #osdev
<immibis>
if you can run 2^20 attacks in parallel that counteracts 20 bits of ASLR
<immibis>
and at a rate of maybe 10 packets every 5 minutes, you *can* run 2^20 attacks in parallel
<immibis>
just not all on the same server, and you can't control which one will succeed first
<zid>
3 weeks per machine average, but the attack is a million times harder, so 3 million weeks, so to get one in a week as long as you're attacking 3 million machines at once I guess
<immibis>
siri what's 10 Gbps / 2500 bytes / 2^20 ? oh. one every 2 seconds. if you can find that many machines to simultaneously attack (you probably can't)
<zid>
Yea no way you can
<zid>
and to get one a day you'd need 20 million machines, one an hour 800 million, etc
<zid>
you'd need a cluster at that point too just to handle that many threads
<immibis>
no, but you need specialized software
<zid>
bear in mind you're on the clock for *writing* this software, also
<zid>
before it's patched
<immibis>
my benchmark for bandwidth is that i managed to download useful data from the internet at 5Gbps, and i generated 7.5Gbps upload for pixelflut ( https://labitat.dk/wiki/Pixelflut-XDR ) on one thread
<bslsk05>
labitat.dk: Pixelflut-XDR - Labitat
<immibis>
(at bornhack on a 10gbps connection)
<zid>
how long did it take to set up and write the software for? :P
<immibis>
pixelflut? a couple hours. surprisingly it worked on the first attempt.
<zid>
haha nice
<zid>
udp does seem very simplerer though, nearly 0 state right?
<zid>
I can just promiscuous listen for udp pretty easily?
<immibis>
well yes. good luck getting the linux kernel to handle 2^20 tcp connections.
<zid>
yea that was my point with the cluster
<zid>
you're going to hit limits on threads, connections, arp tables, etc
<immibis>
or in fact 2^28 which is approximately how many you need to saturate 10gbps by sending 2500 bytes every 5 minutes (estimated values)
<zid>
pretty trivially
<immibis>
threads only if you multithread. connections probably. arp tables no.
<zid>
just, whatever you care about, millions is a lot, I remember the 'c10k' problem :D
<immibis>
for high performance code you want one thread per CPU, not one per connection, and you either assign connections to CPUs *or* you allow any CPU to pick up any connection
<zid>
"Man, how could you ever write something so scaleable it could handle *10000* connections!?"
<immibis>
but there's no way linux will handle 2^28 concurrent connections so you'll be formatting and sending raw packets, anyway
<immibis>
computers are super duper fast now. we hobbled them with software that's super duper slow. the less software you can put between the problem and the solution, the faster it will probably run.
<immibis>
i didn't try porting my pixelflut transmitter to send raw ethernet frames because i was satisfied with 7.5gbps (this leaves 2.5gbps of my shared uplink available for actually useful stuff)
<immibis>
(plus vlan tagging overhead, etc)
<zid>
as a defender though, I just have to hope 800 million immibis's don't exist ;)
<immibis>
could have also multithreaded it. it's pretty cool that one thread calling sendmmsg can generate that much traffic. another fun observation: my CPU has 4 core clusters that run at slightly different frequencies, which are treated the same by the scheduler, and you can see the pixelflut thread migrating between core clusters by the bandwidth graph going up and down slightly
<immibis>
(it's like 5.1ghz vs 5.2ghz)
<zid>
Clearly we need sendmsg that takes a command list
<zid>
so we can pump more packets per syscall
<immibis>
i think that's called io_uring
<immibis>
or AF_XDP
<immibis>
at some time in the future i may or may not try to run a variation of pixelflut over Remote DMA
<raggi>
sendmmsg
<geist>
msg: HELO raggi
<raggi>
howdy!
<immibis>
another cool idea is fpga networking. that reminds me, someone told me amazon was selling a zillion poorly documented 10G fpga nics on ebay...
<raggi>
zid: we use {send,recv}mmsg on linux in tailscale for segment offloading, it works really well (until current stable release where it got temporarily broken kernel side). the api isn't ideal at all, in that it's still far too specific to offloading, in particular it only really improves throughput on a socket with few peer destinations, if you're communicating traffic with lots of different endpoints it's no good
<zid>
til sendmmsg
fkrauthan has joined #osdev
<raggi>
what i really want to see is uring-like stuff hidden behind a much more ffi friendly api exposed via a vdso, and was really glad recently to hear redox heading in that direction
<immibis>
it wouldn't be in vdso, it would be a normal user mode library, right?
eddof13 has quit [Quit: eddof13]
<raggi>
there's a good chunk of performance sensitive layout and behavior that's deeply coupled to whatever the protocol is to the kernel, so sure you could put it in a userspace library, but that is likely to just introduce a three way versioning problem, while the user facing surace could otherwise be pretty stable immediately and avoid that
<immibis>
is the io_uring protocol not fixed by now?
<raggi>
considering every integrator i've spoken to has followed a rite of passage which involves reporting BUGs upstream, not really
<immibis>
vdso wouldn't make that better
<raggi>
sure, i dont' really care much for a vdso vs. other approaches debate, that's never ending
<raggi>
more my point is, hide the protocol more behind a higher level api that has good compatbility wiht user runtime environments
xx0a_q has joined #osdev
eddof13 has joined #osdev
<raggi>
urings user facing api is terrible for integrators, and it's not really necessary to expose that much, it's the protocol that provides the main advantages, not the gritty details
<immibis>
example from https://kernel.dk/io_uring.pdf : sqe = io_uring_get_sqe(&ring); io_uring_prep_readv(sqe, fd, &iovec, 1, offset);
<immibis>
seems not much worse than readv(fd, &iovec, 1, offset) - of course you need more work to handle the results, but that's unavoidable in any asynchronous api
eddof13 has quit [Quit: eddof13]
<raggi>
immibis: if only that example was actually functional :D
SupUser has joined #osdev
rbox has quit [Remote host closed the connection]
rbox has joined #osdev
heat has joined #osdev
heat_ has quit [Ping timeout: 258 seconds]
xx0a_q has quit [Quit: WeeChat 4.3.5]
SupUser has quit [Quit: Leaving]
asarandi has quit [Quit: WeeChat 4.2.2]
hwpplayer1 has joined #osdev
asarandi has joined #osdev
heat has quit [Read error: Connection reset by peer]
heat has joined #osdev
<zid>
You're all web developers
<zid>
why does Header do nothing in my .htaccess
<zid>
nothing in the error log, pages in that directory load as normal
<zid>
no syntax error from not having mod_headers loaded, I am AllowOveride All'd, but I figure that might just cause an error anyway
<zid>
I have.. silence, and no header
<sskras>
> You're all web developers
<sskras>
now that's a pretty obvious trolling :D
<heat>
immibis, fwiw i've been using qemu's io_uring support (for disks) in great effect
<zid>
sskras: Best way to get help on the internet is apparently to be annoying
<zid>
It works great for everyone else I see
<heat>
io_uring is exploitable but seems to work well here, at least for simple usage on (presumably?) well trodded paths
<zid>
the more annoying they are, the more fuz/jester/etc patiently try to 'fix' them :p
<sskras>
or maybe just join a right channel :)
<heat>
dont try it on google prod though :P
<zid>
This is the right channel!
<zid>
osdev isn't a career, everybody here is legitimately a webdev
<nikolar>
zid: doesn't seem to work
<zid>
nikolar: It didn't try hard enough, one sec
<zid>
sskras: You're just not clever enough to help
<mjg>
maybe you can admit winapi is not all bad and microsoft is doing its best
<zid>
CORS is dumb and I hate it
<heat>
is that quote your copium after killing itanium
<mjg>
yes i'm distractingm yself
foudfou has joined #osdev
<heat>
mjg, mofer freebsd swap, does it suck
goliath has joined #osdev
khimaros has quit [Read error: Connection reset by peer]
khimaros has joined #osdev
leon has quit [Quit: see you later, alligator]
<mjg>
heat: probably
dennisschagt has quit [Quit: No Ping reply in 180 seconds.]
<mjg>
you have to assume everything bsd does unless proven otherwise
dennisschagt has joined #osdev
<heat>
idk boss i was reading through mastodon earlier and found someone glazing linux swap and saying how "it starts swapping even when you're not totally out of memory"
<heat>
assuming this person is correct my world has been rocked and i no longer no what to do
<heat>
know*
<zid>
nikolar: I've decided that javascript, browsers, and servers are dumb, and the correct way to do this is <?php curl_simple(); ?>
<nikolar>
correct
<mjg>
that's known
freakazoid332 has joined #osdev
<mjg>
see the swappiness parameter
leon has joined #osdev
<mjg>
vm.swappiness
<heat>
did you forget the /s
<zid>
(I got the no-cors working but the remote server sees my Host: and then refuses to serve me)
<heat>
linux _never_ does proactive reclaim on a typical kernel/userspace setup
<zid>
how am I supposed to steal your shitty web game and make a local version, greasemonkey? yea right
netbsduser has quit [Ping timeout: 248 seconds]
hwpplayer1 has quit [Quit: ERC 5.5.0.29.1 (IRC client for GNU Emacs 29.4)]
eddof13 has quit [Quit: eddof13]
<immibis>
proactive reclaim isn't the same as swapping. it can still decide to swap out anonymous pages instead of file-backed pages, right?
<immibis>
proactive reclaim would be going and swapping stuff that isn't actually under pressure yet?
antranigv has quit [Ping timeout: 244 seconds]
netbsduser has joined #osdev
eddof13 has joined #osdev
eddof13 has quit [Client Quit]
<geist>
there's also a subtlety that i'm not sure linux has: preemptively writing pages to swap but keeping the original in memory
Matt|home has joined #osdev
<geist>
and then just putting those pages as a higher tier of reclaimability
<Matt|home>
<3
<geist>
that may no longer be a generally good idea with SSDs and the relative speed of cpus and SSDs, but i guess you could do a simialr thing with preemptively compressing. depends on what the numbers are
goliath has quit [Quit: SIGSEGV]
<zid>
just do what windows xp did
<zid>
aggressively swap things to disk
<zid>
There we go, that was the end of the strategy.
<geist>
well sure but what i'm pointing out is that 'swap' can be a bit more subtle than write stuff to disk and remove the original page
dude12312414 has joined #osdev
<zid>
and I was making fun of xp
<zid>
if it did what you said that'd be WAY smarter
* geist
smacks you with a wet trout
netbsduser has quit [Ping timeout: 255 seconds]
eddof13 has joined #osdev
karenw has joined #osdev
FreeFull has quit [Ping timeout: 260 seconds]
<heat>
geist, yes that's kind of how linux swapping works
antranigv has joined #osdev
<geist>
really? thought they'd then have trouble tracking the swapped page in the page table if the original is thee
<heat>
it never blocks, it just removes the page from the page tables (while colecting the D bits, there it actually matters!)
<heat>
the page isn't reclaimed until the writeback completes, it's completely async
<heat>
so if it happens you refault the same anon page, you'll be able to get that same page back
FreeFull has joined #osdev
xenos1984 has quit [Read error: Connection reset by peer]
gildasio has quit [Ping timeout: 260 seconds]
<heat>
note: writeback itself doesn't reclaim the page, it just moves it to the LRU's head
dude12312414 has quit [Quit: THE RAM IS TOO DAMN HIGH]
gildasio has joined #osdev
<heat>
immibis, well I was considering "out of memory" in the page allocator sense (everything is in use for the page cache or anon or what have you)
<heat>
a normal linux system will never do reclaim unless in that situation
<zid>
nikolar: I have developed an inexpensive and bio friendly method for dealing with crane flies getting into my room
<heat>
there's certainly no funny page aging and swapping out proactively, which I believe windows does, for instance
<nikolar>
zid: what is it
<zid>
He's a spider and his name is "fucking hell, is that a dog or a spider, he's huge"
<nikolar>
nice
<nikolar>
easy to remember
<immibis>
heat: but that situation is the normal situation, or you're wasting the memory you paid for
<immibis>
although free -h tells me i currently have 84Gb of memory that linux can't even find a use for.
<heat>
no, swapping proactively is not wrong if you can make more educated decisions
<immibis>
so i guess that happens when you have a lot of memory and only a few hours ago
<immibis>
rebooted*
eddof13 has quit [Quit: eddof13]
<heat>
vs being in intense memory pressure (with a gun pointed to your head, basically) and no ability to really track what page is more optimally reclaimed
<immibis>
(256GB! at least software bloat hasn't reached the point of needing all of it yet)
<heat>
like, maybe you could proactively swap out most of pid 1 with no trouble (since it's unlikely to get accessed anyway)
<immibis>
it could also be swapped out under memory pressure. that's a quick determination and any LRU probably determines that too.
<heat>
it is not
<immibis>
proactive zswap makes more theoretical sense
<heat>
linux reclaim will _never_ swap out mapped executable pages
<immibis>
really? so all i have to do to cause OOM is map a really big executable?
<heat>
you'll get OOM killed
<immibis>
well obviously
<immibis>
i didn't expect executable pages would be treated differently from any other mapped pages
<heat>
as far as I know there are some hacky heuristics under memory pressure, which include: not swapping out executable pages, not swapping out shared pages
<heat>
it may be that it ignores them if really really desperate, but IIRC that's not the case
<immibis>
and memory pressure is the normal operating condition and reclaim is the normal way to swap things out, if i understand those words?
<immibis>
weird
<heat>
ok it seems to ignore these heuristics sometimes but i can't exactly understand when
xenos1984 has joined #osdev
Dead_Bush_Sanpai has joined #osdev
eddof13 has joined #osdev
<heat>
well in all regards i hope you understand how hacky and fast these decisions need to be. which would not be the case when proactive reclaim is employed
cuppajoeman has quit [Ping timeout: 244 seconds]
Benjojo has quit [Ping timeout: 245 seconds]
<heat>
particularly because e.g a good chunk/most of file pages can be reclaimed with no IO at all
<heat>
and under an SSD doing swap isn't particularly more expensive than reclaiming a dirty file page
jbowen has quit [Ping timeout: 264 seconds]
coolcoder613 has joined #osdev
cuppajoeman has joined #osdev
Benjojo has joined #osdev
jbowen has joined #osdev
Ameisen_ has joined #osdev
Ameisen has quit [Quit: Quitting]
vdamewood has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
eddof13 has quit [Quit: eddof13]
JupiterBig has joined #osdev
eddof13 has joined #osdev
catten has joined #osdev
catten has quit [Client Quit]
catten has joined #osdev
Dead_Bush_Sanpai has quit [Read error: Connection reset by peer]