<mjg>
now i wonder if a nasty user which just jumps to a kernle address would fuck this up
<heat>
how?
<mjg>
they literally jmp $something_in_kernel
<mjg>
but before they crash
<mjg>
yours ampling func finds it
<mjg>
i would say something to worry baout later
<heat>
but they're crashing immediately
<mjg>
if there is no time window for this to get a false positive that's finew ith me
<mjg>
haha wtf, see aix-perl.pl in the repo
<mjg>
holy shit
<mjg>
this has to be SO BAD
<heat>
what's it doing?
* heat
can't read perl
<mjg>
+ foreach my $pid (@proc){
<mjg>
+ my $command = "/usr/bin/procstack $pid";
<mjg>
+ print `$command 2>/dev/null`;
<mjg>
+ }
<mjg>
you don't need to know perl for this one
<mjg>
and this happens in a loop
<heat>
getting stacks for userspace processes?
<mjg>
getting stacks for everything i would think
<mjg>
but this is so many forks and execs it has to be disfiguring the shit out of everything
<mjg>
also how often can you sample
<mjg>
defo not 1000 per second
Persephone has joined #osdev
Iris_Persephone has quit [Ping timeout: 252 seconds]
vdamewood has quit [Read error: Connection reset by peer]
vdamewood has joined #osdev
gog has quit [Ping timeout: 246 seconds]
frkzoid has joined #osdev
srjek has quit [Ping timeout: 244 seconds]
elastic_dog has quit [Ping timeout: 244 seconds]
elastic_dog has joined #osdev
nexalam_ has joined #osdev
Persephone is now known as Iris_Persephone
nexalam__ has quit [Ping timeout: 260 seconds]
Iris_Persephone has quit [Quit: Leaving]
freakazoid332 has joined #osdev
frkzoid has quit [Ping timeout: 244 seconds]
freakazoid332 has quit [Ping timeout: 260 seconds]
epony has quit [Ping timeout: 252 seconds]
saltd has quit [Remote host closed the connection]
saltd has joined #osdev
[itchyjunk] has quit [Remote host closed the connection]
vdamewood has quit [Read error: Connection reset by peer]
vdamewood has joined #osdev
<heat>
mjg, flamegraphs need a stable frequency right?
<heat>
I can't just plop a sampling point
<heat>
yeah that sounds about right
<heat>
...or does it
<heat>
yeah probably
<heat>
else it wouldn't really correlate to time
<kof123>
flamegraphs, like winamp ? j/k...but not really...
<mrvn>
heat: you could divide the amounts by the time interval to normalize it. But the flames wouldn't fade out right unless you tell it the time passed too
<mrvn>
you loose resolution on the flame then when you add samples too slowly.
<bslsk05>
faultlore.com: Compiler Optimizations Are Hard Because They Forget - Faultlore
<kazinsal>
me: man the early research unix filesystem sucks, I can probably design something better off the top of my head
<kazinsal>
also me: [accidentally shits out a design that looks eerily close to the minix filesystem]
<clever>
> And really at what point does the cost outweigh the benefit and aaAAaaAAAaAAAAA!!!!!
<clever>
lol :D
<MelMalik>
kazinsal, hehe
<zid>
I already designed the best possible filesystem, dw
<sham1>
kazinsal: thus is the hubris
<MelMalik>
you probably can, though, just don't think that you can
<kazinsal>
the research unix fs(5) optimizes for fast search of free block and inode lists through the use of a block list in the superblock + additional indirect blocks whereas the minix fs(5) -- and mine, apparently -- just allocates a contiguous block bitmap on disk and uses the contiguous list of inodes themselves to signal free/used status
<MelMalik>
i wish you great luck
<gog>
meow
* kazinsal
gives gog headpats
* gog
prrr
<gog>
i'm doing inadvisble stuff with c++ and it's fun
<gog>
namely, using c++ at all
<clever>
kazinsal: zfs doesnt store the actual state of the free space map, but rather has a log, where every allocation/free is recorded
<clever>
and when loading a spacemap, it creates a set of buckets, with holes of 2^n long in ram
<kazinsal>
this filesystem is intended to run on an 8088 lol
vdamewood has quit [Read error: Connection reset by peer]
vdamewood has joined #osdev
StoaPhil has quit [Quit: WeeChat 3.6]
[itchyjunk] has joined #osdev
SpikeHeron has joined #osdev
dude12312414 has joined #osdev
srjek has joined #osdev
demindiro has joined #osdev
dude12312414 has quit [Quit: THE RAM IS TOO DAMN HIGH]
demindiro has quit [Ping timeout: 252 seconds]
epony has quit [Remote host closed the connection]
epony has joined #osdev
<vin>
Is it possible to make writes slower (by injecting some delay) than reads to memory? For example memcpy(region2, region1, size) should be 2x faster than memcpy(region1, region2, size).
<sortie>
vin, there's some caching options available e.g. through the Page Attribute Table on x86, which may be interesting or useful to you. Can you tell us more about your use case?
<sortie>
E.g. write combing is usually turned on for RAM by default, so writes happen asynchronously in deduplicated combined manners. For video memory, e.g., you would often want to turn it on to make writing to it more efficient. Meanwhile you definitely don't want it for memory mapped registers.
<sortie>
*write combining
<vin>
Thanks I'll check it out sortie. I am trying to emulate a device where reads are faster than writes.
wootehfoot has joined #osdev
wootehfoot has quit [Remote host closed the connection]
wootehfoot has joined #osdev
<vin>
Actually its not the latency that I want to change but rather the bandwidth. Where write latency should be worse of than read.
<vin>
I am thinking of running parallel workloads to keep the memory controller when write operations are happening (inducing the slow down)
frkazoid333 has joined #osdev
<sortie>
If you turn off write combining, you might slow them down considerably
<sortie>
But idk feels a bit futile
<jimbzy>
Hey sortie!
<jimbzy>
How have you been?
<sortie>
eyyo jimbzy!
<sortie>
Oh man been out there traveling the world, partying to the best music :)
<jimbzy>
Nice!
<jimbzy>
Sounds like time well spent
<sortie>
Plus you know working a bunch and even doing some osdev :)
<sortie>
I just finished up my new init system, going through code review now!
<sortie>
Only took me like half a decade to finish it
<jimbzy>
Sweet!
<jimbzy>
Hey, it's just sands in the hourglass, amiright?
<sortie>
Wait who told you that's privileged information!
<jimbzy>
XD
gxt has quit [Ping timeout: 258 seconds]
xenos1984 has quit [Ping timeout: 246 seconds]
xenos1984 has joined #osdev
wootehfoot has quit [Ping timeout: 246 seconds]
<geist>
woot
<gog>
woot
xenos1984 has quit [Ping timeout: 246 seconds]
<zid>
..w-woot?
<zid>
I am like the dog wagging its tail when the humans get good news
<zid>
just happy to be included
<GeDaMo>
I bought some RAM, you can 'woot' about that if you want :P
<zid>
yay, did you buy me any?
<GeDaMo>
Certainly didn't
<zid>
I need 2x8GB PC3-14900
<zid>
I don't have any paypal dollars atm
<zid>
and amazon wants triple the price
wootehfoot has joined #osdev
<zid>
GeDaMo: you're supposed to brag about what you got
<GeDaMo>
I bought 16GB, now I have 20GB! :P
<zid>
sick
<zid>
dual channel is just a scam by big ram to sell you more dimms afterall
<gog>
i added 8 to my machine last week
<zid>
I'll trade you 32GB of DDR4 for 4x8GB DDR3 41866MHz URDIMMs
<gog>
i don't have
<zid>
1866*
<zid>
yea nobody does unless they happen to own a mac pro 2013
<mjg>
re sched_is_preemption_disabled, i wonder if the func is fine, but you are just calling it a metric fuckton
<mjg>
also why does it exist to begin with
<mjg>
as in the code should know
<mjg>
hm now that said it, are you sure you are allocating per-cpu memory in a manner which avoids cache bouncing with other threads?
<mjg>
bare minimum you want per-cpu bufs with sizes rounded up to a multiple of 128
<mjg>
oh man you use linker sets for this fuckery?
<mjg>
i would argue waste of cpu time
<mjg>
[not that freebsd is better :]]
<mjg>
heat: ok man, your cred code needs a revamp
<mjg>
heat: stock standard approach is to have copy-on-write objs
<mjg>
heat: your rw lock around them is a serious problem
<mjg>
heat: well will be once you fix the current stuff :)
<zid>
we don't take kindly to people who know what they're talking about mjg
demindiro has quit [Ping timeout: 252 seconds]
<mrvn>
Is there something worse than ordering food online, getting a ETA with count down clock, confirmation that the food is on the way and yet nothing arrives. Do they just fake all the status updates, seriously?
<gog>
grubhub?
<mrvn>
lieferando. They even have a marker on the map where the delivery bike is supposed to be.
<gog>
hm
<gog>
they might be faking it
<mrvn>
Oh, the status has changed: "Moaz ist mit Deiner Bestellung bei Liki Burger auf dem Weg." Was just confirmed to be on the way before.
<gog>
auf dem Weg
<gog>
possible that the driver jumped the gun on confirming pickup
<gog>
or bicyclist
<mrvn>
now the bike is moving....
<mrvn>
If only the food where free if they take too long to deliver.
bauen1 has joined #osdev
freakazoid332 has joined #osdev
C-Man has joined #osdev
gxt has joined #osdev
freakazoid332 has quit [Ping timeout: 244 seconds]
<geist>
yeah and honstly.. i dunno. there's mmore to life than speed
<vin>
sortie: Replying to you late. Wouldn't dissabling write combining make all writes slower. Anychance writes to a particular memory region can be made slower?
nexalam__ has joined #osdev
<geist>
everyone has their thing, but performane performance prformance scaling, etc is not everything
<vin>
For context: I want to limit just the write bandwidth made to far numa node memory.
nexalam_ has quit [Ping timeout: 246 seconds]
<mjg>
geist: well enjoy your openbsd manuals in a vm man :)
* geist
shrugs
<geist>
just sayin
<mjg>
i have my kinks, you have yours
<mjg>
look i'm happy to shut about solaris if it is seen as a problem
<mjg>
believe it or not :->
<geist>
well anyway
<geist>
it's late!
<heat>
noooo
<heat>
i like this channel's diversity
<geist>
okay, sorry never mind
<heat>
what have you been up to geist?
<geist>
oh just general home maintenance, etc
<geist>
thinking of doing some of FS hackery here in a sec
<heat>
nice nice
<heat>
still fat32?
<geist>
yeah gotta finish it up
<geist>
hate write wrorking the other day, now have to wire up all the remaining ops, run some stress tests and declare it v1
<bslsk05>
github.com: fstools/fsx.c at master · apple/fstools · GitHub
<geist>
i remember there being basically zero stress tests, so i started building a thing and then handed it off when they pulled me into iphone
<heat>
also you were on the fs team?
<heat>
you've been everywhere! :P
<vin>
FAT32 did not have a driver? I am trying to understand the purpose behind the driver
<geist>
ah no that predates it
<heat>
"gcc -arch ppc -arch i386 -arch ppc64 -arch x86_64" what are these -arch switches?
<heat>
are they for gcc-disguised clang?
<geist>
ah actually yeah i did have a bit to do with this
<geist>
i was there are 2005, write some standalone tool that did something similar. looking at the history of it they rolled some of the machinery into this in 2006
<geist>
yeah this is sort of a distant derivative of some stress test code i wrote at the time
<heat>
very cool
<geist>
well portions of
<geist>
most of it predates it, if nothing else because the style is not mine at all
<mjg>
heh solid
<mjg>
well your old code can come back to haunt you in a new way then
<heat>
lk isn't posix
<heat>
might be hard to do
<geist>
but basicall what i wrote is fairky simple. had somehting like it at Be: spawn a crapton of threads, each thread goes and does a bunch of random stuff, run until something falls over
<geist>
oh gosh no am i using any apple code in my projects
<geist>
*that* is a stress test. try to find all the edge conditions
<heat>
:D
<geist>
i remember writing this tool in 2005 and it would almost instantly fatally corrupt HFS+. a bug was found and fixed
<geist>
i remember the tseting folks were like 'oh this is great!' and they added it to their stuff
<geist>
but it's the usual issue i have with unit testing vs stress testing. lots of places uit test, but dont stress test, because the latter takes time and has harder to define end states
nexalam__ has quit [Quit: Leaving]
<heat>
tbf fsx.c is widely considered more of a unit-test thing these days
<geist>
it's easy to run a unit test every time someone makes a CL, but much harder to say run a bank of machines, beating up on the software, trying to find edge cases
<mjg>
fwiw freebsd has a stress testing machinery and it is pretty good
<geist>
yeah. what i like to look for in a FS implementation is all the internal locking issues, and pushing all of the ops to MT collisions, etc
<mjg>
wait, that's a stale repo
<mjg>
what's the current one
<geist>
also generating crazy lomg journal transations that blow up internals
<geist>
rename() in particular is hell
<heat>
mjg, seems to have been frozen, I don't know
isaacwoods has quit [Quit: WeeChat 3.6]
<geist>
ie, via a crapton of random stuff end up with huge, incredibly fragmented files, then simultaneously rename A onto B onto C while C is renamed into A, etc
<geist>
that sort of stuff finds all sorts of edge cases and failure cases
<mjg>
ye, i even linked to [redacted] system deadlocking with it
<heat>
[TOP SECRET]
<geist>
anyway, that kinda stuff you can't easily unit test, since it relies on lots of heavily threaded slams on a machine
<geist>
anyway yeah i should try to port one of these things to LK, or at least take inspiration. passing that is what i'd consider a V1 release of a FS
DanDan has joined #osdev
<heat>
mjg, haven't tried yet
<heat>
i'm tackling the budget-ass wait tracing rn
<mjg>
+1
<heat>
i can't stress how much I love looking at flamegraphs and tracing shit
<heat>
it just looks soo good
<mjg>
my man!
<heat>
these svgs are particularly satisfying
<heat>
you can even click shit
<vin>
+1
<mjg>
this is not a safe space for such claims though
<mjg>
heat: you should read up on brendan gregg's blog then
<mjg>
in particular there graphs for sleep and wakeup stuff
<mjg>
may i also recommend gil tene talking about latency
<vin>
I am reading his book right now, so much of what he says is something I want to follow "measure and then build"
<heat>
booooring
<heat>
build and then measure 3 years later
<heat>
or in my case, a solid 7 years
<vin>
Haha
<mjg>
overpromise and under deliver
<mjg>
work for a corp for 1 year and you will know what i mean
<vin>
My research has been mostly studying things carefully rather than coming up with some idea and hammiring it all the places
SpikeHeron has quit [Quit: WeeChat 3.6]
<vin>
*hammering
<vin>
Also, regarding my previous question. Is it possible to make just writes to far NUMA memory slow? I have a tmpfs on the other numa node and would like to make all writes to a mmaped region slow by some delay.
SpikeHeron has joined #osdev
dude12312414 has joined #osdev
<vin>
I really want to mimic assymetric read write bandwidth of a device. Any thoughts?
<mjg>
if you strictly control everything, i would just patch the kernel to add an artificial delay
dude12312414 has quit [Remote host closed the connection]
<heat>
that seems... complicated
<vin>
where exactly will I add this delay mjg
<heat>
just add that in user space
<mjg>
provided all the writes happen through write(2) et al, somewhere in tmpfs write
<heat>
tmpfs write isn't a thing
<mjg>
you spot you are in the "screw with it" area and just artificially wait
<mjg>
things get harder if you mmap
<heat>
rather tmpfs_writepages (but it got renamed now, it's readahead iirc?)
<heat>
and also that's useless, writepages is only called when flushing
<heat>
and we'd need to go into the details of linux vfs and... yeah
<heat>
fuck that
<vin>
I would love to just use "wait" in userspace but the thing is I am calling a library that move data around in this mmmaped region.
<vin>
There are other places where I memcpy from a malloced area to the mmaped region where I can add a wait to induce this delay
<mjg>
hm
<mjg>
is this something you need to execute on bar metal?
<mjg>
if you were playing valgrind you could inject delays at that level
<vin>
mjg: Unfortunately yes. I am doing benchmakrs on how something would work on future byte addressable storage devices.
<mjg>
welp good luck :)
<vin>
haha, I will figure something out. Thanks for the discourse :)