[itchyjunk] has quit [Remote host closed the connection]
<klange>
Thankfully, it looks like the refcount thing might be a red herring and this is just good old use-after-free on some memory pages somewhere - likely around thread destruction.
<Jari-->
2 gigs ought to be enough for everyone I want to say now
<geist>
https://echoesbrewing.com/ is the local brewery i just 'discovered' just over the bridge in Poulsbo, WA. if nothing else they have cool looking cans
<bslsk05>
echoesbrewing.com: Brewery - Echoes Brewing Company
<kazinsal>
oh nice
scoobydoo has quit [Ping timeout: 246 seconds]
bgs has quit [Remote host closed the connection]
scoobydoo has joined #osdev
<geist>
ah cute, i see. it's a new brewery formed when a previous brewery (sound brewing) closed up in 2019
<geist>
so see, echos. cute.
<kazinsal>
clever
<geist>
speaking of once again the fan on my rpi4 has started to make a racket
<geist>
when i do, i raise my fist in the air and cry 'clever!!!!!'
<geist>
it's all their faulr!
<kazinsal>
the perils of making your irc nick a common english word
<geist>
fault even
<geist>
yeah i bet clever gets a lot of heat for that
<geist>
(oh heat isn't logged in. probably wise!)
<kazinsal>
*rimshot*
hmmmm has quit [Read error: Connection reset by peer]
Burgundy has joined #osdev
rorx has quit [Read error: Software caused connection abort]
Reinhilde is now known as Ellenor
* kof123
i dont always make clever meta jokes, but when i do, echoes brewery provides the heat
* kof123
steve jerbs mashup stay thirsty and hungry
rorx has joined #osdev
lanodan has quit [Read error: Software caused connection abort]
lanodan has joined #osdev
node1 has quit [Ping timeout: 252 seconds]
Burgundy has left #osdev [#osdev]
vdamewood has quit [Quit: My MacBook Pro has gone to sleep. ZZZzzz…]
yuiyukihira has joined #osdev
<geist>
okay that's nice. i did as i mentioned to heat earlier and fixed the acpi code in LK to run late in the boot process (already did) and stop relying on the physmap, which causes problems with high addresses and i386
<geist>
i have to say it's a little annoying though the way acpi has all these pointers to tables, with no way to determine how big the table is without inspecting it
<geist>
so if you're mapping an acpi table into memory with the mmu, you have to either map a large chunk (say 64K) and hope it's enough, or map the first page, inspect the header to see how big it is, and then if it's not big enough map it again
scoobydoob has joined #osdev
zaquest has quit [Remote host closed the connection]
scoobydoo has quit [Ping timeout: 252 seconds]
scoobydoob is now known as scoobydoo
nyah has joined #osdev
mimmy has quit [Read error: Software caused connection abort]
mimmy has joined #osdev
sikkiladho has joined #osdev
smach has quit [Read error: Connection reset by peer]
flx- has joined #osdev
flx has quit [Read error: Connection reset by peer]
rorx has quit [Ping timeout: 252 seconds]
rorx has joined #osdev
gxt has quit [*.net *.split]
wand has quit [*.net *.split]
Vercas6 has quit [*.net *.split]
gildasio has quit [*.net *.split]
marshmallow has joined #osdev
GeDaMo has joined #osdev
bauen1 has quit [Ping timeout: 248 seconds]
lkurusa has joined #osdev
lkurusa has quit [Client Quit]
lkurusa has joined #osdev
bauen1 has joined #osdev
sikkiladho has quit [Quit: Connection closed for inactivity]
Rubikoid has quit [Read error: Software caused connection abort]
Rubikoid has joined #osdev
awita has joined #osdev
catern has quit [Remote host closed the connection]
FireFly has quit [Ping timeout: 255 seconds]
lkurusa has quit [Quit: I probably fell asleep (or went out). Who will ever know.]
FireFly has joined #osdev
bauen1 has quit [Ping timeout: 252 seconds]
bauen1 has joined #osdev
aejsmith has quit [Read error: Software caused connection abort]
awita has quit [Ping timeout: 246 seconds]
aejsmith has joined #osdev
catern has joined #osdev
dormito has quit [Read error: Software caused connection abort]
[itchyjunk] has joined #osdev
lkurusa has joined #osdev
dormito has joined #osdev
gxt has joined #osdev
gildasio has joined #osdev
wand has joined #osdev
dude12312414 has joined #osdev
sauce has quit [Read error: Software caused connection abort]
sauce has joined #osdev
srjek has joined #osdev
roan has joined #osdev
eroux has quit [Ping timeout: 272 seconds]
dude12312414 has quit [Quit: THE RAM IS TOO DAMN HIGH]
eroux has joined #osdev
eroux has quit [Remote host closed the connection]
zaquest has joined #osdev
Ram-Z has quit [Read error: Software caused connection abort]
Ram-Z has joined #osdev
roan has quit [Read error: Connection reset by peer]
bauen1 has quit [Ping timeout: 255 seconds]
bgs has joined #osdev
rorx has quit [Ping timeout: 255 seconds]
heat_ has joined #osdev
heat_ is now known as heat
<heat>
geist, sure you can. map it for the minimum length and then remap it with the whole size
<heat>
I was just checking and it's what ACPICA does
<heat>
praise be acpica best code!!1!11!11
<heat>
tbf it flawlessly runs with KASAN, which is fairly impressive
<zid>
map 8k so that it can straddle a page, then error if it's larger than 4096 bytes with ERROR: BAD BIOS
rorx has joined #osdev
dennis95 has quit [Read error: Software caused connection abort]
dennis95 has joined #osdev
bauen1 has joined #osdev
<heat>
i've just learnt that /bin/acpisrc does indeed work
<heat>
I'm probably going to use it for my next acpica update
<heat>
acpisrc -l linuxifies an acpica source
<heat>
which makes things like AcpiHwProcessPciList -> acpi_hw_process_pci_list
zhiayang has quit [Quit: oof.]
zhiayang has joined #osdev
koolazer has quit [Read error: Software caused connection abort]
nick64 has joined #osdev
<nick64>
C standard library functions like malloc() gives the userland process space for storing variables, right? Does it first allocate a virtual memory range to be used (or obtain a range from the free bins) and *THEN* ask the kernel to map the range to available physical frame(s), or obtain physical memory frame(s) from the kernel and ask the kernel for a virtual address that can be used for contiguous access on those frames?
koolazer has joined #osdev
awita has joined #osdev
<zid>
The OS does the virtual memory stuff, as C doesn't know what virtual memory *is*
<GeDaMo>
malloc uses mmap to get pages then allocates space within those
smach has joined #osdev
<nick64>
GeDaMo: the "allocate" word you used in the second part of the "then" is not actual memory allocation, but rather a high level abstraction over an already mapped and allocated (or allocated and mapped?) memory range, right?
Ellenor has quit [Read error: Software caused connection abort]
<GeDaMo>
It uses some of the space in the memory pages it got from mmap to fulfill the reqested amount
<heat>
allocate - distribute (resources or duties) for a particular purpose.
<nick64>
heat: the "allocate" you just mentioned is something internally happening within mmap (and not within malloc after returning from mmap) right?
<heat>
it happens everywhere
<GeDaMo>
Allocation happens at multiple levels
<heat>
malloc allocates within what it mmaped, mmap allocates within your address space
<heat>
(and then within your physical memory)
<nick64>
I mean, whatever malloc is "allocating" was in fact already allocated to the process ages ago, just than the malloc's "allocator" chose not to release that for the userland code (eg. a C code) right?
<GeDaMo>
Your program didn't ask for it
<GeDaMo>
If you say malloc(10), malloc asks for a whole page (likely 4K) from the kernel then returns a pointer to 10 bytes in that page
<nick64>
That happens only under memory pressure right?
<GeDaMo>
If you then ask for another 10 bytes, there's still space in that page
<GeDaMo>
It depends how much your program asks for
<heat>
the criteria your malloc uses to allocate and release from the OS is malloc-dependent
<GeDaMo>
You can try asking for memory than you have physical RAM if you want, how successful you are may vary :P
<heat>
in general system malloc's shouldn't be too memory hungry
<GeDaMo>
The kernel can allocate pages in the process page table but not map them to physical memory until you try accessing them
<nick64>
Okay, so mmap does only mapping, and there is no concept of "allocation" in mmap, and that happens only if malloc (or any other allocator like malloc) is involved outside the scope of mmap?
<GeDaMo>
mmap still has to allocate pages from the process address space
<nick64>
That is where I am confused
<heat>
<heat> allocate - distribute (resources or duties) for a particular purpose.
Ellenor has joined #osdev
<heat>
you can need to allocate within multiple layers
<nick64>
"mmap still has to allocate pages from the process address space" : If that is what is allocation, then what is mapping?
<GeDaMo>
mapping can also have multiple meanings
<clever>
mmap has to allocate 2 things, the virtual range within the process, the actual physical pages (which can be lazy), then it has to map those physical pages to the chosen virtual range
<GeDaMo>
You ask mmap for a single page, it needs to place that somewhere in the process address space
<heat>
dude what the fuck are you on about
<heat>
you allocate AND map
<nick64>
Can an mmap call result in an address that is mapped but not allocated?
<heat>
no
<clever>
and if its being lazy, it can just skip the physical allocations, and do them later
<heat>
clever, pls dont
<heat>
that's too much complexity
<heat>
i'll redirect you to <heat> allocate - distribute (resources or duties) for a particular purpose.
<nick64>
I think I know what clever is referring to, but if I pursue that right now my brain will explode (assuming it is about overcommitted alloc)
<heat>
yes
<nick64>
Okay, so is there any datastructure in kernel that keeps track of what pages are allocated? (Let us keep mapping out of the picture now)
<heat>
yes
<clever>
if you ignore overcommit then its simple, allocate a contiguous range in the virtual space, allocate some physical pages, insert those pages into the paging tables so they are now mapped
<nick64>
What would that be? Like we have PD and PT for tracking mapping, what do we have for tracking allocations?
<heat>
the PD and PT dont track mappings
Burgundy has joined #osdev
<clever>
there is likely going to be a physical memory allocator, that tracks what pages are in use, and gives you a list of pages when you ask it to allocate 10 pages
<zid>
They *are* the mappings, but you can walk them manually to determine whether a virtual address is in use already or not
<nick64>
Physical and allocator in one sentence? I think I am confused
<nick64>
We allocate virtual pages right?
<clever>
both
<heat>
you allocate EVERYTHING
<zid>
You allocate both
<heat>
<heat> allocate - distribute (resources or duties) for a particular purpose.
<clever>
2022-11-02 14:01:32 < clever> if you ignore overcommit then its simple, allocate a contiguous range in the virtual space, allocate some physical pages, insert those pages into the paging tables so they are now mapped
<nick64>
Okay heat, really feeling dumb now lol
<clever>
allocate virtual and allocate physical, then map the virtual to the physical
<zid>
Virtual memory is a full range integer, that maps to another full range of integers, that's it.
<heat>
nick64, you keep not reading that sentence
<GeDaMo>
You seem to be hung up on the word 'allocate' :|
<heat>
malloc maps to an mmap segment which maps to physical memory
<heat>
(malloc'd chunks, that is)
<nick64>
Page Table is not for tracking mappings?
<zid>
Page tables *are* the mappings.
<zid>
>They *are* the mappings, but you can walk them manually to determine whether a virtual address is in use already or not
<heat>
so you need to allocate from physical memory to map to an mmap segment (which needs to allocate some virtual space) so malloc can partition that space
<zid>
Or you can track it elsewhere in a different style of tree, up to you
<GeDaMo>
Each process has its own page table which maps from the process virtual address space to physical memory
bauen1 has quit [Ping timeout: 252 seconds]
<nick64>
Ins't "allocating from physical memory" same as mapping? Because you can't really "reserve" a physical frame without mapping it to a virtual page. Or can you? :o
<clever>
you can reserve a physical addr without mapping it
<heat>
mmap does virtual memory allocation, page_alloc does physical, malloc does allocation within virtual memory chunks it asks from the OS (through mmap or brk)
<nick64>
page_alloc allocates a virtual address without necessarily mapping it to a virtual address?
<nick64>
err. physical
<heat>
virtual memory has a lot more trickery so it isn't strictly "mmap gets a chunk of free virtual address space and gets a chunk of physical memory"
<heat>
yes
<nick64>
page_alloc allocates a physical address without necessarily mapping it to a virtual address?
<nick64>
okay
<heat>
mmap allocates virtual addressings without necessarily mapping it to a virtual address
<nick64>
So what does that "allocated" but not mapped physical address mean? What is that allocation attributed to if not a process? Why does someone do that?
<heat>
s/addressings/addresses/
<heat>
mapping = making the CPU know virtual X -> phys Y
<heat>
remotely modern operating systems map lazily, because it's more efficient
<zid>
It may have been more useful to start at 'how this works without an MMU' first, then build up
<zid>
sounds like you're trying to digest 3 steps at the same time
awita has quit [Remote host closed the connection]
<heat>
nick64, ah wait
<heat>
page allocation doesn't need virtual addresses
awita has joined #osdev
<heat>
if you have a range of memory exclusively for DMA for instance
<heat>
page allocation is completely decoupled from the notion of virtual memory except for that bit that virtual memory maps pages
<clever>
dma is also special in that you may need contiguous physical pages
bauen1 has quit [Ping timeout: 248 seconds]
<nick64>
"page allocation doesn't need virtual addresses" heat: I believe you are referring to DMA_ATTR_NO_KERNEL_MAPPING
<heat>
no clue what that is
bauen1 has joined #osdev
<nick64>
A "page" is a range of virtual addresses right?
<heat>
no, i'm referring to alloc_pages
<heat>
no
<heat>
it refers to BOTH virtual addresses and physical addresses
<heat>
confusingly!
<nick64>
Okay, I was calling to range of virts as page and range of phys as frame
<heat>
usually you're talking about physical pages unless the context suggests virtual
<clever>
a page is just a 4096 byte chunk of memory, either virtual or physical
<heat>
linux's page allocator does NOT manage virtual pages
dude12312414 has joined #osdev
<nick64>
I think now I get why I found lot of docs confusing, I was assuming "page" to mean virtual
<clever>
bug page is more just a size thing
<clever>
but*
<heat>
if you read the osdev wiki, they try to really distinguish between page and page frame
<heat>
and that difference really isn't there in the real world
<nick64>
Yeah have been following osdev wiki mostly
<nick64>
That's where I got the idea that page is virt and frame is phys. Wish other docs followed the same
<heat>
<controversial>the osdev wiki is a bad resource</controversial>
<GeDaMo>
You should put that on a sandwich board :P
<heat>
i genuinely do think that the osdev wiki tends to be written by newbs
<heat>
it's the traditional "oh, I know a lot about this, lets write a tutorial"
<GeDaMo>
"I tried random things until it worked, I am now an expert"
<mjg_>
heat: there is a misconeption out there that the best way to learn something is to 1. do it 2. document
<GeDaMo>
Documenting your journey can be helpful
xenos1984 has quit [Ping timeout: 246 seconds]
<clever>
GeDaMo: that is basically what ive been doing with the pi, try random things, document as i go, both to help others and remind myself how it works
<heat>
i believe that teaching is a great way of learning, but teaching is hard when you learned about the subject a month ago
bgs has quit [Ping timeout: 252 seconds]
<heat>
particularly in an authoritative voice like the wiki does
xenos1984 has joined #osdev
<clever>
heat: when i was in highschool, the english teacher was also in charge of teaching students how to use computers
<nick64>
So that's what a sandwich board means :P
<clever>
heat: he didnt even know the basic design themes of windows, that let you solve a problem in any program
<clever>
the whole MDI style of gui's
<clever>
(which MS killed off later on)
<clever>
one student made a new powerpoint project for every single slide, and the teacher couldnt fix it
<heat>
great idea
<clever>
i walk over, 20 seconds of poking around, and find you can drag&drop slides between projects
* kof123
sits at table outdoors, grabs coffee, puts up sign "it will be recreated in-browser. change my mind"
<clever>
i had never used powerpoint before then
bgs has joined #osdev
<heat>
lol
<mjg_>
dude
<mjg_>
i was in "advanced math and computer science" profile in high school
<mjg_>
first time we sat to computers the teacher said everyone needs to create their own dir in the shared space
<bslsk05>
wiki.osdev.org: Detecting CPU Speed - OSDev Wiki
<heat>
oh boy
<mjg_>
so you right-click, create a new dir
<mjg_>
for some reason people were unable to rename the dir they fucking created
<clever>
that was fully automated at my HS, a dir named after $USER was pre-mounted as a shared disk, and walled off so only you can access it
<mjg_>
instead they kept renaming other stuff
<clever>
and that let you access your files from any school pc
<clever>
mjg_: race condition?
<heat>
i just wanna note that that whole article is completely wrong
<clever>
heat: dang, because thats exactly something ive wanted to do, lol
<mjg_>
clever: no, idiots clicking rneame on dirs they did not create
<heat>
clever, modern x86 CPUs have MSRs to give you the current, correct frequency
<heat>
APERF and MPERF
<clever>
mjg_: ah, but last i saw, windows automatically goes into the rename prompt when you make a new dir
<heat>
idk how arm does it
<clever>
heat: arm is a wild-west of custom stuff, for the cpu clock itself
<mjg_>
clever: that was 2002
<clever>
the arm timer has its own dedicated reg, that tells you what clock it should be running at, but the register has no actual effect on anything
<clever>
its basically just a gp reg, that the rules say should hold the clock freq
dude12312414 has quit [Ping timeout: 255 seconds]
<clever>
linux will even divide by zero if you dont populate it :P
<nick64>
mjg_: my school had the very same stuff. A directory for each student, with a public_html folder. I did not upload any HTML files there. I certainly did not upload any PHP file there. I certainly did not get code execution as apache user. I certainly did not find a daily cron that is executed as root. I certainly did not find any shell script that the cron invokes which is Unix world writable
<mjg_>
nick64: dawg
<clever>
lol
<heat>
you should not admit to that stuff
<heat>
it's a crime
<clever>
and i never ran `net send * hello` repeatedly :P
<mjg_>
heat: but he said he did not!
<nick64>
I certainly did not do it
<mjg_>
right call man
<clever>
i also did not discover that every school computer in the province shared a bloody broadcast domain!
<mjg_>
one of the reason i went to that hs was me being too young to know better
<heat>
did you also use GNU nano back then
<mjg_>
i'm from a shit small town which only had 2 high schools considered good, the rest was utter crap
<mjg_>
and i believed the claim that the IT dude from the hs i went to is FUCKING GREAT BRO YOU GONNA LEARN A LOT
<zid>
I like nano
<mjg_>
turns out he did not know shit, as you can imagine
<mjg_>
heat: GNU emacs, true story
<heat>
mjg_ best 10yr old freebsd hacker smartest person ever
<mjg_>
i remembered being hyped about writing my first prog in emacs
<clever>
heat: for measuring cpu speed, ive mostly been assuming the clock divisors are setup the way i expect them to, run a tight assembly loop a million times, and then check a 1mhz timer to see how many ticks elapsed
<clever>
heat: then add a single opcode to that, and see how the loop time changes, and verify its increasing by an integer number of clock cycles
<heat>
clever, but that's tricky to get right and you get a poor man's bogomips
<zid>
if someone told me I had to do all my programming in nano I'd be like "oh okay"
<clever>
yeah
<heat>
which helps you in... almost nothing
<clever>
with time, i can profile how many clocks each opcode takes
<clever>
and then i could do the reverse, measure how long a batch of opcodes take, and compute the clock it was running at
<heat>
nand2tetris but with editors
<zid>
how many bogomips does nano neeed
<heat>
ed2vscode
<clever>
but stalls ruin my plan entirely
<mjg_>
i have to concede i have no idea how to *measure* the clock and i don't know if you can relaiably do it
<heat>
mjg_, x86 can with the APERF and MPERF MSRs
<zid>
oh btw does anybody know why jiffies in linux is 2000 and not 1000, it's just.. double for no real reason
<heat>
with instructions? probably not
<clever>
but a lot of what i just said, assumes your cpu isnt pipelined too heavily
<mjg_>
heat: you mentioned the msrs, i meant without them
<zid>
so proc cpuinfo's bogomips calc is just your_ghz * 2000, rather than being your_mhz
<clever>
i expect that idea to entirely fall apart on a modern cpu
<mjg_>
clever: that's precisely why
<mjg_>
you do an op... or 3 ops... and it all takes a cycle
<mjg_>
except if you do it in a loop where it may be less or more
<zid>
we should just go back to using microprocessors
<zid>
instead of fancyprocessors
<mjg_>
and that's even assuming there is no turbo boost
<clever>
mjg_: the VPU ive been testing with is dual-issue on the scalar side, so yeah, certain combinations of opcodes can run in the same clock
<clever>
so part of what i need to do, is learn the rules of that, and account for them
<zid>
I have icache so.. things run in random amounts of time regardless
<heat>
you could totally warm up turbo boost
<clever>
its single-issue on the vector side, but once you initiate a vector opcode, you can go off and do up to 2 scalr opcodes per clock, in parallel to the vector opcode running
<zid>
you hope
<zid>
turbo has a tau, but it's software configurable
<zid>
so how long do you need to keep the cpu busy before it upclocks?
<clever>
fun fact, that turbo switch was actually for slowing the cpu, not speeding it up!
<zid>
correct
<clever>
but any such boosting and dynamic clocks, are fully under my control on the VPU
<zid>
runtime configurable clockstrap basically
<mjg_>
ah goold old times when cpu clock was at the front of the case
<zid>
rather than fucking with jumpers
<mjg_>
also when you could not just have a sound card
<clever>
i had one case, with a 7-seg display, showing the current clock
<clever>
but it was super dumb
<mjg_>
but instead had to fuck with irq numbers and so on
<zid>
Get yourself a nice 486 mobo with 24 jumpers
<clever>
you used a crap-ton of jumpers, to define on a PER SEGMENT basis, if its on always, on at slow, or on at fast
<clever>
and then the turbo switch toggles it between those states
<clever>
so the display didnt have to match the actual clock rate
<mrvn>
clever: opcodes need a certain number of ticks to get parsed, scheduled, executed, commited. Not all of them being multiples of a clock cycle.
<mrvn>
clever: adding an opcode might add 1 cycle per loop. Adding it again might not change anything.
<clever>
yeah, thats part of what i have to fully characterize
<clever>
turbines, generators, transformers, power grid, all that fun stuff
<clever>
and ive spent hours helping the server reboot the grid after a blackout
<zid>
If your repair doesn't involve "all of the liquid nitrogen in the country" are you really trying?
<clever>
lol
<geist>
heat: yeah re: minimum to max. guess i'll add that logic too
<clever>
if you disconnect a generator from the grid, you can easily get it up to speed, and maintaining voltage
<clever>
but if you then plug it into the grid, it draws enough amps to blow breakers
<zid>
rebooting factorio is fun, in a similar vein
<clever>
but the factorio power grid doesnt catch on fire when you reboot it :P
<zid>
especially in mods where water sources need power
<clever>
the minecraft one does
<zid>
krastorio etc all make pumps use power
<zid>
so rebooting your nuclear is.. impossible
<clever>
if you overload a wire, it will catch fire
<zid>
you have to go scrounge up some shit for some solar panels to power the pump, to get water into the reactors
<clever>
zid: i played a space exploration map recently, it also did that
<zid>
hence the 'etc'
<clever>
i had to make a dedicated solar array to power those pumps
<zid>
it's VERY common outside of vanilla
<zid>
see :P
<clever>
but i also picked the wrong reactor design
<clever>
the reactor was 4x larger then the base itself
<zid>
What's actually cleverer, is to make a one-way valve leading into a storage tank, for your steam
<zid>
then you can uncork it to reboot the system
<clever>
but factorio doesnt have passive check valves
<zid>
it does
<zid>
it's the *only* thing vanilla has
<clever>
it does?
<zid>
yea, 'pump'
<clever>
those need power
<clever>
i said passive :P
<zid>
it is passive
<mrvn>
zid: powering the inserters and pumps for the reactor itself is trivial to do with either solar power or a spare steam tank and turbine.
<zid>
but B&A etc have actual valves which is nice
<mrvn>
zid: pumps without power block steam
<clever>
one major difference between factorio and ELN(the minecraft mod), is what happens during a brownout
<clever>
with factorio, machines just run slower
<zid>
I love the brownout death spiral in factorio though if you're still using coal
<clever>
with ELN, lower voltage means more amps, if you run too many amps thru something, it gets destroyed
<mrvn>
clever: or pause in case of circuit logic
<zid>
inserters and miners run slower, less coal gets into the boilers, inserters and miners run slower, ...
<clever>
so it doesnt run slower, it self-destructs
<mrvn>
brownout spiral of death
<zid>
buy a fuse :p
<clever>
zid: yep, fuse/breaker can prevent that, but then cause more problems on reboot
<clever>
all of the capacitive loads cause a huge surge on reconnection
heat has quit [Read error: Connection reset by peer]
<mrvn>
I'm just writing some code to prevent this problem in real live. In our GUI you can select all nodes of a cluster and then select "Power on". If you do that with all 40 nodes of a rack the fuse just blows.
<clever>
you kind of need to reconnect all generators, while keeping loads disconnected
heat has joined #osdev
<zid>
circuit network in factorio would just make that annoying busiwork :(
<clever>
the more generators you have on the grid, the more interia you have giving your grid stability
<clever>
zid: moonlogic cells, they let you write LUA ingame to do circuit logic stuff
<clever>
the lua code can decide for itself how many ticks between runs
<clever>
so you have to write sensible code, that doesnt run on every tick
<mrvn>
factorio is totally missing triggers for the circuit network LUA interface.
<mrvn>
--> only run the event when the value changes
<zid>
modded factorio is fun until you drop below 60 UPS and find out that the random merger or whatever they include uses 90% of the runtime
<clever>
moon logic does support triggers
<zid>
because they did its logic in lua
<mrvn>
clever: but that is done in LUA, which means you already have the LUA overhead
<clever>
zid: there was a multiplayer map i tried joining years ago, i got 20 UPS whenever i joined it, so i couldnt maintain sync
<zid>
heh
<zid>
I've never actually managed low UPS, but I've watched people play random stuff and have it ahppen
<zid>
helps that I go for high single core perf cpus where possible
<clever>
zid: oh and thats when you get clustorio involved
<mrvn>
clever: multiplayer is a bit stupid there. The server saves the map, it sends it to you and then you havce to catch up to what the game is NOW. If you can't do more UPS than the server then you can't ever join.
<mrvn>
There is no way to make the server pause or slow down during join.
<clever>
zid: ive seen a "server" tour, where they had something like 50 independant factorio servers, sharing items with clustorio
<clever>
just to spread the load and keep the UPS under control
<clever>
they had entire damn servers, dedicated just to mining ore and nothing else
<zid>
heh
<clever>
and with the mining eff upgrades, the miners where dumping ore directly into train wagons
<zid>
I played seablock on a regular map once that was fun
<clever>
and it would fill the entire train in something like a minute
<mrvn>
zid: I have a mod that adds a custom noise function to the map generator to make a fractal map. At start I get 3 UPS till the initial chunks are generated. Exploring causes a noticable lag too.
<clever>
the space exploration pack, adds planets to the game
<zid>
yea I had a save that had gotten TO space
<mrvn>
and space ships
<zid>
but never actually played it
<clever>
each planet has a unique subset of ores
<zid>
watched a guy play a bunch of space though
<zid>
I can't remember which save and set of mods it was
<clever>
i got into the inter-stellar travel phase
<clever>
but then my ram started going flakey and i had to put games on hold
<clever>
linux factorio has a fun feature, async saves
<clever>
it will fork() the whole damn client, and then the child serializes the state to disk, while the parent plays un-interrupted
<clever>
one day, a bitflip corrupted the save routines
<clever>
so the child proc segfaulted
xenos1984 has joined #osdev
<clever>
and now i'm left with a problem, do i keep playing, knowing it cant save? do i just abandon the last 20 minutes of work and restart? ahhhh!
<bslsk05>
github.com: lk-overlay/zfs.svg at master · librerpi/lk-overlay · GitHub
<heat>
nooooo mr ö don't depress
smach has quit [Ping timeout: 252 seconds]
<heat>
found the european zfs: scheiß Dateisystem
<clever>
mrvn: i wrote this diagram to help me understand where to find a given dataset in the tree of objects
smach has joined #osdev
<clever>
each dataset, has its own object set, an array of dnodes
<clever>
and that is basically your inode table
<clever>
but instead of editing the inode table in-place, you store it as just another file in the disk
<kazinsal>
"why is my new laptop taking so long to set up" [inbox contains 24,000 emails] "oh"
<mrvn>
kazinsal: so just an hours worth of new mails?
<heat>
my thunderbird is in a continuous struggle to download emails
<geist>
yah that's like an average tuesday
<clever>
bbl
gildasio has quit [Remote host closed the connection]
<heat>
but srsly i just use gmail
<heat>
my thunderbird only serves as a patch downloader
gildasio has joined #osdev
<heat>
particularly annoying how you can't download emails straight from gmail
<heat>
also annoying: the other tianocore maintainers use an outlook plugin that shits itself on UTF-8, so screw you if you have non-ASCII characters in your name
<GeDaMo>
Is fetchmail still a thing?
<mrvn>
heat: what about +?
<mrvn>
GeDaMo: yes
<zid>
yes.. you can?
<zid>
It has pop3
<geist>
and ima
<geist>
imap
<mrvn>
GeDaMo: getmail is better though
<Ermine>
srlsly using outlook nowadays?
<heat>
Ermine, yeah dude, intel
<heat>
intel is all in on the microsoftium
<geist>
yeah there are tons of companies that use exchange
<mjg_>
oh man
<geist>
it's still one of those silent majority things (though probably more like 30%)
<heat>
although I think they have @linux.intel.com emails for people that collab with linux communities
<heat>
idk what the difference is tbh
<mjg_>
i have no have to deal with outlook for 17 years now
<mjg_>
not
<mjg_>
but when i did, it was such a piece of garbage
<geist>
it's probably much better than i remember. honestly there was a period there, maybe early 2000s, when outlook was a really nice app
<geist>
i seem to remember it got rewritten somewhere along the way and it went downhill, but then i haven't used it since maybe 2005
<Ermine>
Well, looks like there's lack of decent MUAs for Windows
<geist>
i'm probably thinking late 90s. compared to other mail stuff for windows at least, outlook (not outlook express) was a very nice ui and pretty usable
<geist>
competitors were like Eudora or something
<geist>
there was a period there in the early 2000s when it feels like every app windows had got rewritten, probably on top of newer frameworks, and usability took a step back for a while. ex. classic visual studio -> first stab at newer VS
<mjg_>
one common bug was when the mailbox would run out of space
<heat>
see sharp moment
<mjg_>
outlook would et into an infinite loop and fuck you up
<geist>
what i dunno is what most folks use at plaes that are exchange based. do folks ues outlook the app or mostly just the web?
<mjg_>
i would expect if exchange is used, people use something integarted with the calendar et al
<mjg_>
so a big nasty web-based mailer
<heat>
you know, i really like gsuite or google workspace or whatever you call it
<heat>
it Just Works(tm)
<mjg_>
it is ok
<heat>
gchat defaults are kinda annoying sometimes, but that whole thing still works out fine
<geist>
yah i've been using it exclusively for so long i dont really know what other stuff is
<geist>
for better or worse. once i really really learnd all the key commands to gmail i got pretty productive with it
<geist>
the fact that it can be 100% keyboard driven is a win
<mjg_>
i have to note most of the mailing stuff et al targets computer-averse audiences
<heat>
yeah
<heat>
does @google.com play well with plain text emailing?
<Ermine>
web clients became too laggy on my laptop, so I moved to TUI-based clients. I use them through linux VM though.
<mjg_>
marketing, accounting and other folks to whom "this damn machine" is seen as an impediment to get work done any time there is a problem
<mjg_>
Ermine: what's your laptop
<heat>
I've noticed there are plenty of googlers using chromium or kernel.org addresses
<Ermine>
heat: gmail web client allows sending plaintext emails
<geist>
yeah it's a group by group policy. we thought aobut doing the same thing for fuchsia.dev but decided against it
<heat>
i think outlook is also kinda shitty for plain text emailing
<geist>
i think folks that use @chromium said there were some practical downsides to it
<Ermine>
Their UWP mail app can compose HTML-only mail.
<mjg_>
i don't think you can get away with plain text email in corporate environment anymore
<mjg_>
the people which contact you rae using it and automagic conversion likes to *lose* information
<heat>
"we are not affected by all these openssl vulnerabilities because we using boringssl"
<heat>
what's the point here? use boringssl even though they EXPLICITLY say not to use it?
<Ermine>
If google made this post, I would say that this is a nice advertisement.
<heat>
google would not make this post
<heat>
"Although BoringSSL is an open source project, it is not intended for general use, as OpenSSL is. We don't recommend that third parties depend upon it. Doing so is likely to be frustrating because there are no guarantees of API or ABI stability."
<Ermine>
Yeah.
<mjg_>
dude
<mjg_>
is boringgsl even really not affected?
<mjg_>
oh, sorry, crossed wires and confused it with libressl
<heat>
they're both not affected because they forked before 3.0.0
<mjg_>
you know
<heat>
does libressl have a shitton of vulnerabilities no one finds because no one except theo de raadt uses it? probably
<mjg_>
i can make a similar post how one should NOT update
<mjg_>
because an older version is not affected
<mjg_>
heat: right. same arg for boringssl
<heat>
boringssl has a *lot* more usage
<mjg_>
at cloudflare?
<heat>
at google
<heat>
everything google, android, chrome
<mjg_>
huh, ok, i did not know that bit
<heat>
(boringssl is a google project)
<heat>
this is not even an ad, it's just.... pointless
<mjg_>
lack if api/abi guarnatees was very openbsd-esque
<mrvn>
ABI stability isn't a big thing but API changes really suck
<heat>
ABI stability is a big thing
<heat>
it means you can't update the shared library without recompiling everything
<mrvn>
it's not irrelevant but with FOSS you just recompile
* mjg_
laughs in LTS distros
* Ermine
is looking forward for BearSSL
* mjg_
writes his own ssl
<mrvn>
mjg_: in?
<mjg_>
RUST!
<geist>
yah stuff gets recompiled so often at G i can see the ABI stability not being a thing as being not a thing
<Ermine>
They have computing power to do that I guess.
<geist>
yeah there's also basically a policy of recompiling, in order to pick up new compiler stuff, re-run unit tests, etc
<mjg_>
you reminded me that their flagship lang -- go -- recompiles everything anyway
<mjg_>
and is statically linked
<mrvn>
geist: that's what happens when you develope a language that's fast to compile
<mrvn>
go go go go
<geist>
heh at least it's not rust speed
<heat>
rUUUUUUUUUST
<geist>
thought i haven't thought about it, but i dont remember go being particularly slow so yeah i guess it is pretty quick
<geist>
to compile that is
<heat>
sHUT UP RUST RUST UR STUR SU RUST RUST RUSTRUST RUSTRUST RUSTRUST RUSTRUST RUST RUST RUSTRUST RUST RUST RUSTRUST RUST
<geist>
heat: rust.
<mjg_>
DEVELO^WRUST RUST RUST
<heat>
cantrill should snort high amounts of cocaine for the next twitter spaces
<bslsk05>
'Was ist Rust? Rust schnell erklärt!' by RemaKKe (00:02:53)
<heat>
just so we get a RUST RUST UR STUR SU RUST RUST RUSTRUST RUSTRUST RUSTRUST RUSTRUST RUST RUST RUSTRUST RUST RUST RUSTRUST RUST chant
<geist>
so i hear rust rust and then there was rust rust oh an rust rstu rsutursuturustsutstu!
<mjg_>
rust in peace!
<heat>
why isn't illumos written in rust
<mjg_>
dude it totally is
<mjg_>
parts of it
<mjg_>
[well probably not really but the cantrill team likes it]
<heat>
no, it's rustic, not rust
<geist>
solaris is written in rust
<geist>
was back in 1986
<mjg_>
so solaris started as a fork of a 4.3bsd or so, which was already in cobol
<mjg_>
then due to the deal with at&t they took sysv written in rust and rebased
<mjg_>
it is how you got the kernel-level jit cobol compiler in solaris
* mjg_
hits blunt
<heat>
wow thanks for sharing some history
<heat>
fascinating
<geist>
yah after unix v7 switched over to LISP the BSDs went in another direction
<mjg_>
cobol is what gives the unparalled multicore scalability
<mjg_>
geist: they did, but i don't know if php was the right choice....
<geist>
then linux being written in FORTH there was a real fight on their hands
<heat>
wasn't v4 written in forth
<mrvn>
Even the bible said: go forth and multiply.
<mjg_>
no they thought '4th' and 'v4' would lead to confusion
<mjg_>
that's way turbo pascal
<mjg_>
that's why\
<heat>
ah so v3 and v5 were written in forth, but not v4
* mjg_
just went full american
<mjg_>
heat: that's right, yes
<zid>
mjg_: You fired a gun at a schoolkid from your mobility scooter?
<mjg_>
i missed, so what's the harm
<Ermine>
The submarine which explored Titanic had Forth software.
<sham1>
What is this alternative history of OS development
<mjg_>
sham1: earth is FLAT
bgs has quit [Ping timeout: 252 seconds]
bgs has joined #osdev
<zid>
time is a cube
<sham1>
Oh damn it
<mrvn>
zid: follow the inverse linear law?
<zid>
It's nice that we can go back to joking about the earth being flat btw, all the bandwagoners are now qanon
<mjg_>
zid: time is a flat circle
dh` has joined #osdev
<sham1>
As opposed to a non-flat circle?
<mrvn>
mjg_: do you know how much energy it takes to keep the satelites in orbit on a flat earth?
<mrvn>
The teleportation station in greenwich at the dateline is a power hog too
<dh`>
lol
smach has quit []
<sham1>
The point of most of the bandwagoners having gone to QAnon it important. It feels that nowadays conspiracy theories are more "boring" than they used to be
<dh`>
greenwich isn't on the date line anyway
<zid>
greenwich is infact, at 0E
<mjg_>
sham1: true detective reference
<zid>
the observatory anyway
<dh`>
yes, but the date line is at (approximately) 180E
<dh`>
right
<zid>
oh right, date line
<dh`>
oops
<zid>
where le date change
<zid>
date line is fucking screwy these days though
<sham1>
So at Kiribati
<zid>
lots of micronesian islands flipped sides so they could be on US time
<sham1>
Kiribati went the other way, which is why UTC+14 exists and I hate it
<zid>
rather than +23 GMT they went to -1 the day after
<mrvn>
I like +30/+15 timezones
<zid>
+23 is how circles work, yes.
<zid>
I definitely didn't mean +11
bauen1 has quit [Ping timeout: 272 seconds]
MiningMarsh has quit [Ping timeout: 252 seconds]
MiningMarsh has joined #osdev
gog has joined #osdev
ss4 has joined #osdev
wootehfoot has quit [Ping timeout: 252 seconds]
gildasio1 has joined #osdev
gildasio has quit [Ping timeout: 255 seconds]
bauen1 has joined #osdev
bgs has quit [Remote host closed the connection]
awita has quit [Ping timeout: 248 seconds]
bauen1 has quit [Ping timeout: 272 seconds]
bauen1 has joined #osdev
ss4 has quit [Quit: Leaving]
tacco_ has joined #osdev
tacco has quit [Ping timeout: 252 seconds]
bauen1 has quit [Ping timeout: 255 seconds]
bauen1 has joined #osdev
wootehfoot has joined #osdev
bauen1 has quit [Ping timeout: 255 seconds]
bauen1 has joined #osdev
netbsduser has joined #osdev
wand has quit [Write error: Connection reset by peer]
k8yun has joined #osdev
_whitelogger has joined #osdev
GeDaMo has quit [Quit: You are becoming what we French call 'Le Fruitcake'.]
wootehfoot has quit [Read error: Connection reset by peer]
dude12312414 has joined #osdev
Ermine has quit [Read error: Software caused connection abort]
<gog>
hi
Ermine has joined #osdev
<sham1>
ho
<ZipCPU>
hee
<heat>
heehee
srjek has joined #osdev
<sham1>
MJ!?
<heat>
hello it's me, michael jordan
<heat>
let's touchdown some dunks
<heat>
remember kids, don't do drugs, just gamble irresponsably
dude12312414 has quit [Ping timeout: 255 seconds]
<gog>
mfw twitch banned gambling streams
<heat>
booby streamer still unban?
<gog>
idk
<gog>
i don't watch twitch streams
<heat>
same
smach has joined #osdev
<geist>
oh interesting. some color behind why the intel Arc drivers are such a disaster: most of the devs were in .ru
<heat>
oh yeah those instructions seem to be for the all-OSS stuff
<heat>
but with new stuff, at least the kernel part (hence dkms)
<geist>
hmm, okay
Burgundy has left #osdev [#osdev]
Gooberpatrol66 has quit [Quit: Leaving]
<nick64>
heat: you were saying the kernel could allocate physical page frame without creating a mapping, right? What is the name for the datastructure that'd hold the information that a certain PFN is taken by the kernel?
<heat>
does freebsd do the same shit with the direct map?
<nick64>
I thought kmalloc always allocated virtual memory, and the performance gain was from not mapping a page. If it does not even allocate, what does it return?
<heat>
i'll give you the example of linux x86_64
<heat>
it keeps a direct map starting at ffff888000000000
k8yun_ has joined #osdev
<heat>
so from there on up to 64TB, physical page X is mapped at ffff888000000000 + (X * 4096)
<heat>
alloc_page returns your struct pages, very nice, whatever
<heat>
you don't need to allocate virtual memory, nor map anything, since it's already mapped at ffff888000000000 + X
<heat>
since the buddy allocator is exceptionally good at returning contiguous pages, there's no big need for virtual memory there to cover for fragmentation
<heat>
(YMMV, memory can fragment after boot, don't expect a kmalloc(32MiB) to work 4 weeks into a heavily loaded server)
k8yun has quit [Ping timeout: 252 seconds]
<heat>
linux i386 is more complicated. memory is straight up mapped from 0xC0000000 up to <insert address here>. around 800 MiB of it. the kernel is /mostly/ limited to that chunk of memory
<heat>
if you want to access what they call "HIGHMEM" (everything after those 800 and something MiB), you need to kmap*() and kunmap*() the pages
<heat>
but slab allocators, kmalloc will only work on !HIGHMEM
<clever>
and that is where zfs on 32bit linux has so many problems
<clever>
linux allows the file/disk read caches to exist unmapped, but zfs expects everything to be mapped always
<heat>
ZFS is ideally used with 8GB of ram and ECC anyway
<heat>
not a big loss
<clever>
and when that 800mb range runs out of virtual space, problems begin
<heat>
reminds me of those 200k watches that say that the watch's ideal operation conditions is "standing still"
<clever>
that defeats the entire point of it being a watch :P
<mjg_>
heat: freebsd memory support is a victim of long standing lollery
* heat
snorts coke over a bonwick paper
<heat>
VMEM BABYYYYYY WOOOOOOOOO
<heat>
nick64, anyway that's just how linux does it. windows even has paged pool memory that can get swapped out/in
<heat>
my kernel uses a lot more virtual memory because I don't have a SWEEEEEEET buddy allocator (I should maybe look into that)
* mjg_
ingests some fentanyl
<heat>
it relies a lot more on getting vmalloc to map and allocate virtual space because of that
<heat>
essentially slab / kmalloc work on top of vmalloc chunks, and vmalloc() just allocates virtual addressing and asks alloc_page() for pages, and maps them
<heat>
a lot more traditional malloc-ey I guess, but slower
* nick64
nick64_grepme
wand has quit [Remote host closed the connection]
mrvn has joined #osdev
<heat>
it's also arguably safer but who cares about safety, boooooooo
<heat>
go use a microkernel or something, cringe
* mrvn
raises his hand
<heat>
booooooo
<mjg_>
safety? templeos
<heat>
s/templeos/openbsd/
<clever>
heat: yeah, i was also thinking how a microkernel would solve that, it inverts the problem, so now a driver like zfs has access to the 3gig end of the 3g/1g split
<\Test_User>
safety? throw a sledge hammer through the computer, now nothing more will break
<heat>
clever, no, zfs there would absolutely always need to map things. no 800MiB shortcut there
<heat>
unless they want to add (800MiB of resident memory) to the requirements
<clever>
heat: but with zfs running in userland, it has access to the 3gig end of the split, not the 1g end
<heat>
which I guess is already there with the 8GB ram requirement
<clever>
so the limit is ~3g, not ~800m
<heat>
yes, but you need to map things
<clever>
8gig of ram is not required :P
<clever>
my desktop zfs is only using 600mb of ram right now
<clever>
and i'm still getting >98% hit rate
<heat>
"A minimum of 2 GB of free memory is required to run ZFS, however it is recommended to use ZFS on a system with at least 8 GB of memory. "
<CompanionCube>
yep: 'ARC Buffer Data (ABD) - Allocates ARC data buffers using scatter lists of pages instead of virtual memory. This approach minimizes fragmentation on the system allowing for a more efficient use of memory. The reduced demand for virtual memory also improves stability and performance on 32-bit architectures.'
<clever>
mrvn: then why is it only using 700mb, when i have 16gig of ram?
<mrvn>
clever: because you have a newer zfs version
<CompanionCube>
also because it resizes according to memory pressurre
<mrvn>
clever: or never used more
<clever>
mrvn: definitely memory pressure, graphs show the arc shrinking massively when i opened chrome
<CompanionCube>
the '50%' figure is the default 'upper limit'
<clever>
yeah
<mrvn>
clever: or tuned the ARC cache (you or your distribution) to use less
<clever>
50% is the max, and the min is something like 16mb
<clever>
and it will dynamically float within there, based on memory pressure
<CompanionCube>
the min hard limit is calculated from some small percentage, mine's half a gig