eddof13 has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
Bitweasil has quit [Read error: Connection reset by peer]
Bitweasil has joined #osdev
eddof13 has joined #osdev
eddof13 has quit [Client Quit]
<linkdd>
ok, I feel dumb. At some point, in my printk function, I have this declaration: `char num_buf[65] = {0};` (context: this is the buffer for itoa, lowest base is binary, so 64 digits + null character).
<zid>
with you so far
<linkdd>
My printk function in a C kernel: works fine. the same in C++ makes the vm crash (i have not yet dumped the vm's traceback)
<linkdd>
char num_buf[65]; // fixes the issue
<linkdd>
so the = {0}; in C++ have a different behavior than in C ??
<zid>
No idea, I don't know C++, but there's a chance it ends up copying the zeros .data I guess, and that could defo be screwing things up if you're not expecting it, seems unlikely though
<zid>
seems fairly debuggable though
<heat>
i think = {0} is wrong in C++, yes
<heat>
actually, no, seems to work
<linkdd>
memset(num_buf, 0, 65); after the declaration works fine as well.
<heat>
what's the codegen
<linkdd>
if i put a `char foo[65] = {0};` at the very beginning of my entrypoint (kmain), it reproduces the bug
<bslsk05>
github.com: Onyx/kernel/arch/x86_64/make.config at master · heatd/Onyx · GitHub
<zid>
heat: when -mgeneral-but-also-3dnow!
<heat>
this mouthful linkdd
<linkdd>
thx
<heat>
you only need the -mno-{sse, avx, ...} stuff
<zid>
heat why does your kernel have floats in it
<heat>
<heat> want a fun annecdote? your beautiful C++ headers have a bunch of inlined functions that call the __builtin
<heat>
<heat> so there are a bunch of floats there, even if you never call them
<zid>
'the __builtin'
<zid>
the builtin what?
<heat>
__builtin_<whatever standard float function>()
<zid>
okay, and what's the problem?
<heat>
-mgeneral-regs-only errors out at the sight of a float
<zid>
even if it's parsed and then eliminated, before it tries to register allocate?
<heat>
yep
<zid>
sucks to be C++ then I guess
<zid>
but we knew that already
<heat>
which obviously does not work when the toolchain intercepts all your standard C includes
<linkdd>
instead of passing -mno-sse etc... i can enable it in my kernel? i see there are some example code in the osdev wiki page
<heat>
no
<heat>
do not
<heat>
FPU in the kernel is a bad idea for many reasons
<zid>
you'll have to constantly xsave it on every single interrupt etc won't you
<linkdd>
right
<heat>
yep
<linkdd>
ok
<linkdd>
ok i shamelessly copied your CFLAGS, now it works
<heat>
great
<linkdd>
so the explanation, to make sure i understood, is that the compiler saw the opportunity to optimize the initialization of the array using SIMD instructions, which i have not enabled in my kernel startup code, is that correct?
<zid>
std_abs.h:80:12: error: '__builtin_fabsl' requires 'long double' type support, but target 'x86_64-unknown-linux-gnu' does not support it
<zid>
80 | { return __builtin_fabsl(__x); }
<zid>
heh, I tested it
<heat>
linkdd, yes
<linkdd>
does that also mean that if i don't enable those instructions, a program which targets my OS would not be able to use them?
<zid>
You need to enable them globally, but give your kernel never changes the regs
<heat>
yes, you'll want to enable them eventually for user code
<zid>
they'll be fine through syscalls
<zid>
given*
<zid>
so you can basically just ignore them, unless you wanna zero them at process creation/teardown for SECURITY
<linkdd>
ok, cool, another problem for future me (that guy is good, he'll take care of it)
<heat>
you zero them manually when allocating the fpu state
<heat>
set up some fpu regs you need and zero xmm/ymm/zmm etc while you're at it
<heat>
mjg, can you do page LRU without a big global lock?
<mjg>
yes and no
<mjg>
what you *can* do is BATCH a bunch of pages for requeue
<mjg>
so that contention is lessened big time
<mjg>
however, LRU is the cop out mechanism to begin with
<mjg>
"just slap in something plausibly better than FIFO"
<Ermine>
I have feeling that everybody uses LRU
<mjg>
nope
<sham1>
LRU is good
<Ermine>
mjg: who doesn't?
<mjg>
Z-fucking-S for example
<mjg>
(arc)
<mjg>
also check out things like clock, tinylfu and others
<mjg>
every single algo claims to beat the rest
<mjg>
sham1: so one classic anti-LRU is its lack of scan resistance
<Ermine>
There are many algorithms described in Tanenbaum's book
<mjg>
i have not read it, i don't trust the guy
<mjg>
may happen to be decent
<mjg>
but given the age of the book there is utter disregard for smp, so...
<netbsduser`>
Ermine: i am working to migrate to wsclock
<Ermine>
mjg: hm, iirc there's a chapter on smp in 4th edition
<Ermine>
netbsduser`: cool!
<mjg>
i'm sure it discusses smp issues for somethingl ike lru
bauen1 has joined #osdev
<heat>
linux uses MGLRU now, tho
<Ermine>
... which ends in LRU
<Ermine>
Btw I haven't tested it under memory pressure yet
<heat>
but it's quite more complex than "haha list go brr"
<zid>
wait, list does NOT go brr!?
<zid>
:(
<netbsduser`>
i hope mglru is an improvement over their previous state of affairs
<netbsduser`>
linux performance under vm pressure i found to be despicable
gorgonical has quit [Ping timeout: 260 seconds]
<Ermine>
I once run my monte-carlo simulation with large amount of paths and small step size and graphics hung
<Ermine>
Also I tried to decompress nvidia leaked source with the same result
<netbsduser`>
i tried to build something big, might've been LLVM, on a linux box, and it ground to a near halt
<netbsduser`>
graphics was unusable, i was able to rescue the laptop by switching to console, logging in (took 5 minutes or so), and killall -9'ing the needful
<Ermine>
I have just reset my box
<nortti>
did you have swap set up? I've only ever seen that behaviour on swapless systems
<Ermine>
Yes
<heat>
yep, swapping doesn't make a difference
gog has quit [Ping timeout: 256 seconds]
<Ermine>
gog got swapped out
zhiayang has joined #osdev
bnchs is now known as Cindy
MiningMarsh has quit [Ping timeout: 256 seconds]
MiningMarsh has joined #osdev
vdamewood has joined #osdev
<ddevault>
ares is going really well :3
xenos1984 has quit [Read error: Connection reset by peer]
<ddevault>
I think I can install it on my laptop within the month, perhaps dual-boot with linux
<bslsk05>
www.compart.com: “슩” U+C2A9 Hangul Syllable Seunj Unicode Character
goliath has quit [Quit: SIGSEGV]
<FireFly>
SGautam: you'll have to decode the UTF-8 into a stream of codepoints and look those up
<FireFly>
or well, render those
<SGautam>
Okay, well I was stupid to think it was this simple.
<FireFly>
UTF-8 isn't too bad to decode
<SGautam>
Ah
<SGautam>
I must strip those leading bits
<GeDaMo>
C2 A9 -> 1100 0010 1010 1001
<GeDaMo>
0 0010 10 1001 -> 000 1010 1001 -> A9
<FireFly>
right, you wanna mask out and shift the bits into place since the bits making up the codepoint are essentially the "payload", the rest is just framing/encoding it in a nice way
<FireFly>
(with some useful properties like being able to do random seeking and then find a codepoint boundary)
<FireFly>
or recovering in case of corruption
<GeDaMo>
The high bits on the first byte tell you how many bytes there are
<bslsk05>
www.theregister.com: Who has over half all Arm server CPUs in the world? Amazon • The Register
gog has quit [Quit: Konversation terminated!]
<Ermine>
Somewhy I'm not surprised
<moon-child>
mjg: best I can do is a gingerly tap on the shoulder
<mjg>
no thanks, i kissed a girl *ONCE*
<Cindy>
i never kissed someone
<Cindy>
i'm too scared to even touch someone
eddof13 has joined #osdev
xenos1984 has quit [Ping timeout: 240 seconds]
antranigv_ is now known as antranigv
SGautam has quit [Quit: Connection closed for inactivity]
antranigv is now known as antranigv_
antranigv_ is now known as antranigv
gog has joined #osdev
xenos1984 has joined #osdev
<moon-child>
gog: why are you in lowercase now ;-;
<gog>
wdym
<moon-child>
gog (~Ada@user/gog) has quit (Quit: Konversation terminated!)
<moon-child>
gog (~ada@user/gog) has joined #osdev
<gog>
oh
<gog>
my ident on windows is capitalized and i never bothered to fix it
<Ermine>
Expectable on Windows
bauen1 has joined #osdev
goliath has quit [Quit: SIGSEGV]
<sham1>
It's case-insensitive
<Ermine>
But Their APIs Are All Capitalized
<Ermine>
Win32, C#, EFI
gareppa has joined #osdev
dutch has quit [Quit: WeeChat 4.0.2]
<Ermine>
AND FAT LIKES CAPS
<zid>
ALL CAPS ARE PIGS
<gog>
C# C# C#
<gog>
CEEE SHARP
<heat>
mjg, btw i have no way to test the atomic store stuff or whatever, short of patching the patch myself
<heat>
which is an /idea/ but requires effort
<heat>
vlastimil talked about maybe needing spinlocks for remote drain anyway
<heat>
and PREEMPT_RT needs em as well, can't disable preemption there
sortie has quit [Quit: Leaving]
<jimbzy>
I like C#
<heat>
i dont c# i need glases
<heat>
hahahahah ha ah ahhaha ha ha ah ahha haha ha
melonai has joined #osdev
<jimbzy>
It's just c++++
<netbsduser`>
on linux adding per-cpu caches to its allocator: i thought it would already have done so
<netbsduser`>
bonwick and adams describe it in their vmem paper from 2002
<mjg>
it *was* done so
<mjg>
WAY BACK
<mjg>
which maeks the above proposal a wtf
<heat>
netbsduser`, complex question
<netbsduser`>
all the five BSDs also have similar and i think the lineage of each traces directly back to the paper of Bonwick himself
<heat>
their slab allocators have had pcpu allocation since forever
<heat>
SLAB has a magazine-like pcpu cache (like discussed in the patch set i linked, for SLUB)
<netbsduser`>
especially in this age of numa (numa iei) it's valuable
<heat>
SLUB ditched all the queuing SLAB had, because hurr durr it's bad, so it keeps a slab as a percpu thing plus optionally (KConfig) a pcpu partial slab list
<heat>
aw fuck thats a great pun
<heat>
citing dragostea din tei is always a big win in my book
<mjg>
it predates you
<heat>
no it doesn't
eddof13 has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
<sham1>
heat: C# is quite funny, yea
MiningMa- has joined #osdev
MiningMarsh has quit [Ping timeout: 246 seconds]
MiningMa- is now known as MiningMarsh
eddof13 has joined #osdev
GeDaMo has quit [Quit: That's it, you people have stood in my way long enough! I'm going to clown college!]
vdamewood has joined #osdev
SGautam has joined #osdev
<zid>
I went outside today, didn't see any grass though thankfully, my touchgrassginity remains intact
<bslsk05>
eli.thegreenplace.net: Parsing expressions by precedence climbing - Eli Bendersky's website
<zid>
now I can write compilers
<SGautam>
I need to compile the language for the excel clone eventually tho, geist
<geist>
...
<zid>
I knew what needed to happen but not how specifically a decent way to solve the issue this solves was
<SGautam>
To some abstract representation at least.
<heat>
you're all over the place SGautam
<SGautam>
Like a bytecode.
<zid>
me too
<zid>
I half write LOTS of projects, sue me
<heat>
pick something and *focus* on that
<heat>
or don't, it's your life
<zid>
disgusting behaviour, heat
<SGautam>
I'm focusing on the excel clone, yeah, I'm thinking of how to parse and interpret the language I just designed in 5 minutes trying to be as reasonable as possible while not making it look like Forth.
<geist>
if it's an excel clone doesnt that pretty much dictate the language?
<zid>
he means 'spreadsheet program'
<zid>
rather than direct clone of excel
<zid>
TABULAR DATA PROCESSOR.elf
<geist>
i suppose that's true
<SGautam>
Sorry, I really shouldn't use the term "excel clone", more like a spreadsheet program where every sheet has a script attached to it.
<heat>
microshit winblows officrap eshitcel
<geist>
been a while since i have thought of 'excel' as the defacto spreadsheet
<geist>
but that's because i mostly end up using either google sheets or the mac one just because that's what i have in front of me
<sham1>
heat: micro$hit
<SGautam>
Google Sheets is really good now
<SGautam>
I stopped using Excel coz my uni subscription expired
<zid>
I've never used excel on anything but an office computer that wasn't my own
<geist>
hmm, did they ever port excel to mac classic...
<geist>
that would be a fun experience
<sham1>
Excel is the most frequently used database software
<zid>
I had open office installed for a bit before google sheets became a thing
* sham1
runs
<SGautam>
Are any of you guys into retro computing? I kind of expect osdev to be retro PC enthusiasts
<heat>
ITAAAAAAAAANIUUUUUUUUUUUUUUUUUUUUUUUM
<heat>
is itanium retro yet
<gog>
RUST ITANIUM
<gog>
yes
<gog>
well
<geist>
i am yes
<gog>
maybe
<heat>
ROST
<SGautam>
IA64?
<heat>
yes
<heat>
(IA128 in the future)
<gog>
the legendary microarch that was cool and everybody liked
<geist>
that's why i was just thinking huh it'd be fun to dig up an old version of excel for my mac SE
<heat>
gog, it's NOT a microarchitecture gog wtf
<SGautam>
Wait why do I have this feeling that there are many packages suffixed with ia64 instead of x86_64 even though they mean x86_64?
<gog>
i mean ISA
<heat>
get your terminology straight
<gog>
i don't get anything straight ok
<heat>
get your terminology gay
<SGautam>
I've seen ia64 suffixes a lot of times.
<geist>
oh huh interesting, actually excel 1 was released for macintosh *firsT*
<geist>
then 2.0 was released for windows as windows was developed
<zid>
heat: x86s's correct terminology name is "Itanium 3" don't foret
<SGautam>
Yah cuz I think Windows copied Mac's UI
<zid>
forget*
<heat>
you mean winblows
<geist>
re: ia64 being misused, i haven't seen that at least
<geist>
most folks just dont know about ia64 the moniker
<bslsk05>
git.kernel.org: kernel/git/torvalds/linux.git - Linux kernel source tree
<heat>
x86/bugs: Increase the x86 bugs vector size to two u32s
<heat>
fucking hilarious
<nortti>
:D:
<zid>
Top tip, cpus with speculation are not secure in multiuser configs
<zid>
stop trying, guys
<Ermine>
I've got the same feeling tbh
<nortti>
honestly a bit surprised there's not been a push to just fully clear microarchitectural state on context switches. guess modern caches etc are so big that they'll still have some of your data next time your process gets to execute
<moon-child>
the cost of context switches is already high enough ..
<heat>
so what's the strategy? no multiuser systems no more?
<zid>
who needs em
<zid>
computing hw is cheap
<zid>
kick users off between switching customer if you're doing hw pools
<nortti>
heat: does itanium speculate?
<nortti>
ah, I guess it probably does, because branch predictor
<Griwes>
yes, it speculates that it is a useful architecture
<Ermine>
But really, are those vulnerabilities inevitable?
<moon-child>
nortti: I am pretty sure it does not
<heat>
itaniuuuuuuuuuuuuuuuuuuuuuuuuuuuuum
<kof123>
obviously the compiler/programmer does the "speculation" j/k
<moon-child>
Ermine: by definition, there will always be side channel attacks. Having applications share as much as they do when they run on shared hardware just gives you a lot more opportunities to exploit some such
<moon-child>
but I can still steal your ssh key by shining a laser at your window to catch harddrive vibrations, so
stolen has quit [Quit: Connection closed for inactivity]
heat has quit [Remote host closed the connection]
heat has joined #osdev
austincheney_ has quit [Ping timeout: 260 seconds]
Turn_Left has quit [Read error: Connection reset by peer]
<linkdd>
i'm using limine as a bootloader, more specifically the SMP request to detect the bootstrap processor and application processors. limine gives me their lapic_id and processor_id. I saw that both those ids seem to be 0<id<cpu_count, is that a guarantee or is it just a coincidence in qemu?
austincheney has joined #osdev
vdamewood has quit [Quit: My MacBook Pro has gone to sleep. ZZZzzz…]
<mcrod>
hi
SGautam has quit [Quit: Connection closed for inactivity]
<geist>
linkdd: that's a coincidence
<geist>
apic ids are not guaranteed to be compact, they can and do have gaps in them in real hardware
<mcrod>
another day, another compiler bug
<geist>
i think the only thing that's really guaranteed (though i can't tell you where it's written down) is the boot cpu is apic id 0
<linkdd>
ok, so i won't use them as index lol
<linkdd>
thx
<linkdd>
a simple O(n) search through the array will do the trick
<linkdd>
i don't expect a computer to have millions of CPUs anyway
<geist>
yeah in general thats why logical cpus numbers are all you need
<geist>
in practice i've found that logical cpu -> physical id (apic on x86) is a far more common search, and you can yourself make sure the logical id is compact and based at 0
<geist>
because that's your numbering
<geist>
it's fairly rare to go from physical -> logical, so if it happens to be O(n) then so be it
<geist>
and when i say 'logical id' i just mean 'whatever numbering convention your OS uses' which is almost always 0...N-1 where N is the number of known cpus
<linkdd>
well, that's because i can read the apic registry to get the physical id of the currently running cpu, which allows me to search the array of "cpu_locals" (which contains the cpu-dependent data, like the virtual memory page map, etc...)
<linkdd>
this is a very dumb way to do this, i remember heat talking about mapping the same virtual address to different physical address for each cpu
<geist>
right, but there's a better way to do that. on x86 the defacto solution that virtually all OSes do is use the gs: segment to point at a per cpu structure
<linkdd>
but i'm not there yet
<geist>
and every other arch has some sort of similar 'register/mechanism per cpu to point to a local cpu structure'
<linkdd>
geist: as in, store the lapic_id (32 bits) in the gs registry?
<klange>
x18 my beloved
<geist>
linkdd: as in use the gs: segment to point at it, such that you can reference it via gs:offset
<geist>
and you can put a pointer to the strcture itself in slot 0 if you want, for example
<geist>
such that gs:0 -> pointer to structure
dutch has quit [Quit: WeeChat 4.0.2]
<geist>
and then have a per cpu structure wher eyou can anchor all this stuff on
<geist>
on x86-32 this would e via a regular segment, on x86-64 this is why there's a whole MSR for it, and swapgs the instruction exists, etc
<linkdd>
right, i think my dumb way will do fine for now, it can still be easily refactored later on :)
<bslsk05>
github.com: toaruos/base/usr/include/kernel/process.h at master · klange/toaruos · GitHub
<geist>
yep. my only suggestion there is treat it as that: a per cpu structure, even if there's only one
<geist>
and then you can hide the 'get a pointer to my current cpu' behind some function that you can optimize later
<geist>
get_current_cpu()->some_stuff, etc
<linkdd>
i already have this function. i implemented the O(n) search in a cpu, but seeing the values in QEMU i was wondering if
<linkdd>
erf
<geist>
gotcha
<linkdd>
i already have this function. i implemented the O(n) search in a cpu_locals[] array, but seeing the values in QEMU i was wondering if i could use the apic_id as index, but as said earlier, they are not compact
<geist>
yah, and anyway reading the apic_id off the local cpu in x86 is expensive anyway
<linkdd>
right, so i will want to optimize this anyway later on
<linkdd>
problem for future me :D
<linkdd>
make it work, make it right, make it fast, in that order :)
<geist>
yup. you dont have to optimize it now of course, it's just generally nice to structure your stuff so it can be easily optimized, which sounds like what you're doing
<geist>
that's where experience, etc comes in. building thigns such that you know you'll eventually optimize it in some way, and then at least make sure you didn't paint yourself into a corner in v1
<geist>
and this is not so much directed at you, btw. i'm just sort of having a little soliloquy as i tend to do
<linkdd>
years ago i was working on a 32 bits kernel in C for x86 only. recently, i was interested in doing it with multi-arch in mind, even if i only implement x86_64, and adding the niceties like smp support, acpi, not doing the task scheduling on the PIT interruption, etc...
<linkdd>
the multi-arch forces me to consider the interfaces of my data structures and how i access them
<linkdd>
what to put in the "generic" code, and what to put in the "target specific"
<linkdd>
how to glue them together, etc...
<linkdd>
i also want to make it a microkernel, and implement capabilities (i've been reading the source code of ddevault's kernel for some inspiration and as a way to learn hare at the same time)
<linkdd>
i also want to try this idea of a VFS based on urls instead of a file/folder tree
<linkdd>
driver://device:partition/path
<linkdd>
ext4://hd0:3/hello/world.txt
gog has quit [Quit: byee]
<geist>
nice
<kazinsal>
similarly I do mountpoint:path/to/file
Burgundy has quit [Ping timeout: 245 seconds]
<kazinsal>
simple enough that I can eventually add virtual mountpoints for things like tftp: and https:
<kazinsal>
which will honestly probably reside in userspace drivers because I'm not baking SSL into the kernel
<moon-child>
no ktls? laaaaaaaaame
<moon-child>
not webscale
<linkdd>
lol
<linkdd>
as a microkernel, all vfs drivers will be in userspace as well
<linkdd>
so i can imagine a https:// vfs path as well
<linkdd>
but i have no concept of "mountpoint" nor the commands mount/umount from unices. I do like windows in that aspect, if the device is plugged, there is a letter for it (well, here, an URL)
<geist>
i always kinda like thwe way VMS does it, where you end up with generated mount points, somewhat like what you're saying
<geist>
and then the ability for the system to set aliases to them for nicer use
<geist>
ie, boot: could be an alias for <some long device path>:
<geist>
and stuff like home: could be boot:/some/path/within/it
<geist>
really it's fancy environment variables, but environment variables that the VFS understands intrinsically, not just some local user space construct
<linkdd>
well, i'm not there yet :) i currently have an initrd (tar archive), a physical memory manager (with a bitmap, classic), a heap (with liballoc), framebuffer with flanterm, the beginning of SMP support (i'm implementing the cpu local data), and a draft of a virtual memory manager
<bslsk05>
zserge/jsmn - Jsmn is a world fastest JSON parser/tokenizer. This is the official repo replacing the old one at Bitbucket (732 forks/3345 stargazers/MIT)
<linkdd>
> no dependencies (even libc!)
gildasio has quit [Ping timeout: 240 seconds]
m3a has quit [Ping timeout: 245 seconds]
<heat>
percpu is hard
<heat>
having a get_current_cpu() works but it's kind of messy, you get a big struct full of nothing, and non-optimal instruction sequences to get it
<heat>
full of everything*
<heat>
i play linker script + inline asm tricks to get pcpu data
<heat>
but it doesn't fully work, linux for instance has a percpu allocator for dynamic data
<heat>
which gets painfully needed when you realize that for dynamic structures you either reserve MAX_CPUS copies (properly padded to avoid cache line sharing), or you're screwed
gildasio has joined #osdev
<kof123>
"but environment variables that the VFS understands intrinsically" maaaaaaaaaaaaaaaaaaaaaaagic symlinks
<linkdd>
heat: yeah, that would be a concern if my goal would be "optimized, ready for production, totally not a fun experiment". i'm good with using a single global allocator with a spinlock