<ddevault>
what's the overhead of everyone's page allocator
<ddevault>
design I'm looking at now is about 3.5%, which is more than I'd like
<ddevault>
ah, can get that down to 1.5%
gog has joined #osdev
<Mutabah>
Mine uses a bitmap (and freelists), so the overhead is something like three bits per page
<Mutabah>
Hard to calculate, because as soon as a page is referenced twice - it gets a reference counter allocated for it
<zid`>
Do just a raw ass freelist and it's 0 :P
wlemuel has quit [Ping timeout: 250 seconds]
<Mutabah>
Yep
wlemuel has joined #osdev
<gog>
hi
nyah has joined #osdev
* moon-child
pets gog
* gog
prr
<mrvn>
ddevault: I have 0% and 0.2% overhead depending on whether you want per-pöage infos or not.
<ddevault>
thanks mrvn
<mrvn>
A freelist will be 0%.
<ddevault>
aye
<mrvn>
How do you get 3.5%? Thats nearly 40 bytes per page
<mrvn>
aeh, 150 I mean
<ddevault>
the estimate was off
<ddevault>
got it down to like 0.5% by rechecking my math
<mrvn>
So 20 bytes?
<ddevault>
didn't work it out by page
<ddevault>
in any case, I'm not going with this approach (yet)
<ddevault>
solved my problems in other ways and will keep the free list for now
<mrvn>
where does it come from?
<ddevault>
buddy bitmaps plus capability storage for the subdivided memory areas
<mrvn>
and you need that why?
<ddevault>
easier allocation of continuous ranges of pages and different allocation sizes (e.g. large/huge pages)
<mrvn>
which will break down fairly quickly due to fragmentation and become pointless
<ddevault>
I would not be so sure
<ddevault>
maybe for huge pages
<ddevault>
but large pages should be fine imo
bauen1 has quit [Ping timeout: 240 seconds]
<mrvn>
Have multiple free lists. Start with a free list of 1GB pages. Then as needed you split this up and put it in the free list for 2MB pages. Again split it for the 4k free list.
<ddevault>
not a bad idea! will think about it
<mrvn>
0 overhead and basically the same advantage of easy allocation as long as the fragmentation doesn't get you.
<mrvn>
I believe the only sustainable solution for contigous allocations over time is to be able to defragment physical memory. That's where my per-page infos and 0.2% overhead come from.
<mrvn>
8 byte per page, a few control bits and then a pointer to the owning process or a refcount in the rest.
<mrvn>
or pointer to the next free page when unused.
<mrvn>
ddevault: If you think you need a lot of contigous pages have you considered putting free pages into a balanced binary tree? If you make a tree for every 512GB chunk of memory you have 27 bit page number * 2 (left/right) = 54 bit leaving 10 spare bits for meta data for the tree structure and only need 8 byte per page.
<mrvn>
or go with 16k pages. 1/4 the overhead right from the size increase.
_xor has quit [Quit: brb]
<ddevault>
yeah perhaps
<ddevault>
I basically want large pages for more efficient use of capability space
<ddevault>
for small allocations in userspace
<ddevault>
then need continuous pages for DMA, large pages should be sufficient to start
<ddevault>
not too worried about efficiency or fragmentation this early on
<ddevault>
but the large page capability space issue is more pressing
<ddevault>
right now each bin of small allocations for userspace takes up 256 capabilities
<ddevault>
can reduce that to 1 with large pages
<gog>
does your target platform have iommu?
<ddevault>
gog: yes
<ddevault>
will add it later
<gog>
if you're under some kind of time pressure that makes sense, but it saves you from having to care about fragmentation
<gog>
as far as DMA goes anyhow
bauen1 has joined #osdev
wootehfoot has joined #osdev
Brnocrist has quit [Ping timeout: 260 seconds]
goliath has joined #osdev
<mrvn>
ddevault: iommu or scatter gathering makes contigous pages for DMA unneeded
wlemuel has quit [Quit: Ping timeout (120 seconds)]
<ddevault>
true
wlemuel has joined #osdev
<mrvn>
My userspace malloc will only allocate 2MB chunks as smallest unit. Would really make sense to use large pages there.
<ddevault>
where can I read up on iommu? any good resources?
<zid`>
I keep meaning to get around to implementing it
<zid`>
but then I find a weird blocker like my device not supporting it in qemu right
<zid`>
so I have to write some other driver first etc
<zid`>
so I just keep putting it off
<ddevault>
I get that there's this MSI/MSI-X capability in each PCI device
<zid`>
It *seems* simple at least
<ddevault>
which have some memory addresses I can write to in order to get it to write interrupt stuff there
<zid`>
It just DMA's a single write to the apic, I get an IRQ, simples.. in theory
<zid`>
Just need to convince the actual device to use it instead of PIN interrupts
<ddevault>
but what IRQ is raised? should I set the interrupt pin in the device header?
<ddevault>
then I can plug multiple devices into that pin and scan the message area to identify the interrupting device?
<zid`>
I think you can just give everything its own number just fine?
<ddevault>
do not have APIC yet fwiw
<ddevault>
so this would be through the PIC
<zid`>
I think the minimum is 255 IDs
<zid`>
PIC isn't memory mapped though, how's it going to react to a write if it can't be written to? *think*
xenos1984 has quit [Read error: Connection reset by peer]
<ddevault>
hm
<zid`>
You could probably *test* MSI by polling regular RAM for the write
<zid`>
but you're not getting an IRQ without that write hitting the/an apic
<ddevault>
so I don't really grok APIC yet
<ddevault>
but I'm inferring from this discussion that it provides some area in memory which, when written to, will raise an interrupt
<zid`>
yea
<ddevault>
I'd further speculate that the location is processor-specific
<ddevault>
and IPIs are done by user code writing to that address, too
<ddevault>
and I can configure some kind of routing such that it raises a desirable IRQ?
<zid`>
I wouldn't know how to microkernel
<ddevault>
this is all still generic
Burgundy has quit [Ping timeout: 250 seconds]
<ddevault>
as in I can tell the APIC "when someone writes to this address, raise IRQ $number"
<zid`>
letting user processes write to your lapic to do IPI sounds incredibly microkernel
<zid`>
I can think of nothing more micro
<ddevault>
user code as in kernel code
<ddevault>
as in not the hardware
<zid`>
oh yea, sure then
<ddevault>
sorry for the confusion
<ddevault>
anyway, then I write to PCI config space in the MSI/-X thing to point it at the appropraite address
<zid`>
the sdm is kinda lacking for the acic stuff ngl
<ddevault>
then the APIC raises the appropriate IRQ on the appropriate processor
<zid`>
as far as I know, yes
<zid`>
I've not actually *done* it but I did read all the docs
<ddevault>
and I scan the APIC to figure out which vector raised the interrupt?
<ddevault>
and/or scan PCI space to see what's pending, if it remains ambiguous at that point (unlikely?)
<zid`>
I think you get the 'message'
<zid`>
or you just tell each device to trigger a different irq
<ddevault>
right
* ddevault
pulls up the manual
<zid`>
okay I checked
<zid`>
figure 11-24
<ddevault>
of which document
<zid`>
sdm
<ddevault>
sdm?
<zid`>
software developer's manual
<ddevault>
gross, intel
<zid`>
There's an 8bit vector field, and a 'destination id' field, and you can use the latter for routing and former for which.. vector
* ddevault
pulls it up
<ddevault>
volume 3?
<ddevault>
chapter 11 is memory cache control
<zid`>
figure 11-24
<zid`>
not chapter 11
<zid`>
the figures are squential
<zid`>
so that's the 11th set of figures, I think vol 1/2 might have a couple figures, and not all chapters have them, etc
<ddevault>
we're looking at different documents
<ddevault>
but w/e, I have enough to keep working
<ddevault>
many thanks
<ddevault>
fwiw solution I landed on is to program the interrupt line in the PCI device config and revisit once I look into the APIC
xenos1984 has joined #osdev
<zid`>
I think in practice some weirdness happens if you use PIC like: the pci-e emulates a PIN interrupt by generating an MSI anyway, and the cpu sees the MSI and pretends it was a PIC interrupt, or something, it's all full of legacy and emulations and other nonsense at least
elastic_dog has quit [Killed (platinum.libera.chat (Nickname regained by services))]
elastic_dog has joined #osdev
theboringkid has joined #osdev
[itchyjunk] has joined #osdev
xenos1984 has quit [Ping timeout: 246 seconds]
Arthuria has joined #osdev
wlemuel has quit [Quit: Ping timeout (120 seconds)]
xenos1984 has joined #osdev
wlemuel has joined #osdev
theboringkid1 has joined #osdev
theboringkid has quit [Quit: Bye]
theboringkid1 is now known as theboringkid
xenos1984 has quit [Ping timeout: 265 seconds]
xenos1984 has joined #osdev
xenos1984 has quit [Ping timeout: 265 seconds]
bnchs has quit [Remote host closed the connection]
xenos1984 has joined #osdev
linearcannon has joined #osdev
theboringkid has quit [Read error: Connection reset by peer]
theboringkid1 has joined #osdev
linear_cannon has quit [Ping timeout: 276 seconds]
theboringkid1 is now known as theboringkid
Brnocrist has joined #osdev
antranigv has quit [Read error: Connection reset by peer]
antranigv has joined #osdev
Halofreak1990 has joined #osdev
<Halofreak1990>
does anyone know if GNU AS/GCC defines any macro other than __ASSEMBLY__ when compiling assembly files, and if so, where I can find a list of those?