<azonenberg>
Is there a way to automatically discard unused sections at link time, a la --gc-sections, but somehow also mark specific sections as "never discard even if unreferenced"?
levitating has quit [Remote host closed the connection]
gorgonical has quit [Ping timeout: 256 seconds]
<Mutabah>
Add them to the NODISCARD section in the linker script
<Mutabah>
(I think?)
<Mutabah>
azonenberg: Wrap the reference in the linker script with `KEEP( )`
<Matt|home>
On average, would you say that software bloat has gone up? In general for things like web browsers which get -very- frequent complaints about inefficient memory usage, and that unnecessary additional software you get with modern samsung phones nowadays
<Matt|home>
and if so.. does that also apply to kernel and OS code as well? or is it _absolutely_ needed nowadays to use up that much memory
<Mutabah>
Well, a lot of that can be attributed to the compexity of modern systems, and expectations about them
<Matt|home>
e.g. quick google search says the windows 10 OS is about 15 gigs uncompressed
<Matt|home>
but is that all kernel code or is it like, graphics stuff which are big by necessity
<Mutabah>
... and the amount of assets required to make an OS look pretty
<Mutabah>
ntoskrnl is probably nowhere near large
<Mutabah>
although, drivers can get big
<Matt|home>
right. what im saying is, is there a way to be more efficient with our use of data and memory without sacrificing practicality
<Matt|home>
or is this pretty much as good as it gets rn
<Mutabah>
Especially graphics ones, with lots of features (and per-game hacks)
<Matt|home>
the only reason i brought this up is cuz im doing a _very_ slow write of a windows installer to my usb drive and.. yeah it's taking forever. is it really impossible to do something like the entire windows OS in only a few gigs of data?
<Mutabah>
ntoskrnl.exe on my machine is 10MB - seems kinda chunky compared to a hobby OS, but I assume there's a lot of glue code there for the various drivers
<Mutabah>
Oh yeah, I think you can trim a windows install down to much smaller - there used to be tools for that
<zid>
10M is pretty small given how old it is, that is to say, 'featureful'
<zid>
loot of syscalls in ntoskrnl
<zid>
plus it's effectively an 'allyesconfig'
<Matt|home>
in terms of extremes i assume we're still limited to the gigabytes range? like, you're never ever gonna be able to make a modern full featured web browser only require 128MB of ram or whatever are you?
<azonenberg>
Matt|home: i mean given how big modern web *sites* are
<Matt|home>
yeah no chance
<azonenberg>
you probably need 128 MB of ram just for the HTML :p
<Mutabah>
The browser code? Most likely
<Mutabah>
the browser's working set - Unlikely
<azonenberg>
But like, on my current embedded platform
<Matt|home>
im a little fascinated by efficiency nowadays. it's taken quite an interesting journey.. especially with things like renewable energy now
<azonenberg>
I have a TCP/IP stack, SSH server, SCPI server, NTP client, DHCP client, serial console, and a bunch of other things
<Mutabah>
a 1080p framebuffer is 2MB on its own - and a modern browser engine will have lots of buffers to make rendering efficient
<Matt|home>
but i just have a nasty feeling ten years from now we're gonna have a 20 gig slackware install :\
<azonenberg>
In about 160 kB of code and maybe 128 kB of SRAM
<azonenberg>
it's running on a 500 MHz Cortex-M7 with perfectly acceptable performance, ssh'ing to it (with a little help from a hardware AES block on the MCU and a curve25519 accelerator on the adjacent FPGA) is faster than a Debian x86 box
<azonenberg>
not even a RTOS per se just a single threaded event loop and a few interrupt handlers
chibill has quit [Ping timeout: 256 seconds]
<Matt|home>
sadly i am beginning to feel the "not enough storage space" creep that im sure literally everyone else started feeling in like.. the early 2000s. but i never needed to store like, 4K videos before okay ;_;
<Matt|home>
srsly there is _not_ enough storage space i can afford to keep all my shit on that i really really want to -_-
chibill has joined #osdev
<azonenberg>
Matt|home: i've stopped putting local disks on my boxen
<Matt|home>
As of 2023, the largest capacity of a single SSD is 100TB, while the largest capacity of a single Hard Disck is 22TB. Nimbus ExaDrive DC is the largest SSD on the market, with a capacity of 100TB. It is a PCIe Gen3 x4 NVME drive that uses 3D TLC NAND flash.Dec 14,
<azonenberg>
other than a tiny ssd for the OS
<azonenberg>
All of my bulk data lives on the Ceph cluster
<Matt|home>
hm. yeah i'd need like ten or so of those drives to store my stuff. that'd be clunky as hell
<azonenberg>
The bulk of that data, in terms of volume of storage, is a few things... my personal photo/video archives dating back to 2004
<azonenberg>
FPGA toolchains (almost a terabyte worth of just those, lol)
<Matt|home>
lol.. do you guys remember when we needed those bigass floppies to store our stuff, we carried them around in long boxes :D
<Matt|home>
i kinda wanna do that with SSDs now
<azonenberg>
and high resolution microscope photos
<Matt|home>
The list price for the 100 TB ExaDrive DC drive is $40,000, or $400 per TB <-- bahahahahahahahahahahahahaha
<azonenberg>
if you ever want to burn disk space fast, get yourself a microscope on a CNC XYZ table and start scanning specimens at 2500 photos an hour
<Matt|home>
oh yeah.. i know how huge some images can be
<azonenberg>
i can see spikes in the disk usage chart on my cluster every time i turn it on lol
<Matt|home>
iirc the upper limit was somewhere around like, 4 gigs or so for the highest resolution/quality frame you could possibly get
<azonenberg>
These are stitches not single frames
<azonenberg>
single frames are like 4 megapixels coming off my system
<Matt|home>
right, but im saying i know how huge that data _can_ theoretically be :d
<Matt|home>
:D
<azonenberg>
with the highest mag objective the physical sensor resolution is about 45 nm/pixel although the optical diffraciton limit is around 300 depending on wavelength
<azonenberg>
but some of my datasets of large specimens are 60K frames stitched and focus stacked lol
<azonenberg>
you run into file format or tool size limits very quickly
<Matt|home>
average SSD size iiiis.... 500GB, which isn't too bad. so i could carry like 20 of them around in a backpack or something
<azonenberg>
JPEG: max 64K in one axis
<azonenberg>
as in uint16 pixel dimensions on each axis
<azonenberg>
TIFF: 32 bit pointers to chunks so max 4GB file size
<azonenberg>
ImageJ stitching tools: 32 bit signed integer pixel indexes so max 2^31 pixels regardless of aspect ratio
<azonenberg>
I have hit all three of those limits
<Matt|home>
:D gj
<Matt|home>
btw what were you scanning
<Matt|home>
and which microscope were u using :D
<azonenberg>
PCBs and ICs, depending on objective in use. It's a LIP-X1 from Labsmore, a purpose built mass imaging microscope combining a CNC XYZ table with a Mitutoyo optical column
<Matt|home>
i have kind of a shitty one i spent.. quite a bit of money on sadly. i really need someone else to take a look at it and help me with the damn lubrication and settings -_-
<Matt|home>
ah
<Matt|home>
i use mine for bio slides mainly.. been really wanting to finish my chlorophyll video for a while now
<azonenberg>
You can see every transistor of the approximately 5000 gates
<Matt|home>
it does the board trace etchings?
<azonenberg>
The DIMM was with a 1x objective, the die shot was with a 100x
<azonenberg>
both were
<azonenberg>
both were stitches of many many frames*
<Matt|home>
rofl.. that _is_ impressive. i legit did not know that gates were literally made up of actual transistors
<Matt|home>
that's wild, i thought it would've been a different design or something at least
<azonenberg>
I do (among other things) silicon security work for my day job, but we use a SEM since modern chips are too small to see all the fun stuff optically
<azonenberg>
same idea of thousands of photos stitched though
<Matt|home>
btw i should mention im _really_ high right now so you showing me this acid blot trip of the back of the PCB is also incredibly wild :D
<Matt|home>
any idea what the purple coloring is from?
<Matt|home>
oh is this like a thermal image?
<azonenberg>
no thats color optical of the silicon die itself
<Matt|home>
ahh
<azonenberg>
those are interference fringes from thin layers of partially etched glass insulator
<Matt|home>
i have very important instructions i need you to follow
<azonenberg>
here's the top layer before etching
<Matt|home>
i need you to come here, to where i live, give me your brain just put it in my hands and you can walk away :D
<Matt|home>
and i will study it
<Matt|home>
sigh.. little under 1/3 of the way done with my friggin usb write.. jesus christ it shouldn't be this slow even at 5MB/s
<zid>
smh you spilled a blob of acid onto it
<zid>
This is weird
<zid>
It's got like, horizontal rows of pads and then just a bunch of vertical links connecting them.. but also a bunch of U shapes
<zid>
Is this some standard type of design I'venot seen before
<zid>
or is it legitimately weird
MrCryo has joined #osdev
<azonenberg>
zid: this looks like a very textbook standard cell 2 metal ~600nm design
<azonenberg>
maybe 500
<azonenberg>
On 3 metal and up processes, normally you have the gates on poly and M1, then alternating layers of horizontal and vertical routing
<azonenberg>
With only 2 layers available, you have the gates and horizontal routing both on metal 1 with rows of routing and cells alternating, then vertical routing on metal 2
<azonenberg>
then at the top you can see a single P-channel MOSFET with two vias each for source and drain (the structure is symmetric, doesn't matter which is which)
<azonenberg>
and a single N-channel at the bottom with one via each for S/D
<zid>
ah yes, of course I can
<azonenberg>
P-channel tend to have higher resistance than N-channel for physics reasons i dont fully understand
<azonenberg>
So they tend to be bigger (effectively more channel width in parallel) to compensate
<azonenberg>
here's what it looks like in schematic
<azonenberg>
the vertical bar with the tab is Vin
<vin>
azonenberg: ??
<azonenberg>
then the left or right side of each of the two transistors connects to power (top side) or ground (bottom side) while the other side is the output
<azonenberg>
here it is with the metal 1 connectivity drawn over it
<azonenberg>
top blue bar is power, bottom is ground, red is input, right hand blue is output
<azonenberg>
each of the blue-red-blue sandwich structures is a transistor
<azonenberg>
vin: sorry i meant V_{in} lol
<azonenberg>
Matt|home: does that make any sense?
<azonenberg>
you can see a second, mirror image NOT gate at the left of the image, and then in the middle there's what looks like an AND2 (an inverter paired with a NAND2) although i can't be absolutely certain without seeing the corresponding metal layer
<azonenberg>
(e.g. it might be a NAND2 with one *input* inverted rather than the output)
<azonenberg>
But it clearly contains a NAND stage and an inverter stage
<Matt|home>
mh?
<Matt|home>
oh god no, im tripping balls right now
<azonenberg>
lol
<Matt|home>
but i get the general gist of the fact that you know everything behind pcb and computer engineering
<azonenberg>
i wish i did :p
<azonenberg>
the rabbit hole goes deeeeep
<Matt|home>
oh if you're that smart i could get you to answer a question for meh
<Matt|home>
so.. when i say "computers" i mean in the most generalist vague sense. for me the most common type of computer i can think of is an IBM
<Matt|home>
but im including like macs and whatever
<Matt|home>
so in general, for computers, is there a way to physically prevent pressing the power button and holding it down from doing anything? can you interrupt something that low level?
<Matt|home>
i.e. the only way to turn it off is to unplug it
<Matt|home>
i asked this b4 here and the answer i got was iirc "not really" but im curious if u know the specicifs:d
<azonenberg>
I don't know how that is actually implemented on desktop PCs
<Matt|home>
aw
<Matt|home>
sadness
<azonenberg>
On laptops, I would expect this is a GPIO input to the embedded controller
<azonenberg>
And the EC actually implements the "hold down for X seconds" check
<azonenberg>
and then shuts things down
<azonenberg>
So with custom EC firmware you could theoretically override it
<azonenberg>
On my own hardware, I have soft power on/off functionality implemented like this, I have a dedicated Cortex-M0 or M4 microcontroller that connects to the enable and power-good signal of every power rail in the system
<azonenberg>
plus the reset of the main processor, the FPGA, etc
<Matt|home>
could you brick a computer hard enough that it rebooted too fast constantly for you to do anything about.. or is there no real way to prevent entering the bios like that
<azonenberg>
when you ask it to turn on it'll turn on each rail in the correct sequence then sit in the background waiting for you to push the off button
<azonenberg>
(or for a rail to drop out of the safe voltage range in which case it does a panic shutdown)
<azonenberg>
um, i mean if nothing else you could overwrite the flash that the bios is stored on
<azonenberg>
the CIH virus famously did this years ago
<Matt|home>
yeh im just being silly
<Matt|home>
hm. yeah bios flash would suck
frkzoid has quit [Ping timeout: 260 seconds]
<azonenberg>
a lot of more modern systems have the bios as a socketed DIP chip so its still field swappable if completely hosed
<azonenberg>
(vs soldered down)
<Matt|home>
in terms of malware im actually kinda enamored with worms. i just really think the idea of self propogation is awesome, but apparently they're stupidly complex
<azonenberg>
Depends on the platform and what libraries are available etc
<Matt|home>
i've heard of the CIH virus, but i forget the name of it.. it was a worm that was just a school project or something but it was so effective it infected half the internet in the 90s :D
<azonenberg>
and how hard getting into the target is
<azonenberg>
the morris worm? the original one?
<Matt|home>
buster worm?
<Matt|home>
oh yeah morris lol
<Matt|home>
i need to think of a better name for my virus..
<Matt|home>
right now it's just called santa's workshop but i kinda don't like the origins of that name so
<azonenberg>
What's more fun is the whole cat and mouse between malware and AV developers
<Matt|home>
https://en.wikipedia.org/wiki/Morris_worm <-- ..... rofl. can you imagine some jackass museum curator asking if he can keep shit from your house to put on public display? the guys still alive :D
<azonenberg>
For example, since malware started to use packers to obfuscate themselves, pretty much any modern AV actually contains a sort of emulator
<azonenberg>
that basically runs the target malware in a simulated system to see what it does
<azonenberg>
and looks for either it unpacking a known malware payload, or doing various things like writing to certain system files that raise red flags
<Matt|home>
It is usually reported that around 6,000 major UNIX machines were infected by the Morris worm. Graham claimed, "I was there when this statistic was cooked up, and this was the recipe: someone guessed that there were about 60,000 computers attached to the Internet, and that the worm might have infected ten percent of them".[10] Stoll estimated that "only a couple thousand" computers were affected.[5]
<Matt|home>
The Internet was partitioned for several days, as regional networks disconnected from the NSFNet backbone and from each other to prevent recontamination while cleaning their own networks.
<azonenberg>
But these simulations are imperfect and if you know enough about how the emulator is implemented, you can detect you're not on a real PC and just sit tight and not do anything malicious, in hopes of not being picked up by the scanner
<Matt|home>
bahahahahahahahahahahahahaha :D they had to turn off the internet to prevent infecting each other :D luuuuuls
<azonenberg>
So the emulators are all obfuscated and locked down to make it harder to fingerprint them
<azonenberg>
and of course a bunch of my classmates years ago blew that wide open :p
<azonenberg>
unrelated: apparently on risc-v with recent gcc, -fno-exceptions does not imply -fno-threadsafe-statics
<azonenberg>
so if you declare a static object in a function g++ will still try to create a whole bunch of mutexing that can throw exceptions and brings in like 50+ kB of libstdc++ i dont want
<zid>
do we know any serenityos people as a hole?
<zid>
whole*
Matt|home has quit [Quit: Leaving]
<vin>
I am trying to understand if/when linux merges or divides VMAs (/proc/pid/smaps). I read https://landley.net/kdocs/ols/2009/ols2009-pages-19-28.pdf but it only talks about sharing pages across virtual machines and nothing about a VMA merges in a single process
stolen has quit [Quit: Connection closed for inactivity]
Jari-- has joined #osdev
Matt|home has joined #osdev
Arthuria has quit [Ping timeout: 256 seconds]
slow99 has quit [Quit: slow99]
slow99 has joined #osdev
Matt|home has quit [Quit: Leaving]
vdamewood has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
<geist>
VMA as in a single mapping
<geist>
?
<geist>
i think it splits and merges then opportunistically. like if you map something and unmap something in the middle of it, it has to split it
<klys_>
after switching to freedos everything works, and it can chainload w95b of course, i should have used freedos sooner.
<klys_>
well the first thing that worked was to try grub1 on the vm
<geist>
i've been reinstalling my main server box with proxmox
<geist>
actually have a little cluster of two proxmox servers right now, pretty neat
<klys_>
i've been through this cycle a few times, freedos -> grub4dos -> linux -> kexec grub4dos -> grub2 -> linux -> kexec grub4dos -> repeat
<klys_>
proxmox is known for some kind of use case i forget what exactly
<klys_>
otoh, I know this is ovmf, except it seems to ignore /EFI/boot/bootx64.efi
<klys_>
right, i'll try putting the gpt mbr back now...
<klys_>
yeah it goes straight to mbr
xenos1984 has quit [Read error: Connection reset by peer]
MrCryo has quit [Remote host closed the connection]
<klys_>
right iirc proxmox is a distribution featuring qemu
<Mutabah>
Iirc it's a KVM/qemu wrapper UI/framework for distributed virutalisation
<klys_>
oh I found efi
<klys_>
the shell says a lot of the .efi files are "not an application."
xenos1984 has joined #osdev
<klys_>
then if I load grubx64.efi -> linux -> kexec grub4dos -> I get an unresponsive grub prompt with Memory: 0K / 0M, MenuEnd: 0x47FFD
mubluekoor has quit [Ping timeout: 260 seconds]
mubluekoor has joined #osdev
<vin>
Yes geist I wanted to know what "opportunistically" actually means and if there is a way to disable it
<chiselfuse>
is this a valid assembler file that i can re-assemble again with an assembler? if so, which flavor/dialect is it and which assembler can do it?
<Mutabah>
it's intel/nasm format by a quick look
<Mutabah>
That said, the `__bss_start db ?` may not compile
<chiselfuse>
i'll try nasm then, i wonder if it includes absolutely all info from the original file or if some info is lost
<chiselfuse>
Mutabah: what does that line mean?
<Mutabah>
Probably that IDA knows that some data could be there, but has no idea what
<Mutabah>
It's in the BSS section, which is "Blank State Segment" - a block of writable memory that is initialised to zero
<chiselfuse>
i'm guessing that in nasm language it's <name of thing> <size> <value> and there should be a value in place of '?' ?
<Mutabah>
Yes, or a `name: resb <size>`
<chiselfuse>
resb?
<Mutabah>
"reserve byte"
<Mutabah>
are you using this to learn assembly?
<chiselfuse>
i don't remember nasm requiring that i specify all that elf information in order to assemble an elf from a few instructions. does nasm automatically add elf information unless explicitly specified as in this file?
<chiselfuse>
Mutabah: i don't know, i was just using ida for something and i decided i'd try every button it has
<chiselfuse>
so now i'm trying this "produce asm" button
gsekulski has joined #osdev
<Mutabah>
Most assemblers will/can emit ELF files - I assume this is intended to be emitted as a raw binary instead
<bslsk05>
github.com: xnu/osfmk/kern/smr.c at main · apple-oss-distributions/xnu · GitHub
<netbsduser>
looks like apple implemented something like freebsd's safe memory reclamation
<netbsduser>
the freebsd people are very happy with SMR and think it a real novelty and the better of RCU and epoch in a lot of cases (less space overhead, faster.) i have no idea how it works yet
<chiselfuse>
looks like it's some hybrid syntax that doesn't work with most assembler and needs fixing
X-Scale has joined #osdev
vai has joined #osdev
X-Scale has quit [Ping timeout: 250 seconds]
X-Scale has joined #osdev
bauen1 has quit [Ping timeout: 268 seconds]
MrCryo has joined #osdev
X-Scale has quit [Ping timeout: 250 seconds]
neo has quit [Ping timeout: 260 seconds]
gsekulski has quit [Ping timeout: 268 seconds]
zxrom has quit [Quit: Leaving]
housemate has quit [Quit: "I saw it in a TikTok video and thought that it was the smartest answer ever" ~AnonOps Radio [LOL]]
<geist>
netbsduser: interesting
Matt|home has quit [Quit: Client closed]
<netbsduser>
geist: a possible thing to look at for yourself given these are not patent-encumbered (and apple using it is a good guarantee)
<geist>
indeed
<netbsduser>
though mjg will turn up (any minute now) to explain why it's pessimal
<geist>
it even says it's derived from the freebsd version
<geist>
apple has the property that they only have to scale to as big as hardware they make
<geist>
so at least for some time they wont have to worry about 256 or 512 core machines, presumably
zxrom has joined #osdev
<nikolapdp>
likely ever if you ask me
<nikolapdp>
unless they change their mind again on the server side of thigns
gsekulski has joined #osdev
<mjg>
they implemented it based on the freebsd code
<mjg>
it is not pessimal, but it is highly deficient
<mjg>
it probably wont matter at the scale os x is running at though
<mjg>
the code can end up in a state where all cpus are pulling on the same line
<mjg>
when smr was being created i tried to explain to the author this will cause massive trashing but he would not listen
<nikolapdp>
mjg what's the difference between rcu and gus
<mjg>
what's gus
<nikolapdp>
that freebsd thing?
<mjg>
freebsd thing is "smr"
<nikolapdp>
that one then
<mjg>
rcu is a general "did everyone pass the grace period" mechanism
<nikolapdp>
> This SMR scheme is directly FreeBSD's "Global Unbounded Sequences".
<nikolapdp>
from the apples' code
<mjg>
lol i never heard "global unbounded sequences"
<mjg>
anyhow smr is kind of like epoch, except freed memory is batched
<vai>
thank you all for the ELF loaders, looks decent code, lots of migration work needed, going to get someone to do somethings, so I can speed it up
<mjg>
it is tightly coupled with the allocator
<vai>
I just want my Windows games to run on top of my OS
<nikolapdp>
while rcu is an independent thing for synchronization
<vai>
(source code in games by me)
<mjg>
oh lol dude this indeed use that in the comment