<bslsk05>
forum.osdev.org: Do you use compiler optimizations? - OSDev.org
<heat>
bzt was goated
<the_oz_>
before zfs was GEOM gates, and until you'd set that shit up you dunno pain
<the_oz_>
comparatively
levitating has joined #osdev
<zid>
That's an incredible essay
<zid>
with many many words
<zid>
to say "I don't know C"
<zid>
I actually met a guy like this in a discord semi-recently, he's amazing
<karenw>
"I mapped something at 0x0 and now my C compiler doesn't do what I expect" oh how cute
<karenw>
Also the alignment rules for %rsp in the sysv abi are dumb and require the stack to be intentionally misaligned... but still, it's a known dumbness
stolen has quit [Quit: Connection closed for inactivity]
<the_oz_>
he just expects the compiler to let him be dumb
<heat>
he should use msvc for that
<the_oz_>
"Why won't you do exactly as I say!?"
<the_oz_>
they have a way to do that. asm.
waffelo has joined #osdev
<zid>
"as someone who has done all this on multiple platforms and compilers, then anyone is free to disagree all they want. you can't argue with working software"
<zid>
'please stop doing UB' -> "you can and for a lot of software you must"
<zid>
he's a great guy
<the_oz_>
>philosophical argument with observers why you're justified in wanting reeality to be different
<the_oz_>
ok but ...
<zijjgfs>
well why is reality like that in the first place??
<the_oz_>
someone decided to
<zid>
like what
<zijjgfs>
like whatever oz was saying
<the_oz_>
now you have a take it or leave it proposition
<zid>
oh he's insane so I don't listen
<zijjgfs>
sometimes you're better off leaving it
<the_oz_>
*turns head sideways at zid*
<zijjgfs>
what you gotta take often times sucks and is completely inane
<zijjgfs>
i *used* to be a practical person :)
<zijjgfs>
*used to* key phrase
<zid>
heat: More places shouldbe troll-caves full of personal insults in which form discussion is useless, imo
<heat>
agreed.
<zijjgfs>
erm. MSVC is the guiltiest one at taking insane liberties whenever UB is present
<heat>
is it?
<zid>
clang*
<zid>
clang famously deletes entire functions such that they fall through to the next function
<heat>
clang fucking deletes the whole function if it sees you sneezed some UB
<heat>
yeah
<zijjgfs>
yes I did see clang do stupid stuff with pointer aliasing in like one version, and then they rolled it back
<zid>
the response to many many people telling them it was dumb was "no u"
<heat>
MSVC at least keeps stuff like NULL ptr derefs
<the_oz_>
lul
<zijjgfs>
so is C "portable assembler" or is it not?
<heat>
*((int *) 0) -> mov rax, [0]
<zid>
clang does not see itself as a useful tool for writing software with, it sees itself as a bastion of utopian code optimization that must not be sullied by reality
<heat>
msvc on arm64 also funnily enough adds an artificial branch + trap on every division, to emulate a div-by-0 trap instead of just allowing UB to do its thing
<zid>
so it will happily compile void f(void) { *((int *)0 = 0; } void secret_admin_only_function(void){ system("rm -rf / --no-preserve-root"); } into f: g: push str; call system;
<heat>
(for context, an arm64 div-by-0 gives you all-1s as a result, IIRC)
<zijjgfs>
convinced they only do that to make extra chaos. it's not like your software is gonna go faster because a NULL dereference was omitted
<zid>
# The use of this compiler is only permitted for internal evaluation purposes
<zid>
I like how it spits that out
ramenu_ has joined #osdev
<heat>
yeah they slapped that on every msvc godbolt compiler now
<zid>
and we're just like "yep, this is obviously pirated, thanks godbolt"
<heat>
nah, this is new
waffelo has quit [Ping timeout: 260 seconds]
<zid>
Do they mean I am not allowed to copy paste the output
<zid>
because it reads like the warning on a DVD screener
<heat>
idk ianal
<zijjgfs>
isn't cl.exe free to use? it comes with various SDKs
<zid>
how you use a tool != having access to a tool
<heat>
YOU WOULDN'T STEAL A COMPILER OUTPUT
<Maja>
zijjgfs: it might go faster if it can figure out that in a particular inlined instance of a function, the null check isn't necessary because the caller dereferences the pointer
<the_oz_>
We dare not breach godbolt's liucense with M$
<the_oz_>
I'm just reading a book
<zid>
I would love to see some software that was faster due to an elided null pointer check ngl
<\Test_User>
zijjgfs: could easily be smaller, though, since the space for that function is no longer in there
<zid>
It's kind of funny tbh
<zid>
There's this arms race of clang trying to get increasingly more aggressive at eliding UB etc
<zid>
and an entirely different set of people making the codegen slower and slower to add security
<zid>
fixing half the issues the former introduced
<the_oz_>
I've been upgrading my -std from gnu99 to c23
<the_oz_>
what a crazy update
vdamewood has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
<karenw>
Pretty sure the Windows SDK is free and contains cl.exe. It's VS that's $$$ for 'enterprises', which they use a very broad definition of.
<zijjgfs>
it's more important to stay true to the author's original intentions even when they do a UB
<zijjgfs>
otherwise you're kinda failing your job as a compiler
<the_oz_>
wat?
<the_oz_>
compiler does what compiler does
<zijjgfs>
I'm saying that the half of clang devs that focus on --omg-optimize are silly
SGautam has quit [Quit: Connection closed for inactivity]
<jjuran>
zid: Wow, so UB really can erase your hard drive!
<zid>
clang only feature so far
<zijjgfs>
I used to be a regular in a C discussion channel, the joke would be that UB could launch nuclear missiles, and i bought it. Now that i'm older and have more perspective I really disagree, and it's not just for 'security' purposes either
<zid>
clang only nuclear missile launches
craigo has quit [Quit: Leaving]
<Mutabah>
UB means that the compiler _can_ do anything, but most likley it'll abuse UB to just make its life easier, or the code faster/smaller
<zijjgfs>
it's all fun and games until you realize they used clang for the nuclear "football" firmware
<Mutabah>
So, if that means letting code run into an unrelated function which ends up firing ze missiles, that'll happen
craigo has joined #osdev
<the_oz_>
code actually executes? oh my word.
<the_oz_>
someone warn the coders
<the_oz_>
doing things DOES THINGS]
<Matt|home>
Good evening.
<karenw>
the_oz_: Ah, a classic example of MS07-052 - Code execution leads to code execution
<heat>
there's no excuse for having a literal empty function not have a return of any kind
<heat>
really, it's probably the trivialest shit to detect
<heat>
if you see llvm opt passes (i looked at it the other day), at some point the llvm bytecode for void foo(){*(int *)0;} becomes empty
<karenw>
"controls reaches end of non-void function" warning is on by default though?
<heat>
empty? add return instruction. problem solved, nuclear missiles averted
<the_oz_>
you could have a construct wherein interfaces assume implementation of functions
<heat>
that's for C code, not LLVM bytecode
<the_oz_>
but ... if that's the necessity ... maybe macro that gets elided
<heat>
OR, if you don't want to return (can't?): invalid instruction
<heat>
fucking crazy ideas right
<jjuran>
You could have a single illegal instruction in the exe that all empty functions map to.
<zijjgfs>
i mean, MSVC pads functions to 16 byte boundaries with 0xCCs
<the_oz_>
hlt: jmp hlt
<karenw>
Yeah, I can't think of a situation where converting "Control reaches end of function" to "Screw you, #UD" is a bad thing
<karenw>
It's still within the bounds of what developers should expect from UB (abnormal termination)
<heat>
int3 is an illegal instruction, hlt is an illegal instruction, ud2 is an illegal instruction
<heat>
this is just for x86.
<heat>
karenw, problem is that this happens without any sort of "control reaches end of function"
<zijjgfs>
int3 is not an illegal instruction
<heat>
it's a trap instruction
<karenw>
hlt isn't an illegal instruction in ring 0
<heat>
ohno
<heat>
musl uses hlt as a trap instruction fwiw
<zijjgfs>
back in my day which isr handler got triggered mattered
<karenw>
In any asm I write, int3 is my "You should never have got here, what are you smoking" instruction
<zid>
int3 came about because you could pad inbetween functions with it but it wouldn't fill the uop caches
<zid>
with garbage
<karenw>
Is that why MSVC loves filling unused parts of .text with it? TIL.
<heat>
now that straight-line-speculation became a thing, ret; int3; is the only way to avoid that shit
<zid>
yea, so when you call blah and blah is only 4 bytes, and the cpu fetches 16 bytes, 12 of which are int3
<zid>
it doesn't speculate, flush your caches, etc
<karenw>
Speculation considered harmful
<heat>
mharden-sls=all will add the int3 stuff after every ret
<zid>
it does if you know for a fact that the code it's speculating on is guarenteed not to run
<zid>
or rather, is
<karenw>
I mean, it's considered harmful by the bi-monthly "spectre/metldown" style exploits that keep being found. But also for that reason, yes.
<heat>
meh, fake exploits
<heat>
don't really matter
<zid>
fake exploits that don't matter is exactly how I dealt with spec/melt
<zid>
I'm not a VPS provider
<heat>
yeah bro, if your instruction sneezes into the cache, i can use a high resolution ns-accurate timer to measure it
<karenw>
Eh, depends. If your hardware runs untrusted, unsandboxed code. But as zid beat me to, I'm not a VPS provider
<zid>
just vmexit rtdsc if you care so much, I don't
<zid>
tdtsc
* zid
gives up
<heat>
meanwhile my mom uses a keyboard app with ad popups and probably logged her keystrokes for the longest time
<karenw>
If you install a custom keyboard on android it gives you a big scary warning about keylogging that I guarentee 99% of users click past without reading.
<heat>
eu4-CRACKED-SETUP.exe needs admin? ok boss, there you go, have fun
<zid>
if someone turns a ring3 payload into a ring0 payload on my desktop, they get to install printer drivers
<zid>
rather than just ruin my life
<heat>
at that point
<heat>
they do deserve it
<heat>
fuckin dedication and skill right there
<karenw>
Getting ring0 isn't scary. Getting persistance that can surive past "backup, format, reinstall" is slightly more scary.
<heat>
the gang moves the LAPIC to SMRAM
<zid>
which is why uefi is scary and insecure
<zid>
compared to bios
<karenw>
That was the most stupid exploit I read in a long time.
<zid>
I miss my pop ss exploit my last cpu had
<zid>
incredibly good bug
<heat>
bios also had SMM, dont worry
<karenw>
Lmao. "POP SS; SYSENTER" gg you are now ring 0?
<zid>
yea, pop ss had really high latency
<zid>
and the sysenter would run without serializing to it
<zid>
so the pop ss ran after the sysenter finished
<zid>
I think it didn't actually work with sysenter though, it only worked with debug trace exceptions or something, but yea, forgot a flag that pop ss shouldn't speculate across ring changes
<karenw>
Speculation considered harmful again
zijjgfs is now known as ring0_star
<zid>
oh I think I remember what it was now? the pop ss actually would *cause* a debug exception, but wouldn't retire until *after* the swapgs
<zid>
so the debug exception would then swapgs again
<zid>
swapgs best instruction, 0 exploits
<zid>
FRED is useless
<kof673>
> it's not like your software is gonna go faster because a NULL dereference was omitted i dunno, stuff like that makes me think it made a difference...on a 286 :D
<kof673>
with ack, i can write if (0) IIRC and it will generate code for that.......... some of this stuff made sense on older compilers...
eddof13 has quit [Quit: eddof13]
<the_oz_>
repeat it a jillion times and happen to sit on cacheload the exe N or N+1 and THEN MAYBE!!! *yanks hair out*
<kof673>
well one time i know, i mean 30 functions, called in a loop, at some point i allow it may matter
<kof673>
i always do null checks, so i am glad if it does not amtter. and bail at the first sign of trouble. if a function was invoked wrongly, bail as early as possible
<kof673>
but then people complain you shouldn't return from a function from multiple points, that is not "structured" <pshaw>
<kof673>
*matter
<the_oz_>
that's silly
<kof673>
you are supposed to indent your code into giant nested ifs i suppose /s
<kof673>
and then one return at the end /s
troseman has joined #osdev
troseman has quit [Client Quit]
heat has quit [Read error: Connection reset by peer]
<the_oz_>
also, caveat, don't be a retard like passing in malformed data
ramenu_ has quit [Quit: ZNC 1.8.2+deb3.1+deb12u1 - https://znc.in]
eddof13 has joined #osdev
eddof13 has quit [Client Quit]
heat has quit [Ping timeout: 260 seconds]
levitating has quit [Remote host closed the connection]
levitating has joined #osdev
levitating has quit [Ping timeout: 260 seconds]
Arthuria has quit [Ping timeout: 260 seconds]
sly has joined #osdev
sly has quit [Client Quit]
frkazoid333 has quit [Ping timeout: 276 seconds]
goliath has joined #osdev
<the_oz_>
ss
netbsduser` has joined #osdev
karenw has quit [Ping timeout: 244 seconds]
xenos1984 has quit [Quit: Leaving.]
xenos1984 has joined #osdev
emntn has quit [Quit: WeeChat 4.4.2]
GeDaMo has joined #osdev
heat has joined #osdev
spareproject has joined #osdev
xenos1984 has quit [Quit: Leaving.]
xenos1984 has joined #osdev
xenos1984 has quit [Client Quit]
xenos1984 has joined #osdev
levitating has joined #osdev
levitating has quit [Ping timeout: 252 seconds]
emntn has joined #osdev
Left_Turn has joined #osdev
the_oz_ has quit [Read error: Connection reset by peer]
<netbsduser`>
once again i am working on the vm system
<netbsduser`>
i am replacing the big page tables locks with a per table lock
<netbsduser`>
vm objects will partake in a similar locking scheme (but they have tables configured as direct pointers in the VMO, indirect, doubly-indirect, triply-indirect, quadruply-indirect, which is good for the common case of only a page or two of data being incore)
<sortie>
Does anyone know any hobby operating systems suitable for server usage? Because honestly, I can't think of any. Haiku is very much meant as a single user desktop experience. Managarm is just not there at all, didn't manage to get it running with a sshd. SerenityOS about being a desktop too and they don't provide disk images which instafails it as a server.
<netbsduser`>
managarm has good underlying technology and will probably make an excellent server whenever someone devotes some time to clearing it up for that use
<sortie>
Yeah I am impressed with managarm and I believe it will get that far. Sortix is still quite ahead there in the server use cases for now
<bslsk05>
wiki.osdev.org: Notable Projects - OSDev Wiki
<Mutabah>
I did have a TCP stack... but no servers, just clients
<sortie>
A big part of a server story is also how to install & update the OS in addition to an init system for managing daemons and servers
<sortie>
Plus having ports of key servers that people might want to run
vursc has quit [Ping timeout: 248 seconds]
vursc has joined #osdev
vursc has quit [Client Quit]
j00ru has quit [Ping timeout: 252 seconds]
Turn_Left has quit [Read error: Connection reset by peer]
Turn_Left has joined #osdev
Turn_Left has quit [Remote host closed the connection]
Turn_Left has joined #osdev
cloudowind has quit [Ping timeout: 272 seconds]
cloudowind has joined #osdev
<Ermine>
if i port go to onyx i can give it a server try theoretically
Arthuria has joined #osdev
ramenu has joined #osdev
bauen1 has quit [Ping timeout: 252 seconds]
j00ru has joined #osdev
heat has quit [Read error: Connection reset by peer]
heat has joined #osdev
the_oz has joined #osdev
annamalai has joined #osdev
edr has joined #osdev
kl1n3 has joined #osdev
kline has quit [Ping timeout: 630 seconds]
heat_ has joined #osdev
heat has quit [Read error: Connection reset by peer]
<sortie>
Ermine, is the availability of Go something you consider important for a server?
<Ermine>
sortie: no, but some stuff I use is written in go
<sortie>
That's cool. What stuff is that? Pure curiosity
<sortie>
I am wanting to port Go too. It's one of the more doable ports in terms of bootstrapping
<Ermine>
e.g. soju
<Ermine>
also a telegram bot I've written
<sortie>
Right so to run some personal programs and servers, bots
<Ermine>
I also want to deploy Xray server
<Ermine>
Yeah, it's personal stuff
<sortie>
It's why I want to package a bunch of langs for Sortix, like python, perl, php, lua, etc. Because I think the proglang availability is what will really determine if people can run their toys on my OS which is where they might find it actually useful
Left_Turn has joined #osdev
<Ermine>
Well, that's what are hobby OSes are for
<sortie>
I will do go too. node.js actually kinda succeeded although not a nice port and a lot left to make it work well. rust is bootstrap hell sooo maybe not this release
<Ermine>
Ah, also there's one thing I use which is written in Rust
Turn_Left has quit [Ping timeout: 245 seconds]
_ngn has quit [Ping timeout: 260 seconds]
<Ermine>
sortie: you can try to port Hare as well. Should be relatively easy
<sortie>
Yeah took a look at it
<sortie>
Mostly I think it might be too small a language for the main distribution but think it may be a good fit for the community ports collection I want to do
heat_ has quit [Read error: Connection reset by peer]
heat_ has joined #osdev
<Ermine>
Also, if I wanted to run a code forge that would be either sourcehut or gitea/forgejo, but gitlab would be a more popular choice
<Ermine>
In either case an RDBMS would be required
<Ermine>
Also yeah, when people are presented with a new operating system, they would first ask how to run their (existing) workloads on it
<nortti>
aiui forgejo can just use sqlite
<Ermine>
Didn't know, but that would simplify things indeed
<Ermine>
Ah, also for server it would be nice to have a firewall
<ThinkT510>
If you are tackling harelang you could also try to port zig
Turn_Left has joined #osdev
housemate has joined #osdev
Left_Turn has quit [Ping timeout: 272 seconds]
goliath has quit [Quit: SIGSEGV]
SophiaNya has quit [Remote host closed the connection]
ptrc has quit [Remote host closed the connection]
<sortie>
Ermine: Firewall definitely makes sense. Although I imagine a lot of such servers would be running in the cloud where the cloud usually provides a firewall at the cloud level
<heat_>
i get really excited when someone mentions RUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUST
<heat_>
i have a strong urge to rewrite everything in it
<heat_>
rust-zfs please
<heat_>
sortie, onyx tries to be performant for insane (read: server) workloads
<heat_>
but i am missing features and packages and...
<sortie>
I mean that's where we've been different. My priority is to even be able to run these workloads before I make them optimal :)
<heat_>
i can build gcc with speeds somewhat competitive with a linux system, and that makes me happy
<sortie>
My live ramfs environment build speeds are kinda comparable with -.j1
bauen1 has joined #osdev
<Ermine>
sortie: I have firewalls enabled on my VPSen
<Ermine>
nikolar: what's what
<heat_>
he's asking what the "rust" thing is
<sortie>
I'm curious, what firewall rules do you find useful for a server?
<nikolar>
i asked heat what he was quoting
<heat_>
i was quoting the Ermine man himself
<heat_>
sortie, all of them?
<Ermine>
sortie: my case is the simple one, filter anything but ports where the services are actually running
<heat_>
one mustn't assume there's a cloud in the way
<heat_>
also you can add custom rules with error reporting, etc with iptables
<sortie>
Ermine: Well if nothing is running on those ports, it doesn't matter much of they're filtered? Nice hardening though
<sortie>
heat_: Beself but also yeah just listening and learning here
heat_ is now known as heat
<heat>
I AM HEAT
<Ermine>
Yeah, that's "just in case" hardening
<sortie>
I imagine that most people would run my OS in a virtual machine or cloud instance rather than an actual dedicated machine (but also want to support that)
<Ermine>
But... nmap uses closed ports to detect OS
<heat>
netbsduser`, two tricks for page tables: 1) don't require locks in the fast case for upper page tables 2) you only really need per-pgtable locks for tables that will actually be contended (pmd/pte level)
<Ermine>
Renting VPS is the simplest option
<heat>
linux page table locking is funny, basically you _always_ know page tables are stable if you hold the mmap lock in any case. page reclaim can't zap page tables, only ptes/pmd entries themselves (and they'll hold the per-pgtable lock there)
<Ermine>
At home you can only have SBC unless you have a room with good sound isolation, and you're ready to pay a bill for full-fledged server
<netbsduser`>
heat: i need to have a look at what linux is doing
<netbsduser`>
i have some intuitions of my own peculiar to the way i've done things
<heat>
btw sharded vm_object locks are probably not really worth it
<Ermine>
Or... maybe some older computer will do the job, but you need to put it somewhere and keep it online 24/7
<netbsduser`>
(i rely a lot on the type stability of page structs, the first type stable thing i ever touched)
<heat>
you can get away with a big lock for writing and RCU for reading, it'll get you very far
<heat>
like, linux 2024 far
<netbsduser`>
with vm objects i was thinking, those *are* somewhat different to process page tables
<netbsduser`>
since most process page faults require installing a PTE
<netbsduser`>
but in the happy and probably quite common case, VM object lookup ends in finding a page already there just not mapped in the faulting process yet
<heat>
sortie, software wise, i'd consider: apache, nginx, mysql/mariadb, *postgres*, various language runtimes (java, golang, rust at least)
<heat>
python and perl ofc
<sortie>
I have sqlite atm
<Ermine>
why is postgres highlighted
<sortie>
Java is definitely a need but really afraid of it
<heat>
because postgres is really important and widely used
<heat>
same for apache and nginx
<sortie>
I have nginx. I wasn't sure apache was really needed with nginx
<heat>
postgres is probably the best open-source DB, possibly best DB
eddof13 has quit [Quit: eddof13]
<heat>
mongodb for javascript webscale fans
<netbsduser`>
i do already have a mechanism that makes it so no locks proper are held while vm object readin (and this goes for process pages too) is underway, viz. "busy pages" whose vm_page has a pointer to a refcounted structure containing a semaphore signalled when the I/O is done
<Ermine>
mongodb is nonfree tho
<netbsduser`>
and in fact there is currently a strict discipline that anyone who sees one of those waits until told to proceed (so readin does not generally need to reacquire locks, only to unbusy the pages and signal the semaphore)
<heat>
server side public license goated
<heat>
node.js ofc is also important, but you kind of already have that
<netbsduser`>
mongodb is still free, they just use their own PPL-style licence instead of whatever they used before (AGPL?)
<heat>
that's basically it for really basic software. then you want solid observability support, iptables/firewall stuff
<heat>
OPENSSL
<netbsduser`>
if it became a paid product they would certainly face revenge at the end of a fork
<heat>
also, systemd
* heat
runs
<heat>
zig is mostly irrelevant, hare is also, so if you want to push hard for "oooh yeah sortix server so gooooood" then i'd recommend you don't spend time on those
<Ermine>
Are you targeting Real Prod Workloads
<sortie>
heat: That is basically what I meant by saying community ports collection :P
<heat>
though i'd expect hare to not need much work from your side, though idk
<Ermine>
Also, postfix
<heat>
the real challenge is to make everything work really smoothly and fast and stable
<sortie>
Yeah I should port postfix and dovecot
<sortie>
Self-hosting sortix.org email would be awesomesauce
<heat>
don't be like the oracle cloud free VM that OOM's on a software update
<sortie>
heat: That only happens in the live environment :P
<sortie>
As for systemd, I do have Sortix init :)
<heat>
sortix init does not have cgroups nor weird containerization nor seccomp support nor <insert other fancy systemd features here>
<sortie>
You don't need that
<sortie>
But I will build such features later. Sortix was always meant as a container friendly recursive nested platform
Starfoxxes has quit [Ping timeout: 252 seconds]
<netbsduser`>
cgroups in service management were useful, that's why solaris did it 9 years earlier (contracts)
<sortie>
I am definitely running into limitations with process groups and sessions being woefully unfit for actually grouping and managing processes in init
<heat>
contracts, like the ones you signed with sun so they made sure you didn't show solaris benchmarks
<netbsduser`>
i think in terms of kernel support for service management, something like contracts/cgroups is the one thing that i think the lowest common denominator in unix is sorely lacking
housemate has quit [Quit: Nothing to see here. I wasn't there.]
housemate has joined #osdev
housemate has quit [Client Quit]
frkzoid has joined #osdev
housemate has joined #osdev
Starfoxxes has joined #osdev
housemate has quit [Ping timeout: 244 seconds]
Left_Turn has joined #osdev
Turn_Left has quit [Ping timeout: 260 seconds]
annamalai has quit [Ping timeout: 248 seconds]
eddof13 has joined #osdev
housemate has joined #osdev
housemate has quit [Remote host closed the connection]
<ring0_star>
not to shit on everybody's unix clones but has anybody tried anything different?
<ring0_star>
for example doing something different other than the filesystem heirarchy standard
<ring0_star>
in some ways i am nostalgic for the way Windows used to shove everything related to a certain application into C:\Program Files\
<ring0_star>
it is technically bad, but an overlooked aspect of computing i think is discoverability. you could learn a lot by simply poking around in that one directory, but these days the right way of doing things has everything related to a single application spread amongst 50 directories, many of which in ~/.local or something, and there's a lack of consistency
<zid>
yea I prefer the diretory blob approach too if I am honest
<ring0_star>
the reason why it's no big deal for people like ourselves is because we already know what to look for if not already have the answer to where something is or ought to be. it's hard to think of things from a novice point of view when you're not a novice
<zid>
but the way linux software operates tends to be much different
<zid>
simple binary, public headers, public documentation, etc
<zid>
and directory blob is a TERRIBLE approach to that fileset
<ring0_star>
it is technically bad, but using FHS has some cons that i've yet to see addressed by anything
<ring0_star>
could be some kind of virtual filesystem on top that organizes data a different way. oh, Windows also tried to do something like this with the Documents directory and people got pissed off about it i recall
Turn_Left has joined #osdev
hwpplayer1 has joined #osdev
Left_Turn has quit [Ping timeout: 265 seconds]
<nikolar>
Also files that belong to a package are also discoverable through your package manager
<nikolar>
Even though they aren't shoved into a single dir
housemate has quit [Quit: Nothing to see here. I wasn't there.]
housemate has joined #osdev
annamalai has joined #osdev
hwpplayer1 has quit [Quit: I'll be back later thanks all !]
<Ermine>
heat: I don't think such advanced systemd features are that useful in hobbyist setups
<Ermine>
and for professional stuff I'd like to have OS audited for security
xenos1984 has quit [Ping timeout: 245 seconds]
<zid>
Imagine if nikolar was a better person
<zid>
you can't imagine it either huh
netbsduser` has quit [Ping timeout: 252 seconds]
<Ermine>
why to imagine something that is real
<kof673>
> has anybody tried anything different? well, that's double-sided w.r.t. do you want "native"-grown apps (who will write them? limited resources) or do you want to port existing stuff, that was written to run elsewhere? not all one way or the other, but for simplicity.............call it momentum
xenos1984 has joined #osdev
<kof673>
same thing, do you expect people to run their existing "workloads" by porting some things over, or will you do things a "new" way, and expect them to use your "new" "native" stuff ?
<kof673>
limited resources, path of least resistance is to go with the existing flow :D
<kof673>
if you had unlimited resources, conway's law would favor otherwise :D
<ring0_star>
there is no concern about "what people will run" and "path of least resistance"
<kof673>
i agree, i think "research" is a good name for that :D
<ring0_star>
OS dev has always been a hobby. if you have some sort of delusion about there being a critical mass of people using your thing over Linux or whatever, you're dreaming
<ring0_star>
there is absolutely no reason to use anything anybody makes here unironically. so that shouldn't even be considered a motivating factor.
<kof673>
preaching to the kof :D
<zid>
shouldn't your nick have starr with two rs btw
vdamewood has joined #osdev
<ring0_star>
yeah it should, i was going to put it like that but then idk i just didn't
ring0_star is now known as ring0_starr
<kof673>
and "research" may or may not result in a "product" but that is not the end-all be-all or the primary motivation per se
netbsduser` has joined #osdev
<ring0_starr>
but back to the file pile discussion. package managers will give you listings of files belonging to a certain pkg, which isn't really the same as there being a directory with all those things in it
<zid>
yes, but neither is 'better'
<ring0_starr>
you can make a collection of symlinks I suppose
<zid>
so who cares
<zid>
just pick something and distribute your shit in a way that makes sense with it
<zid>
not that a hobby osdever has anything to distribute :P
<ring0_starr>
malware !!
<ring0_starr>
package managers also won't list user created files. ~/.config/foobar.cfg might be a location mentioned in some doc somewhere but it won't be part of the pkg list
<zid>
good, you don't actually want your package manager deleting those
<zid>
weirdly enough
<ring0_starr>
which gets back into the thing about discoverability
<ring0_starr>
the more documentation one needs to read in order to do a simple thing, the less they're gonna want to do it
<zid>
I don't see the difference, tbh
<zid>
it's just that linux packages TEND to ship with some public files, and windows ones don't
<zid>
if I install quake3 I still end up with a directory with quake3 in it
<ring0_starr>
should go into /opt but pfht
<zid>
only shared stuff ends up spread around in /usr/share and stuff
<zid>
the alternative would be super shitty, I go to use quake3/api.h and have to specify the full path, or clog my gcc include path with thousands of things in /opt
<zid>
or do man /opt/games/quake3/doc/manual etc
eddof13 has quit [Quit: eddof13]
<ring0_starr>
how often do you make new quake3 projects that it needs to be optimized for that exactly
<ring0_starr>
libraries i can see. but
<zid>
libraries are the 'annoying' ones that leave files everywhere, and 'software' is the stuff that makes a dir full of files that are standalone
<zid>
both systems do it the exact same way, except the *types* of stuff you install on each are more distinct
<zid>
both use a big dir for each piece of software, both use a smattering of small shitty files for libraries
<ring0_starr>
the Windows way of handling it would be to create a symlink in some kind of centralized directory where man quake3 would work the same as if you specified the full path.
<zid>
like, the .NET installer makes 8000 files and 40000 registry entries
<zid>
in thousands of directories
<zid>
and the quake3 installer makes one in Program Files
<zid>
linux is the same
* kof673
points at nextstep ghost, mac, does their app.dir and framework thing and whatever else
<ring0_starr>
what, different applications do different things?
<kof673>
*program.app
<zid>
yes, your perceived difference is covered almost entirely by just that
<kof673>
a minor thing i will say is, not all programs may like shared nfs /home, esp. different cpus or osen running the same app :D
<kof673>
so even within unix-style, there are levels...
<ring0_starr>
the .NET installer has almost this personality to it where it feels like it's part of the OS, so it leaves its little poop trail everywhere
<kof673>
s/app/program/
<bslsk05>
<kof673*> a minor thing i will say is, not all programs may like shared nfs /home, esp. different cpus or osen running the same program :D
<zid>
yea, I have issues with the way software works on both systems, but neither is the fault of how the filesystems are laid out
<zid>
i.e. windows programs tending to be too self contained and refusing to share or co-operate with other things
<zid>
it's very much just the people who make the software
<ring0_starr>
and i don't really have an issue with either, it's just a thought experiment of what could be improved by using a different paradigm
<zid>
there are memes in how things are packaged
eddof13 has joined #osdev
eddof13 has quit [Client Quit]
<ring0_starr>
but then the next logical conclusion is that if the app developers package things their way, and the OS wants to organize things a different way, and the end user wants things in their own way too, the only real solution is to present the files with as much contextual information as possible so that the user can do whatever they want with it
eddof13 has joined #osdev
<zid>
You mean, mash 'Next ->' :P
<ring0_starr>
so the package manager as existing only sees /etc/foobar/foobar.conf that needs to get unzipped and placed there, whereas this new thing would say "this is a global config file, in the traditional unix layout it'd go here, but..."
<zid>
(advanced options in windows installers are very much a thing)
eddof13 has quit [Client Quit]
<zid>
don't install this for all users, instead install it into Users/zid, don't install the redist, instead, I will run it to extract it to C:/zid/bin etc
<ring0_starr>
one "view" would be a directory heirarchy with a bunch of hardlinks to said files with one form of organization that the user perfers
<ring0_starr>
which might be the same or different than what they developer intended
<ring0_starr>
yeah and on linux you can do the same thing, you just need to search for the right command line switch and/or config file to edit first. and then it might not work because you edited the wrong file it doesn't read from or you made a syntax error unintentionally. GUIs eliminate that issue
<kof673>
yes, some package managers are more equal than others (meaning, have to decide who is authoritative, os or the author of the program/language-specific package manager, etc.)
<ring0_starr>
my point is that there would be no ultimate authority, just different ways of organizing the same file
<kof673>
fair
<ring0_starr>
because what the file is or does would be based upon some kind of "tag" metadata. i think microsoft tried to do this several times, first with alternate file streams in NT and XP, then there was that ReFS that didn't go very far
<kof673>
well, sounds MVC to me, different views even simultaneously :D
<kof673>
"personalities" or subsystems or whatnot even
spareproject has quit [Remote host closed the connection]
<kof673>
and even before it gets that far, packaging things, you can argue there is something parallel w.r.t. which build system to use....should you "import" things and maintain a local copy, try to get them all using the same build system, etc.
<SystemPrompt>
what's actually needed is a database that allows multiple keys. fetch config: select from files where user='bob' and app='quake3' and filename='config'; delete user: delete from files where user='bob'; uninstall app: delete from files where app='quake3';
goliath has joined #osdev
the_oz has quit [Remote host closed the connection]
<SystemPrompt>
but... it doesn't compose as well as hierarchies - if an app supports sub-accounts do you add a sub-account column to the files table?
heat has quit [Read error: Connection reset by peer]
heat_ has joined #osdev
griddle has quit [Quit: griddle]
<heat_>
Ermine, the features are kind of important for prod though
<heat_>
which, you know, srs bzns server operating system
griddle has joined #osdev
<Ermine>
again, nobody puts hobbyist OSen to prod
<heat_>
this whole discussion is about hobby os for server usage
<heat_>
these are features you want in server usage
Arthuria has quit [Ping timeout: 260 seconds]
<heat_>
like, technically you can run windows 95 as a "server", but that's not the point is it
<Ermine>
There are different servers
<Ermine>
Like, there's a server with my toys, (as sortie have well said) and some google-ish webscale stuff
<heat_>
cgroups and containers and namespaces and seccomp and <...> are all perfectly valid, useful features for every kind of server
<heat_>
even desktops
<Ermine>
yes, they are perfectly valid, but it easier to get job done without them if you don't require too much. Hence it's drastically easier for hobbyist OSes to meet the requirements of such simple setups
housemate has quit [Quit: Nothing to see here. I wasn't there.]
cloudowind has quit [Ping timeout: 252 seconds]
GeDaMo has quit [Quit: 0wt 0f v0w3ls.]
housemate has joined #osdev
cloudowind has joined #osdev
vdamewood has quit [Quit: Life beckons]
netbsduser` has quit [Ping timeout: 272 seconds]
<sortie>
Ermine: I literally put Sortix in production all the time. It's scary, it's trial by fire, but it's the _only way to get production ready_ :)
<sortie>
Hosting sortix.org reliably on Sortix is such a crazy thing, it's exposed to the actively hostile internet, and if I fuck up, it takes down myself
heat_ is now known as heat
<ThinkT510>
not being a popular target probably helps
<sortie>
But how can anyone else possibly trust me for production quality if I don't even run my own stuff in production? Most alternative operating systems don't even run themselves.
<sortie>
ThinkT510, indeed, but still lots of hostile scans on the http port, probes on sshd all the times, ddos possibilities, hostile network packets, ssl vulnerabilities
<sortie>
Had various kernel panics when I first put Sortix on the real internet
<ThinkT510>
glad you didn't stick with the fake internet
<sortie>
I do for other stuff. There it's proxied via qemu slirp and such
housemate has quit [Quit: Nothing to see here. I wasn't there.]
griddle has quit [Quit: griddle]
goliath has quit [Quit: SIGSEGV]
<cloudowind>
qemu is grub together taking me insane
<nikolar>
call the bondulance
<cloudowind>
if i am emulating linux i use systemd-nspawn instead it is very hard to give qemu macvlab ipvlan stuff ,
xenos1984 has quit [Read error: Connection reset by peer]
griddle has joined #osdev
griddle has quit [Client Quit]
heat has quit [Read error: Connection reset by peer]
heat has joined #osdev
griddle has joined #osdev
netbsduser` has joined #osdev
xenos1984 has joined #osdev
housemate has joined #osdev
<cloudowind>
spending $500 with ambulance?nope , i would rather contact the developers of qemu and grub and p-ay 250 each and get a detalailed report about my issues
<heat>
good news: i got bash-completion
<heat>
bad news: i got rpm db corruption again
<heat>
good news: i can repro the db corruption again
<kof673>
the frogurt is also cursed, that's all you really need to be 90s fluent heat
griddle has quit [Quit: griddle]
griddle has joined #osdev
griddle has quit [Client Quit]
<nikolar>
heat: is it different than last time
<heat>
nope, same kind of corruption. block looks erroneously zeroed
<nikolar>
sadge
<zid>
is it repro
<zid>
or just it happened again
<FreeFull>
I wonder if maybe one of your memory pages is getting zeroed first, and then that's getting written to disk?
griddle has joined #osdev
<heat>
it's pretty/100% consistent, though last time i accidentally unfucked it by giving it a good copy
<zid>
If you can get it working off a filesystem checkpoint
<heat>
FreeFull, wdym?
<zid>
you can at least add an if(page_to_be_written == 0x89483943) stacktrace();
<zid>
I debugged my gameboy emulator a *lot* like that :D
griddle has quit [Client Quit]
* FreeFull
shrugs
<heat>
the gist here is: page cache looks good, and you can totally read the db properly as long as it's all in the page cache. after a reboot (even properly sync'd, etc, etc), it goes bad in one single block. blocks around it look good in the diff
<nikolar>
is a only a single block corrupted in the whole fs
<heat>
i think sqlite uses memory mapped IO on onyx, and that gives me some hints in what might be going fuckywucky
<heat>
yes, fsck passed
<nikolar>
interesting
<heat>
quite possibly what could be happening is that somehow the mmap write isn't dirtying the page (shared pages are write protected, first write sets the dirty flag in struct page, on writeback all those mappings get reset to write protected)
<heat>
and thus writes look like they're present in the page cache, but obviously since the page was never dirty, it never gets written back
<heat>
but all of those paths have gone through some stress tests, so i'd hope it's correct...
<heat>
also: sqlite's code style is COMPLETE AIDS WHO THE FUCK WROTE THIS
<bslsk05>
github.com: sqlite/src/os_unix.c at master · sqlite/sqlite · GitHub
Dead_Bush_Sanpa1 has joined #osdev
<heat>
????
netbsduser` has quit [Ping timeout: 252 seconds]
Dead_Bush_Sanpai has quit [Ping timeout: 255 seconds]
Dead_Bush_Sanpa1 is now known as Dead_Bush_Sanpai
<sortie>
heat: Still not as bad as GNU brace placement and rules for tabs and spaces
<sortie>
heat is osdeving like normal not realizing that I'm finishing up some really, really important stuff that makes me win
<heat>
im actually not osdeving
<sortie>
While you sleep, sortie is working to take over the world
<heat>
the david goggins of osdev
<heat>
while im sleeping you're waking up at 5 am to write some init code
<sortie>
heat, Sortix has port sets like 'minimal', 'basic' and 'all' as a way to easily install a supported subset of ports. However it was never remembered which one you installed, and tix-upgrade doesn't install anything extra added to such sets later on. ... So last night and today I have actually made those sets into real port packages and added tix-upgrade support.
<sortie>
Why am I telling you this? IT'S THE LAST MISSING FEATURE IN MY NETWORKED PACKAGE MANAGEMENT :D
<heat>
:)
<heat>
i got NETWORK PACKAGE MANAGEMENT too
<heat>
thanks dnf and rpm
<sortie>
Unblocking merging tix-upgrade once it's fully documented and tested and finished -- that means I have actual shipped networked package management :D :D :D
<sortie>
As much as tix is kinda a hack, this part of it is actually to not be hacky
<sortie>
*starting to not be hacky
<zid>
sortie adding portage profiles
<sortie>
The really cool part of this work was making it real pure packages and eliminating special cases
<heat>
also ports are lame, real men use *packages*
<zid>
dang, portage has fuck loads
<sortie>
Ports go in, packages come out
<heat>
think about all the garbage you can install