<bslsk05>
github.com: Page not found · GitHub · GitHub
<sonny>
oof
<sortie>
None of this is advanced computer science, the complexity lies in many different upstreams organizing their releases weirdly and varied, so the difficulty lies in managing this variance and reducing it to something expressible in a simple manner
<sortie>
UPSTREAM_VERSION_PAGE="https://www.python.org/downloads/source/" ← Instead I define that the script should scrape the official website to find the actually published versions
<sonny>
I can't think of a solution other than hoping these all use git and having the ability to be able to build from a tag, but that's a lot more work even if it's easier to automate
<sortie>
Unfortunately then you run into a lot of stuff using e.g. autoconf where ./configure needs to be generated with the right autoconf version, and you want the upstream to do it for you
<sonny>
grrr
<sortie>
Plus, well, lots of stuff doesn't actually use git..
<sonny>
yeah, that too
<sortie>
So there's a few weird cases like this and I'm sure I'll run into more in the future. The good news is that in most cases, the tarball has a neat systematic name in a single directory with directory listings on, so it can be easily scraped by default.
<sonny>
ok, that's good to know. It seemed like it would be more chaotic
<sortie>
So MOST ports really just need to define the upstream site and the name of the tarball, and my script can figure out the rest
<sortie>
E.g. https://ftp.gnu.org/gnu/ is a pretty wonderful place in most cases where it's organized and is super easy to scrape
<bslsk05>
ftp.gnu.org: Index of /gnu
<sortie>
(the contents of those tarballs may leave something to be desired but at least they're pretty well organized)
<sortie>
sonny, yeah so I have 67+ ports, so I run into LOTS AND LOTS of weird stuff where the upstream deviate from the norms and fuck up in all sorts of ways. It requires a whole lot of good design work and happily patching workarounds for problems, but I'm very proud I managed to WIN and get these kittens herded
<sonny>
good work
<sortie>
E.g. I can 1) cross-compile all of those ports (totally does NOT work out of the box, took a lot of patching and investment 2) compile all of those ports natively on my OS.
<sonny>
interesting so, this forms a test for posix compliance?
<sonny>
or is it just software you needed?
<sortie>
It more is a test for meeting the de-facto Unix spirit
<sonny>
I see
<sortie>
Definitely I need to do POSIX properly enough, but it also tests a lot of those assumptions baked into things that aren't part of POSIX
<sonny>
I keep wondering what a posix subsystem would be like, I can only see something that is very system specific
<sonny>
ah, in that case you report a bug?
<sortie>
Well if I don't implement POSIX properly I do have to fix my bug, or add my own little patch temporarily working around the problem
<sortie>
If the upstream software assumes something non-POSIX such as the filesystem layout or whatever extensions, I either have to provide that extension, or if I don't like it, try to remove that bad assumption
<nomagno>
POSIX should probably at the very least guarantee /bin/sh existing.
<sortie>
E.g. you might see programs including <poll.h> which is non-standard and the standard POSIX header is <sys/poll.h>. Or things including <sys/param.h> for no reason as it's not standardized to contain anything in particular and rarely anything from it is actually needed, so I don't have that header, but so much uses it.
<sortie>
nomagno, that is literally one of the very FEW paths POSIX requires to work, along with /dev/tty
<nomagno>
Does it?
<nomagno>
I read up on it not that long ago and found POSIX expliticly says there's no way to run a script portably without modifying the shebang during installation
wgrant has quit [Ping timeout: 256 seconds]
<nomagno>
Might be misremembering
<sortie>
I could've sworm but I don't spot it right now
<sortie>
I'll check in the morning when I'm clear in the head :)
<sonny>
I'm surely going to ignore posix
gog has quit [Quit: byee]
<sortie>
I have a design principle that my OS is *just itself*. It must not try to fight who it is. I either embrace an aspect into my OS or I actively fight it.
<sortie>
E.g. that means the kernel lives in kernel/, libc in libc/, init is init/init.c.
<sortie>
It's not branded names. These things are just themselves in the trust form. It's just ls.c.
<sortie>
Likewise with system calls, in libc they are just plain wrappers that directly and immediately unconditionally call the actual system call with the very same function signature.
<sonny>
don't you want to be independent of your source layout?
<sortie>
There's no ‘POSIX subsystem’ because that'd violate the principle of embracing what it is.
<sortie>
If I had some other better native API, that should be used instead to embrace what the system is
<zid>
I think your system calls doing the same thing your 'OS' does makes sense
<klys>
libc is somewhat less a cohesive category than if one were to categorize its functionality. app/ls.c
<zid>
linux and posix try, at least
<nomagno>
I get this is osdev, but I just get dizzy thinking how much work it is to code up an OS that starts to be usable.
<zid>
but I'm not certain how well it pulls it off, certain things are definitely 1:1, some aren't
<sonny>
define usable ;)
<nomagno>
I suffer from quite a bit of analysis paralysis...
<sortie>
E.g. my libc only supports my kernel, and the versions need to match ABI versions. There's no #ifdef Linux. The libc and kernel are built to work together. It's simple and things can be relied on because it's not portable to other systems.
<klys>
what's next a libpy/
<sortie>
I do have a python3 port :D
<zid>
Only thing I'd worry about is overspecialization, lest you end up with .NET inside your kernel :P
<sortie>
nomagno, I use my OS to run an irc channel where people are only allowed to say ‘:D’ or they are auto kicked
<sortie>
The rest of it is just a tech loop for that purpose
<bslsk05>
sortix.org: The Sortix Operating System
<sortie>
It also runs its own website
<klys>
app/ls-l--color.c ; dat/ls-b.c
<sortie>
nomagno, I like to think I've actually reached the point where my OS can be useful because I have networking (in a WIP branch I'm finishing up) and I make it easy to spin up a VM running a server
<sortie>
In practice, your OS being a daily driver for you and the people is really difficult and hard to be useful
<sortie>
My current bet is on being being easy to deploy my OS to the cloud and being a fun development platform that's really hackable
<kazinsal>
yeah you really need to have a super specific goal in mind in order to get anywhere near developing an OS that's "useful"
<sortie>
Because these properties means people can actually set up some online services running on my OS (even made people run a few bots on there)
<nomagno>
My current project definitely doesn't know what/who it is.
<sortie>
That's OK :) osdev can be a journey of self-discovery
<klys>
useful software is a journey or a destination?
<nomagno>
I know it's a racing game. I know the racing game is portable enough it'll eventually be fit to be its own operating system, as well as a userspace program
<sortie>
Yeah klys makes an EXCELLENT POINT: You don't need to be osdev.
<sortie>
*to be useful
<sortie>
osdev can be all about the journey if you'd like.
<sonny>
I need a language for osdev first
<klys>
that kind of sounded like me
<sonny>
I think better in terms of environments
<sortie>
Right now I'm enjoying the journey but I'd like to be useful, defined as people trying out my OS and setting up some random services to run on it
<klys>
so I started hacking together a simple parser for rpn -> asm
<sonny>
nice
<nomagno>
How would I organize a codebase that is horrible unfit to be performant but works on a toaster, into progressively less portable but more performant ports?
<nomagno>
In such a way it's not a mess to navigate, I mean.
<sortie>
nomagno, gradually?
<zid>
rewrite it nicely
<nomagno>
Well yeah, but the rewrites won't replace the portable version
<zid>
I generally go for what I would consider the ideal (in the platonic sense), by thinking about what it *would* look like if it were nice
EtherNet has quit [Quit: WeeChat 3.5-dev]
<sortie>
Completely overhauling something or rewriting usually means you end up in a state where you have something that doesn't work or you can't release
<zid>
"I'd just have some file that has a couple of calls out of it and there's a few small files that implement those things in a different way for each platform" or whatever
* sonny
actually likes rewrites
<sortie>
It's generally the better option to evolve in the right direction in chunks where everything works at the same time
<zid>
local maxima/minima are a thing though
<sortie>
Rewrites are fine. You just don't want a situation where the old thing is competing against the new thing.
<zid>
sometimes it's just outright better to rewrite it, but that complexity also scales with how nicely it was written in the first place
<zid>
If it was shit to begin with, rewriting it will feel shitty and hard
<zid>
if it was nice to begin with, rewriting it will be nice
<nomagno>
My project is already structured what I'd call nicely. Issue is I'm unsure how to organize different implementations of interfaces
<sortie>
nomagno, my OS began back in 2011 and honestly it was a mess for a while, and it's been a long evolution to become well organized
<zid>
nomagno: arch/blah implements a common API, typically
<sortie>
With time, I came to see the principles guiding my OS and doubled down on them
<zid>
and you just link against the correct implementation, and that spits out the code specialized for that arch
<nomagno>
My project is lowkey overspecified already and the bulk of the code is yet to be written
<kingoffrance>
go on :)
<sortie>
Well it was fun talking to some new people in here I haven't said much hello to before :)
<zid>
so there is no "sound.c" with a thousand sound hardware implementations, there's "sound.c" which routes calls to a *specific* arch/blah.o depending on what it links to
<kingoffrance>
explain :) "overspecified" :)
<sonny>
klys: this file is rather spartan, what does sis() do?
<sortie>
It's 2 AM and STORMING. That's right. It's a dark and stormy night. And I'm off to bed :)
<sonny>
cya
* sortie
. o O (the inverse of the bro function)
<klys>
sonny, main loop
<nomagno>
kingoffrance: The project derives from the desire to specify something in the first place, so essentially the bulk of its merit is in interfaces being crystal clear and clean
<nomagno>
Just as data structures
<nomagno>
This also results in writing code not being the most time-consuming task
skipwich has quit [Ping timeout: 268 seconds]
<klys>
sonny, the loop in sis() does some lexing
<nomagno>
I'm stuck in an almost-over analysis paralysis period, but I'm unsure I can keep tackling the rest from this bottom-up approach
gog has joined #osdev
<kingoffrance>
i guess i feel similar, but i never have a "what to do next?" issue. there are infinite things to do, and many skeletons with no implementation, etc.
<klys>
if it finds whitespace it skips that, if it finds a valid token (any token which has been duly processed) it runs push( token );
<kingoffrance>
but i dont regret overdesign, slowly is forming
<zid>
nomagno: I didn't hear what your actual thing is, do you have a brief description?
<zid>
or a long one I guess :p
<sonny>
klys: more than 3 letters are allowed! `enum { NIL, NUM, LTR, OPS, QUO, ETC, SPC, REG, PRN, PRM, BKT, IDX, CLN };`
skipwich has joined #osdev
<zid>
nope, ask my assembler
PyR3X_ is now known as PyR3X
<sonny>
this isn't an assembler :P
<sonny>
just lexical elements
* kingoffrance
jedi handwaves, there is no assembler spoon
<zid>
Nope it's literally not possible, my assembler is extant, therefore nothing else is possible
<zid>
qed
<zid>
tla
<klys>
nil, number, letter, operator, quote, everything else, space, register name, open paren, close paren, open bracket, close bracket, colon
<sonny>
klys: are you content with just rpn?
<klys>
sonny, to a point, mostly
<sonny>
hardcore
<sonny>
I think my goal was just a uniform way to get concurrency
<klys>
uniform among assemblers?
<kingoffrance>
what languages are you targetting nomagno ?
<sonny>
klys: no, for my OS implementation
<klys>
so, you have a portable os idea, and what's the difference for uniformity?
<sonny>
no, my ideas are not very portable
<klys>
so, you mentioned uniformity, is that a goal?
<sonny>
I want the representation to be consistent
<sonny>
yeah
<klys>
the uniform representation of concurrency, except what's the background?
<klys>
architectures?
<sonny>
there is none, that language will provide concurrency primitives, which in turn builds the OS
<klys>
ah, so you could proceed to make a meta thing all you need is a parser that suits you
<sonny>
yeah
<sonny>
seeing the smalltalk demo from back in the day is mind blowing, I think an OS can be that easy :-)
epony has quit [Ping timeout: 240 seconds]
wgrant has joined #osdev
sonny has quit [Remote host closed the connection]
* moon-child
wonders how klange is getting on with only 0MiB of ram
srjek has quit [Ping timeout: 240 seconds]
<zid>
It's just running out of rom and has its stack in sram, which is close but doesn't count as ram
<klange>
I haven't done RAM discovery, the function that exposes total RAM to userspace is stubbed, and I should probably do that stuff next.
<clever>
for a moment, you had me thinking about how much i could do from just rom and the L2 cache-as-ram, lol
<zid>
I mean, basically everything
<clever>
but then i remembered, the rom is true rom, so the 128kb L2 is all i got
<clever>
not even a 640x480xRGBA8888 framebuffer would fit there!
<zid>
shove that in device memory?
<zid>
external vram
<clever>
zid: the 2d scanout engine of the rpi cant read memory over a non-standard interface, so my only option to expand is the proper ddr dram controller, which i lack drivers for on some models
<zid>
plug in a voodoo 2
<clever>
i was more thinking, how much can i do if i turn off the dram
<zid>
You'll just need devices that don't need dma
<zid>
and everything should *work* at least
<clever>
hmmmm, could i drive an isa bus? .....
<clever>
with some level shifters, maybe!
Irvise_ has joined #osdev
paulusASol has joined #osdev
nopenope[m] has joined #osdev
junon has joined #osdev
gog has quit [Quit: byee]
the_lanetly_052_ has joined #osdev
Irvise_ has quit [Quit: Client limit exceeded: 20000]
nopenope[m] has quit [Quit: Client limit exceeded: 20000]
paulusASol has quit [Quit: Client limit exceeded: 20000]
Brnocrist has quit [Ping timeout: 240 seconds]
Brnocrist has joined #osdev
freakazoid333 has joined #osdev
mahmutov has quit [Ping timeout: 256 seconds]
zaquest has quit [Remote host closed the connection]
zaquest has joined #osdev
lleo has joined #osdev
lleo has quit [Quit: institute budget]
g1n has quit [Read error: Connection reset by peer]
GeDaMo has joined #osdev
Burgundy has joined #osdev
mahmutov has joined #osdev
mahmutov has quit [Ping timeout: 256 seconds]
g1n has joined #osdev
<g1n>
hello
Irvise_ has joined #osdev
paulusASol has joined #osdev
nopenope[m] has joined #osdev
biblio has joined #osdev
<GeDaMo>
Hi g1n :)
<g1n>
whats up GeDaMo?
<GeDaMo>
Hmmm ... not much, you?
<g1n>
just woke up, going to continue orion rewrite :)
<g1n>
currently on global constructors stage
<klange>
geist: have you spun up anything graphical in hvf on an m1 yet?
<geist>
negative
<klange>
After much work I got things to boot and run under it, but my framebuffer is super slow
<geist>
might try an ubuntu install or something though
<geist>
ah interesting
<geist>
how are you mapping the framebuffer?
<geist>
this may be a case of actual hardware actually honoring caching parameters and whatnot that are complicated to set up in the page tables on arm
<geist>
since presumably you haven't tested on qemu-kvm?
<klange>
I've tried all sorts of caching and noncaching configs to no avail, but for all I know they're trapping every write or something
biblio has quit [Ping timeout: 260 seconds]
<geist>
true, though possibly just didn't get it right. how are you setting up the page tables? specifically the MAIR and how you're using those indexes in the page tables
<klange>
I know I'm setting and referencing MAIRs correctly because I had to set normal memory for exclusive loads to not trap on this hardware.
<geist>
kk. and you're mapping the framebuffer as normal memory i guess
<klange>
And I've tried every combination of caching options for the framebuffer with no effect; showed a video to marcan and he thinks it's totally trapping
ephemer0l has joined #osdev
<geist>
yeah kinda curious now. i should try installing ubuntu or whatnot on it
togooroo has quit [Remote host closed the connection]
sdfgsdfg has quit [Quit: ayo yoyo ayo yoyo hololo, hololo.]
<klange>
The doc page for the 'virt' machine makes a claim that virtio-gpu-pci is the only display device that works "correctly" with KVM; bochs one seems to work fine in actual KVM, but maybe that's a hint that it is missing something critical to memory access performance in other accel backends
eryjus has quit [Remote host closed the connection]
heat has quit [Ping timeout: 245 seconds]
<klange>
i pushed the branch
g1n has quit [Read error: Connection reset by peer]
mahmutov has joined #osdev
mahmutov_ has quit [Ping timeout: 250 seconds]
gog has joined #osdev
fwg has quit [Quit: .oO( zzZzZzz ...]
nyah has joined #osdev
dennis95 has joined #osdev
fwg has joined #osdev
g1n has joined #osdev
dennis95_ has joined #osdev
dennis95 has quit [Ping timeout: 256 seconds]
srjek has joined #osdev
gorgonical has joined #osdev
gorgonical has quit [Ping timeout: 252 seconds]
jjuran has quit [Quit: Killing Colloquy first, before it kills me…]
g1n has quit [Read error: Connection reset by peer]
<Griwes>
mine was more embarrassing, no macros in sight
<zid>
I'm surprised I don't write more of them, my memory of the actual precedence is weak
<zid>
I guess I just managed to judge precisely *how* much I don't know correctly and put the right amount of defensive () in :P
<GeDaMo>
I always add more parentheses in :|
<GeDaMo>
Or I use Forth :P
<geist>
same. but parenthesis around arguments in macros are a requirement
<zid>
I think the only thing I actively like, have to 'remember' relative to just going on instinct is right side of shifts, and (*a)->b
<geist>
also ternarys will mess you up because they're always lower precedence than you think
Raito_Bezarius has joined #osdev
Raito_Bezarius has quit [Max SendQ exceeded]
Raito_Bezarius has joined #osdev
GeDaMo has quit [Remote host closed the connection]
[itchyjunk] has quit [Remote host closed the connection]
sdfgsdfg has quit [Quit: ayo yoyo ayo yoyo hololo, hololo.]
sdfgsdfg has joined #osdev
sdfgsdfg has quit [Client Quit]
dennis95_ has quit [Quit: Leaving]
sdfgsdfg has joined #osdev
sdfgsdfg has quit [Quit: ayo yoyo ayo yoyo hololo, hololo.]
<geist>
klange: doing an install of ubuntu server on a qemu-hvf image on m1 now. thus far screen seems fine
<geist>
but it's doing text mode and i'm currently using virtio-gpu
<geist>
will switch to bochs-display once it gets installed
<geist>
also i really can't find the desktop installers like i remember in the past. i wonder what happened to them
<klange>
I did take a quick look and virtio-gpu is doing its video memory completely differently than how the bochs-display backend does it
<geist>
exactly
<geist>
you send commands to blit explicitly
<klange>
and it's regular system memory while the bochs display has to be behind PCI
<geist>
right
<geist>
like i said, i know it's different, but i'm doing the isntall now and i know the UEFI stuff likes virtio-gpu
<geist>
but i dont think it initializes bochs-display, so i'm using what works for the installer
<geist>
yap, with bochs-display it's mega slow
<geist>
and seems to have lots of corruption
<geist>
(doing a ls -lR on the console)
<geist>
looks like it has cache flushing issues even (or lack of memory barriers)
<klange>
thanks for helping to check, guess it's time for a virtio-gpu driver :)
<geist>
what are you using for input/output?
<geist>
i have virtio-input here but didn't know if you had virtio support for that stuff
<klange>
Nothing yet. I was thinking about just going in on USB now...
<geist>
yah or xhci yeah
<geist>
i'd invest a bit in virtio. once you get the basic stuff working you can then have net/block/input/etc mostly for 'free'
<geist>
gpu is not too bad, in the simple mode you just set up a scanout buffer and then the main difference between that and a standard framebuffer is you have to explicitly kick it to blit the output
<geist>
but you can just set up a 60hz timer that kicks it if you want
<geist>
fwiw this is the command line i'm using to drive this:
<geist>
the commented out .iso image was what i used to isntall it
<geist>
and i got the QEMU_EFI.fd from an ubuntu package
<geist>
the highmem=off switch if oudn i need on the M1 but not the M1pro. it tells the virt machine to not emit the high PCI ECAM and the mmio aperture up there, because it tries to configure it > amount of virtualized physical address space on the M1 (36 bits)
<geist>
seems to be a cleaner way than doing -M virt-2.12 or whatnot we were doing before i think
<klange>
Anyway, surprisingly productive weekend, now back to work work. Really didn't think I'd have the GUI running this quickly.
fwg has quit [Ping timeout: 260 seconds]
Burgundy has quit [Ping timeout: 250 seconds]
dude12312414 has quit [Quit: THE RAM IS TOO DAMN HIGH]
fwg has joined #osdev
<gog>
i keep meaning to put aside some time to pick my memory management apart and refactor it but i've just been busy as hell and too exhausted to think clearly
<gog>
trying to abdicate all responsibilities and make time for it wednesday
<zid>
Wednesday ends in y though so I couldn't possibly do anything productive
<gog>
i just want one fucken day to myself T_T
sdfgsdfg has joined #osdev
vdamewood has quit [Quit: My MacBook Pro has gone to sleep. ZZZzzz…]