<geist>
seems like that'd be a problem though, since would it' pick up a ground off all of the computers it plugged into?
<geist>
seems like you could get a good ground loop with that
<geist>
unless it was explicitly neutral shield i guess
<mrvn>
BNC is uncoupled from the computer, no shared ground.
<geist>
ah
<mrvn>
You don't want a shared ground with systems on different breakers and different phases.
sonny has joined #osdev
<geist>
so probalby i nternal to the nic they have the whole transformer to decouple it like they do with twisted pair
<geist>
what AUI did is i think simply move the PHY outside of the nic, similar to SFP nowadays or so
thinkpol has quit [Remote host closed the connection]
<mrvn>
pretty much the same except with just one non-twisted pair.
thinkpol has joined #osdev
<kazinsal>
yeah basically the MAC spits out AUI logical signals onto an AUI interface and then the MAU on the other end of the AUI interface takes those signals and turns them into physical layer signalling, and vice-versa
<kazinsal>
the MAU also does the collision detection and jabber detection, being 10Mbit ethernet
<heat>
kazinsal, i kinda asked you this a few days ago but how do you implement a proper FIB? do you just have a plain array of routing table entries and then a radix tree as a cache? or can you use the radix tree as the actual routing table?
<kazinsal>
you keep them separate and build and maintain the radix tree as a cache from the routing table
<kazinsal>
you can also cache layer 2 header information in the radix tree for efficiency
<geist>
i was tinking about this. presumably a radix 512 is for ipv4 only (or only makes sense there) and you build some sort of 'page table' looking structure that has an entry for every possible address?
<kazinsal>
as well as whatever rewrite information you might want
<mrvn>
heat: I think what linux does is cache the routing entry index in the connection and when it gets invalidated you just do a linear search top to bottom.
<geist>
was thinking that'd be terribly expensive memory wise, but you can get massive sharing of nodes i guess
<geist>
since entire ranges of the internals would be identical so you can reuse the same inner node?
<heat>
mrvn, linux also needs to maintain a proper FIB for routing
<heat>
there's no connection there
<geist>
ie, if all of say [0...10].x.x.x is the same, yo can use the same 11 top level tables?
<geist>
and all the internals?
<mrvn>
heat: connection tracking?
<kazinsal>
geist: precisely
sonny has quit [Quit: Client closed]
<geist>
so i *guess* you could do the same thing with the first 64bits of ipv6, though the layout of that would probablylet you further suboptimize
<heat>
geist, you can just have special radix tree entries that say "this is a value, not a sub tree"
<heat>
that's how I've seen it done
<geist>
heat: yeah basically like a 'large page' in a page tables
<heat>
just like a page table hugepage for instance
<heat>
yeah
<geist>
yah makes sense
<kazinsal>
yep
<heat>
how faster is the radix tree?
<mrvn>
geist: if [0...10].x.x.x is the same you would have 0-7, 8-9 and 10 nodes in the tree.
<kazinsal>
noticeably.
<heat>
even for the average host?
<heat>
or consumer router
<kazinsal>
for forwarding? yeah
<geist>
mrvn: why so?
<mrvn>
geist: matching 3bit, 4bit, 5bit prefix.
<geist>
it's not a binary tree, i'm assuming it's a bunch of tables of 256 entries
<geist>
so a linear run of the same thing would just be N things copied
<kazinsal>
even if your routing table has only a few different prefixes you still need to do a lookup for each packet without a trie FIB
<mrvn>
geist: if your radix tree uses bytes instead of bits then yes.
<kazinsal>
and then you need to then do ARP cache lookups etc
<geist>
right, i think that's the point, since it's organized according to a.b.c.d
<heat>
kazinsal, more or less how many routes are you expected to have?
<geist>
though really, could radix anywhere you want as long as it is a common division of 32
<mrvn>
geist: I would have it organized by the netmask
<geist>
yah this is a router level thing, so netmask isn't that interesting
<geist>
or at least the netmask can be done already for local networking stuff
<heat>
every route has a mask
<kazinsal>
I'll dig out my big fun book of router internals in a bit
<geist>
i'm assuming this is more for inter-router stuff. like how routers decide how to forward stuff
<kazinsal>
few things to take care of first
<geist>
with this sort of radix tree they simply start with the address and drill into it
<mrvn>
the route woulde be 0.0.0.0/248.0.0.0 -> somewhere
<kazinsal>
yeah the big use for this is to effectively make your route lookup almost fixed-time
<geist>
such that it's at worst 4 hops
<geist>
right
<kazinsal>
and to be able to cache useful information in the same structure
<geist>
indepdenent of any netmasks
<kazinsal>
yeah
<geist>
netmasking would feed into how you build it, since it allows you to share chunks of the tree
<kazinsal>
maintaining your FIB becomes a bit more complicated, with some callbacks needed on routing table changes
<geist>
or use 'large pages'
<mrvn>
kazinsal: any tree form will be fixed time O(number of bits in the IP format) so that doesn't say much.
<kazinsal>
but it's fine, really
<geist>
yah could do some sot of copy on write almost, like build a new tree and switch to it
<geist>
like a diuble buffered tree basically
<heat>
right but I would expect that a vector with few entries would be faster than a radix tree
<geist>
maybe, but that's where the N in O(N) matters
<geist>
this is a cse where you throw memory/complexity at a problem such that all lookups become O(1)
<mrvn>
You can probably also compare and match a number of entries in parallel with SSE.
<geist>
but probably the actual numeric constant behind the '1' has a crossover point with real Ns for traditional tables
sonny has joined #osdev
<heat>
geist, yup, which is why I asked the avg number of routes in a router
<heat>
I actually have no idea
<mrvn>
If you have just your 2-4 basic routes bof your home system then a straight up array will be fast.
<geist>
especially given that tables like that probably have relatively poor cache locality. or at least one can actually throw packets at a router to try to trigger it
<geist>
ie, throw packets at the router that explicitly try to get it to have to thrash its cache
<mrvn>
If you throw packets at a router many cheap ones fallback to HUB mode.
<geist>
this table thing is also probably much more straigthforward for hardware to implement
<geist>
some software can build the table, hardware uses it. ie, just like page tables on cpus
<mrvn>
N-way lookup table in hardware. That's what small routers use.
<heat>
which is why intel should introduce new x86 routing extensions for hardware accelerated lookups
<heat>
great idea
<geist>
anyay this is all probably Serious Router 101, but it's fun to work through it as a mental exercise
<heat>
AVX512ROUTER
<kazinsal>
the real fun routing in hardware is with ternary content addressable memory
<geist>
yah i assume a whole new series of techniques were needed for ipv6
<mrvn>
Do you want to route ethernet frames or IP frames?
<geist>
or... actually possible ipv6 is generally much simpler, because it was designed to be that way
<kazinsal>
you don't route ethernet frames so
<mrvn>
kazinsal: bridging does
<kazinsal>
that's not routing. routing is explicitly a layer 3 concept.
<geist>
bridging i assume just has a different set of tehniques
<mrvn>
Do you want to bridge/forward/whatever ethernet frames or route IP frames?
<geist>
probably somewhat simpler, since the size of N in those cases is probalby largely fixed by physics
<heat>
how would you go about connecting a router to another router?
<mrvn>
geist: you could have a "route" for every 6byte MAC
<geist>
well, first you put them in neighboring cages
<heat>
do you just turn it on and start speaking ospf or so?
<geist>
and you let them see each other through the wire
<kazinsal>
I'm mostly interested in layer 3 but I have some layer 2 features that I am continously improving
<geist>
they'll get used to each other's presence
<kazinsal>
heat: depends on if they're both in the same autonomous system or separate ones
<mrvn>
geist: so you map seen MACs to ports where you saw them?
<heat>
kazinsal, lets assume the same, no eBGP here
<geist>
mrvn: i dunno, my brain only has space for one concept at a time right now
<kazinsal>
yeah you just configure stuff like OSPF or EIGRP or IS-IS or whatever tickles your fancy
<kazinsal>
and depending on what hardware you're using, what vendor you like, and what requirements you have for little nitty gritty details
<mrvn>
geist: Hence why I asked if this is for MACs or IPs. the solution for them is quite different.,
* geist
shrugs
<geist>
yah
<heat>
hm
<geist>
always wanted to build a switch and/or a router in fpga for lulz
<geist>
just to stumble through all of this like a caveperson
<heat>
so you can literally just add a router to your home network and have NAT in your local network, behind your own NAT?
<geist>
but then i'd rather build something like a sprite engine in fpga first
<kazinsal>
sure, no reason you can't
<mrvn>
With IPs you have network/netmask to give you a nice structure and usualy a verry compact tree form for it. Wiht MACs is basically totally random 6byte strings.
<kazinsal>
what happens in RFC 1918 stays in RFC 1918
<mrvn>
geist: oh have fun with adding multicast support
<heat>
i was afraid a router could just reject any other router
<geist>
that's why you let them get used to each other first
<kazinsal>
I mean you have to configure them the same overall way
<mrvn>
and don't forget loops betgween routers
<geist>
you dont want one router to attack the other on sight
<kazinsal>
in a proper OSPF environment you can't just shove a router in and have it start getting your whole network's routing tables
<geist>
since afterall you really want little baby routers, to propagate the router species
<kazinsal>
there's HMACs and stuff
<mrvn>
kazinsal: wouldn't that be fun when the janitor plugs in a foreign router to hijack the network
Likorn has quit [Quit: WeeChat 3.4.1]
<kazinsal>
yeah that's why there's levels of security involved in most routing protocols
<heat>
ah, so you cant
<kazinsal>
there are some really old routing protocols that still exist (in theory, at least) that just blast out routing tables to whoever asks
<mrvn>
I never delt with anything more complex than managed switches where you telnet/ssh into the switch and assign MACs to ports, VLANs and routes statically.
<heat>
real routing is insecure routing
<kazinsal>
this is literally my job :)
<mrvn>
viral routing is fun but very bandwidth limited
<geist>
i can totally see the satisfaction in that
<geist>
in just my little one router, v4/v6, 5 switch, 3 vlan, 3 AP home network it's super pleasant to know where everything is
<heat>
you do it manually?
<mrvn>
You can do it in RL too. Just tell all your friends about a message to xyz and let them tell all their friends till it reaches xyz.
<geist>
and that it's operating pretty much ideally
<heat>
do consumer routers just have one route?
<heat>
mine seems to
<mrvn>
heat: most people have an ISP and that's where all network traffic goes to by default if it isn't local.
<mrvn>
s/an/one/
pretty_dumm_guy has quit [Quit: WeeChat 3.5]
<mrvn>
heat: you should have 2 routes, the local network and default gateway
<heat>
I see routes for my local NAT'd network, the router's local network, and a default gateway
<geist>
yah probaly
<geist>
lokay, just moved vid card from one slot to another, starting test again. average time to failure seems to be about a day or two
<geist>
watch -n1 'uptime; dmesg | tail -40' seems like a pretty good indicator of what's going on, since the screen sticks
kingoffrance has joined #osdev
<mrvn>
lucky you. I'm debugging a problem that happens on ~5% of a 1000 node cluster once a week.
<bslsk05>
'1 MILLION FPS - The Slow Mo Guys' by The Slow Mo Guys (00:18:38)
<geist>
yah i'm being pretty slow about it since it's not a critial machine, and i only want to change one variable at a time
<geist>
and of course it's just some derpy desktop motherboard that's not really intended to be used as a server, with a beefy power hungry cpu in it, so it's entirely possible it's just the motherboard getting loaded down or whatnot
<geist>
or may be that reseating stuff will cause it to fix itself
Ali_A has joined #osdev
nyah has quit [Ping timeout: 276 seconds]
zaquest has quit [Remote host closed the connection]
zaquest has joined #osdev
gog has quit [Ping timeout: 260 seconds]
qubasa has quit [Remote host closed the connection]
dude12312414 has quit [Quit: THE RAM IS TOO DAMN HIGH]
Clockface has joined #osdev
srjek has quit [Ping timeout: 250 seconds]
<Clockface>
does the stack do anything weird on linux besides pushing an 8 byte address to the stack upon CALL?
<Clockface>
*in x86_64
<Clockface>
im using purely assembly, so i dont care what the compilers are doing
<Clockface>
i dont think it does, but for some reason im not getting data off the stack the way im expecting
<Mutabah>
you still need to follow the right calling convention
<Mutabah>
what are you trying and expecting?
<kazinsal>
you should be caring what the compilers are doing
<kazinsal>
because what the compilers are doing is "following the ABI"
<moon-child>
well, only if you are calling into c code
<moon-child>
if it's all assembly (syscalls included), then you don't need to care about the abi (except the kernel abi, but that doesn't care about the stack)
<heat>
it's useful to follow some ABI in assembly as well
<moon-child>
yes
<heat>
Clockface, code?
<moon-child>
(and I will add, if it's not an abi issue from talking to c, it's probably just a bug in your code)
<Clockface>
pop RAX
<Clockface>
push mydata
<Clockface>
push RAX
<Clockface>
and
<Clockface>
pop RAX
<Clockface>
pop registercontainingmydata
<Clockface>
push RAX
<Mutabah>
...
<Mutabah>
yeah, going to need to see more than jsut that (in a gist or pastebin?)
<Mutabah>
That seems... really strange, pop/load/push/push is quite off
<Mutabah>
What would be in `rax` here?
<Mutabah>
Is this meant to manipulate your own arguments, without clobbering your return address?
<Clockface>
yeah
<Mutabah>
why?
<kazinsal>
this seems like one of those times where the answer to your problem is the question "what exactly is it you're trying to accomplish"
<Mutabah>
Any why make macros to do it instead of writing out the rather-simple code directly
<Clockface>
im slapping together myself a high level language out of macro's
<Clockface>
its going better than i expected, im only having issues with function calls now
<Clockface>
if i used any assembly instead of macro's to do stuff, i wouldent be writing a language anymore
<Clockface>
i saw someone did that with some form of BASIC too?
<Clockface>
i never found the project, but someone said someone did it
<Mutabah>
If you're writing a language, why not use a proper parser/code-generator
<Mutabah>
Instead of abusing nasm's macro system to produce VERY inefficient code
<Mutabah>
(e.g. use python to create a translator)
<Clockface>
but i can write my bad compiler with macro's then compile the compiler when the compiler is capable of compiling
<Clockface>
*good compiler
<Clockface>
i intend to do that eventually, also i do regret this
<Clockface>
im 95% done, so im not stopping now
<klys>
finding that it's easy to stop when it seems you've reached a milestone
<geist>
kazinsal: oh crap i used to have a name for that sort of problem
<geist>
there was a name for it. where someone is asking about some esoteric solution and the problem probably has a much simpler solution
<Clockface>
noob tangients?
<geist>
heh, well you did come up with a rationale
<heat>
XY problem?
<geist>
ah yeah that's it!
Ram-Z_ has quit [Ping timeout: 246 seconds]
<klys>
so in the year 1992, what would have been the best way to devise an os to avoid the w95 trap
<heat>
what's the w95 trap?
<klys>
obviously you'd need lgdt/lidt and a 386
<geist>
you just fell in it!
<geist>
oh.
<klys>
and as anyone might recall w95 was vulnerable to call gates, so that could be a way to swap out the swapper
<kazinsal>
sit in a college dorm and bang out an email to comp.os.minix about how you're working on an operating system (won't be big and professional like gnu)
<klys>
yes
<geist>
+1
<heat>
code-review +2
<kazinsal>
but what exactly is the "w95 trap"
<kazinsal>
all I get for searching it is trap remixes of the microsoft sound
<klys>
minix was probably the best way to avoid w95 at the time
<heat>
"anyone might recall" most people do not have the details of an almost 30 year old OS in the back of their mind
<heat>
especially *windows 95*
<kazinsal>
despite literally using windows 95 today I have to concur
<geist>
yeah i mean i know quite a bit of estoeric stuff about things, but i wouldn't know about any single 'w95 trap'
<heat>
you could use 386BSD
<heat>
that existed
<klys>
ah yes, bsd
<geist>
but i guess maybe can be interpreted not as a trap as in where you fall in and your leg is stuck, but literally 'the way to do syscalls'
<heat>
also netbsd, freebsd
<heat>
BSD/386
<klys>
well what you would have wanted is an invasive system which could replace w95 at a relatively low level
<heat>
SVR4
<kazinsal>
what
<klys>
it's a ui thing, the next generation went the way of the mouse, then the way of the technophobe
<heat>
what
<heat>
technophobe?
<klys>
so as to keep the commandline interface is success at avoiding the trap as thus defined
<klys>
loads of phobic users have cellfones today, and will interact if you allow them to avoid keyboard usage
<kazinsal>
are you... under the impression that before august 1995, nobody had ever used a consumer-oriented graphical user interface before
<klys>
that's a mistake in the history
<heat>
phobia of what exactly?
<klys>
right, technology is still ill defined, though I think it represents literacy as an aspect of trust
<heat>
deh kernel of deh unix drawing windows on the screen
<heat>
with a maus
<kazinsal>
ah, good ol' Unix System Laboratories
Burgundy has joined #osdev
<klys>
I saw a book about unixware at the local thrift store where my current occupation resides, one time, it was published by NOVL and had a baseball player on the front cover
<klys>
NOVL is now "micro focus" and lives in south Provo, they don't do anything big like NOVL was trying to do with their NDS database, though.
<Clockface>
there are too many UNIX ripoffs!
<klys>
NDS was just a project like the current 389-ds package
<Clockface>
im sick of it!
<kazinsal>
time to write one new unix ripoff to supplant all other unix ripoffs
<klys>
srs.
<heat>
hi, github.com/heatd/Onyx
<bslsk05>
heatd/Onyx - UNIX-like operating system written in C and C++ (2 forks/41 stargazers/MIT)
* geist
calms down Clockface
<geist>
there there
<heat>
i did not rip off unix
<heat>
i ripped off linux
<heat>
which ripped off minix, which ripped off unix
<heat>
cuz of that, i never ripped off unix
<heat>
suck it
<heat>
unix? what's that
<sonny>
turns out files are a pretty good interface
<klys>
if you are familiar, you'll write an os with a "c library." c libraries are kind of messy though, and the interface has been standardized. tcp/ip requires it. manipulating data portably requires it. if you're like me you want to put the functions in separate namespace categories. in the end you have to rewrite the c compiler.
<heat>
you don't need a C library
<sonny>
Clockface: what OS inspires you?
<heat>
tcp/ip doesn't need a C library
<heat>
namespaces exist in C++
<heat>
no rewriting is needed
<klys>
that's an xy solution
<heat>
no it's not
<Clockface>
i was inspired by DOS and 9x mostly
<Clockface>
*windows 9x
<heat>
you're telling me rewriting the C compiler is less of an XY solution than "use another language"?
<sonny>
did NT come out in 9x?
<heat>
you do *not* need a libc as an interface (see: Go in Linux)
<klys>
using forth is still impractical and there needs to be a better other language in the rpn space
<sonny>
I don't know about windows design before NT
<Clockface>
no, but i found NT later and its what made me think about moduler kernels
<Clockface>
im probably inspired more by NT than 9x now that i think about it
<sonny>
I read about namespaces in plan9 and I wonder if NT missed out on that
<heat>
define UNIX btw Clockface
<Clockface>
i initially wanted something DOSy that isnt stuck in the past
<sonny>
if so then the object manager would be super cool
<sonny>
otherwise, it is similar to unix virtual file system I must admit
<geist>
<insert token comment about openvms>
<geist>
though i think the objectnamespace thing is more of a derivative of PRISM or whatnot than VMS
<sonny>
Clockface: I thought about it, but it's the best/eventual design
<sonny>
if osdev started over we'd just rediscover unix
<heat>
y'know lots of OSes don't have unixy interfaces right?
<Clockface>
DOS is nice in its own way though, i like how it just shuts up and gets out of the way of the current program
<heat>
windows, a good chunk of macOS, fuchsia
<geist>
i dunnom i think modern designs would still generally rediscover a global heirarchy namespace
<Clockface>
its like writing an OS but all the boring stuff is premade
<geist>
that is a really compelling thing
<sonny>
ya
<geist>
i think in general more modern stuff tends to gravitate towards a heirarchy of ndoes
<geist>
nodes
<sonny>
that's what I mean
<geist>
where nodes are objects that support some sort of interface that you can discover
<sonny>
unix == plan9
<geist>
ie, not necessarily just files
<geist>
but i mean even if you weren't trying to recreate unix, it's hard to not gravitate towards that part of it at least
<geist>
unless you're explicitly trying not to have a global namespace, etc etc
<heat>
otoh global namespaces are bad for concurrency
<Clockface>
i would at least differentiate between real files on drives and "everything is a file" files
<geist>
sure, and that might be a reason not to
<Clockface>
and call both of them as a whole objects or something for clarity
<geist>
even if there isn't aglobal namespace, having some sort of tree based namespace, even if it's zoned and private to processes or whatnot is still pretty compelling
<geist>
vs, i dunno, flat string based naming where you just have to know the name
<geist>
or everything is a UUID you have to either just know
<geist>
or discover via another interface (ie, UEFI)
* heat
laughs in UEFI
<heat>
everything is a UUID
<geist>
yah that'skinda an odd design. i guess it *does* make sense for maybe the common use case: little binary that just wants to get a handle to a thing that's either there or not, and use it and move on
<geist>
having a 'get me this interface by UUID' is pretty convenient
<heat>
yeah but like
<heat>
maybe magic numbers?
<geist>
isnt' that basically the same thing?
<heat>
why do you need a 128-bit number?
<geist>
oh well, yeah that's just MSFT at the time
<geist>
but there are worse things
<heat>
i think this is plain and simple wintel GUID extravaganza
<sonny>
lol
<geist>
well, sure
<geist>
there's all sort of MSFT all over it
<geist>
16 bit unicode, uuids, PE binaries, FAT filesystem, etc
<heat>
the coding style
<sonny>
Clockface: I saw a smalltalk demo today, made me feel backwards ... but I think objects probably need more hardware support
<kazinsal>
yeah, UEFI's cool but I sometimes wish it were more generally approachable
<heat>
well, it's not that hard
<heat>
for instance, filesystem operations: get filesystem, open file (you get a handle to a file), use the file, close, close the fs
<heat>
pretty similar to an OS
<klys>
so I guess uefi still has something like PXE boot?
<heat>
yes
<Clockface>
i dont think filesystems should be a bios thing
<klys>
do you, "open network device?"
<heat>
Clockface, have you tried reading a filesystem in 1024 bytes?
<Clockface>
no
<Clockface>
but most proggrammers never touch the disk directly
<heat>
klys, network booting is more automatic than that, but it generally ends up using the HTTP/TCP, UDP protocols I believe
<Clockface>
so, convinience at that level isnt too important
<heat>
I don't know how you get the protocols, probably by looking at network devices
<klys>
heat, just curious what access uefi has to the network, is it because the network device firmware is in place?
<heat>
yup
<heat>
firmware publishes handles and protocols to stuff
<heat>
same with the GOP, that's all published by the GPU's UEFI firmware blob
<kingoffrance>
"PRISM (surveillance program) " lol lemme know if you ever see the code for "PROMIS" database it was public domain originally lol
<klys>
so that's why everyone is still using intel/broadcom/realtek network interfaces?
<kingoffrance>
supposed to be very flexible
<heat>
Clockface: UEFI is made with the purpose of reusing code and making life easier for everyone
<klys>
kingoffrance, that prism never made it into the topic
<heat>
less duplicated code, more maintainable software
<heat>
it turns out that writing an ext2 driver in 1024 bytes of an ext2 bootsector isn't easy
<kazinsal>
everyone still uses intel/broadcom/realtek network interfaces because those are the ones that are most commonly produced
<heat>
klys, no
<kazinsal>
and they're all quite simple to work with
<klys>
because the network device vendor has to produce firmware capable of interacting with uefi on a high level
<klys>
no?
<heat>
what's a "high level"?
<klys>
osi model
<klys>
media access control
<kazinsal>
UEFI just has a network stack like anything else
<heat>
all UEFI firmware is just a UEFI driver
<heat>
a UEFI driver is exactly like a UEFI app (your bootloader, your linux kernel's EFI stub) but slightly different
<Clockface>
only 1024 bytes of non-files in ext2?
<klys>
so you can open the network device from uefi code
<heat>
all the vendor needs to do is write a simpler driver
<heat>
Clockface, yes
<heat>
of "non-filesystem-stuff"
<Clockface>
that sucks!
<Clockface>
fat32 blessed me with 32k
<klys>
so just use a blocklist
<klys>
grub2-mbr uses a blocklist
<heat>
there's an inode allegedly reserved for bootloader stuff
<heat>
but you need to read that first
<kazinsal>
yeah, inode 5 is reserved as "boot loader inode"
<heat>
anyway, UEFI is still relatively simple
<heat>
no interrupts, no threads for example
<kazinsal>
so I guess you can go: bios loads your first 512 bytes -> you load the next 512 for more code space -> poke at the superblock and find the inode table -> read the contents of inode 5 -> use that to load a second stage bootloader
<Clockface>
that sounds like a major flaw, i like how fat32 just gives you 32 kilobytes
xenos1984 has quit [Ping timeout: 276 seconds]
<Clockface>
simple stuff that doenst change is better than complicated stuff that does
<Clockface>
unlike my spelling of doesnt apparently
<Clockface>
im going to use that from now on, its funny
heat has quit [Read error: Connection reset by peer]
heat has joined #osdev
janemba has quit [Quit: WeeChat 3.1]
janemba has joined #osdev
<mrvn>
klys: at work we boot grub via network on uefi and have it load the kernel and ramdisk because it's network code has no problem with e.g. 1GB ramdisks.
<clever>
i think ive done similar on ipxe, but your way sounds a lot more modern
<mrvn>
Clockface: does your language has partial aplication and composition of function?
<mrvn>
We used ipxe for bios boots. Can't remember why we switched to grub exactly but something just worked better there.
<clever>
when i last did grub netboot, i was cheating
<clever>
i had ipxe map bios disk io into iscsi
<clever>
so grub thought it was accessing a bios legacy disk
<mrvn>
How long ago was that because grub does netboot just fine natively
<clever>
more, i just didnt want to bother with finding out :P
<clever>
the ipxe thing was simpler
<mrvn>
dnsmasq is helpful there too. You can set it up to check if the system boots via bios or uefi and give it different configs. Or check other stuff to pick a config.
<mrvn>
dhcp-boot=grubnet.i386-pc
<mrvn>
dhcp-boot=tag:efi-x86_64,grubnetx64.efi
<clever>
dhcpd can do that too
<mrvn>
Booting with bios fetched the first, uefi the second via tftp and then grub.cfg says what kernel + initrd to load.
<bslsk05>
github.com: nixos-configs/netboot_server.nix at master · cleverca22/nixos-configs · GitHub
<clever>
and this deals with legacy pxe vs efi
<mrvn>
mines shorter, na na nana na :)
<clever>
lol
<clever>
mrvn: mines reproducible!
Likorn has joined #osdev
sonny has left #osdev [#osdev]
<zid>
My builds are always reproductiaicaible, just use the same compiler, same flags. Because I don't use __DATE__ anywhere, easy
<zid>
(Note, I don't know which compiler I used, or flags, good luck)
<clever>
zid: thats what nix solves
<zid>
It solves nothing good day sir
xenos1984 has joined #osdev
<stephe>
do u guys know any good books about building a x86-32 os from scratch?
<zid>
Depends if the intel software developer's manual counts
<mrvn>
or the barebones tutorial in the wiki
<zid>
depends what you already know, I don't know anything holistic
<zid>
do you know x86? do you know what kind of OS you wanna make? do you know how to use binutils? etc
<mrvn>
and you shouldn't want to make a x86-32 os. It's a dead architecture that just won't quite die.
<mrvn>
go 64bit
<zid>
surprised you have a pentium pro laying around you wanna revive though over something interesting like a desktop PC or a weird ARM board
heat has quit [Remote host closed the connection]
heat has joined #osdev
<stephe>
partly just curious about x86 and maybe would help to understand that if i eventually get more interested and want to build something 64bit
<zid>
amd64 is about 10x easier
<zid>
as long as you were intending to do 32bit paging
<stephe>
really?
<stephe>
i thought it would have been more complicated :D shows how much i know
<zid>
Yea 32bit mode supports hardware task switching and segmentation and stuff
<zid>
that amd64 just forces you to hardcode to 0
<stephe>
i see
<zid>
so there's like, 8 chapters explaining what a whole bunch of structures do in the 32bit manual, then a single line underneath for the 64bit equivalent that says "Write a 0 to field 8, all other fields are ignored, not setting it to 0 will cause #GP when long mode is entered." or whatever
<stephe>
do you recommend the intel manuals for 64 bit too?
<stephe>
is it volume 3?
<zid>
3 is the system one yea
<zid>
amd have manuals too, some people prefer them
<zid>
but I am super used to the intel manual style from them.. existing for 20 years prior to amd64's changes
<stephe>
hmm i've only seen the intel ones i'll search for the amd ones
<stephe>
i remembed years ago intel would send out their manuals for free
<zid>
yea I have hard copies of the 32/64 manual
<zid>
fedex'd from america for free
<stephe>
:D
<GeDaMo>
I requested a set just as they were stopping sending them out so I didn't get a complete set :(
<zid>
I didn't ask for the optim one for.. reasons, so I have 1-3
<GeDaMo>
I have 2A, 2B, 3A, 3B
<zid>
yea I might not even have 1, I think we have the same sets
<mrvn>
Read both the intel and amd manuals and pick the one more to your liking. Sometimes a feature is better described in one or the other.
<GeDaMo>
Maybe they'll do that in a later standard :P
<zid>
doesn't mean every cpu that uses signed 2's comp also has an O flag
<mrvn>
probably not. It's the part that allows a lot of compiler optimizations.
<zid>
and there's no grammar to check it in C++ or C still
<zid>
SETS_O_FLAG(a+b);
<mrvn>
zid: why would you need an O flag? unsigned int doesn't have that either.
<zid>
because it's defined and the 'check' is the < operator
<zid>
if you're defining it, presumably you want to define it so that you can use it in real programs
<mrvn>
zid: b = a + 10; it (b < a) overflow();
<mrvn>
IF signed overflow is defined that codeb just works like it does for unsigned.
<zid>
I'd have to work through all the cases to prove that to myself
<zid>
right what if you don't know the second operand to +
<zid>
can you still do it?
<zid>
without sorting a and b for absolute magnitude, and then comparing
<zid>
where a and b can have arbitrary sign
<mrvn>
sure. you just have to check for the second argument to be positive or negative and then check for over or under
<zid>
can already do that by doing those comparisons as unsigned
<mrvn>
But that is your problem to solve, not the languages.
<zid>
I'd rather jut check the o flag :p
<mrvn>
I would also rather have a Carry bit.
<zid>
same :(
<mrvn>
but totally different problem
<zid>
I'm not sure how you could work tuples into C though
<zid>
and without them it's sort of much less of an issue to begin with because you have to write two expressions regardless, one for the value and one for the carry
<zid>
making it like errno maybe?
<zid>
but now you're just literally writing assembly, with flags
<zid>
it's ugly
<mrvn>
If you write your tupple add with unsigned and compute the carry yourself gcc actually can optimize that into addx.
heat has quit [Remote host closed the connection]
<zid>
yea and it'll probably do jo for the case where I do the 'does a and b signed overflow' too, but I don't want the code to be peppered with the checks
<zid>
I think in reality I'd rather they left it undefined, and let gcc have an UBSAN option that makes it call abort()
<mrvn>
doing big int with signed integers wouldb waste 1 bit. nobody sane would do that.
<zid>
good job unsigned exists then
<mrvn>
doesn't gcc have a __builtin for arithmetic with carry?
<bslsk05>
'CppCon 2019: Marshall Clow “std::midpoint? How Hard Could it Be?”' by CppCon (01:02:32)
<zid>
no thanks
<GeDaMo>
"A study published in 1988 shows that accurate code for it is only found in five out of twenty textbooks.[63] Furthermore, Bentley's own implementation of binary search, published in his 1986 book Programming Pearls, contained an overflow error that remained undetected for over twenty years." https://en.wikipedia.org/wiki/Binary_search_algorithm#Implementation_issues
<bslsk05>
en.wikipedia.org: Binary search algorithm - Wikipedia
<mrvn>
The proposal for std::midpoint took 4 (iirc) revisions before it actually worked for all corner cases.
<bslsk05>
'CppCon 2019: Marshall Clow “std::midpoint? How Hard Could it Be?”' by CppCon (01:02:32)
<zid>
Dr Pepper if he has any sense
<zid>
but he's a C++ guy so I have counter-evidence
<geist>
could be tea
<geist>
it's the usual standin for whiskey when filming movies and TV
<zid>
because shaking bottles of dr pepper to make them flat uses up a runner's time for too long
<mrvn>
too dark for apple juice
<mrvn>
geist: Note: the code still fails to compute the midpoint of a 3GB char array on 32bit archs.
<mrvn>
with pointers
* geist
nods
<mrvn>
makes me wonder if qsort() works with arrays > 2GB.
<geist>
it is one of the nice side effects of 64bit address spaces. no hardware (that i know of) supports a single aspace that covers more than half of the space
<geist>
and thus no signed math issues
<mrvn>
geist: amd64 pointers go from -xxxxx to xxxxx
<geist>
sure, but no real system has a single aspace that crosses that boundary
<mrvn>
Only problem is that nullptr there right in the middle
<geist>
but you're right, i guess i should rephrase
<geist>
no combination of 64bit hardware & software has an aspace that crosses a signed boundary
<geist>
that i know of. i'm actually kinda curious what modern POWER or z/machines can do. if there's any machine that actually enabled a full 64bit virtual aspace it'd be those
<mrvn>
On 64bit archs a-b can always fit in a ptrdiff_t
<mrvn>
(so far)
<geist>
POWER in particular with their hashtable was always more intrinsically set up to handle a full 64bit aspace than page tables
<geist>
since you just add more bits to the TLB and the hash
<geist>
i forget how big a PPC64 (ie, POWER4) let you go
<jjuran>
I met Marshall Clow at MacHack :-)
bliminse has joined #osdev
orthoplex64 has quit [Remote host closed the connection]
orthoplex64 has joined #osdev
FreeFull has joined #osdev
GeDaMo has quit [Remote host closed the connection]
Dyskos has quit [Quit: Leaving]
<sbalmos>
geist: tangentially along the same lines, I was always fascinated by the OS/400 / z/OS native bytecode software distribution system. The whole switch from 32 to 64 bit was a non-issue, since it just transparently recompiled all software on next load. Always made me wonder whether, maybe not WASM, but some OS that "natively" distributed its libraries and apps in LLVM IL format would work.
<zid>
real compliation is slow
<j`ey>
LLVM IR isn't fit for that, since it already has encoded stuff like ptr size etc
<mrvn>
LTO output could be
<zid>
If you just want 'most of the speed, but no 40 minute pause while it optimizes during the rebuild', you can just run the bytecode
<zid>
gcc takes me 15 minutes to link
<mrvn>
sbalmos: why not just distribute source code? '!/usr/bin/gcc' :)
<mrvn>
sbalmos: why not just distribute source code? '#!/usr/bin/gcc' :)
<zid>
My favourite cross platform bytecode is a zip of the github
<sbalmos>
mrvn: gotta have something of a binary format for the folks who don't/can't distribute source
<mrvn>
base64?
<sbalmos>
j`ey: random curiosity hasn't gone deep enough yet... LLVM the instruction set doesn't define a pointer width, but the IR does?
<mrvn>
sbalmos: something like sizeof(void*) is probably already evaluated
<sbalmos>
mrvn: grr
<mrvn>
Would any constexpr/consteval/constinit code remain in the IR?
<sbalmos>
mrvn: I was even thinking of Rust for a moment, but then you still have to define the platform tuple, memory widths, etc for rustc for anything not already defined
<j`ey>
sbalmos: I dont really understand that question, but also stuff like what mrvn said
<zid>
and again, it all just comes back to optimization taking time
<zid>
you don't want to redo them all every time you run it, that's called javascript
<zid>
and it's slow and uses shit loads of memory
<sbalmos>
obviously not
<sbalmos>
like I first said, was just fascinated by OS/400's not-quite-source bytecode distribution
ptrc_ has joined #osdev
ptrc has quit [Ping timeout: 260 seconds]
ptrc_ is now known as ptrc
ptrc has quit [Remote host closed the connection]
ptrc has joined #osdev
Likorn has quit [Quit: WeeChat 3.4.1]
srjek has joined #osdev
<energizer>
if you want to specify cpu pipelining sequence for a program, is that expressible in uops, or not even there?
<moon-child>
zid: this is well studied by eg java, js
<moon-child>
you do it incrementally
<energizer>
iow, is there a language that can customize pipelines on a per-program basis, or is it fundamentally baked into the hardware and you cant touch it
<moon-child>
and unlike java, you can cache. But you might not want to, since the whole reason java is fast is that it dynamically recompiles based on runtime stimuli
<mrvn>
energizer: For ia64 you have to pipeline yourself.
<energizer>
i see, cool
<mrvn>
gcc/clang will customize pipelines sometimes by inserting a NOP. But it's really a cpu specific optimization.
<mrvn>
add eax, ebx; nop; mul eax, edx or so can be faster than without NOP kind of thing.
<mrvn>
C++ is allowed to merge calls to new. Is C allowed to merge malloc calls?
<mjg>
how would that work vs free later?
dh` has quit [Quit: bbiab]
<mrvn>
it has to see free calls for it
Likorn has joined #osdev
<mrvn>
int *p = new int; int *q = new int; ... delete q; delete p; can be optimized into allocating 8 byte and a single delete
Likorn has quit [Client Quit]
<mrvn>
or none and put 8 byte on the stack
<mjg>
i know c allows for malloc/free to be elided altogether
<mjg>
or at least that's how clang sees it
<jjuran>
Seems like a similar merging of malloc calls would be allowed under "as if" behavior
<mjg>
so while i don't have a definitive answer, it does seem that "yes" would work in special cases of that sort
<mjg>
[i know about clang because it keeps fucking up the malloc microbenchmark from will-it-scale, making it a nop]
<mrvn>
jjuran: can't be under "as if" because malloc/free have side effects
<mjg>
see above
<mjg>
at least clang elides malloc + free altogether in this loop:
<mjg>
while (1) {
<mjg>
void *addr = malloc(SIZE);
<mjg>
assert(addr != NULL);
<mjg>
free(addr);
<mjg>
(*iterations)++;
<mjg>
}
<mjg>
making the side-effect note moot
<mrvn>
the side-effect note means the standard has to specifically allow this
dmh has joined #osdev
<jjuran>
mrvn: The only side effect I can think of is mutating the malloc pool, and that's a detail not specified by the language
<jjuran>
Well, the exact bits of the address are observable
<mjg>
again see above. clang elides malloc + free altogether in the pasted loop. so either the standard allows this or clang is violating it and i'm guessing it's the former
<mrvn>
depending on your malloc and sizes the addresses might stay the same
<mrvn>
mjg: yeah, me too. Just aren't sure about it.
<kingoffrance>
eh, if it inlined it that would scrwe up the microbenchmark too, no?
<kingoffrance>
i mean, it would be constant screw up seemingly, but point being if you really wanted to call a function, other ways that gets "elided" too
<mrvn>
kingoffrance: like when malloc/free are static inlie functions?
<kingoffrance>
well someone like me who wants to intercept everything, i do worry about these things lol
<kingoffrance>
i dont know practically if ppl do
<mrvn>
glibc intercepts malloc/free. it has hooks that get called on every call.
<mrvn>
And in mjg's example suddenly no hooks get called.
<kingoffrance>
yep
<moon-child>
mjg: see, this is why it's nicer to write such benchmarks are written in asm :P
<mjg>
it's not my code man fwiw
<mjg>
i just suffer because of it
<mrvn>
mjg: what does your code get turned into? "1: b 1b"?
<mrvn>
or is iterations volatile and printed in a SIG_ALARM?
<mjg>
said empty loop
<mjg>
running the bench only gives you 0s
<mjg>
funny part is that clang aligned the loop target to 16
<heat>
mrvn, no. more like new int behaves exactly like e.g new (std::nothrow) int
<mrvn>
ahh, always or only on nothrow?
<heat>
always
<heat>
basically the global operator new loses the implicit nonnull attribute
<geist>
yah from fiddling with it, basically everything but a plain new (nothrow, or even custom ones) can return null according to the compiler, thus it wont elide a check
<mrvn>
That seems wrong. I would only want that on nothrow
<geist>
gcc added this switch to make it work in all cases
<mrvn>
A super feature would be for implicit checks for new == nullptr with stack unwinding in constructors.
<mrvn>
Foo(size_t size) : data(new uint8_t(size) { } ... Foo *foo = new Foo(); if (!foo) panic(); should work
<moon-child>
I use -falign-functions so I can stick data in the low bits of function pointers
<moon-child>
am I a bad person?
<zid>
yes but that's not related
<mrvn>
yes
<mrvn>
how do you even access the bits in a function pointer?
<moon-child>
cast it to an integer
<mrvn>
function pointer can be larger than intptr_t
<moon-child>
not on the platforms I care about
<moon-child>
and see app. J.2
<zid>
It's IDB, so you do it in an IDB way that works on your I
<mrvn>
on basically not x86 you don't even have to align the functions.
<mrvn>
(for 1 bit)
<moon-child>
2 bits, no?
<moon-child>
afaik arm and riscv ops are 4 bytes each, so you get 2 bits
<mrvn>
16 bit opcodes gives you even addresses
<moon-child>
what arches have 2 byte ops?
<zid>
fum
<geist>
riscv
<geist>
and arm32 with thumb
<geist>
but still even at least
<mrvn>
MC68000
<moon-child>
riscv is 4 byte ops
<mrvn>
ia64 has 256bit opcodes, right?
<geist>
nope. there's a 16bit instruction set
<moon-child>
there are compressed 2 byte ops, but you have to pair them
<geist>
nope
<moon-child>
oh
<geist>
they are and can be totally unaligned
<geist>
well 2 byte unaligned
<geist>
there are no such restrictions with branching to aligned-on-two offsets
<geist>
i think they decided that even though very low end modern cpus will probably fetch at least 32 bits at a time aligned on 4 bytes, it's not a lot of hardware to deal with decoding the instruction in the second half of the word, or an instruction that crosses that threshold
<geist>
there may be some penalty to branching to a 4 byte instruction unaligned, but probably only on very low end cpus, before it has a chance to prefetch
<heat>
do the riscv/arm instructions need to be 4 byte aligned?