<davidlt[m]>
djdelorie: we don't have an image, but with the kernel from yesterday it should boot on Unmatched.
<davidlt[m]>
djdelorie: we have no SPI-NOR bootmode available in U-Boot, but the patches for doing it were posted maybe 3 weeks ago for a review.
<davidlt[m]>
So the firmware is loaded from the microSD (U-Boot SPL, OpenSBI, U-Boot proper) at that point you get PCI and thus NVMe and USBs working (or a SATA card too).
<davidlt[m]>
btw, someone figured out a nice way to handle reboot
<davidlt[m]>
To be exact seems that fedpkg sources is failing with timeouts, but on a large sources
<rwmjones>
ok /me tries fedpkg sources in kernel
<davidlt[m]>
Yeah, kernel is one of those which failed
<rwmjones>
when dj wakes up we're going to try and go through the process of booting Fedora on the unmatched
<davidlt[m]>
maybe I could create an image for Unmatched
<rwmjones>
that would be hugely helpful
<davidlt[m]>
kernel build finished overnight thus all the bits for Unmatched are now in the repository
<rwmjones>
even instructions on what we need to do would help
<davidlt[m]>
the rest is exactly the same as Unleashed
<davidlt[m]>
The we upstreamed U-Boot is that it will attempt to boot from NVMe by default, then USB, then MMC, then PXE, then DHCP IIRC
<davidlt[m]>
Ah SATA was added too (not sure if it landed already and it's probably 2022.01 material)
<rwmjones>
is there a sata port on unmatched? I don't recall one
<davidlt[m]>
The firmware must be on the microSD card (no support for SPI NOR Flash yet, probably in 2022.01)
<davidlt[m]>
No, but some people use PCIe SATA cards
<davidlt[m]>
Some people want multiple drives (don't ask why)
<rwmjones>
ah I see
<davidlt[m]>
I assume I will be able to start hacking a new disk image this evening, but again I might have missed some stuff, but I think the basics are in
<davidlt[m]>
I still want to pull in SPI-NOR bootmode support in U-Boot, DT changes to enable more things, irqchip fixes (issues with PLIC) which also causes issues with RTC, maybe reset (well delayed reset) support if there are any feedback on it, etc.
<davidlt[m]>
There are a bunch of interesting things posted for a review, but no reviews yet on them :/
bkircher has quit [Quit: Leaving]
<rwmjones>
so I finally got around to running fedpkg sources, and it's not hanging for me, will repeat it a few times to see
<davidlt[m]>
maybe it's only via kojid that it happens
fede2cr has joined #fedora-riscv
<davidlt[m]>
okay, kernel-tools will be a problem
<davidlt[m]>
looking at Koji logs, some BPF hell, probably some chicken and egg situation here
<davidlt[m]>
the config seems to be fine, thus all debug crap should be in the kernel build
<davidlt[m]>
but it seems bpftool didn't generate what was needed and struct bpf_perf_event_value is missing
<davidlt[m]>
maybe it needs a newer Clang version
<davidlt[m]>
I don't really want to debug it via Koji (just costs extra time... and you loose work directory)
<sharkcz>
FWIW there is a koji plugin that preserves the whole chroots for failed builds
<djdelorie>
davidlt[m]: I think part of my problem is getting an initial image onto the nvme card
<djdelorie>
rwmjones: I'm awake :-)
<rwmjones>
djdelorie: hey
<rwmjones>
give me a few mins, got to send some emails
<djdelorie>
np, I'm still eating breakfast
<davidlt[m]>
this will be boring, but I am generating distro repos
<rwmjones>
I'm also going to send an email to kevin fenzi, see if he can join us
<davidlt[m]>
I just had dinner thus time is ready
<davidlt[m]>
Hmm.. need to make a tea while this is cooking
<davidlt[m]>
new image is cooking in Koji (still needs some post-steps)
<davidlt[m]>
this will take some time, the last one took 4 hours
<davidlt[m]>
nope, we all got DT from the same place
<rwmjones>
ok
<davidlt[m]>
upstream kernel
<rwmjones>
let's see then ..
<rwmjones>
I mean is there a newer dtb?
<rwmjones>
another question: how can I make the mac address stable?
<rwmjones>
and can you run these fanless? the case fans I got have an annoying whine but I'm not sure if I should run without a case fan (except the PSU fan)
<davidlt[m]>
So
<davidlt[m]>
Not really, but there are a bunch of stuff that's not merged or aren't posted for a review
<davidlt[m]>
So DT will have changes at some point, but that's nothing critical
<davidlt[m]>
If MAC address is not stable I assume because there is no I2C EEPROM driver
<davidlt[m]>
No, fan is needed
<davidlt[m]>
(well you can replace it, I am running Noctua fans)
<davidlt[m]>
The limit (not upstreamed) is 85C for critical
<davidlt[m]>
The best wouldn't go beyond 40C (but it's fine to above)
<davidlt[m]>
Probably 40-60C is fine
<rwmjones>
btw I found a bug that was specific to riscv today (well, it turned out it affected arm after we did some more investigation)
<rwmjones>
davidlt[m]: so the DT it's using is not the one off disk, it's:
<davidlt[m]>
all should be good here
<rwmjones>
## Flattened Device Tree blob at ff73d020 Booting using the fdt blob at 0xff73d020 Using Device Tree in place at 00000000ff73d020, end 00000000ff746e0f
<rwmjones>
is that from the vmlinuz itself, or from a rom?
<davidlt[m]>
U-Boot use a slightly modified DT, but it's not passed to Linux (yet)
<rwmjones>
ok back to the ubuntu kernel
<davidlt[m]>
It has to contain things like DDR init which are not needed for Linux
<davidlt[m]>
But in general it's synced with the kernel so at some point it will be passed to the kernel
<rwmjones>
ok I think my extlinux conf was wrong which is why it didn't pick up the DT, trying again ...
<davidlt[m]>
rwmjones: does your PCIe work in the U-Boot?
<davidlt[m]>
Ok
<rwmjones>
yes
<davidlt[m]>
So far it from my point of view all things seem to be in places, the patches are also upstream that are required
<davidlt[m]>
djdelorie: any feedback on glibc / kernel RISC-V vectors support?
<rwmjones>
looks like no network again
<davidlt[m]>
could you test networking on U-Boot side?
<rwmjones>
sure, any particular commands to try?
<davidlt[m]>
I think "ping <ip_address> should work
<rwmjones>
ok one sec
<davidlt[m]>
you would also need dhcp commands most likely
<djdelorie>
davidlt[m]: The argument continues. I'm all for an ELF bit that says "be paranoid" as a cover-all but the setjmp side is still being discussed
<djdelorie>
not knowing how many registers you might have adds considerable difficulty
<rwmjones>
ping failed; host 192.168.0.1 is not alive
<sorear>
it would help if there was a privileged register to override vlmax, so you can run threads with the vector state limited to <N> kB until a higher value has been negotiated
<davidlt[m]>
rwmjones: run "dhcp" first
<sorear>
but this ties in to the migration issue (because you're changing vlmax) and the decision to get rid of the explicit vuncfg
<rwmjones>
yeah it works fine
<rwmjones>
Using ethernet@10090000 device
<rwmjones>
ethernet@10090000: link up, 1000Mbps full-duplex (lpa: 0x7c00)
<rwmjones>
host 192.168.0.1 is alive
<djdelorie>
davidlt[m]: the base problem is that jmpbuf is allocated at compile time
<djdelorie>
s/davidlt/sorear/
<djdelorie>
there are other places we need to preserve registers that are dynamic
<rwmjones>
let me log the full boot sequence
<sorear>
the vector registers are all call-clobber specifically so that jmpbuf doesn't have to change, unless someone changed that...
<djdelorie>
whew
fede2cr has joined #fedora-riscv
<djdelorie>
does the chip itself have a hardware config register that tells you what it "is" ? or do you just have to know?
<rwmjones>
no network device found at all as far as I can see
<sorear>
if you're asking specifically about the max vector length, yes you can read that with instructions
<djdelorie>
sorear: I was
<djdelorie>
well, and total number of vector registers
<davidlt[m]>
rwmjones: did you install all the kernel modules? macb driver is set to m, not y
<sorear>
but in general no, you need the device tree to know what extensions are supported (the V opcodes might be used by a non-standard extension if V is not supported)
<sorear>
total number of vector registers is always 32
<davidlt[m]>
rwmjones: is it loaded?
<sorear>
the length varies, not the count of them
<rwmjones>
davidlt[m]: there are only 3 modules loaded: nvme, nvme_core and i2c_dev
<djdelorie>
so you can auto-detect the size of the vector save space, but not IF there's a vector save space
<rwmjones>
what does "macb driver is set to m, not y" mean?
<rwmjones>
oh you mean CONFIG*
<davidlt[m]>
rwmjones:
<sorear>
n.b. i've read the 1.0 spec but am not on the mailing lists and could have missed very recent changes
<davidlt[m]>
it's loadable module
<rwmjones>
yeah, got it, what's the name of the module? not macb
<davidlt[m]>
So if your modules don't match the kernel version it probably won't load it
<sorear>
djdelorie: i think the plan is that you can getauxval(AT_HWCAP) & (1 << ('V' - 'A'))
<sorear>
which ultimately comes from the device tree
<rwmjones>
oh wait, no I don't have any kernel modules for this kernel, that's weird
<davidlt[m]>
obj-$(CONFIG_MACB) += macb.o
<davidlt[m]>
obj-$(CONFIG_MACB_PCI) += macb_pci.o
<davidlt[m]>
I think, it's macb
<sorear>
this is going to be a big mess with the multi-letter standard extensions, I don't know what the plan is for supporting more than 2 words of hwcap
<davidlt[m]>
did you install all kernel-modules* rpms?
<davidlt[m]>
We will have TONS of extension very soon
<davidlt[m]>
I am not sure how to keep track of those in RISC-V land
<davidlt[m]>
Hmm.. Maybe I should consider to macb=y in the later kernels
<rwmjones>
no I didn't install kernel-modules, ok I'll d othat
<davidlt[m]>
I think that's your problem :)
<davidlt[m]>
You don't have network driver :)
<rwmjones>
no it's even weirder than that
<rwmjones>
something has deleted /lib/modules/5.14.16...
<davidlt[m]>
I can only think of one thing, kernel install script deleting that, but I doubt it
<rwmjones>
I think anything where it's built into the board should really be non-module
<davidlt[m]>
I don't think that's the what Fedora kernel maintainer would like :)
<davidlt[m]>
But yeah, I can do that :)
<rwmjones>
be nice to have a theory for what deletes the directory; on the other machine the dir is intact
<rwmjones>
ok reboot again
fede2cr has quit [Quit: Ping timeout (120 seconds)]
<djdelorie>
ok, I have davidlt's fedora image mounted on my pc, and the usb stick with the OE stuff also mounted. Can I just swap the rootfs's or do I need to tweak the /boot also?
<djdelorie>
(sorry, distracted by actual work-work too)
<davidlt[m]>
rwmjones: ^^ I think you have a better guide :)
<davidlt[m]>
I am still hours before assembling the disk image :)
<rwmjones>
djdelorie: do you have /boot/extlinux/?
<rwmjones>
davidlt[m]: I don't believe it, the module directory has been deleted again!!!
<djdelorie>
rwmjones: yes
<davidlt[m]>
what?
<rwmjones>
I booted it into the ubuntu kernel, ran dnf reinstall kernel-modules-5.14.whatever
<rwmjones>
then rebooted into the fedora kernel and something either during shutdown or reboot had deleted /lib/modules/5.14...
<rwmjones>
djdelorie: so in /boot/extlinux/extlinux.conf I have ...
<davidlt[m]>
there is a thing that shouldn't be a problem
<davidlt[m]>
on system shutdown NVMe considers it incorrect sutdown
<davidlt[m]>
(if you look at the SMART data)
fede2cr has joined #fedora-riscv
<rwmjones>
of course usb doesn't work because no modules ...
<davidlt[m]>
:D
<fede2cr>
BTW, yesterday I was complaining about the fedora images for the nezha board from rvboards.
<davidlt[m]>
Okay, another thing to =y
<fede2cr>
Today I tried the oldest image in the download repo (for the nezha) and it's working fine.
<davidlt[m]>
At least you get PCIe working 😉
<rwmjones>
do we send flush commands properly through to nvme?
<fede2cr>
I was able to dnf update all the packages. It's not booting with 5.14, but with the defualt 5.4, but that is probably because it's bypassing grub or something like that. But it is working fine.
<davidlt[m]>
I personally never hit issues, but again SMART data suggest that it's ungraceful shutdown
<davidlt[m]>
power_cycles: 886
<davidlt[m]>
power_on_hours: 231
<davidlt[m]>
unsafe_shutdowns: 708
<davidlt[m]>
From: nvme smart-log /dev/nvme0
<davidlt[m]>
There are no reports about this being a problem, but I don't like it.
<rwmjones>
let's hard reboot in case it's something in the ubuntu kernel
<davidlt[m]>
it's always a few times :)
<rwmjones>
now booting fedora
<davidlt[m]>
share lsmod if network and usb works
<davidlt[m]>
so I could find the modules easier to enable as =y
<rwmjones>
it looks like the directory has been deleted again, it's having trouble mounting my NFS drives
<rwmjones>
this time I downloaded the kernel-modules rpm to the local disk, so I should be able to recover it without the dance
<rwmjones>
but there's something very wrong in a start-up script
<rwmjones>
yup, confirmed
<davidlt[m]>
startup script?
<rwmjones>
must be I suppose, what else?
<rwmjones>
unless it really is nvme losing the writes even though I did sync 3 times
<davidlt[m]>
what do you mean by "startup script"? :)
<rwmjones>
it couldn't be that actually, because the RPM didn't get lost
<rwmjones>
I mean a systemd unit
<davidlt[m]>
U-Boot environment? Systemd unit file mess?
<rwmjones>
could u-boot see into an ext4 filesystem?
<davidlt[m]>
yes
<davidlt[m]>
you can ls
<davidlt[m]>
or ext4ls
<rwmjones>
I still can't modprobe macb, I get lots of:
<rwmjones>
[ 437.234115] macb: Unknown symbol phylink_set_pcs (err -2)
<rwmjones>
[ 437.238896] macb: Unknown symbol phylink_set_port_modes (err -2)
<rwmjones>
etc
<davidlt[m]>
2725 CONFIG_PHYLINK=m
<davidlt[m]>
2726 CONFIG_PHYLIB=y
<rwmjones>
I don't have a phylink module, unless it has another name
<davidlt[m]>
hmm
<rwmjones>
oh I see
<rwmjones>
it's in kernel-core, and those drivers got deleted too, but I didn't reinstall kernel-core
<rwmjones>
let me fix all that mess too
<rwmjones>
reboots ...
<davidlt[m]>
yes, I checked and it seems to be kernel-core
<rwmjones>
let's reinstall all the kernel pgs
<davidlt[m]>
I assume Koji will take another 1-2 hours to finish disk image before I can start assembling it (and probably writing instructions how to do it)
<davidlt[m]>
it's still cooking, so I am going to the bed
<rwmjones>
this is really weird
<rwmjones>
it deleted only some of the files in /lib/modules/5.14...
<davidlt[m]>
what deleted it?
<rwmjones>
I don't know - the thing that deletes /lib/modules!
<rwmjones>
but apparently only partially
<rwmjones>
let's get rid of /.autorelabel in case that could be anything to do with it
<djdelorie>
well, I've gotten to "starting kernel" and it hangs... using vmlinuz-5.10.6-200.0.riscv64.fc33.riscv64
<rwmjones>
djdelorie: yeah that kernel won't work
<rwmjones>
you need the 5.14 kernel that davidlt[m] built
<davidlt[m]>
you need a different kernel
<djdelorie>
ok
<rwmjones>
5.14.16-101.0.riscv64.fc33
<djdelorie>
thought it was something like that
<davidlt[m]>
and you need to protect your kernel modules apparently :)
<djdelorie>
heh
<fede2cr>
Nice, I've just changed two pi4s for another beagleV.
<fede2cr>
changed as in swapped.
<djdelorie>
sigh, out of space on /boot
nb has quit [Read error: Connection reset by peer]
nb has joined #fedora-riscv
<rwmjones>
ok for my next experiment, I've reinstalled the kernel packages again (from Fedora) and I'm going to reboot to see if that deletes /lib/modules/
<rwmjones>
before reboot
<rwmjones>
# find /lib/modules/5.14.16-101.0.riscv64.fc33.riscv64/ -type f | wc -l
<rwmjones>
3311
<rwmjones>
huh, same after reboot
<rwmjones>
I wonder if it was /.autorelabel after all?
<rwmjones>
anyway the good news is I seem to have a working Fedora system
* rwmjones
tries to remember what was on p2 & p3 ...
<rwmjones>
interestingly it's an MBR partition scheme and p2 and p3 are really missing
<rwmjones>
in answer to your question, it seems like /boot is fine on ext4, uboot can read ext4 ok
<djdelorie>
yay
<rwmjones>
nirik: fyi it seems like we've worked out how to make fedora boot, so once you've got the hardware assembled ping me or dj and we'll talk you through it
<nirik>
awesome. My parts are supposed to trikle in over the next week or two... I think the board showed up today.
<djdelorie>
it got "Unrecognized filesystem type"
<rwmjones>
when you used ext4 on /boot?
<rwmjones>
let me check mine again
<djdelorie>
yup
<rwmjones>
$ mount | grep /boot
<rwmjones>
/dev/nvme0n1p1 on /boot type ext4 (rw,noatime,x-systemd.mount-timeout=300s)
<djdelorie>
ah, I have that, but only used it for the usb and nvme
<djdelorie>
so I should be able to dd from the usb to the mmc, just that partition?
<rwmjones>
ok I'm booting mine quite differently
<rwmjones>
I'm booting from mmc (SDHC card)
<rwmjones>
then have configured uboot to read the /extlinux/extlinux.conf from nvme0:1
<rwmjones>
not using usb at all
<djdelorie>
right, I mean, I can dd /dev/sda2 to /dev/mmcblk0p2
<rwmjones>
mmc is only used to store the firmware
<rwmjones>
no I dd it over the whole device
<djdelorie>
that's how I got /dev/sda in the first place :-)
<rwmjones>
so I'm unclear what we're talking about, what's /dev/sda?
<djdelorie>
the usb stick
<rwmjones>
usb in the hifive, or in a PC?
<djdelorie>
(or the nvme could be used, they were both created with that xzcat command
<djdelorie>
on the hifive
<rwmjones>
so here's what I did:
<djdelorie>
the usb stick is sda
<rwmjones>
took an SD card, put it into my laptop, wrote 32 MB over the whole device, plugged SD card into the hifive and booted from that
<djdelorie>
my brain wants to know why "do it on a pc" is important
<djdelorie>
all that data is already on the hifive
<rwmjones>
it's not, but that's how I did it, just to be clear
<djdelorie>
I'm asking if anything outside the first two partitions is important, or if only the second partition needs updating
<djdelorie>
I don't want to corrupt the GPT
<rwmjones>
ok, I'm not sure what you mean by partition - partition on what? I'm treating the first 32 MB of the SD card as a blob, don't care if it contains a partition table
<djdelorie>
the first 5M spans two partitions; the docs say they're the uboot SPL and ITB, respectively
<djdelorie>
I suppose I could just try it, what's the worst that could happen? ;-)
<rwmjones>
right, doesn't matter
<rwmjones>
just grab the first 32 MB from the image, splat it on the SD card (whole device)
<rwmjones>
apparently it does have a partition table, but that's not material
<djdelorie>
dd'ing those two partitions did the trick, but it still says unsupported, so... on to the next bug!
<rwmjones>
hmm ok, i don't know in that case; can you use ext4ls to list nvme0:1 ?
<rwmjones>
I think the command is:
<rwmjones>
ext4ls nvme 0:1 /
<djdelorie>
0:1 and 0:2 say unsupported, 0:3 (which is /boot) works
<rwmjones>
ah, I see
<rwmjones>
so that should be fine
<rwmjones>
you'll need to set the uboot variable to boot off 0:3 (which I cannot remember how to do), and save the config
<rwmjones>
probably printenv and look through all the variables for a likely candidate
<djdelorie>
looks like "bootable flag not set"
<rwmjones>
that's unexpected :-)
* rwmjones
checks mine
<djdelorie>
I had to resize /boot so ended up deleting and recreating the partition as a larger ext4
<rwmjones>
hmm, bootable flag isn't set on mine
<rwmjones>
maybe it defaults to p1?
<djdelorie>
that didn't help :-(
<djdelorie>
"part list nvme 0 -bootable devplist" doesn't pick partition 3 as bootable...
<rwmjones>
I'm fairly sure this will be controlled by a uboot variable, did you look at the output of printenv (which will be very long)?
<djdelorie>
but it finds it on usb and mmc
<djdelorie>
yeah I have it in an emacs buffer for searching
<djdelorie>
hmmm... turns out the fdisk "bootable" flag doesn't set the attr bit I wanted...
rwmjones is now known as rwmjones|afk
<djdelorie>
supper time...
<djdelorie>
had to set legacy_boot attr flag in parted
King_InuYasha has quit [Read error: Connection reset by peer]