00:23 <rtwfroody> I just got a Beaglebone Black Wireless, but don't have an SD card. What should I get? It looks like a micro SD card, but I can't find people say what size it should be.

00:36 <PhotoJim> rtwfroody: depends on what OS you want to put on it, and what you'll be doing with it. but these days, cards are cheap so I tend to get a little bigger than I need to avoid problems and hassle later. I'd suggest 16 or 32 GB.

00:37 <PhotoJim> I have a 32 in mine. 30% full.

00:37 <zmatt> you have 10G in use? o.O are you storing large media files or something?

00:37 <zmatt> rtwfroody: note that an sd card is not required, the beaglebone black and black-wireless have 4GB of internal eMMC

00:38 <zmatt> but yeah it's a microSD card, and how much space you need is completely dependent on what you're doing with it

00:38 <PhotoJim> fail2ban tends to use up a fair bit of storage. I'd guess it's the biggest culprit.

00:39 <zmatt> uhh, how?

00:40 <zmatt> I've never used it but it sounds like something that has no reason to require significant storage

00:40 <PhotoJim> I have a public IP address on mine.

00:40 <zmatt> rtwfroody: generally I'd recommend using eMMC unless you really need more space for your use-case

00:41 <PhotoJim> eMMC will be faster. but it's tight.

00:41 <zmatt> depends on what you do on it

00:41 <PhotoJim> Fair point.

00:41 <PhotoJim> If you were careful, you could probably make it work.

00:41 <PhotoJim> I've had to upgrade storage on a few systems (an Alix 2D3, some Raspberry Pis) so I tend to go bigger than needed to delay or prevent that.

00:42 <zmatt> the only beaglebone here that's ever been at risk of running out of space is our main development beaglebone, and that's mostly because 3 different devs work on it and tend to accumulate cruft, including countless separate installs of the same NPM module :P

00:42 <rtwfroody> . eMMC is on-board storage, right?

00:42 <zmatt> yes

00:43 <rtwfroody> I was just following instructions to update it, because no storage showed up when I plugged it into my Linux box. But maybe it is programmed and I need to figure out why nothing auto-mounted.

00:43 <zmatt> I'd generally recommend booting from eMMC unless you absolutely need more space and are willing to sacrifice performance to get it

00:44 <rtwfroody> zmatt: You've convinced me. Thanks.

00:44 <zmatt> note that the storage that shows up via usb is just a small read-only disk image containing some documentation and such, it's not of any use for updating or whatever

00:44 <PhotoJim> rtwfroody: you can always migrate to SD later. and I presume you could have / on eMMC and use an SD card for /home or additional storage if needed.

00:45 <zmatt> if the system on eMMC is in a broken or unknown state and you don't have stuff on it you want to preserve, I recommend just reflashing it

00:45 <zmatt> the easiest way to do that is by grabbing a ready-made flasher image, e.g. "AM3358 Debian 10.3 2020-04-06 4GB eMMC IoT Flasher" in the "Flasher Debian images" section of https://beagleboard.org/latest-images

00:46 <zmatt> write that to microSD card (4GB or larger), it's highly recommended to use Balena Etcher for doing so

00:46 <zmatt> if you then boot the beaglebone from that sd card, it should automatically proceed to wipe and reflash eMMC

00:48 <zmatt> (you can tell it's reflashing from the leds: the flasher shows a back-and-forth sweep (aka "cylon" or "knight rider" pattern) across all four leds)

00:51 <rtwfroody> Thanks, so 4GB is the magic number if I need buy an SD card.

00:51 <zmatt> in rare cases (specifically, if the existing bootloader on eMMC is really ancient or weird) it may be necessary to use the S2 button to force it to boot from SD card. this is done by holding the S2 button (the one closest to the SD card slot) while connecting power to the beaglebone, you can let go of the button once the power led turns on

00:51 <zmatt> (it doesn't hurt to do so just in case, but usually it's unnecessary)

00:52 <zmatt> yeah, 4GB is the minimum for flashing the "IoT" image, which has a lot of stuff preinstalled

00:54 <PhotoJim> what sort of free space does that image leave?

00:54 <zmatt> there are also very lean "console" images for people who prefer to start with a lean and clean system and are comfortable with simply installing whatever packages they want

00:55 <zmatt> the IoT image has 1.9G in use

00:55 <PhotoJim> that's not bad.

00:55 <rtwfroody> My problem was that I was using a "charging" USB cable when I first tried.

00:56 <zmatt> whenever you discover such a cable, cut it in half and throw it away

00:56 <zmatt> such cables are even bad for charging

00:56 <zmatt> (charger detection uses the data lines)

01:08 <zmatt> PhotoJim: and the console image uses 400M

01:08 <PhotoJim> zmatt: that'd be the better way to do it. then one could keep it as lean as possible.

01:12 <zmatt> our production beaglebones use 310M excluding our custom stuff, and maybe 100M of custom stuff or so

01:13 <zmatt> but those are very sparse systems used as appliances

01:13 <zmatt> (no development tools)

01:16 <PhotoJim> what kernel are you running, 4.19.0? I've run into some interesting issues with the 5.11.12 on my Arch install (that also happens with my Debian Bullseye 5.14.0 on a SheevaPlug) where IPv6 causes weird delays in response, but IPv4 doesn't. No idea how to isolate it enough to file a bug report. Clearly not hardware, exactly, though they're both 32-bit ARM. (my Pine A64+ with 64-bit aarch64 doesn't

01:16 <PhotoJim> have the issue though.)

01:17 <zmatt> we're still on 4.14, I should probably put some time into moving to a newer kernel, but it hasn't been a priority

01:18 <PhotoJim> well, a heads-up.

01:18 <PhotoJim> maybe it's the kernel. maybe it's something in the networking stack. no idea. so weird.

01:19 <zmatt> there's no logical reason for the hardware to be relevant for what you're describing... to the ethernet driver a packet is a packet, it doesn't know or care about the IP layer

01:19 <PhotoJim> exactly.

01:19 <PhotoJim> but it's super consistent. and repeatable. and before I realized the SheevaPlug was doing it too, I changed Ethernet cables, changed ports, even plugged it into another switch, and added a USB NIC (which had the same issues).

01:20 <PhotoJim> so it really doesn't seem to be hardware.

01:23 <zmatt> I mean, if it affects two totally different SoCs as well as a USB NIC then you've demonstrated successfully that it's not hardware-dependent (as expected) :P

01:24 <PhotoJim> I feel like I have :)

01:24 <PhotoJim> Pretty sure it's the same problem.

01:24 <PhotoJim> It's slightly annoying on the Sheeva. Makes the BB Black unusable with IPv6 though. It's much worse. But on IPv4... it's fine.

01:25 <PhotoJim> if I had another BB Black and knew someone else with IPv6, I'd test one there.

01:25 <PhotoJim> That's the ideal next step. Reproduce it somewhere that isn't on my network.

01:26 <zmatt> I can test 5.10, or does it have to be 5.11 ?

01:27 <PhotoJim> No idea to be honest.

01:27 <zmatt> (and the network that beaglebone is on only has link-local ipv6, no global ipv6, if that matters)

01:28 <PhotoJim> And no idea, to be honest. :)

01:28 <zmatt> what problem are you observing exactly?

01:28 <PhotoJim> IPv6 connections act as if there is huge packet loss.

01:28 <PhotoJim> Huge delays on ssh connections, e.g.

01:29 <PhotoJim> a couple of characters in a command might appear, then nothing, nothing, nothing, then a bunch of characters you typed a few seconds ago, then nothing...

01:29 <zmatt> hmm

01:29 <PhotoJim> if you had a connection with really bad packet loss... it would be like that.

01:30 <PhotoJim> and if it matters, I use systemd-networkd for IP configuration.

01:30 <zmatt> I use systemd-networkd too

01:30 <zmatt> oh wait not on this beaglebone

01:31 <PhotoJim> you could possibly deploy a private IPv6 range too. fc00:/12, I think, is all private. I use an fdde:: subnet locally (plus a public 2605:: one)

01:31 <zmatt> I don't see how that would make a difference compared to link-local

01:31 <rtwfroody> Hmm... beaglebone connects to USB, and then fails with some USB error (in dmesg output) a few minutes later.

01:31 <PhotoJim> less hassle than adding the %nic extensions. but in theory it shouldn't matter.

01:33 <PhotoJim> I'm not sure I tested if the problem exists on fe80:: addresses. I'll test.

01:34 <PhotoJim> hmm, seems okay.

01:35 <PhotoJim> I'll make another session to the 2605:: address.

01:35 <PhotoJim> first ssh attempt failed. second succeeded. evidence of the problem.

01:35 <PhotoJim> Yup, has the problem.

01:36 <PhotoJim> and another connection to the fdde:: address does too

01:36 <PhotoJim> but the fe80:: session is fine.

01:36 <PhotoJim> so that's another piece of evidence.

01:37 <zmatt> that makes absolutely no sense, wtf is going on on your network

01:37 <PhotoJim> and only on 32-bit ARM systems that aren't Raspberry Pis.

01:38 <zmatt> rtwfroody: nobody can possibly give any useful feedback on a statement that vague btw

01:38 <zmatt> rtwfroody: (what error? share using pastebin. have you already reflashed it to the latest image? does the beaglebone seem to boot normally based on led activity?)

01:41 <PhotoJim> zmatt: networkctl shows eth0 (and eth1, which is still connected) as "configuring" sometimes, and "configured" at other times. so I wonder if there's some issue where IPv6 stateless autoconfiguration is getting purged or reset somehow, then restored. that would explain the delays. the system may be without that IPv6 address for a moment.

01:41 <zmatt> have you checked journal

01:41 <zmatt> ?

01:41 <PhotoJim> No. Doing that now.

01:42 <PhotoJim> Nothing obvious on journalctl -xe

01:43 <PhotoJim> looking without the -xe now

01:43 <PhotoJim> Nothing obvious.

01:44 <zmatt> well, good luck with it! :)

01:44 <PhotoJim> if you get any inspiration or ideas... let me know

01:45 <PhotoJim> or if you find the same problem and need me to help diagnose or investigate... also willing

01:45 <PhotoJim> I wish we had IPv6 at work. I'd take it there for awhile. (No one would care. Our IT guy there trusts me.)

01:46 <zmatt> afk

01:47 <PhotoJim> I should change systemd-networkd to debug logging.

02:08 <PhotoJim> also afk