ArmbianHelper changed the topic of #armbian to: armbian - Linux for ARM development boards | www.armbian.com | Github: github.com/armbian | Commits: #armbian-commits | Developer talk: #armbian-devel | Forum/Twitter feed: #armbian-rss | Type 'help' for help | Logs: -> irc.armbian.com
<[TheBug]>
FYI Armbian.com (including forum.armbian.com and the redirectors for apt) are down and may be for up to 2 hours from now, emergency maintenance at DC where it is hosted. Hope to have things back up ASAP guys, sorry for any inconvience.
<ReThan>
So i am almost there 99% i think, i just need to disable the ttys0 console on boot as i am using this for sensor data, i cannot access the forums to see how the others did it
<ReThan>
i tried systemctl disable serial getty etc but on reboot it starts up again
<ReThan>
if anyone knows i would apprecaite :D
SallyAhaj_ has joined #armbian
SallyAhaj has quit [Ping timeout: 260 seconds]
schwarz_Kat has left #armbian [Offline]
aasami_ has joined #armbian
aasami_ has quit [Client Quit]
aasami_ has joined #armbian
xoan0 has joined #armbian
xoan has quit [Ping timeout: 246 seconds]
xoan0 is now known as xoan
<ReThan>
sorted :)
ReThan has quit []
BCMM has joined #armbian
Net147 has quit [Quit: Quit]
Net147 has joined #armbian
archetech has quit [Quit: Konversation terminated!]
pablocastellanos has quit [Ping timeout: 250 seconds]
<[TheBug]>
From what I can see the issue with the site is that there was a large fiber cut in Santa Clara, CA affecting several large DCs in that area and where Armbian.com is hosted, they have been working all night to resplice fiber and get things back up since that but it take a lot of time to do. Hopefully things will get back online soon or we will start looking at bringing things up on a back-up
<[TheBug]>
location as time allows.
xoan8 has joined #armbian
SallyAhaj has joined #armbian
xoan has quit [Ping timeout: 245 seconds]
xoan8 is now known as xoan
SallyAhaj_ has quit [Ping timeout: 246 seconds]
SallyAhaj_ has joined #armbian
SallyAhaj has quit [Ping timeout: 272 seconds]
<BCMM>
[TheBug]: thanks for keeping us updated!
SallyAhaj has joined #armbian
SallyAhaj_ has quit [Ping timeout: 246 seconds]
<Armbian-Discord>
<bret> Multiple datacentres having their entire service out because of a single fibre cut seems.. unfortunate?
<Armbian-Discord>
<bret> wouldn't be expecting everything to be unavailable because of a single fibre being cut, though I have no idea which provider it is so :p
<FergusL>
Re: C2 picky about sdcards: it appears that one of the card I tested is reported as not working and c2 and the other one has had no report and his an less common brand (Philips). I just got a SanDisk Ultra A1 today, hopefully it will confirm that the device is fine!
pablocastellanos has joined #armbian
<Armbian-Discord>
<macstibs> you guys know your server is down?
<Armbian-Discord>
<macstibs> 🤪
<IgorPec>
we haven't slept whole night and guys who are doing repairs neither ... server is down? :)
archetech has joined #armbian
<IgorPec>
some infra we have HA only withing data center, but for the next time we are going to handle this kind of outage too
<Armbian-Discord>
<bret> ah, don't take my comments as a dig at you guys, was just curious that one fibre could bring down a datacentre like that
<Armbian-Discord>
<bret> unless you're running the whole datacentre thennn 😉
<[TheBug]>
Not a single fiber cut, multiple
<[TheBug]>
taking down multiple carriers
<[TheBug]>
Our provider uses Zayo and Internap and both had fibers cut
<Armbian-Discord>
<bret> ah, well that's different and makes more sense then :p when you said a large fiber cut I thought it was singular, my bad
<[TheBug]>
Appears there is ongoing construction in the area and construction crews dug up fiber
<[TheBug]>
so I am sure someones paying a large amount of fines per hour till its fixed
<[TheBug]>
I forget what it is, something like 10k an hour when you mess that up
<Armbian-Discord>
<bret> sounds about right, at least you're not waiting for boats to head out and find it 🙃
<Armbian-Discord>
<bret> I always enjoy when those sub-marine cables in the APAC region get cut, makes for fun times
<[TheBug]>
FergusL: biggest issue I had with c2 and Sdcard is if you don't cool the c2 well it will bake your SDcards, I had one SDcard totally fail before I fixed the cooling for it. I now have a custom metal case that is a good passive heatsink, previously was using the OEM case and there wasn't enough air flow to keep it cool enough.
<Armbian-Discord>
<Myron> Is it worth scheduling a cron task to "fstrim -all" to try extend the life of an SD card.
<nekomancer[m]>
<Armbian-Discord> "<Myron> Is it worth scheduling..." <- there are systemd timer. just enable it.
<nekomancer[m]>
`fstrim.rimer`
<nekomancer[m]>
* `fstrim.timer`
<BCMM>
apt.armbian.com is back, but it seems to redirect to a random mirror, which makes apt upset if it's an https -> http redirect
<BCMM>
(and my armbian.list had the https url by default)
Herc has quit [Remote host closed the connection]
<IgorPec>
apt was moved to another location
Herc has joined #armbian
<IgorPec>
just for website and forum we don't find time
<Armbian-Discord>
<Myron> Thanks nekomancer.
<BCMM>
IgorPec: yeah, figured it was something like that
<BCMM>
anyway, just mentioning that it doesn't actually work on the default configuration, in case anybody else wants to take an 's' out of their armbian.list
<[TheBug]>
lanefu ^ may be a quick config fix for redirector needed?
<[TheBug]>
I believe recall that was an issue before at some point and there was some change
<Armbian-Discord>
<Myron> @nekomancer[m] Just had a look through the filesystem. fstrim.timer is not there. I've got it running on a cron. Still does the same thing. */6 * * * /usr/bin/echo -e "$(date)\n\r$(/usr/sbin/fstrim --all --verbose)" 1>/tmp/fstrim.out 2>&1
<[TheBug]>
BCMM: pretty sure all the mirrors support https its likely a config issue from quick redeployment but lanefu would have to check that when he can in a bit
<BCMM>
[TheBug]: that does seem to be true for at least the random selection of http mirrors it's given me so far
Guest4645 has joined #armbian
Guest4645 has left #armbian [#armbian]
Guest4645 has joined #armbian
Guest4645 is now known as Myron
archetech has quit [Quit: Konversation terminated!]
Myron has quit [Killed (NickServ (GHOST command used by guest6191!~Myron@94.6.204.7))]
archetech has joined #armbian
alekksander has joined #armbian
<FergusL>
My hopes for the C2 to come to life are low again: I misread my own card and I was indeed already using an ultra card. So I basically just wish an Ultra SDXC A1 will work and that the Ultra SDHC Class 10 is also not compatible (to be clear the device just doesn't boot at all, it recognizes the card in the slot 1 out of 10 boots and it's very
<FergusL>
unreliable, I'm already suspecting the slot itself is broken or damaged etc... but I sincerely hope it is not
<[TheBug]>
are you sure your writing image correctly?
<[TheBug]>
im not sure I can say I have had much issue with any specific sdcard in c2
<[TheBug]>
used mostly A1 card from adata, sandisk
<FergusL>
[TheBug] tested both cli and desktop imge, using etcher and dd on linux, checked with 3 different HDMI screens and cable and also using UART
<FergusL>
have reflashed the image about 4-5 times now I guess
<[TheBug]>
you have decompressed the file before writing?
Werner changed the topic of #armbian to: +++ Major services outtage. Working on it +++ armbian - Linux for ARM development boards | www.armbian.com | Github: github.com/armbian | Commits: #armbian-commits | Developer talk: #armbian-devel | Forum/Twitter feed: #armbian-rss | Logs: -> irc.armbian.com
Myron has joined #armbian
glsorre has joined #armbian
<Armbian-Discord>
<Myron> Does etcher automatically decompress before writing the image to the SD card?
<nekomancer[m]>
Armbian-Discord: it does
<Armbian-Discord>
<bret> It does on mine, I see a decompressing stage before it starts writing if it’s a compressed archive
<BCMM>
if using dd, you can use cmp to verify the write afterwards
<BCMM>
(might be useful if you're trying to narrow down which machine isn't playing nice with the SD card)
<FergusL>
UART is obviously the method that gave the best results because I could see what was happening, sometimes it just wouldn't even boot into uboot. the error I got most of the time was "Card did not respond to voltage select!"
<FergusL>
[TheBug] yes I have decompressed the xz file
<FergusL>
sometimes I do get past the initial step before ramdisk loading and (on the desktop image at least) it stops at "Starting kernel..."
<Myron>
I bet the failover was in that same bundle.
<[TheBug]>
"Fiber repair crews are working to run temporary cables around the damaged area to allow faster restoration of services. This involves working with three major backbone cables, and is being performed alongside several other carriers, as a total of over 30 fiber conduits and their associated fiber cables were destroyed. This will complicate and slow the work as we will need to maneuver around
<[TheBug]>
the other crews on site. We appreciate your continued patience as we work to restore services."
<[TheBug]>
note the "DESTROYED" part
<[TheBug]>
...
<[TheBug]>
30 fiber conduits at 3 cables per conduit = 90 cables.
<[TheBug]>
Most carrier cables have 144 fibers per cable. 90x144=12960
<[TheBug]>
That means there is 12960 fibers currently out of service.
<[TheBug]>
To get them all back, they will need to place new sections of fiber and fuse 25,920 splices.
<[TheBug]>
It takes 30-45 minutes to prep a cable end for splicing, and about 5 min per fiber to actually fuse it.
<[TheBug]>
That works out to 12.5 splicing hours per cable. So the fastest they would be able to splice a cable is about 6 hrs. with 2 guys working the splice points on each end. They are going to need to have 30 splicers out there to get this done in the next 24 hrs. and that doesn't include conduit replacement, fiber relocation, etc.
<[TheBug]>
The more you know* :)
<Armbian-Discord>
<ManoftheSea> $DIETY_IN_VAIN
<FergusL>
curious to know how big the effect is on other unrelated websites.
<FergusL>
?*
<FergusL>
[TheBug] I'm using the Armbian 22.0.1 current and edge images
<[TheBug]>
FergusL: well one thing I would suggest to really see where the issue is, is I would test using an older image -- sometimes newer images do have issues and it sounds more like you are hitting an issue there, you could probably find out some more by hooking USB TTL serial cable up to board and watching on boot, but I would guess something is breaking in kernel possibly. Trying with older
<[TheBug]>
image / kernel would help you to better know if it is an kernel issue or an sdcard issue as you are thinking.
<FergusL>
thanks for the suggestion, IgorPec suggested the same
<Myron>
Over 360 domains will be unreachable and that's only on one IPV4 subnet!
<ArmbianHelper>
^ List of all domain names in network 173.45.128.0/19
<FergusL>
I am using at USB TTL -- should it be different with the desktop images? I was wondering if the tty switched to the hdmi output for the desktop image
<FergusL>
any way to get an older image during the outage? :D
<FergusL>
you guys are really helping, thanks a lot!!
<Werner>
debug uart is always enabled by default across all images. what you can do is to increase verbosity of boot process to get further information about what happens after "starting kernel..."
<FergusL>
ok, thanks, then it's just that it isn't booting, although I do get the heartbeat blue blink
<Werner>
mount sd card to any linux computer, go to /boot and edit armbianEnv.txt and change verbosity to 7
<Myron>
... and if you've only got Windows systems then "Linux File Systems for Windows by Paragon Software" is a life saver.
aasami_ has quit [Quit: leaving]
archetech has quit [Quit: Leaving]
<stipa>
lol, fibre cut
<stipa>
i guess half of america is out down
<stipa>
yeah, forum is still down
<stipa>
that fix could last some time
<stipa>
[TheBug]: what's that, just one data centre down or?
<stipa>
or a backbone
<stipa>
or both
SallyAhaj has quit [Remote host closed the connection]
<Armbian-Discord>
<cats> I wonder if TheBug is of the Linux variety
<Armbian-Discord>
<cats> Ahhh they are
<Armbian-Discord>
<cats> :)
<Armbian-Discord>
<cats> stipa probably a full backbone. I used to work for the same host and they have redundancies in place, but not for a massive fiber cut like this is
<IgorPec>
some backup will be estabilished "within the next 2 hours."
<stipa>
cats dang, that's the reason why to keep idiots away from such stuff
<IgorPec>
we will build proper redundancy for website / forum when time permits
<stipa>
it's hard to predict something like that
<stipa>
the good thing is that 5G haters don't know about that stuff still
<stipa>
they would excavate all the backbones there are
<Myron>
You would think a DC would have two bundles of fiber cables coming out of the building from two points as opposed to one bundle. That way if one bundle of cables get destroyed, the other bundle survives and the DC stays connected and on-line.
<Armbian-Discord>
<RichN> 5G waist of radio band space
<Armbian-Discord>
<RichN> had bad for humans
SallyAhaj has joined #armbian
<stipa>
RichN , those fucks would burn themselves alive if someone told them that they're the reason for their own suffering
<stipa>
Myron: that sounds expensive
<Myron>
Yep. The better the disaster plan the more it will cost.
<stipa>
Myron: the downtime now could even cost more than the "two bundles of cables"
<stipa>
they're lucky, there's an excavator to blame
<Myron>
Yep
schwarz_Kat has joined #armbian
samythemany has quit [Remote host closed the connection]
samythemany has joined #armbian
schwarz_Kat has quit [Quit: Offline]
<Armbian-Discord>
<cats> I love the Chicago mirror
<Armbian-Discord>
<cats> Mostly because I run it and it's insanely fast lmao
<Armbian-Discord>
<cats> AT&T Fiber has decent routes to it so I can do 50MB/s easily
Myron has quit [Ping timeout: 256 seconds]
SallyAhaj has quit [Remote host closed the connection]
SallyAhaj has joined #armbian
<Manouchehri>
Is there any sort of signed/secure boot on the RK3399?
<nekomancer[m]>
two bundle of cables mean a system that will solve routing|switching. data or power. and this is point of failure. very expensive point of failure.
<nekomancer[m]>
you can't "just connect" two different power source, you know.
<ArmbianHelper>
^ Secure Boot on Arm systems – Building a complete Chain of Trust upon existing industry standards using open-source firmware - SFO17-201 - Linaro Resources Hub
<ArmbianHelper>
^ arm-trusted-firmware/rockchip.rst at master · ARM-software/arm-trusted-firmware · GitHub
<Armbian-Discord>
<ManoftheSea> But I don't know if that's a "we could do this" or a "we did do this"
<FergusL>
[TheBug] I just tested with the new known to be working model card and still as unreliable :/ I have flashed 2021.01, should I go further back?
<FergusL>
I'm really worried the board is damaged somewhere... erghhhhh
<FergusL>
Werner thanks, just done that and since then I haven't been able to reach the point where it loads the kernel... after about 5 time unplugging/replugging the micro usb power
<FergusL>
I'm also skeptical the serial FTDI adapter I got keeps a few signal highs and I don't get actuall complete cold reboot it I don't un*plug it too
<[TheBug]>
are you using the original adapter that came with the c2?
<FergusL>
[TheBug] it came with one but not sure it's the original one, micro USB with a switch on the cord
<[TheBug]>
um?
<[TheBug]>
hmm
<[TheBug]>
what is the adapter rated for
<[TheBug]>
look on it, 5v 2A?
<FergusL>
Yes 3A even I've tested with 2 other 3a phone chargers and also powering through the header
<[TheBug]>
well I always would advise using the barrel jack to power but it should technically work that way so I can't fault you on that.
<[TheBug]>
but underpower can also sometimes cause some fun
<[TheBug]>
why I asked
<stipa>
also, you could blame a shitty sd card
<[TheBug]>
well it more so sounds like the sdcard reader isn't powering up or initializing correct at boot time
<[TheBug]>
so it then can not find any dick to boot from
<[TheBug]>
disk*
<[TheBug]>
LOL
<[TheBug]>
man teach me to try and type on irc fast
<stipa>
khm
<stipa>
FergusL: forum is full of kinda useful resources
<[TheBug]>
yeah hoping it will be back up any minute
<[TheBug]>
eesh
<[TheBug]>
basically DC says any minute fiber should be back up
<[TheBug]>
story of my day
<stipa>
10 000 cables, idk
<stipa>
they probably don't know where to start still
<stipa>
+[TheBug] | Most carrier cables have 144 fibers per cable. 90x144=12960
<stipa>
oh my
<stipa>
maybe it's not just one cut
<[TheBug]>
what I provided is literally what the fiber provider sent, with some added floursih to demonstrate the complexity there
<stipa>
yeah, put some debugging on top of everything, it won't be the first time for sure
Myron has joined #armbian
<FergusL>
do you know what size the barrel is on the C2? It is so tiny I don't have anything so small lying around
<Armbian-Discord>
<Tonymac32> Censor [TheBug]
<Armbian-Discord>
<Tonymac32> 😆
Myron has quit [Ping timeout: 256 seconds]
samythemany has quit [Ping timeout: 240 seconds]
<Armbian-Discord>
<ajfriesen> Is your website down at the moment?
<steev>
seems like
<Armbian-Discord>
<IgorPec> yes. we are waiting for good news for a while now
alekksander has quit [Quit: Konversation terminated!]
<FergusL>
I ordered an emmc module for c2 from a local seller, I hope it will work better than with the sdcard
<IgorPec>
now discord doesn't want to accept my messages :)
Armbian-Discord has quit [Remote host closed the connection]
Armbian-Discord has joined #armbian
<nekomancer[m]>
discord spies to your processes and windowses
<IgorPec>
i only run terminals ;) lol
<IgorPec>
and browser which is spyware anyway
Armbian-Discord has quit [Remote host closed the connection]
Armbian-Discord has joined #armbian
Armbian-Discord has quit [Remote host closed the connection]
Armbian-Discord has joined #armbian
Armbian-Discord has quit [Remote host closed the connection]
Armbian-Discord has joined #armbian
Armbian-Discord has quit [Remote host closed the connection]
<nekomancer[m]>
discord steal window titles from browsers and terminals! awww!
Armbian-Discord has joined #armbian
Armbian-Discord has quit [Remote host closed the connection]
Armbian-Discord has joined #armbian
Armbian-Discord has quit [Read error: Connection reset by peer]
Armbian-Discord has joined #armbian
Armbian-Discord has quit [Remote host closed the connection]
Armbian-Discord has joined #armbian
<IgorPec>
yes, true
ajfriesen9 has joined #armbian
ajfriesen has quit [Ping timeout: 256 seconds]
ajfriesen9 is now known as ajfriesen
<Manouchehri>
IgorPec: unless you’re a fellow security researcher or are into cryptocurrency, you’re safe.
flyback has quit [Quit: Leaving]
flyback has joined #armbian
schwarz_Kat has joined #armbian
xoan7 has joined #armbian
<schwarz_Kat>
FergusL: "what's weird is that I reflashed the desktop version and it did boot into the kernel once (flashing blue heartbeat), but couldn't reproduce..."
<schwarz_Kat>
I had something similar on a slightly power hungry Pine64. The power supply was rated 5v3A and worked well on other boards (made in China). Wnen I eventually plugged in a USB tester, it showed only 4.7v. Rigged it to my bench PSU and it worked. Tried another 5v3A supply and it worked.
<schwarz_Kat>
Your issue sounds very familiar. I wasted days on something similar.
<schwarz_Kat>
Are you using a supported build that is known to work on that board? Bad u-boot device tree could cause that.
xoan has quit [Ping timeout: 252 seconds]
xoan7 is now known as xoan
<FergusL>
schwarz_Kat I've only used the official armbian builds from the website
<FergusL>
and upon suggesting here also tested a build from january 2021
<FergusL>
I should add that the board is second hand and was sold as "not working but it's just the sdcard" which I supposed was easy fix, but the problem might be deeper
<FergusL>
I'm quite familiar with sbcs. Yes the Pine64 is power hungry! got used to a big phone charger and powering through headers
<FergusL>
i'll try with the bench psu tomorrow, thanks schwarz_Kat !
<lanefu>
OMG it's that freaking bug in redirect that makes it suddenly forgot teh requestor thing