<re_irc>
<@huntc:matrix.org> : I guess there's an implied data integrity with COBS. And I guess it is good enough, right? Good questions though.
<re_irc>
<@huntc:matrix.org> +check
<re_irc>
<@adamgreig:matrix.org> : Cobs won't detect most errors in anything except the substitution bytes, and even then might not detect errors there, so you definitely also want a crc. One question is whether you put it inside or outside the cobs...
<re_irc>
<@firefrommoonlight:matrix.org> Example, for Ublox (GPS etc):
<re_irc>
/// See interface manual, section 3.4: UBX checksum
<re_irc>
/// excluding, the checksum fields (see the figure UBX frame structure).
<re_irc>
/// "The checksum is calculated over the message, starting and including the class field up until, but
<re_irc>
/// The checksum algorithm used is the 8-bit Fletcher algorithm, which is used in the TCP standard
<re_irc>
/// RFC 1145)."
<re_irc>
///
<re_irc>
/// This is a standalone fn due to needing a ref to the buffer in question.
<re_irc>
<@huntc:matrix.org> : I’d think inside the COBS given the transmission purpose it has…?
<re_irc>
<@huntc:matrix.org> : Do you use COBS with that?
<re_irc>
<@firefrommoonlight:matrix.org> No; not familiar
<re_irc>
<@boiethios:matrix.org> I have an array of "Option<u8>", and I'd like to optimize the memory: is there a way to tell to use "u8::MAX" as the None variant, just like 0 with "Option<NonZeroU8>"?
<re_irc>
<@boiethios:matrix.org> In any case, I can use "Option<NonZeroU8>" and then decrement the value by 1 when I need to use it
_whitelogger has joined #rust-embedded
<re_irc>
<@jamesmunns:beeper.com> : That was my opinion, but adam pointed out that by transforming the data after applying a CRC, you might lessen some of the protection characteristics
<re_irc>
<@jamesmunns:beeper.com> Like CRC coefficients are selected to provide particular resistance against certain burst errors, for example. If you transform the bits, then what would be "four bits in a row" is now spread out, and maybe even value flipped
<re_irc>
<@adamgreig:matrix.org> I think it ends up just depending on what protection you need and how you're storing or transmitting the messages, crcs usually lose effectiveness when they're not basically right at the bottom layer anyway
IlPalazzo-ojiisa has quit [Ping timeout: 248 seconds]
IlPalazzo-ojiisa has joined #rust-embedded
emerent has quit [Ping timeout: 252 seconds]
emerent has joined #rust-embedded
starblue has quit [Ping timeout: 256 seconds]
kenny has quit [Quit: WeeChat 3.8]
kenny has joined #rust-embedded
<re_irc>
<@shakencodes:matrix.org> : I think inside the COBS. Putting the CRC outside the COBS breaks the guarantee of having the 0x00 byte be the end of packet (as the CRC might also have a 0x00), which is the big advantage of using COBS.
<re_irc>
<@fullauto:matrix.org> Unboxing variables is fake and gay and you halted developement of chips for 10 years.
starblue has joined #rust-embedded
<re_irc>
<@jamesmunns:beeper.com> : Practically, yes, but you may be losing some (?) Effectiveness of your CRC. Will still be a better signal of corrupted data than no CRC at all, just maybe not the strength specified on the label :D
starblue has quit [Ping timeout: 250 seconds]
Vorpal has joined #rust-embedded
lehmrob has joined #rust-embedded
starblue has joined #rust-embedded
<re_irc>
<@hyahoos:matrix.org> Friends, for embedded rust, if we want to setup some lower level control like for example, we want to move data over to the I2c peripheral using DMA and then ask the DMA to generate an interrupt for the microprocessor when our external sensor responds, would we be better served using a hal such as the stm32f3xx_hal or should we instead juust use a peripheral access crate?
Vorpal has left #rust-embedded [Konversation terminated!]
<re_irc>
<@dirbaio:matrix.org> and it automatically uses DMA and interrupts internally
<re_irc>
<@dirbaio:matrix.org> so the core can run other tasks while it's waiting for DMA to finish, or sleep if there's nothing else to do
<re_irc>
<@dirbaio:matrix.org> same as if you wrote it manually
<re_irc>
<@hyahoos:matrix.org> Okay, im honest kind of at odds with whether to use embassy or not
<re_irc>
<@dngrs:matrix.org> automagic dma?! how cool is that
<re_irc>
<@hyahoos:matrix.org> cause i really dont like the async model
<re_irc>
<@hyahoos:matrix.org> because it keeps on polling the task doesnt it?
<re_irc>
<@hyahoos:matrix.org> We have a queue of tasks to perform and then we keep polling until the task becomes blocking, and after that we then move on to another task right?
<re_irc>
<@hyahoos:matrix.org> did i get that wrong?
<re_irc>
<@dirbaio:matrix.org> sort of
<re_irc>
<@dirbaio:matrix.org> it doesn't do busy-loop polling. The i2c driver calls a "waker" when an interrupt fires, to tell the executor that the task can run again
<re_irc>
<@dirbaio:matrix.org> until that fires, the executor knows never to poll the task
<re_irc>
<@dirbaio:matrix.org> * the irq
<re_irc>
<@hyahoos:matrix.org> alright id dint know that
<re_irc>
<@hyahoos:matrix.org> so it only polls when the waker is fired?
<re_irc>
<@hyahoos:matrix.org> and the waker is triggered via an interrupt once the i2c peripheral sees data has been communicated back?
<re_irc>
<@dirbaio:matrix.org> yes!
<re_irc>
<@hyahoos:matrix.org> okay so I hope you can clarify some stuff for me, what I dont understand is why does the async model need to poll?
<re_irc>
<@dirbaio:matrix.org> async's "Future::poll" is not really "busy-loop polling", it's more like "do work"
<re_irc>
<@hyahoos:matrix.org> Like I dont really understand. Typically right if we want to send some data over we would use DMA to move our chunk of data to the I2c peripheral transmit and then ask the DMA to interrupt the microprocessor once data has been written back, and this interrupt is what we use to service the data that has come back right?
<re_irc>
<@hyahoos:matrix.org> when we do stuff with embassy-32 what is the difference with the async model?
<re_irc>
<@hyahoos:matrix.org> sorry i emant the embassy-stm32
<re_irc>
<@dirbaio:matrix.org> not sure what you mean
<re_irc>
<@dirbaio:matrix.org> embassy-stm32 implements async drivers for stm32
<re_irc>
<@dirbaio:matrix.org> using rust's async model
<re_irc>
<@dirbaio:matrix.org> same as every other rust async crate
<re_irc>
<@hyahoos:matrix.org> alright, within the async model, so when you say busy loop polling its more like do work correct?
<re_irc>
<@hyahoos:matrix.org> So suppose I am transferring data A to some off chip sensor and I am expecting data B from that off chip sensor right. If we use embassy, we would do something like have the microprocessor "do work" by transferring data A to the I2c peripheral, and then once that is done, we see that the task is blocking and so the task will sleep, and we do another task
lehmrob has quit [Ping timeout: 265 seconds]
<re_irc>
<@dirbaio:matrix.org> the way it works is:
<re_irc>
if it returns Poll::Pending, the task has decided it has no more work to do now and wants to wait for some interrupt. The executor doesn't poll it again.
<re_irc>
the executor calls "poll" on the task only once.
<re_irc>
If it returns Poll::Ready, the task has finished running (returned from its top-level "async fn"), and gets removed.
<re_irc>
<@hyahoos:matrix.org> oh ok
<re_irc>
<@dirbaio:matrix.org> on "poll", the executor gives a waker to the task. The task saves it somewhere.
<re_irc>
<@dirbaio:matrix.org> when the interrupt fires, "waker.wake()" is called.
<re_irc>
<@dirbaio:matrix.org> * the interrupt handler calls "waker.wake()".
<re_irc>
<@hyahoos:matrix.org> alright I thought that the executor keeps on polling the task agian and again and again all the time
<re_irc>
<@hyahoos:matrix.org> nevermind
<re_irc>
<@dirbaio:matrix.org> this notifies the executor the task can be polled again
<re_irc>
<@hyahoos:matrix.org> thats why I thought it was being wasteful
<re_irc>
<@dirbaio:matrix.org> so it polls it again, only once
<re_irc>
<@dirbaio:matrix.org> the task returns Pending/Ready again, rinse and repeat
<re_irc>
<@hyahoos:matrix.org> alright
<re_irc>
<@hyahoos:matrix.org> no overhead right?
<re_irc>
<@hyahoos:matrix.org> as if there is no overhead doing things like this compared to if we program the DMA itnerrupts and whatever manually right?
<re_irc>
<@dirbaio:matrix.org> so in the end it behaves similarly as if you manually wrote interrupt-driven state machines. the core sleeps or does other stuff while waiting
<re_irc>
<@hyahoos:matrix.org> yeah if it really HAS NO TASK TO DO then it sleeps, but as long as we keep it busy with computations it will keep on doing those right?
<re_irc>
<@dirbaio:matrix.org> yes
<re_irc>
<@hyahoos:matrix.org> : I see, that is a relief to hear/
<re_irc>
<@hyahoos:matrix.org> it sounds awesome tbh
<re_irc>
<@hyahoos:matrix.org> Was it you who recommended that I debug with just print messages instead of using gdb integration?
<re_irc>
<@dirbaio:matrix.org> there _is_ some overhead:
<re_irc>
with async, the interrupt handler calls the waker, then the executor has to run and poll the task again, which then does "the next thing"
<re_irc>
with manually written state machines, you'd do "the next thing" right in the interrupt handler usually.
<re_irc>
<@hyahoos:matrix.org> : yeah whatever, it doesnt seem that significant anyways
<re_irc>
<@dirbaio:matrix.org> "call waker, then the executor polls the task" is ~110 cycles in "embassy-executor"
<re_irc>
<@hyahoos:matrix.org> it sounds good
<re_irc>
<@hyahoos:matrix.org> thats still okay
<re_irc>
<@hyahoos:matrix.org> vs the powerful abstraction capabilities it gets us
<re_irc>
<@hyahoos:matrix.org> very fair price to pay
<re_irc>
<@dirbaio:matrix.org> not zero, but the abstraction is worth it for most use cases
<re_irc>
<@dirbaio:matrix.org> yep
<re_irc>
<@hyahoos:matrix.org> agreed
<re_irc>
<@dirbaio:matrix.org> because writing state machines by hand is very annoying :(
<re_irc>
<@hyahoos:matrix.org> : I dont understand you
<re_irc>
<@hyahoos:matrix.org> what do you mean state machine?
<re_irc>
<@hyahoos:matrix.org> How can we consider our microcontroller as a state machine?
<re_irc>
<@dirbaio:matrix.org> when writing code with manual interrupt handlers, you have to convert your code to state machines
<re_irc>
<@dirbaio:matrix.org> store the "state" of your work in some variables, when the interrupt fires read that to figure out what were you waiting for and what the next action is, do it, update the state, return from the interrupt handler
<re_irc>
<@hyahoos:matrix.org> yepp
<re_irc>
<@hyahoos:matrix.org> ah okay
<re_irc>
<@hyahoos:matrix.org> wait do you mean something like we have to figure out what the microcontroller has to do afterwards?
<re_irc>
<@dirbaio:matrix.org> that's what I mean with a "manually written state machine"
<re_irc>
<@hyahoos:matrix.org> like oh after this interrupt fires but before that other interrupt, what shall this machine do?
<re_irc>
<@dirbaio:matrix.org> with async, Rust transforms your code into a state machine for you
<re_irc>
<@dirbaio:matrix.org> so you write code that looks like blocking code, except with added ".await"s
<re_irc>
<@hyahoos:matrix.org> alright
<re_irc>
<@hyahoos:matrix.org> I am still kinda confused about the state machine though
<re_irc>
<@hyahoos:matrix.org> but I think that the interrupt thing for embassy works great
<re_irc>
<@hyahoos:matrix.org> Wait just to be sure it is also relatively trivial to setup an interrupt for our main control loop right?
<re_irc>
<@hyahoos:matrix.org> like say I wanna setup an interrupt that fires 2khz to start my main control loop, with embassy it is relatively trivial right?
<re_irc>
<@dirbaio:matrix.org> blocking:
<re_irc>
do_foo();
<re_irc>
println!("done!");
<re_irc>
do_baz();
<re_irc>
do_bar();
<re_irc>
<@dirbaio:matrix.org> manually-written state machine:
<re_irc>
Foo,
<re_irc>
enum State {
<re_irc>
Bar,
<re_irc>
Baz,
<re_irc>
Done,
<re_irc>
}
<re_irc>
static mut STATE: State;
<re_irc>
fn main() {
<re_irc>
STATE = Foo;
<re_irc>
start_do_foo();
<re_irc>
}
<re_irc>
#[interrupt]
<re_irc>
fn MY_INTERRUPT {
<re_irc>
match STATE {
<re_irc>
Foo => {
<re_irc>
start_do_bar();
<re_irc>
STATE = Bar;
<re_irc>
}
<re_irc>
Bar => {
<re_irc>
start_do_baz();
<re_irc>
STATE = Baz;
<re_irc>
}
<re_irc>
Baz => {
<re_irc>
println!("done!");
<re_irc>
STATE = Done;
<re_irc>
}
<re_irc>
}
<re_irc>
}
<re_irc>
<@dirbaio:matrix.org> async
<re_irc>
do_foo().await;
<re_irc>
do_bar().await;
<re_irc>
do_baz().await;
<re_irc>
println!("done!");
<re_irc>
<@shakencodes:matrix.org> : If wrapping data+CRC inside COBS diminishes CRC effectiveness unacceptably, why would you use COBS at all? If the CRC is appended to COBS data, you can no longer use COBS (alone) for framing. You have to have some other code to differentiate whether a 0x00 in the CRC is part of the CRC or part of another frame. (Maybe this is simpler than framing non-COBS encoded data. But I don't love it.)
<re_irc>
I'll need to see if I can find some reading to help understand how much integrity protection is lost by wrapping data+CRC in COBS. My intuition is that this is not going to be too bad, and that there is still very little chance of mis-accepting bad data. (My actual intuition is that COBS wrapping the data+CRC is no different than the bare data+CRC... but I respect the expertise of the folks here and want to learn more!)
<re_irc>
<@dirbaio:matrix.org> async code reads like blocking code, but executes interrupt-drive, like manually-written state machines
<re_irc>
<@hyahoos:matrix.org> oh ok I think i get it
<re_irc>
<@hyahoos:matrix.org> wait when u write #interrupt
<re_irc>
<@hyahoos:matrix.org> that is an interrupt generated by a timer or by a hardware event?
<re_irc>
<@hyahoos:matrix.org> or does it not matter?
<re_irc>
<@hyahoos:matrix.org> so if we are writing I2c, then #interrupt would be telling us that we are receiving data back from our peripheral and hence the match state block would be what we would do in the case we have received information back from the peripheral?
<re_irc>
<@dirbaio:matrix.org> it was just a dummy example
<re_irc>
<@shakencodes:matrix.org> Am I correct that the only way to do what I am suggesting (encode to Postcard, add CRC, COBS encode) is a two pass process? Postcard + CRC in one step, COBS encode in another step. (I.E. Postcard's mixing flavors doesn't support this combination of flavors.)
<re_irc>
<@hyahoos:matrix.org> yeah
<re_irc>
<@dirbaio:matrix.org> but yeah the interrupt handlers would be "i2c received data" or "dma transfer complete"
<re_irc>
<@hyahoos:matrix.org> ah I see
<re_irc>
<@dirbaio:matrix.org> so for example on DMA transfer complete, you'd go process the received data, then maybe fire another transfer, then update the state and return
<re_irc>
<@dirbaio:matrix.org> etc
<re_irc>
<@dirbaio:matrix.org> hyahoos: use embassy_time::{Duration, Ticker};
<re_irc>
async fn ticker_example_1() {
<re_irc>
let mut ticker = Ticker::every(Duration::from_millis(1));
<re_irc>
loop {
<re_irc>
#[embassy_executor::task]
<re_irc>
run_control_loop();
<re_irc>
ticker.next().await;
<re_irc>
}
<re_irc>
}
<re_irc>
<@hyahoos:matrix.org> : wait im kinda confused with this example, when we do something like this, do_bar will not execute until d_foo finishes right?
<re_irc>
<@dirbaio:matrix.org> yep
<re_irc>
<@dirbaio:matrix.org> it's "do foo, then do bar, then do baz"
<re_irc>
<@dirbaio:matrix.org> if you want them to run concurrently instead, you have to put them in separate tasks
<re_irc>
<@hyahoos:matrix.org> but then we can declare in a separate block other stuff that is supposed to run concurrently to the task we have right?
<re_irc>
<@dirbaio:matrix.org> separate task, yep
<re_irc>
<@hyahoos:matrix.org> ah okay
<re_irc>
<@hyahoos:matrix.org> I see
<re_irc>
<@hyahoos:matrix.org> : How do you debug your code?
<re_irc>
<@hyahoos:matrix.org> do you use gdb?
<re_irc>
<@dirbaio:matrix.org> async tasks are sort of like threads
<re_irc>
<@hyahoos:matrix.org> do u just print messages using RTT?
<re_irc>
<@hyahoos:matrix.org> what do u do?
<re_irc>
<@hyahoos:matrix.org> : I see
<re_irc>
<@dirbaio:matrix.org> printing stuff with defmt, yes...
<re_irc>
<@dirbaio:matrix.org> i've never had luck with interactive debuggers
<re_irc>
<@hyahoos:matrix.org> Well now that I thought about it, async tasks are kinda like RTIC istn it?
<re_irc>
<@hyahoos:matrix.org> : I see, but no I meant like stepping through ur code using gdb cli
<re_irc>
<@dirbaio:matrix.org> RTIC v1 tasks are more like interrupt handlers, you still need to manually write state machines with these
<re_irc>
<@hyahoos:matrix.org> oh ok
<re_irc>
<@hyahoos:matrix.org> wait
<re_irc>
<@dirbaio:matrix.org> RTIC v2 adds support for async tasks
<re_irc>
<@hyahoos:matrix.org> RTIC is just an interrupt handler?
<re_irc>
<@hyahoos:matrix.org> oh ok so we still need to write our state machines with RTIC
<re_irc>
<@hyahoos:matrix.org> so that's it then you just debug by printing messages using RTT?
<re_irc>
<@dirbaio:matrix.org> yes :D
<re_irc>
<@hyahoos:matrix.org> alright, I love that philosophy
<re_irc>
<@hyahoos:matrix.org> Im probably gnna do that too then
<re_irc>
<@per.lindgren:matrix.org> With RTIC 2 you can both interrupt driven tasks and async.
<re_irc>
<@hyahoos:matrix.org> : oh okay thanks
<re_irc>
<@dirbaio:matrix.org> also, the nrf's bluetooth stack explodes if you pause the core with breakpoints...
<re_irc>
<@hyahoos:matrix.org> lmao
<re_irc>
<@dirbaio:matrix.org> because it's timing sensitive
<re_irc>
<@dirbaio:matrix.org> * the radio stuff is
<re_irc>
<@hyahoos:matrix.org> I also debug code by just printing stuff, but the issue with embedded systems was that I was scared I cannot communicate the printing errors back to me, so thats why I invested in gdb
<re_irc>
<@dirbaio:matrix.org> with RTT/defmt the printing goes over SWD, so if you can flash it you can see the error messages
<re_irc>
<@hyahoos:matrix.org> because i was scared that how am i supposed to have the code print to me if its on another machine it just sounded like a recipe for disaster
IlPalazzo-ojiisa has quit [Remote host closed the connection]
<re_irc>
<@dirbaio:matrix.org> it pretty much never fails
<re_irc>
<@hyahoos:matrix.org> okay
<re_irc>
<@hyahoos:matrix.org> printing NEVER FAILS RIGHT?
<re_irc>
<@dngrs:matrix.org> you can also send the data over e.g. usb uart
<re_irc>
<@hyahoos:matrix.org> like ONE OF THE MOST RHOBUST DEBUGGING METHODS is just LITERALLY PRINTING
<re_irc>
<@hyahoos:matrix.org> right?
<re_irc>
<@dngrs:matrix.org> and if you're a madman like me you receive it in a browser window and print it to the debug console
<re_irc>
<@hyahoos:matrix.org> : nonono, i dont want that
<re_irc>
<@hyahoos:matrix.org> I want it to print over SWD
<re_irc>
<@hyahoos:matrix.org> no extra setup
<re_irc>
<@dirbaio:matrix.org> logging over uart/usb is another story, if the firmware crashes USB stops working, so you don't see the error message when you need it the most
<re_irc>
<@dngrs:matrix.org> just don't crash™
<re_irc>
<@hyahoos:matrix.org> hahahahahahaha
<re_irc>
<@hyahoos:matrix.org> wait guys
<re_irc>
<@hyahoos:matrix.org> when do we ever prefer floating points over fixed point numbers?
<re_irc>
<@dngrs:matrix.org> when you have an FPU :V
<re_irc>
<@dngrs:matrix.org> also, as far as I understand it floats have a better, uh, dynamic range so to say
<re_irc>
<@hyahoos:matrix.org> yeah
<re_irc>
<@hyahoos:matrix.org> so are FPU's slower than our Algebraic logic unit on our main microprocessor?
<re_irc>
<@dngrs:matrix.org> and ... Rust has builtin floats so there's less typing (as in keyboard) and typing (as in type system) overhead
<re_irc>
<@hyahoos:matrix.org> I imagine it would be slower since we are not using 2's compleemnt right?
<re_irc>
<@dngrs:matrix.org> FPU ops will still be slowar than integer ops yeah
<re_irc>
<@huntc:matrix.org> : I guess your serial reader could read the cobs encoded message and then look for another 4 bytes for, say, a 32 bit crc…
<re_irc>
<@dngrs:matrix.org> I think around a factor of 10 usually?
<re_irc>
<@hyahoos:matrix.org> alright
<re_irc>
<@hyahoos:matrix.org> thanks
<re_irc>
<@hyahoos:matrix.org> so fixed points are faster, but then we need to figure out the range of values we can even tolerate with our fixed point numbers right?
<re_irc>
<@hyahoos:matrix.org> and its around faster by a factor of 10 right?
<re_irc>
<@hyahoos:matrix.org> give or take
<re_irc>
<@dngrs:matrix.org> don't quote me on that factor
<re_irc>
<@hyahoos:matrix.org> hahahaha okay
<re_irc>
<@dngrs:matrix.org> it's off the top of my head... might be more
<re_irc>
<@hyahoos:matrix.org> alright sure
<re_irc>
<@hyahoos:matrix.org> wait
<re_irc>
<@hyahoos:matrix.org> you cannot multiply 2 fixed point numbers where the "decimal point is different" right?
<re_irc>
<@dngrs:matrix.org> it can't be different
<re_irc>
<@hyahoos:matrix.org> we got to left or right shift it first to align the decimal point before we can perform the fixed point operation right?
<re_irc>
<@dngrs:matrix.org> it's ... fixed :D
<re_irc>
<@hyahoos:matrix.org> Wait
<re_irc>
<@hyahoos:matrix.org> i thought shifting decimal points is as trivial as asking the hardware to do bit shifting?
<re_irc>
<@hyahoos:matrix.org> either left shift it or right shift it?
<re_irc>
<@dngrs:matrix.org> but yeah, if you have different types, like "Fixed<8,24>" and "Fixed<16,16>" you'd need to convert
<re_irc>
<@hyahoos:matrix.org> yeah that conversion
<re_irc>
<@hyahoos:matrix.org> is it trivial like left shifting or is it hard to do in hardware?
<re_irc>
<@jamesmunns:beeper.com> I'll follow up more later about CRC, line is busy now :)
<re_irc>
<@huntc:matrix.org> : You should be able to compose using the lower level flavor api ie pretty much what the convenience functions do.
<re_irc>
<@dngrs:matrix.org> a shift is going to be binary, but AIUI fixed point math is decimal
<re_irc>
<@hyahoos:matrix.org> ah
<re_irc>
<@dngrs:matrix.org> but a mul or div by a power 10 isn't gonna end the world
<re_irc>
<@hyahoos:matrix.org> wait so the number 3.75
<re_irc>
<@hyahoos:matrix.org> is gnna be smthng like 11.11
<re_irc>
<@hyahoos:matrix.org> so if we want to decrease the number accuracy to smthng like 3.5 then we cannot just left shift it
<re_irc>
<@hyahoos:matrix.org> because left shifting yields 111.1 which isnt waht we want
<re_irc>
<@hyahoos:matrix.org> we would somehow do smthng to get 11.1
<re_irc>
<@hyahoos:matrix.org> wait no
<re_irc>
<@hyahoos:matrix.org> left shifting would still work right?
<re_irc>
<@hyahoos:matrix.org> I guess my question is this:
<re_irc>
Suppose we have a complicated mathematical operation that we are performing and we can guarantee each part of the operations ranges and relative size.
<re_irc>
<@grantm11235:matrix.org> I think you would need to keep the integer part the same, and shift the fractional part
<re_irc>
<@hyahoos:matrix.org> But the parts are of different sizes, so would it still be faster for us to use fizxed point numbers and convert them to reconcile the decimal points or would it just make sense to use floating point unit on the machine?
<re_irc>
<@jamesmunns:beeper.com> : It's like 1-3 cycles for most float ops on cortex m iirc if you have an FPU btw
<re_irc>
<@dngrs:matrix.org> hyahoos: so for some context, I just checked the docs for fixed (https://docs.rs/fixed/latest/fixed/) and they say they're using binary, not decimal, math. Might wanna check out the source to be sure really
<re_irc>
<@dirbaio:matrix.org> but it's part of the core, and when enabled it runs on the same clock as the core
<re_irc>
<@hyahoos:matrix.org> oh ok
Foxyloxy has quit [Ping timeout: 255 seconds]
<re_irc>
<@hyahoos:matrix.org> so in that case we wouldnt even notice a performance overhead using floating point vs fixed point do we?
<re_irc>
<@hyahoos:matrix.org> since FPUs work very fast anyways
<re_irc>
<@jamesmunns:beeper.com> hyahoos: The answer is "it depends", but like all optimizations, you should have benchmarks first before assuming
<re_irc>
<@hyahoos:matrix.org> yeah
<re_irc>
<@hyahoos:matrix.org> that is fair
<re_irc>
<@jamesmunns:beeper.com> Sometimes it does really help. Sometimes all the complexity for shuffling from/to fixed point types ends up being slower
<re_irc>
<@hyahoos:matrix.org> no but as in floating points are not as bad as like 10x slower than fixed point operations
<re_irc>
<@jamesmunns:beeper.com> Yes, but only for divides :)
<re_irc>
<@hyahoos:matrix.org> :) fair
<re_irc>
<@hyahoos:matrix.org> fair
<re_irc>
<@jamesmunns:beeper.com> I mean I'm doing some fixed point math stuff right now, and it is faster, but certainly a bit more math and requires some care. But the big reason is because it has to work on devices with and without a hardware floating point
<re_irc>
<@jamesmunns:beeper.com> Software floating point will be like hundreds of times slower to emulate in software
<re_irc>
<@hyahoos:matrix.org> alright
<re_irc>
<@jamesmunns:beeper.com> Try it with floats if that's easier, it'll be easier to iterate on like that anyway. If it's too slow, then write good tests and benchmarks and give fixed point alternatives a try :)
<re_irc>
<@hyahoos:matrix.org> but the key here is that you needed code to work on devices with and without hardware floating point units right?
<re_irc>
<@hyahoos:matrix.org> but if we had a floating point unit onboard, then floating point operations are nearly as fast as our fixed point operations, though the division is substantially slower?
<re_irc>
<@jamesmunns:beeper.com> hyahoos: This seems generally correct!
<re_irc>
<@huntc:matrix.org> Fixed point is also important when trying to preserve accuracy, having accepted accuracy loss in the first place in serving such values. Things like metering values for example.
<re_irc>
<@huntc:matrix.org> * determining
<re_irc>
<@hyahoos:matrix.org> Alright thanks
<re_irc>
<@huntc:matrix.org> : I’m struggling to understand why composing a crc within a cobs would diminish the crc’s effectiveness. None of the input to a crc is changed - we simply append the crc to the output.
<re_irc>
<@jamesmunns:beeper.com> CRC polynomials are chosen based on certain characteristics, for example catching ALL N bit errors of some kind
<re_irc>
<@jamesmunns:beeper.com> This gives them a STRONGER than 1/(2^n) bit collision probability
<re_irc>
<@jamesmunns:beeper.com> But those polynomials are chosen assuming things like "data bits are stored sequentially", which is not the case if you've "scrambled" the data bits with cobs framing or other techniques like bit stuffing!
<re_irc>
<@jamesmunns:beeper.com> I _think_ you can't make it worse than the 1/(2^n) random collision chance, but you lose the "better than statistically expected" characteristics crcs with specific polynomials are chosen for
<re_irc>
<@jamesmunns:beeper.com> Disclaimer, I am not an expert on this, and have mostly learned from citing Koopman and other papers on this topic.
<re_irc>
<@huntc:matrix.org> But CRCs have nothing to do with governing the transmission itself, only verifying that was transmitted was correct… right?
<re_irc>
<@huntc:matrix.org> +what
<re_irc>
<@jamesmunns:beeper.com> The CRC polynomials do assume you have a linear stream of bits
<re_irc>
<@jamesmunns:beeper.com> So any mutations you make to that for sending on the wire need to be compensated for, IF you expect the calculated guarantees for that polynomial to hold
<re_irc>
<@huntc:matrix.org> Right, but if you’re calculating the crc before you stream via cobs… then all is well…
<re_irc>
<@jamesmunns:beeper.com> Nope!
<re_irc>
<@jamesmunns:beeper.com> When you mutate a value, the "data bits" are no longer sequential in the stream
<re_irc>
<@jamesmunns:beeper.com> Which means that a "burst error", say 5 sequential bits, are flipped
<re_irc>
<@jamesmunns:beeper.com> This will PROBABLY cause the decode to fail, because it will mess with both the framing and CRC "layers"
<re_irc>
<@huntc:matrix.org> But we don’t mutate the value - we just derive a crc from the value and append it
<re_irc>
<@jamesmunns:beeper.com> CRC talks about data stream "on the wire". You calculate it BEFORE transforming it to the cobs encoded data stream that is actually on the wire
<re_irc>
<@jamesmunns:beeper.com> It assumes that if you have a CRC32 with a guaranteed hamming distance of 6, ANY sequential 6 bit errors will be detected as an error, not just (1 - (1/(2^32)) of them
<re_irc>
<@jamesmunns:beeper.com> But you've shuffled around your data bits (a bit) from when you CALCULATED the CRC (unencoded), and when they actually go out over the wire.
<re_irc>
<@jamesmunns:beeper.com> PRACTICALLY you are probably still fine, as long as "not detecting" an error in 1/4 billion cases is statistically okay for you
<re_irc>
<@jamesmunns:beeper.com> You've "weakened" the crc's guarantees, but perhaps not practically enough to matter.
<re_irc>
<@huntc:matrix.org> I think I’ll put a postcard example together to understand more. I’m still not getting this. Cobs doesn’t care about what the bytes represent- they are just bytes. For decoding, cobs does it’s thing, you end up with bytes. You then validate using a crc as the last n bytes. I’m obviously missing something.
<re_irc>
<@huntc:matrix.org> Thanks for trying to explain it to me though.
<re_irc>
<@jamesmunns:beeper.com> (number 7: "Ignoring interaction with bit encoding")
<re_irc>
<@shakencodes:matrix.org> : This is a great link, James. (We used his online Embedded Systems course as a "level up" for some junior engineers a while back. Good stuff.)
<re_irc>
<@huntc:matrix.org> Thanks. I don’t quite understand how this relates to number 7. We aren’t ignoring the interaction with bit encoding. I don’t see that there is _any interaction_ with bit encoding. I’ll read more.
<re_irc>
<@jamesmunns:beeper.com> lemme write it out
<re_irc>
<@shakencodes:matrix.org> You don't often read the phrase "on my CRC and Checksum Blog" ... http://checksumcrc.blogspot.com/
<re_irc>
<@jamesmunns:beeper.com> Okay :
<re_irc>
0x00 0x12 0x00 0x34
<re_irc>
Let's say we're sending this data on the wire:
<re_irc>
We append that to the message:
<re_irc>
0x00 0x12 0x00 0x34 0x00 0x56
<re_irc>
Let's pretend the CRC16 we calculate for this is "0x00 0x56"
<re_irc>
<@jamesmunns:beeper.com> all the "^"s are places where we changed the "data bit stream" from the "time of calculation" to "what was sent on the wire
<re_irc>
<@jamesmunns:beeper.com> Because the polynomial is "doing math" assuming that values will be adjacent, any changes we make to the bitstream is not "considered" by the CRC
<re_irc>
<@jamesmunns:beeper.com> We could end up weakening our hamming distance from a theoretical value of 5 (any 5 sequential flips) to something like 2 or 3 (totally made up numbers), once you have considered the change. This is what happened with CAN-FD, where after bit stuffing (a much more invasive encoding method than COBS), the theoretical HD=6 was reduced to HD=2
<re_irc>
<@jamesmunns:beeper.com> I have _no idea_ practically how much COBS encoding could affect CRC hamming distances! but it theoretically could, and I am unlikely to write a thesis on it. And in the WORST case you still have a (1.0 - (2^(-N))) probability of detecting errors, where N is the length of the CRC32 (effectively HD=1)
<re_irc>
<@jamesmunns:beeper.com> so this is all WAY too much analysis for cases where you weren't selecting a CRC based on it's actual hamming distance guarantees on the wire, and just want a "is it probably not corrupted?" check for basic comms
<re_irc>
<@jamesmunns:beeper.com> it WILL still do that, even when cobs encoded, or bit stuffed! Just _potentially_ not at the "fully rated hamming distance strength", if that makes sense.
<re_irc>
<@huntc:matrix.org> This now makes sense. Thanks for your patience in explaining it. Given an effective HD of 1 then, we're probably ok, but this deserves a further look.
<re_irc>
<@huntc:matrix.org> Fixed len records FTW ;-)
<re_irc>
<@jamesmunns:beeper.com> tbf: some of this was also me "synthesizing" the knowledge that has been sharing "at" me for the last weeks while we've been discussing this in the anachro room :D
<re_irc>
<@jamesmunns:beeper.com> "message with crc all wrapped in cobs" is probably a _very reasonable and convenient_ choice for things you are throwing together and generally expect to work. Premium duct tape, if you will. Probably not good enough for like... safety critical purposes, like CAN-FD might be.
<re_irc>
<@jamesmunns:beeper.com> : if you trust your medium. Otherwise you might be stuck accumulating the next 2^16 or 2^32 bits due to a corruption of the length header :)
<re_irc>
<@jamesmunns:beeper.com> IF you have something like say, TLS, then yeah! don't use cobs! just do "[ len_u32 ] [ message bytes ]", it'll be fine!
<re_irc>
<@jamesmunns:beeper.com> but postcard is more "serial ports and actual wires" than "encrypt + MAC" transport :)
<re_irc>
<@jamesmunns:beeper.com> Anyway. It'd be cool if someone wrote a better "framed and integrity checked" wire encoding than "message with crc all wrapped in cobs", that supports variable length payloads. If anyone here would like to sponsor me to write it, I would love to come up with one. If someone else writes one, I would love to merge it as a postcard flavor. Until then, I'll probably use the premium duct tape model for personal projects until...
<re_irc>
... I do something more critical :D
<re_irc>
<@jamesmunns:beeper.com> And thanks for coming to my TED talk. I should probably go to bed now :D