<JamesMunns[m]>
then you could look at integrating ekv or sequential-storage, using your second bank as a backing storage?
<dirbaio[m]>
stm32h743 has humongous 128kb flash sectors
<RockBoynton[m]>
yes, I have been, ekv seems to only store bytes, and sequential-storage is only async, so I will have to implement embedded-storage-async for the stm32h7xx-hal's flash peripheral
<RockBoynton[m]>
but I think sequential-storage can actually store structured rust data
<dirbaio[m]>
> ekv seems to only store bytes
<dirbaio[m]>
what do you mean?
<JamesMunns[m]>
you'd typically pick a serializing format (like postcard or json or something), and write that "as bytes" to the storage
<dirbaio[m]>
you can always serialize/deserialize to bytes with something like postcard yes
<JamesMunns[m]>
Rust doesn't have a stable layout, so you can't generally just memcpy structs to flash and back
<dirbaio[m]>
that's what sequential-storage is doing under the hood anyway
<JamesMunns[m]>
(or very carefully write it with repr(c) structs, or your own direct serialization formats)
<dirbaio[m]>
* sequential-storage is serializing your stuff under the hood anyway
<RockBoynton[m]>
with serializing though, if I ever want to change a value of the config, I will have to rewrite the entire thing, no?
<JamesMunns[m]>
with flash you generally need to erase the whole sector before writing anyway
<dirbaio[m]>
if you're serializing the entire struct yes. if you're storing each value as a separate key in the key/value storage then no, you just change that key.
<JamesMunns[m]>
flash is written by turning 1 bits into 0 bits. The only way to make them 1s again is by erasing the whole sector.
<RockBoynton[m]>
This is quite a bit of data we are talking about here, on the order of 500 KB, so it definitely spans more than a few sectors, even for the stm embedded flash
<JamesMunns[m]>
so what you'd like to do is have the 500KiB in place, and be able to change a few bytes in the middle, say from 1, 2, 3, 4 to 10, 11, 12, 13; without rewriting the whole thing?
<RockBoynton[m]>
yes, with the understanding that it may have to rewrite more than that due to the physics limitations of NOR flash, but I don't want the application code to care about that if possible
<JamesMunns[m]>
I don't think there are any libraries that make that Just Work today. and as dirbaio said, with 128K pages, you need to read that all into ram, change it, erase the page, then write it all back
<dirbaio[m]>
and if device powers off in the middle of it, data is gone :D
<JamesMunns[m]>
A different approach is to use ekv or s-s, and instead of one big 500K struct, have 100 or something much smaller fields
<JamesMunns[m]>
then you can overwite a single field in an append-structured log
<RockBoynton[m]>
with something like sequential-storage, I should be able to update a key's value and it will handle doing the minimal set of read/modify/write?
<RockBoynton[m]>
the data is essentially a 2d array if that helps. I'm still trying to figure out how to best represent that most efficiently due to the physical and ergonomic limitations of storing things in a nonvolatile manner
<JamesMunns[m]>
honestly, it sort of depends on your access patterns
<JamesMunns[m]>
like, how much you need to load at once, how much you can afford to read/write at once, etc.
<RockBoynton[m]>
I need to access the "table" basically as a for loop over the rows, then access some fields of the table to do some calculations
<RockBoynton[m]>
but I will access it all at once during the actual use of it in the code and won't modify it in code logic, but this table needs to be read/written to flash in a serial interface as well
<JamesMunns[m]>
Do you write it in the loop too? Or only occasionally, like writing params?
<RockBoynton[m]>
no, only reading in the loop
<RockBoynton[m]>
the only modification comes when we get a command over the serial interface to change something in the table (which should be rare)
<JamesMunns[m]>
could be easiest to just have an A/B side, and some kind of counter to decide which is the newest. This is like a very coarse version of what sequential storage does
<JamesMunns[m]>
so at boot, see which one has a valid CRC AND has the highest sequence number. pick that as primary, and the other one as secondary
<JamesMunns[m]>
when you get a command, erase the secondary, write it completely, use the next sequence number, finally write the crc
<JamesMunns[m]>
that way you always have 1 or 2 valid ones, and you pick which to use based on the highest sequence number
<JamesMunns[m]>
you can use a u32 sequence number and not worry about rollovers because your flash won't last 4 billion erases
<RockBoynton[m]>
is there a downside to just using sequential-storage? where the key is the row index and the value is the column? Or even the key is the (row, col) tuple and the value is the actual value?
<JamesMunns[m]>
Could also work!
<JamesMunns[m]>
depends on whether you can pay for paging in all the values you need in your loop
<RockBoynton[m]>
well, I can just read them all into a 2d array (what I actually want) in RAM at bootup, right? Then if it changes, I can change it both places
sroemer has joined #rust-embedded
k86td has joined #rust-embedded
k86td has quit [Remote host closed the connection]
Foxyloxy has quit [Read error: Connection reset by peer]