12:35 <sterni> dnkl: have you ever investigated the combination of fish, foot and grapheme clusters?

12:35 <sterni> the combination breaks something in a very annoying fashion

12:36 <sterni> which is most likely due to our refusal to implement grapheme clusters in an incorrect way, as it is working properly in e. g. kitty

12:36 <sterni> I finally have a bit to motivation to check exactly what the problem is

12:37 <sterni> but I am fairly sure that fish assumes grapheme clusters only take up a certain size (one cell?!), but for an emoji + presentation selector it is two cells in foot

12:38 <sterni> stuff like this convinces me bit by bit that we really need to move past terminal emulators in the future

12:40 <dnkl> sterni: no, I haven't. But suspects it comes down to a disagreement of what the width of the cluster is

12:41 <dnkl> I've been considering picking up the grapheme clustering branch and adding an option to enable/disable clustering

12:41 <dnkl> Still not sure how to handle width though

12:43 <dnkl> Parse Unicode data files ourselves and provide our own implementation is wcswidth

12:43 <dnkl> Might be the best option

12:44 <dnkl> I think that's what fish does

14:38 <sterni> I'll check it out

14:38 <sterni> a thing that may already help is skipping the presentation selector altogether

14:39 <sterni> or add an option for it?

14:39 <sterni> this is such an annoying topic there is really no way forwards nor backwards

14:39 <dnkl> sterni: if we just skip it, we're just as likely to emit a one-column glyph when fish expects two...

14:40 <sterni> maybe

14:40 <dnkl> Yeah, I think a single option to enable best-effort grapheme clustering is the way to go

14:40 <sterni> I guess good idea is checking out the fish source code later

14:41 <sterni> hopefully I don't put it off again

14:42 <dnkl> I think we can get a lot of combinations to work. But presentation selectors, at least, will require plumbing all the way down to fcft

14:51 <sterni> otoh we could just figure out what fish does differently than say GNU readline and try to patch it

15:03 <dnkl> sterni: 'it' being foot or readline? :)

15:04 <dnkl> the way I see it, no matter what we do, we'll break something. Whatever we do in foot to "fix" fish will break bash. And/or zsh. Or something else.

15:05 <dnkl> that's why I think an option to enable/disable grapheme clustering is the only way. I also think doing this the "right" way, instead of trying to monkey patch foot is the right thing to do.

15:06 <dnkl> we could even add a custom escape to switch grapheme clustering on/off, to be able to support both "old" and "new" applications in the same foot instance

15:18 <dnkl> sterni: but, considering foot hasn't supported grapheme shaping at all, it would be perfectly ok to add things incrementally. No need to do *everything* correct from the beginning

15:49 <sterni> dnkl: no fish

15:50 <sterni> dnkl: I mean our initial endavour has convinced me that implementing this in a correct way is not feasible pretty much

15:51 <sterni> it seems there is some kind of consensus among ncurses based stuff what kind of incorrect leapfrogging to do to get emojis to work to an extent

15:52 <sterni> but my current frame of mind is more like, eh just switch to some GUI editor and don't use fish anymore

16:00 <dnkl> sterni: my guess is doing basic grapheme clustering (instead of printing the emoji to one cell, and the selector/skin modifier/whatever to the next cell), and then using max-wcwidth-of-all-codepoints will probably take us pretty fast.

16:00 <dnkl> But I don't see how we can get around grapheme clustering

16:01 <dnkl> If more applications start handling grapheme clusters then foot needs to change too. Having broken grapheme cluster handling is only fine as long as all applications are broken in the same way...

16:03 <dnkl> So, while I agree it's more or less impossible to do correctly in a terminal, we can at least try to provide support for the things that _can_ be done

16:05 <dnkl> Or we tell people to stop using fish. But I have a feeling there will more applications that expect this to work

17:34 <dnkl> sterni: fwiw, looks like fish just does wcswidth() on the whole grapheme cluster. But I just tested this very briefly, with a single emoji, the 👩‍👩‍👧‍👧

17:34 <dnkl> the fish prompt broke down when I used two cells in foot, rather than 4

17:42 <dnkl> sterni: going to test latest fish from git, because so far, I don't see fish being any better than any other shells. Which isn't what I've been led to believe

17:50 <dnkl> hmm, no difference

17:50 <sterni> dnkl: I had problems today with 🕰️ wheras ⌚ is fine, so I'm fairly sure that this specific issue is caused by the variant selector somehow

17:57 <dnkl> sterni: yeah, it is. The base emoji has a wcwidth == 1, and the selector has wcwidth == 0. But we want wcwidth=2. And yeah, it does appear fish understands this, as this particular case works fine in kitty

17:57 <dnkl> grapheme clustering by itself wont help here, but tweaking our width calculation will

18:00 <dnkl> sterni: anyway, I've resurrected the grapheme clustering branch. Will see what I can do about the width calculation.

18:08 <sterni> dnkl: the double width character lie comes back to haunt us

18:08 <sterni> I wonder why ppl introduced this in the first place a double width glyph is such an odd concept

18:10 <dnkl> sterni: I believe they initially were single width. Some fonts still have them as single-width glyphs. But to then suddenly render them as double width... obviously they who thought that was a good idea weren't terminal emulator users... :)

18:10 <sterni> dnkl: does the (glibc) wcswidth(string) == cells occupied in foot invariant hold currently?

18:11 <sterni> dnkl: yeah well honestly I am more inclined to blame whoever started to emulate this behavior with terminal emulators

18:11 <sterni> instead of forcing the glyphs into single cells

18:13 <dnkl> sterni: about wcswidth, yes, also in the resurrected grapheme clustering branch

18:13 <dnkl> Except when we see an emoji variant selector

18:13 <dnkl> Then we hardcode it to 2 😅

18:16 <dnkl> sterni: hmm, except that in master we don't do clustering, so we never call wcswidth on a string of characters. Only wcwidth in single codepoints

18:17 <sterni> do we reserve two cells for codepoints with wcwidth == 2?

18:25 <dnkl> Yup

18:26 <dnkl> But note that zero-width characters can be appended to the previous cell. This is mainly to deal with composed characters, not emoji sequences