Track_Shovel@slrpnk.net to Lemmy Shitpost@lemmy.worldEnglish · 1 day agoHexadecimalslrpnk.netexternal-linkmessage-square91fedilinkarrow-up1890arrow-down122
arrow-up1868arrow-down1external-linkHexadecimalslrpnk.netTrack_Shovel@slrpnk.net to Lemmy Shitpost@lemmy.worldEnglish · 1 day agomessage-square91fedilink
minus-squaremorrowind@lemmy.mllinkfedilinkarrow-up1·18 hours agoNot really a concern. It’s basically translation, which language models excel at. It just needs a mapping of the hex to byte
minus-squareGissaMittJobb@lemmy.mllinkfedilinkarrow-up1·17 hours agoIt is a concern. Check out https://tiktokenizer.vercel.app/?model=deepseek-ai%2FDeepSeek-R1 and try entering some freeform hexadecimal data - you’ll notice that it does not cleanly segment the hexadecimal numbers into individual tokens.
minus-squaremorrowind@lemmy.mllinkfedilinkarrow-up1·17 hours agoI’m well aware, but you don’t need to necessarily see each character to translate to bytes
minus-squareGissaMittJobb@lemmy.mllinkfedilinkarrow-up1·16 hours agoIt’s not out of the question that we get emergent behaviour where the model can connect non-optimally mapped tokens and still translate them correctly, yeah.
minus-squarekautau@lemmy.worldlinkfedilinkarrow-up1·30 minutes agoI’m confused, is the concern when the model doesn’t properly identify when it is using software to identify something like a hex pattern?
Not really a concern. It’s basically translation, which language models excel at. It just needs a mapping of the hex to byte
It is a concern.
Check out https://tiktokenizer.vercel.app/?model=deepseek-ai%2FDeepSeek-R1 and try entering some freeform hexadecimal data - you’ll notice that it does not cleanly segment the hexadecimal numbers into individual tokens.
I’m well aware, but you don’t need to necessarily see each character to translate to bytes
It’s not out of the question that we get emergent behaviour where the model can connect non-optimally mapped tokens and still translate them correctly, yeah.
I’m confused, is the concern when the model doesn’t properly identify when it is using software to identify something like a hex pattern?