Track_Shovel@slrpnk.net to Lemmy Shitpost@lemmy.worldEnglish · 1 day agoHexadecimalslrpnk.netimagemessage-square97fedilinkarrow-up1895
arrow-up1895imageHexadecimalslrpnk.netTrack_Shovel@slrpnk.net to Lemmy Shitpost@lemmy.worldEnglish · 1 day agomessage-square97fedilink
minus-squareGissaMittJobb@lemmy.mllinkfedilinkarrow-up1·17 hours agoIt is a concern. Check out https://tiktokenizer.vercel.app/?model=deepseek-ai%2FDeepSeek-R1 and try entering some freeform hexadecimal data - you’ll notice that it does not cleanly segment the hexadecimal numbers into individual tokens.
minus-squaremorrowind@lemmy.mllinkfedilinkarrow-up1·17 hours agoI’m well aware, but you don’t need to necessarily see each character to translate to bytes
minus-squareGissaMittJobb@lemmy.mllinkfedilinkarrow-up1·17 hours agoIt’s not out of the question that we get emergent behaviour where the model can connect non-optimally mapped tokens and still translate them correctly, yeah.
minus-squarekautau@lemmy.worldlinkfedilinkarrow-up1·60 minutes agoI’m confused, is the concern when the model doesn’t properly identify when it is using software to identify something like a hex pattern?
It is a concern.
Check out https://tiktokenizer.vercel.app/?model=deepseek-ai%2FDeepSeek-R1 and try entering some freeform hexadecimal data - you’ll notice that it does not cleanly segment the hexadecimal numbers into individual tokens.
I’m well aware, but you don’t need to necessarily see each character to translate to bytes
It’s not out of the question that we get emergent behaviour where the model can connect non-optimally mapped tokens and still translate them correctly, yeah.
I’m confused, is the concern when the model doesn’t properly identify when it is using software to identify something like a hex pattern?