68
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
this post was submitted on 14 Mar 2024
68 points (91.5% liked)
Technology
58761 readers
4199 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related content.
- Be excellent to each another!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, to ask if your bot can be added please contact us.
- Check for duplicates before posting, duplicates may be removed
Approved Bots
founded 1 year ago
MODERATORS
GPT4 is pretty good now. I'm not convinced it will be secure until we can run it locally on our own hardware.
As soon as we can run it locally, I plan to do so. Even if it means using a GPT4 quality LLM when far better exists if I use a cloud service.
Sure it would be nice to have something that hallucinates less than GPT-4, but I kinda feel like striving for that is making perfect the enemy of good. I'd rather stick with GPT-4 quality, and focus on usability/speed/reliability/etc and let people keep working on the fancy theoretical stuff in the background as a lower priority.
A Steve Jobs said, Real Artists Ship. They don't keep working on it forever until they can't think of any more improvements. You'll never ship.
Seems like it would be trivial to just place tokens in a buffer on the server and send output to the client in say 1KB chunks (a TCP packet can't be much bigger than that anyway, and it needs a bit of space for routing metadata).
And if the entire output is less than 1KB... pad it out to that length. Pretty standard to do that anywhere you care about security... e.g. if you were to dump the password table databases... they're all 256 bits. That's obviously not the real length - most will be shorter, some will be longer. Whatever they are it's cryptographically expanded (or shortened) to 256.